I have some large tables in CCK (one is over 500,000 entries), and the update for Text #6003 keeps failing. Not only do I have numerous fields and tables, but many of them are TEXT or VARCHAR fields. My server has an internal timeout per page load, and I keep encountering that error. Although I can change that setting, some other systems may not be able to.

Looking at the update (#6001), I'd think that would be easy to convert to a full-on batch.

Comments

fractile81’s picture

Title: Text Update #6003 Fails » Update Text Update #6001 to Batch

Here's a modification of the update to use the built-in batching functionality:

function text_update_6001(&$sandbox) {
  include_once('./'. drupal_get_path('module', 'content') .'/content.install');
  $ret = array();

  if (!isset($sandbox['progress'])) {
    if ($abort = content_check_update('text')) {
      return $abort;
    }

    drupal_load('module', 'content');
    // Get the latest cache values and schema.
    content_clear_type_cache(TRUE, TRUE);

    $types = content_types_install();

    if (empty($types)) {
    	return $ret;
	}

	$sandbox['fields'] = array();
	foreach ($types as $type_name => $fields) {
	  foreach ($fields as $field) {
	    if ($field['type'] == 'text') {
	      $sandbox['fields'][] = $field;
		}
	  }
	}

	if (empty($sandbox['fields'])) {
	  return $ret;
	}

	$sandbox['progress'] = 0;
	$sandbox['visited'] = array();
  }

  $field = $sandbox['fields'][$sandbox['progress']];

  // We only want to process a field once -- if we hit it a second time,
  // that means it's its own table and it should have already been updated.
  if (!in_array($field['field_name'], $sandbox['visited'])) {
    $db_info = content_database_info($field);
    $table = $db_info['table'];
    foreach ($db_info['columns'] as $column => $attributes) {
      $attributes['not null'] = FALSE;
      $column = $attributes['column'];
      db_change_field($ret, $table, $column, $column, $attributes);
      // TODO: errors on text/blob columns: no default value allowed (!)
      db_field_set_no_default($ret, $table, $column);
      if ($attributes['type'] == 'varchar' || $attributes['type'] == 'text') {
        $ret[] = update_sql("UPDATE {". $table ."} SET ". $column ." = NULL WHERE ". $column ." = ''");
      }
      else {
        // TODO: replace format = 0 with format = NULL ?? Is this right ?
        $ret[] = update_sql("UPDATE {". $table ."} SET ". $column ." = NULL WHERE ". $column ." = 0");
      }
    }
    $sandbox['visited'][] = $field['field_name'];
  }

  $sandbox['progress']++;
  $ret['#finished'] = $sandbox['progress'] / count($sandbox['fields']);

  return $ret;
}

Then you also have to update the 6003 update as well:

function text_update_6003(&$sandbox) {
  return text_update_6001($sandbox);
}

With these changes, I've been able to complete the update with my data. There's also an issue where shared fields will have their table updated for each node type that shares it, which is somewhat wasteful. I admit that the $sandbox['visited'] part could probably be done a bit better, but this can hopefully be a starting point for something like this. The real question is whether this is the right approach for the update?

karens’s picture

The idea makes sense, but I don't want to commit this without testing and I don't have a setup that can test this. I'll let yched weigh in on whether or not to add this in.

yched’s picture

+1 on the principle. This update should be multipass. I'll try to test ASAP.

yched’s picture

Status: Active » Fixed

Tested, works fine.
I applied the same treatment to nodereference_update_6000(), which could potentially suffer from this as well.

Committed. Thanks !

PS : IMO we don't need to force sites already upgraded to re-run this update.

Anonymous’s picture

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.