db_columns blank for some number fields after upgrade

I would say... a) make sure it works in the source site, then b) upgrade manually and see if there is any error reported. Just in case, review the upgrade instructions and follow the steps carefully.

Many sites have upgraded successfully, so this seems something odd in your environment. Switching to support request...

I had exactly the same problem. As suggested by raintonr above , manually selecting number_update_6000 to run when running update.php solves the problem.

Thanks a ton for the tip raintonr. May be this is actually a bug after all.

Abhaya

I also ran into the same problem during D5->D6 migration. I can provide a few more details that might help to reproduce it.

An example field that was not migrated properly (db_columns blank in D6 after migration) is an integer field with widget type checkbox and two values listed in the "Allowed values" field. Other (apparently - all) integer fields with select list widgets and "Allowed values" were similarly corrupted. Here is the SQL for the original field's definition in D5:

INSERT INTO `node_field_instance` (`field_name`, `type_name`, `weight`, `label`, `widget_type`, `widget_settings`, `display_settings`, `description`) VALUES ('field_orga_intern','organisation',4,'Interne Nutzer','options_onoff','a:2:{s:13:\"default_value\";a:1:{i:0;a:1:{s:5:\"value\";i:0;}}s:17:\"default_value_php\";s:0:\"\";}','a:0:{}','');
INSERT INTO `node_field` (`field_name`, `type`, `global_settings`, `required`, `multiple`, `db_storage`) VALUES ('field_orga_intern','number_integer','a:9:{s:6:\"prefix\";s:0:\"\";s:6:\"suffix\";s:0:\"\";s:15:\"append_position\";N;s:9:\"db_length\";N;s:11:\"db_unsigned\";N;s:3:\"min\";s:0:\"\";s:3:\"max\";s:0:\"\";s:14:\"allowed_values\";s:20:\"0|\r\n1|Interne Nutzer\";s:18:\"allowed_values_php\";s:0:\"\";}',1,0,1);

The D5 version of CCK used was 5.x-1.12. The target D6 version of CCK was 6.x-2.8. I followed the recommended migration steps, and in the end update.php did not offer any more automatic updates. As the original poster mentioned, forcing update 6000 from number.module manually fixed the problem. I conclude that there must be some state on the migration path in which executing this very update does not have the intended effect (ordering issue?), so you are left with an inconsistent database, which is rather tough to debug because the problem simply pops up in Views UI (for instance, I could not select the field as a filter). If this bug cannot be tracked down and fixed, I would suggest that an assertion is placed somewhere to shorten the bug hunt for anyone affected.

FWIW, I just ran into exactly this bug. I was starting with a D5 site with 8 CCK number_integer fields, some using a textfield widget, others select lists. I've been scripting the upgrade, which looks something like this:

- Grab a clean copy of 6.x core
- drush updatedb -y (no errors)
- clone cck and checkout the end of the 6.x-2.x branch
- drush updatedb -y (no errors) -- it says number_update_6000() was run (although I don't see anything that would suggest that would impact this bug)

At this point, the number fields were all basically broken. I was getting a ton of PHP warnings when trying to view or edit nodes, and none of the values were there. After some debugging to figure out why, I discovered that {content_node_field}.db_columns was empty for most of them:

mysql> select field_name, db_columns from content_node_field where db_columns = '';
+------------------------------+------------+
| field_name                   | db_columns |
+------------------------------+------------+
| field_show_related           |            |
| field_frontpage              |            |
| field_dateline_behavior      |            |
| field_story_weight           |            |
| field_image_crop_type        |            |
| field_byline_has_affiliation |            |
| field_story_label_has_names  |            |
+------------------------------+------------+
7 rows in set (0.00 sec)

After a bit of experimentation, I discovered that creating a new dummy content type with some number fields was working fine. In that case, db_columns was set to: 'a:1:{s:5:"value";a:3:{s:4:"type";s:3:"int";s:8:"not null";b:0;s:8:"sortable";b:1;}}'. Makes sense, and from what I can tell, all number_integer fields, regardless of widget, share these values. Therefore, I ran this query:

UPDATE content_node_field SET db_columns = 'a:1:{s:5:"value";a:3:{s:4:"type";s:3:"int";s:8:"not null";b:0;s:8:"sortable";b:1;}}' WHERE type = 'number_integer' AND db_columns = '';

This seems to have repaired the site and things appear to be working fine again. All the old values for these number_integer fields are loading and working, etc.

I have no explanation for why these columns were empty, so I don't really know what the bug is, but it really does appear to be a bug. If I knew the internals of CCK better, I'd try to dig further and fix this, but I really don't have time for that now. Just wanted to update here about my experience and work-around.

@CCK maintainers: can there ever be a number_integer field that doesn't use the above value for db_columns? If not, perhaps a number_update_N() that just runs the above query would be safe, and could at least paper over this bug, even if we don't understand why it's needed? ;)

Cheers,
-Derek

This is something that didn't exist in D5, we added it in D6 because we found we had to have this information even if the field module that provided the field was no longer available. Somewhere in the original D5 -> D6 upgrade path that column gets populated. Since you are jumping straight from D5 to D7 you're missing it.

It requires that the D6 module be available to find out what values should be in those columns. I have no idea how you can do that if you're skipping D6.

And it is needed for *all* field types, because the Content Migrate module uses those values to figure out where the data is.

I am migrating from D5 to D6 first. I'm neither skipping nor jumping. ;) Look at my steps in comment #5...

My struggles with D7 content_migrate can be found in other issues, in particular:
#1092840: Allowed values for list_number and list_boolean not converted to arrays
#1066268: PHP Notice: undefined index when migrating decimal number (which I should rename, it's much more serious than a PHP notice in my case)
#1059214: Batch processing problem causes migration to fail with "1062 Duplicate entry" error
#1078804: 'longtext' columns converted to varchar(255) on migration

But this was just to get things working again in the D6 version... (which I might fall back to, this D7 upgrade has been hell so far).

It *did* work on most fields, just apparently not the number.module fields. I definitely had number.module enabled when I was trying any of the cck-related updates. I left this out of my steps in the issue (but it's there in my script):

drush en -y content fieldgroup filefield imageapi imagecache imagefield optionwidgets nodereference number text

(which comes just before the drush updatedb). Maybe I should run updatedb before trying to enable?

Any other suggestions I should investigate? I'm about to start another whole D5 -> D6 -> D7 data migration again this morning.

Thanks!
-Derek

Okay, I'm running through from the original D5 site, and I hit this again. I couldn't run updatedb without enabling everything, since the updates fail in that case. So, my steps really are:

1. Grab a clean copy of 6.x core
2. drush updatedb -y (no errors)
3. clone cck and checkout the end of the 6.x-2.x branch
4. grab latest 6.x versions of filefield, imageapi, imagecache, imagefield
5. drush en -y content fieldgroup filefield imageapi imagecache imagefield optionwidgets nodereference number text
   At this point, the following are enabled:

   drush pml
    Package          Name                         Type    Status         Version  
    CCK              Content (content)            Module  Enabled                 
    CCK              Content Copy (content_copy)  Module  Disabled                
    CCK              Content Permissions          Module  Not installed           
                     (content_permissions)                                        
    CCK              Fieldgroup (fieldgroup)      Module  Enabled                 
    CCK              FileField (filefield)        Module  Enabled                 
    CCK              FileField Meta               Module  Not installed           
                     (filefield_meta)                                             
    CCK              ImageField (imagefield)      Module  Enabled                 
    CCK              Node Reference               Module  Enabled                 
                     (nodereference)                                              
    CCK              Number (number)              Module  Enabled                 
    CCK              Option Widgets               Module  Enabled                 
                     (optionwidgets)                                              
    CCK              Text (text)                  Module  Enabled                 
    CCK              User Reference               Module  Not installed           
                     (userreference)                                              
   

   Note: Version is blank since I'm cloning from Git but haven't yet enabled git_deploy at this point. However, here's the relevant chunks of the drush make file I'm using:

   projects[cck][download][type] = "git"
   projects[cck][download][module] = "cck"
   projects[cck][download][revision] = "6.x-2.x"
   
   projects[filefield][download][type] = "git"
   projects[filefield][download][module] = "filefield"
   projects[filefield][download][revision] = "6.x-3.9"
   
   projects[imageapi][download][type] = "git"
   projects[imageapi][download][module] = "imageapi"
   projects[imageapi][download][revision] = "6.x-1.9"
   
   projects[imagecache][download][type] = "git"
   projects[imagecache][download][module] = "imagecache"
   projects[imagecache][download][revision] = "6.x-2.0-beta10"
   
   projects[imagefield][download][type] = "git"
   projects[imagefield][download][module] = "imagefield"
   projects[imagefield][download][revision] = "6.x-3.9"
   

6. drush updatedb -y (no errors) -- it says number_update_6000() was run:
   

   ...
   Executing number_update_6000                                           [success]
   UPDATE {content_node_field} SET type='number_float' WHERE    [success]
   type = 'number_decimal'
   ...
   

7. Inspect DB, find badness 10000:
   

   mysql> SELECT field_name, type, db_columns FROM content_node_field WHERE db_columns = '';
   +------------------------------+----------------+------------+
   | field_name                   | type           | db_columns |
   +------------------------------+----------------+------------+
   | field_show_related           | number_integer |            |
   | field_frontpage              | number_integer |            |
   | field_dateline_behavior      | number_integer |            |
   | field_story_weight           | number_integer |            |
   | field_image_crop_type        | number_integer |            |
   | field_byline_has_affiliation | number_integer |            |
   | field_story_label_has_names  | number_integer |            |
   +------------------------------+----------------+------------+
   7 rows in set (0.00 sec)
   

8. Find that one of the number fields *was* properly migrated: (weird!)
   

   mysql> SELECT field_name, db_columns FROM content_node_field WHERE type = 'number_integer' AND db_columns != '';
   +--------------------------+-------------------------------------------------------------------------------------+
   | field_name               | db_columns                                                                          |
   +--------------------------+-------------------------------------------------------------------------------------+
   | field_original_image_nid | a:1:{s:5:"value";a:3:{s:4:"type";s:3:"int";s:8:"not null";b:0;s:8:"sortable";b:1;}} |
   +--------------------------+-------------------------------------------------------------------------------------+
   1 row in set (0.00 sec)
   

Any ideas? I'm just going to manually repair again and keep going, but I wanted to document all this for clarity.

Thanks!
-Derek

Sorry, dww, I did read it but answered too quickly :)

Anyway I dug into where this got set originally and here's the scoop:

There is a very important function that gets run when you enable a field module in D6: content_associate_fields(); That function does things like populate the columns in that table and setting the name of the module that controls the field and widget (also needed for the D7 migration).

There are some cases where because of module weight it didn't run when it should, and in those cases we added it to the field .install file (i.e. text module has this).

In a normal update it seems to work fine to get all those fields populated. At least I have not seen any problems. But maybe when you do it the way you are doing it things are not triggering in the right order.

Try enabling the content module first, then doing back and enabling the individual field modules. That should ensure that the content module is ready and waiting to generate that function when the field module is enabled.

And be sure to run updatedb *after* enabling the content module and *before* enabling the field modules. Then run it again after all the field modules are enabled.

If you still have trouble (that column is still empty) after doing all that, you can try running content_associate_fields('number') from drush and see if that helps.

Yeah, still not happening automatically. Even if I split step #5 above into these:

drush en -y content
drush en -y fieldgroup optionwidgets nodereference
drush en -y filefield imageapi imagecache imagefield

it still fails.

However, after the updatedb, this works fine:

drush php-eval "content_associate_fields('number');"

Any objection/harm in adding a number_update_6001() that does that automatically? I'd be happy to roll such a patch if that was a good idea. ;)

Thanks!
-Derek

Go ahead and roll a patch. We don't want to run that function on a site that has those tables populated already because we might change other values that should not be changed, so the update should only run if the table column is empty.

Oddly, adding that in number_update_6001() still doesn't work. :( Something funny must be going on with ordering and weights, although all the CCK-related modules in the D5 DB have weight 0 except for fieldgroup (which is at 9).

Note that there's no way to *just* run content's DB updates without number and the rest of CCK itself. Once I put CCK on the site, as soon as you run updates, all updates are run, even for modules that aren't yet enabled. I sort of understand why that behavior is necessary in some cases, but it sure makes things difficult for a lot of others. ;)

Anyway, for now, I'm just leaving that drush eval step in my migration and it's working. I wish I could figure out a way to fix this upstream in CCK itself so others could benefit, but I just don't have time to keep trying. Alas.

I had the same issue, doing exactly the same thing (upgrading from Drupal 5 to 7). dww's drush command fixed it for me too, I just put it into my enormous bash script that doubles in length by the second. :O

If adding this to an update_6001 doesn't work, I'm out of ideas. I don't have the problem and can't relicate it, so the only thing I can do is apply a patch that works, and apparently there is no patch that works.