The block in d5/d5.inc, from lines 95-121, seems to rely heavilly on a column called db_columns in the table node_field_instance. This column does not exist for drupal 5 (it is not created by cck).

Because of this, the foreach loop at 108 does not execute, and on line 116, the $columns array is empty.

To me, it seems the entire if block (95-121) does not do anything if db_columns does not exist. As a result, I commented out 95-121 and it seems to be working for two drupal 5 to drupal 7 migrations.

To check to see if the latest 5.x version of cck has db_columns in its node_field_instance table:

git clone --recursive --branch 5.x-1.x http://git.drupal.org/project/cck.git
cd cck
grep -nr 'db_columns' . 
# no results

# Although 6.x-3.x does.
git checkout 6.x-3.x 
grep -nr 'db_columns' . // Lots of results, 

Conclusion: db_columns doesn't exist for cck in drupal 5.

I'm not sure if there is a better way to do this.

I have submitted a patch which deletes these lines, based on the latest checkout of migrate_d2d.

Please let me know if I am misunderstanding something or if I can help.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

mikeryan’s picture

Status: Active » Needs work
FileSize
1.35 KB

I don't have any D5 sites to analyze, so it's hard to say exactly how to determine the column names to pull. Simply deleting the whole section, though, isn't going to do it - $this->sourceFieldInfo will never get populated with the field info. The basic problem here is that it appears CCK on D5 didn't store column info in the DB, it only obtained it via hooks, which are no good to us here. The only option I can see is to inspect the tables for columns matching the field name to determine what columns they have.

In the meantime, the attached patch (or something like it) should at least work for fields with a single 'value' column.

roberttstephens’s picture

Thanks a lot for the patch, you're absolutely correct. Deleting the code just got rid of the error, and (obviously, in hindsight) doesn't add cck fields.

The patch looks like enough to get working. I'll update if I find a more robust way to do it.

Thanks for replying!

mikeryan’s picture

Priority: Normal » Major

I consider this a release blocker - unfortunately, I don't have a D5 database to test with (and my attempt to install D5 locally failed miserably, not sure what the max PHP version it works with is but I don't have it convenient...). Is there anyone doing D5 imports that can take a crack at this?

meramo’s picture

I'm doing the D5>D7 migration at the moment, same issue here. Will provide a detailed feedback after applying patch, thanks for it!
UPD. It worked

mikeryan’s picture

OK, I'm actively looking at this again - D5 does work with PHP 5.2, although it's very helpful to ignore E_NOTICE in error_reporting, so I've got a functional source installation to work with. I think the best approach is simply to hard-code knowledge of the columns provided by the built-in CCK fields and the most popular external CCK modules (link, filefield, etc.), and make _value the default assumption otherwise.

KarenS’s picture

The lack of column names in the table in D5 CCK was a huge problem for D5 to D6 'normal' migrations. That is the point where we realized that info had to be in the database and we added it in. But there is nothing that can be done about the fact that it wasn't there earlier :(

I agree that the best you can do is to hard code info for known fields. Hopefully most of the one-off fields stuck with the choice of using 'value'. The "good" news is that there were not a huge number of custom CCK field modules in D5, that really took off in D6. So hitting the most-used fields should be a pretty comprehensive list of the possibilities.

KarenS’s picture

Hmm, there is something wrong with that list of D5 modules because Date is not on the list, and Date definitely had a D5 release. Other than that, I guess that list is the best you can do. I notice that some modules on the list don't actually provide fields, they either provide utility functions or widgets, and you won't need either of those. So the list of 'field' modules is relatively small.

KarenS’s picture

FWIW, I started a list of field columns, using the above list. I only went down a couple pages into the 8 page list, but you can see that a pretty good percentage aren't fields that need migration (although there may be migration info related to widgets or formatters.) And some of them have no D7 release, which raises the question of what you would migrate them into.

  • Date: value, value2, timezone, offset, offset2, rrule
  • Link: url, title, attributes
  • Filefield: fid, description, list
  • Imagefield: fid, title, alt
  • Emfield: embed, value, provider, data (+ anything added by emfield_field_columns_extra() in other modules)
  • Contemplate: Not a field
  • Content Taxonomy: value
  • Fivestar: rating, target
  • Filefield Paths: separate table - filefield_paths with type, field, filename, filepath
  • Hierarchical Select: Not a field
  • Conditional Fields: Not a field
  • Phonefield: value (No D7 release)
  • Viewfield: vname, vargs
  • CCK Blocks: Not a field
  • Checkbox Validate: Not a field
  • Editable Fields: Not a field
  • Multiselect: Not a field
  • Unique Field: Not a field
  • Link Image Field: fid, title, alt, url (No D7 release)
  • Imagefield Tokens: Not a field
  • CCK Fieldgroup Tabs: Not a field
  • Colorpicker: value
  • Flicker: id, type, nsid; separate flickr_users table with uid, nsid, identifier
  • Nodereferrer: Doesn't create database entries
  • Jammer: Not a field
  • Embed GMap: value (No D7 release)
  • Flash Video: Not a field
  • Search and Replace Scanner: Not a field
  • Node Access Userreference: Not a field
  • Field Indexer: Not a field
  • Relevant Content: Not a field
  • UC Node Checkout: Not a field
KarenS’s picture

In case anyone is interested in finishing that list, here's what you have to do:

- Go to the module page and look for the 'repository viewer', then roll down and dig into the D5 release at the bottom of the viewer page. This avoids the need to download a local copy of all these modules, but you could do that as an alternative.
- See if this is a module that provides a field at all by looking for an implementation of hook_field_info() in the .module code. If it doesn't implement that, it's not a field.
- If it is a field, look for the column definitions. They will be in hook_field_settings using the $op of 'database columns'. Generally you can just scan the code for the word 'columns'. This operation tells you the columns this field implements, which is what we need.
- Also look for a .install file to see if other tables are defined in hook_schema, because they probably need to be taken into account.

mikeryan’s picture

Priority: Major » Normal

I've committed a bunch of work to fix up the D5 support. The one gotcha is we don't have a way to handle fields with optional columns (I'm looking at you, Date;). Having done the hard-coded-column route (thanks Karen for your research), maybe we should just query the tables and see what pops up...

KarenS’s picture

Yeah, my basic idea for optional columns is you look for each possible column and use it if it exists. Sorry about the optional column complexity. Date is not the only module that does an optional column, FWIW :)

mikeryan’s picture

Status: Needs work » Fixed

Now why didn't I just query the tables and see what columns were there in the first place? So simple, so flexible...

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

Anonymous’s picture

Issue summary: View changes

Adding code brackets around code.