CCK Imports

I am updating the content.inc file to properly handle CCK imports. I am adding in the ability to import into all field columns for fields that have more than one column, to properly clean up dates and other special types, and to allow a way to import multiple values into multiple value fields. I have something working fairly well except for the multiple values, which I am still working on.

Here is what I have so far. I'll post an updated version when I get multiple values working right. This is still beta quality, so don't use it anywhere that is mission critical until it's completely debugged.

Well the problem with multiple value fields will not be so easy to fix right now, so here is the updated file with a TODO on getting multiple values working. The problem is that CCK is inserting three blank values before the legitimate values in all multiple value fields, so a fix to CCK is needed, but that is part of a whole bunch of fixes to multiple values that needs to be made, so it won't get done right away.

This file should work for all single value CCK fields, though.

Changing status.

I have done more research and found that multiple value fields will import correctly with two small fixes in this module (attached). The other part of the problem has nothing to do with the import, CCK is not displaying the multiple values correctly, so I will work on a fix for that on the CCK side of things.

Just reporting back that the CCK fixes are made, so everything should work with the small patch to Node Import and the altered content.inc file (need to remove the line at the top of that file that says multiple value imports don't work since they now do).

Thanks KarenS for investigating this. If I understand correctly, for CCK to work, we need the content.inc of comment #2 and a small patch of #4?

I'll check it out this weekend.

Correct. Thanks!

Also, be sure to use latest CCK snapshot since I had to make some fixes there, too.

Commited to HEAD. Thanks KarenS.

Needs some more testing! Use with care!

Setting back to "active" until it is properly tested. Note that as KarenS says: multiple value CCK fields are not yet supported.

I noticed that for fields with multiple columns, I could map columns from my CSV, but they weren't being stored. A little trouble shooting showed the problem to be in the code that stored the value into the $node. I've found a solution that seems to work. I think this is basically just a syntax issue. :-)

Patch attached.

The exact patches here didn't seem to work for me, so I did some digging. A quick print_r of the node object just before it gets saved made me notice that depending on the widget type, the construction of each cck field was different. So I fixed it, seems to work for me.

Someone should review this patch, clean it up so it's neater code, verify that it works (but it works for me).

Dave

Just so people know, there is a problem with my patch as well. It doesn't seem to work in all environments. I used it on one server and it works, and on another server it seems to fail. Not sure if another contrib module is causing problems or not.

Dave

the patch from comment 11 seems to work for me.
this is with the following modules installed (and enabled for the used content type, where relevant):
auto_nodetitle cck node_import token views

Hi, the patch on comment 11 does not work for me. The preview shows the multiple-select field but it's not being stored in the node once imported. No, error messages.

Update : the patch does work for me. My "Allowed values list" had a key and a label. Once I removed the label, the patch works as advertised.

Confirmed that this is indeed a bug in the current code. Here's a slightly simpler patch.

The issue occurs when processing multiple columns after the first column. The $delta value is unchanged (we don't have multiple values) so, apparently, our new record gets overwritten; it has the same delta as the previously-set array key.

The attached patch declares arrays as needed and then fills them with the values, keyed by delta. This should work whether or not we have multiple values. Multiple values create a new delta key, multiple columns create a new column one.

The widget type handling in patch #11 appears to address a distinct issue (which I haven't investigated). I've left it out of this patch.

patch #15 works for me but does not fix the key|label issue (at least for me). Was it meant to?

@jt_jones: I don't understand what the key|label issue is. Could you explain?

When I first tried the node import module, text fields were imported just fine but multiple select-list data would not import at all. Now, using patch #11 and #15 select-list data will import if the list is apple, orange, banana (and one of those items is in the cell). But if the list is 0|apple, 1|orange, 2|banana then the import does not work.

I think this is confusing to the user since CCK allows multiple values in a select-list field (and the node import instructions don't specifically say don't use select-list fields). Here is the help text from the Allowed-values-list text area;

"The possible values this field can contain. Enter one value per line, in the format key|label. The key is the value that will be stored in the database and it must match the field storage type, text. The label is optional and the key will be used as the label if no label is specified."

Patch #11 works for me for a single value from a multi-select field, but not for multiple values, till I realised that the module used a double pipe "||" rather than a single pipe "|" to separate list items.

// Explode multiple values to create the $delta and $value for each.
           if ($field['multiple']) {
             $values = explode('||', $node->$dummy_name);

This is about line 83 on my patched content.inc file.

Perhaps as the general documentation suggests using a single pipe this should be changed?

I haven't tested this with Key|Label pairs on, only with Key's.

UPDATE: This was working, but after a DB delete and restore operation has ceased again.

Ok, I think this issue started off with a problem similar to mine.

Basically when I am trying to import a file into a cck node containing userreference, date (with 'to date' enabled) and text fields, only the text fields are being imported.

The date settings are 'datestamp' and 'select' and the user reference just references the uid. The uid in the csv is set as 1 and the dates are set to '20 January 2008', '22 January 2008' respectively.

I have debugged this as much as my skills allow me for now and have outputted the node object so that I can see what's going off. As far as I can tell all the fields are set in the node object as they should be up until node_submit($node); on line 654 of node_import.module, after this function the userreference field has an error in it and the date fields (value and value2) are blank. If I comment this line out, the correct data gets put in.

I believe from reading the API that node_submit prepares the node for save and allows modules to hook in. (forgive my terminology, i'm a bit new to this). I am imagining that CCK Userreference and Date are hooking in at this point and trying in their own ways to prepare the data for the fields and, as they receive the already prepared values and not the raw format they expect, their functions error.

Am I right?

And should node import actually be preparing CCK values that the respective CCK modules expect to recieve rather than preparing values ready to directly insert into the database?

I hope that makes sense and that someone has a clue what I am on about :)

Tom

I think you're probably right, unfortunately I have no idea how to make your suggested changes!

Commenting the node_submit fixed the problem for me however I'd prefer a better solution!

I've also had this problem - text fields import fine, but date fields get blown away - tommo's post #20 helped me track things down, and making this change to content.module (in cck) fixed the problem for me. However, all the usual disclaimers apply - I'm *very* new to drupal, and though I can now import a simple date field with no problems, (no time zone set, no multiple date) I hope it won't affect anything else.

Anyway, I'm just putting this up in hopes that it will help some other people, and maybe lead to a more elegant/permanent fix.

nb: the patch I posted in #22 will make node_import work, but breaks normal
node creation via the web interface. It looks like the node_submit() isn't
formatting the date the way that cck expects, because the date field I'm
working with is a select field. Still looking into this...

Just to let you know that patch #15 works well for me now with data for CCK Address Field (including Canadian support). Thanks.

I tried adding #patch 11 but am not able to do it properly.
I shall be be grateful if anyone who successfully added #patch 11 posts the entire code of the content.inc page
Many thanks

I tried to add #patch 15 but could not do it properly. Could you please post or attach the entire code of the page after you added the patch.
Many thanks in advance.

Froggieone, here's the complete content.inc that has patch #15 already added to it. Make sure you put it in the right place (node_import\supported\cck\) and remove the .txt extension.

THANK YOU IncrnB!
I am so very grateful to you for this help.
I will try this code today.
May God bless you always.

Yes, I second that. Could we get this committed or maybe split this topic up since this patch doesn't solve the date problems, but DOES solve the address issue?

incrn8 said:

Just to let you know that patch #15 works well for me now with data for CCK Address Field (including Canadian support). Thanks.

Yes, I second that. Could we get this committed or maybe split this topic up since this patch doesn't solve the date problems, but DOES solve the address issue?

It looks like the code in cck/nodereference.module is expecting the $items array (@ about line 276):

$items[0]['nid'] = $items['nids'];

to look like:

array {
  [nids] = 47
}

But the data from node_import is already formatted into $items[0]['nid'] and therefore gets set to nothing. And then none of the nodereference columns get set in the db.

I tried just dropping in the above patches but it didn't work for me (or maybe I'm too tired of staring at the same code and made a silly mistake). I'm still getting a feel for Drupal, so I'm not sure if the real problem is in cck/nodereference.module or in node_import/supported/cck/content.inc.

I applied the patch for importing cck select list field and check box field
as I can't import it from node_import module.
After I applied the patch, I 've got the below error message.
Fatal error: Cannot unset string offsets in
D:\AppServ\www\3rkh\modules\link\link.module on line 282
Where can I fix that error?
I think the bug fix should be in somewhere within the patch but I couldn't
find it.
Because the original content.inc dodn't give any error message like that.
Thanks in advance for your advice.

How about just re-downloading the original module from here, uploading it to your server on top of the current files?

Dave

Quick update: the already patched content.inc file in comment 28 works with the new 5.x-1.6 version as well.

I applied that patch from comment 28 and I got that error message.
My drupal version is 4.7.

Subscribing, greetings, Martijn

I don't know about the other people here, but I've been doing all these patches to the 5.x version of the module, so I don't even know if they work for 4.7 anymore.

+1 for nedjo's patch. #15 works perfectly with the CCK Address field version 5.x-2.1 under Drupal 5.9. Thank you!

Could this issue be related to another that I'm having, that Text (only in a dropdown), UserReference and NodeReference don't show on the preview screen!?
Could this be related to #32?

@wwam
I had the same problem. The patch doesn't agree with link.module. I don't know if this works for you, but I simply removed that field from my content type, and replaced it with a simple CCK text field that I labeled "Website URL". After that, I imported fine (after applying the #15 patch above).

Subscribing

Patch #15 + cck_address module, if you are importing the address_state field (full length: ie Indiana), you need to make sure in the cck_address field settings (when managing your content type) is set to State abbreviations. Otherwise, all imports fine, but the state does not show on the node output. I have not taken the time to sort through the code to check why but did notice this to be an issue.

I'm using the content.inc from #28 and still can't figure out how to upload into select list fields. I am willing/able to groom my source csv as needed, but I don't understand what it should look like even after reading a bunch (below).

If I have a field called Organization Type with choices corporation and non-governmental organization, what should appear in the csv? When I use corporation it looks fine on the mapping screen, but does not show up in the preview or imported node.

For a multi select field for countries, how should the input look? A csv entry like India|China|United States gives me the whole string India|China|United States on the mapping screen, nothing in the preview or imported node.

My lists generally use the same string as key and value.

It would be helpful if someone could explain how the csv data should be formatted. Thanks in advance for any help.

For reference, other issues related to this are:
#268728: cck node_import for select list and check box fields
#229776: How to import a CCK checkbox field???
#285384: Importing into select boxes
#273423: Option widget "allowed values" validation
#146158: Request Support For CCK Drop Down List and Radio Button Values
#229552: CCK Imports issues with fields that contain drop down list of data
#165063: CCK_Address Fields

What about India||China||United States? That should work for multiple values.

As for selects, try using the keys of the select, rather than the values. As a secret to finding the keys for newbies, go to node/add/mycontenttype and examine the source of the page, paying particularly close attention to the <option value="key" >, a bunch which should be present inside the select element from your cck form. In this case, key = the thing you want to enter into your csv file in order to get that option. I don't know if this will work, but its worth a shot.

Thanks for the suggestions and quick reply. Unfortunately I'm still having the problem. The data shows up on the mapping screen but there's nothing in the preview or imported nodes. Since this seems to work for others, I wonder if I'm missing something simple.

here's the entire contents of the test csv I'm using -- the only fields that work are Organization name and Primary contact first name:

Organization name,Organization type,Primary contact first name,Countries of operation
Michael's Test Organization,Non-Governmental Organization,Michael,China
test organization 2,Academia,John Doe,Canada||China

+1

Worked perfectly for me (using state abbreviations).

Can you:

1. Go to the node/add/nodetype form that this content is going to be imported into.
2. View source.
3. Copy and paste all of that source into a pastebin.
4. Give us the link to view the source.

Thanks,

Dave

Dave, thanks again. I think I have this figured out.

I added the segment of code from #273423: Option widget "allowed values" validation to content.inc. I didn't expect it to solve the problem I'm having based on the description of what it's supposed to do, but it did. I wish I knew why but for now I'm happy it's working.

If you'd still like to troubleshoot further I'm happy to paste the code from the node/add page.

Best,
Michael

Nope, just trying to help, glad it worked.

Dave

Hi,
I am interested in the code.
By the way, is the code Drupal 4.7?

greetings,
Martijn

Version 4.7.x of the node_import module is no longer supported. Setting this issue as "won't fix".

If you have the same issue with the latest release of the 5.x branch, open a new issue. Thanks.