Add support for content_taxonomy

Great. If you need any help with it, ask here.

My catalog taxonomy isn't importing; definitely interested in what you come up with.

Maybe you could post the .inc even if it is untested?

I am very interested in this feature too and would love to do some testing...

I would also love to get my hands on this / help with testing. Patiently awaiting the inc file -- any updates?

Subscribing.

I have used the latest -dev -release, with checkboxes as taxonomy, with the following import results:
- If the field is set as required, the import will give "Illegal choice in field_name element."
- set to not required, the import works but only the first value is imported (tried with only 2 items).

I already have 9000 nodes in csv file. Every node has assigned taxonomy term. Is it possible with this feature to import nodes with already assigned taxonomy terms ?

So can I just add the content_taxonomy.inc to the module? Or what do I do with it if I want to test it? Node Import works great but like others being able to import content_taxonomy fields is critical to me.

Thanks,

V

So to answer my own question in case any other newbies are wondering: Put the .inc file in sites\all\modules\node_import\supported
Then I ran the update.php - just to make sure and it worked great!

Thanks for writing this code! It saved us about 2 months worth of data entry.

I'm not 100% sure what you mean by widget. I am using a Content Taxonomy Field in the custom content type. The vocabulary does NOT have hierarchy. It is a one level vocabulary. I'm also importing another vocabulary which adds coma separated vocabulary terms to the node. This is also through a content taxonomy field but this taxonomy adds new terms to the vocabulary while the other does not allow addition of terms.

I hope this answers your question - if not I'll try to elaborate.

Kind Regards,

V

Can't get this addon working with content_taxonomy_options (Content Taxonomy Checkboxes)

The data displays fine when I put some print array -codebits inside the previously posted code. I can't find yet why it doesn't save the correct data.My taxonomy field is set as required, so it selects only the first one, even though I have 2-3 in every row. And by the first one I mean, the first listed taxonomy term available, not first of the imported ones.

OK - so to answer the question a few back I am using a few widgets sucessfully:

Autocomplete (Freetagging): works in most cases. There were a few terms it had an isssue with but adding them to the vocabulary before import fixed this - wired bug but can be overcome. one of the terms was "autoloaded".

checkboxes/Radial buttons: From this you can only select one term. No issues but your formatting must be bang on - ensure there are no leading or ending spaces.

Did not use select list widget.

My import is complete. Thank you for all your help.

Also it was almost 5000 nodes and ran for about 3.5 hours with no issues.

Cheers,

V

hmm. we've installed this (and have run update.php) and I'm a little confused. our test content type has three content_taxonomy fields:

State
Region
Resort

All three of those use Autocomplete (Free Tagging) widgets. State and Region are set so that the user can't add new tags, Resort is set so new tags can be added.

On Step 4 - the node_import Map File Columns page - we see the Resort selection as a pulldown menu showing the column headers for the imported CSV file. We don't see anything for mapping State, and for Region (which as of right now doesn't have a corresponding column in the CSV file, we see a multi-select field showing the imported files column headers, rather than a pulldown. I'm not sure why the inconsistency in the types of selection widgets here, but that's what we're getting.

In any case, since "Resort" seems to work normally, we map it to the "Resort" column in the imported file, and we go ahead with the import. The resulting nodes don't have the Resort Content Taxonomy field filled in, and we have to do them manually. By the way, the resort names already exist in the Resort vocabulary, so we're not trying to add them. Any ideas?

quick update on my previous post: this module wasn't working because for some reason my ftp client didn't give it the right permissions. so it wasn't being loaded. In any case, I fixed that and the State content_taxonomy field works. However, the Resort Name content_taxonomy field doesn't work. It's really weird - as far as I can tell there's no difference between the two fields. (I've set Resorts so that users cannot add terms, which is how State is set up), and there are matching resort names in the Vocabulary, imported from the same spreadsheet using the Taxonomy Import module. So I know there are no spelling differences or anything like that.

any ideas why some fields would work but others wouldn't?

subscribe

@rconstantine: that seems to have done the trick. thanks.

I tried the file from #11 and importing data into content_taxonomy fields worked! Thanks for adding support for content_taxonomy.

Subscribing...
Great work done here... :)

#11 Works fine for me.
@rconstantine: thanks a lot for the very useful code.

@rconstantine
hello, big thanks for .inc for taxonomy_content, but it not working for tree widget in my case (i'm make import by change it to select list widget)
i have big, multi parent vocabulary, with 7 level depth, and on step 7 i have error mesage from #7.
But big thanks anyway.

a big thanks indeed!!

Thanks for this, I was looking at a solid couple days of input - this patch was perfect.

Thanks, the patch from #11 worked for me too and saved me a lot of time.

As others have pointed out, all terms need to exist before they get added, otherwise it won't work. Apart from this, it works well.

yeah, the patch from #11 is working for me as well. however, it seems content taxonomy import is case sensitive... ?
maybe there should be an option to choose between case sensitive and insensitive ?

anyway, this is a very useful module nonetheless.thanx

for anyone who's interested. attached is an adaptation of the #11 patch, so it supports content_taxonomy checkboxes as well now.
let me know it this is working for anyone.

Today I tried this inc and it worked great!! Thanks guys!!!!

Subscribing and will test....

Thanks very much for the patch (#11). I have installed it with the latest dev release of node import.

I am attempting to use it to import a content type with 2 taxonomies, both with heirarchies, but running into difficulty. I have prepopulated both taxonomies with terms/ parents etc.

I seem able to import content with taxonomy terms if I just list the child term (eg "banana") in my CSV file against the node but not if I show the parent with the child (eg "banana>>fruit"). Unfortunately because in my taxonomy the same child can have several parents (eg banana>>fruit, banana>>yellow etc), I need to import it with the parent relationship.

Does this patch support this functionality? And, if so, am I using the correct syntax to show the child/ parent relationship? (I am using banana>>fruit , as mentioned).

Also, I notice during the import (step 4) that there is an option to map terms to both the Content taxonomy field and core taxonomy field, even though these fields duplicate one another. I'm not sure what I should do here, and have tried all combinations of importing to one, or the other or both of these fields. However, nothing seems to work when I try to import a parent/child taxonomy term.

Also note that I have set both vocabularies to freetagging (deny new terms). I don't know whether I need to use a different widget to get this to work?

Would really appreciate any advice from someone who understands the module/ patch better than me or who has successfully imported content.

Thanks

for anyone who's interested, attached is an adaptation of the #35 patch, so it supports content_taxonomy free tags as well now :

when a tag does not exist, it was not created, it now creates taxonomy term when not in vocabulary when importing nodes (see lines 155 - 123)

let me know it this is working for anyone.

#39 seems to be working for me when applied to a slightly older 6.x-1.x dev version of node_import - thank you for posting this!

#39 Works perfectly for me. version 6.x-1.0-rc4 thank you very much erwanpia

Thank You! Thank You! Thank You!

#39 works perfectly for me, and in fact if I wouldn't have been able to do my import (4 different vocabularies, one of them freetagging) without it.

Subscribing.

#39 worked for me to, 3 vocabularies, one of them freetagging.

I think this should be marked as "reviewed & tested" and only after committed marked as patch to be ported? right?

#39 Worked for me too

From what I tested, yes.
Thanks!

I just ran patch #39 -- thanks so much. I was about to go postal on Drupal before finding this patch.

What format is required to add multiple tags? I've tried all the following, and each time only one tag is applied. So, I'll get concatenated tags like: "one two", "one two three". But I'm hoping to get separated tags like:"one", "two", "three".

Thanks

I think the default option is a |

As in: one | two | three

I'm not sure though...
I can't remember, if it was something you could specify...
Let us know how it goes...

Probably "||" (double pipe).

If the content field is set to have multiple values, you should be presented with a "Multiple separator" option on the "Set options" wizard page.

Excuse me if this sounds ignorant, but where/how do I need to install this patch? We're developing a large music community site, and are using CCK taxonomy to tag all of our content. We're looking to import large spreadsheets of data surrounding concerts, and one of the fields is a dependent taxonomy term (it must be in the system for a user to add it). I can import all the other content, minus this CCK taxonomy field, but without this field, none of the other data is worth importing.

This is not really a patch...

Just place this file in sites\all\modules\node_import\supported\cck

I don't know if it's necessary, but run update.php, just in case...

I think its going wrong when u use a special character like (&).
In my case it is isnt importing fields which contains such a character.
is there a workaround or could you adjust the code for it?

arievdonk

make sure the file you use to import, is encoded as UTF-8

well it makes the problem funny because it is UTF-8 encoded.
Maybe the problem is somewhere in the sql, i cant see anything what looks like escaping special characters.

bump

erwanpia, thanks!
#39 work for me.

I set up content taxonomy fields that have pre-defined parents. For example, everything falls under "Product". Then I have

Product
-- Product Year
-- Product Type
-- Product Detail

When I set up this patch to import into product year, it just import the NEW terms into the highest level taxonomy. Why? Is there any way to get this module to obey the structure I have set up for hierarchy in CCK/Hierarchical Select if the term DOES NOT exist already, and is being imported anew?

Its strange. It seems to update the CCK Content_Taxonomy path correctly on the node page as product/product_year/1900 but in the stock Drupal taxonomy administrator the term 1900 is at the highest level, equal with product. Strange. Help would be appreciated. Perhaps it would make more sense to import all my Taxonomy term options first, then do this import?

Spent all day trying to import records into content type using content taxonomy.
The setup/wizard seemed to work but I would get 100% errors on import.

Came across this in the issue queue unzipped content_taxonomy.zip from #39 and seeks to work.
Using node_import-6.x-1.x-dev.
three content_taxonomy fields.

Thanks for the module and thanks for the patch....good stuff.

So I tested this out and I'm not sure if it is working for my use case. I could use some help with this...

Basically, we have a taxonomy structure that allows for child terms such as Arcade>>DonkeyKong

In our CSV file we have associated these terms in that manner (using the >>) structure. But I am receiving errors as I think that they are not formatted properly.

This is not documented well as I'm unsure as to whether we should be using the taxonomy id's or the actual names for taxonomy terms. Here is a dump (below) from the validate_tree function... when I use 270>>271 in my CSV it doesn't work properly...

 [field_resource_grade_level] => Array
                (
                    [value] => Array
                        (
                            [270] => Array
                                (
                                    [children] => Array
                                        (
                                            [271] => Array
                                                (
                                                    [checkbox] => 271
                                                )
                                        )
                                )
                        )
                )

Any help would be fantastic!

thanks

In response to this (so that others know how this works)... basically you just need to use the name indicated in the Taxonomy not the actual ID. The code that you can download will resolve everything properly for you.

Also, you can use whatever separators that you wanted as you are prompted in one stage.

The patch that you created is fantastic... but unfortunately I was having some problems where most of my taxonomy items weren't actually being setup properly (and nothing was being imported). After a big of work and debugging I figured out what was happening.

Two cases (patch below):
1. In the code you take the first Taxonomy term found $term[0] which in my case doesn't work b/c we have multiple taxonomies with the same terms
2. You also check for an empty parent data... but since you are searching for the terms by name you probably don't actually need to do this check (and commenting it out works for my child terms)

Hope this code helps others.

51,64c51,57
<         $foundterms = taxonomy_get_term_by_name($value['value']);
< 
< 	// Need to loop through the found terms to find a the proper VID match
<         foreach ($foundterms as $index => $termfound) {
<           if ($termfound->vid == $vid) {
<             $tid = $termfound->tid;
<             $terms[$tid] = (array)$termfound;
<             $children = taxonomy_get_children($tid, $vid);
<             $terms[$tid]['children'] = $children;
<             $parents = taxonomy_get_parents($tid);
<             $terms[$tid]['parents'] = $parents;
<             break;
<           }
<         }
---
>         $term = taxonomy_get_term_by_name($value['value']);
>         $tid = $term[0]->tid;
>         $terms[$tid] = (array)$term[0];
>         $children = taxonomy_get_children($tid, $vid);
>         $terms[$tid]['children'] = $children;
>         $parents = taxonomy_get_parents($tid);
>         $terms[$tid]['parents'] = $parents;
68c61
< //        if (empty($data['parents'])) {//we only want top level terms here
---
>         if (empty($data['parents'])) {//we only want top level terms here
77c70
< //        }
---
>         }

What i can make with '>>' separator between two terms?
When i use '||' i assign two and more terms for one node. - this i know
But what means '>>' separator?

freetag cck field have option:
Freetagging settings:
- Allow and insert new terms by the user into the vocabulary
- Deny any new terms
latest patch don't support this option, terms creatin in any case.

Has there been any work done for the content taxonomy hierarchical select widget?

We have a content taxonomy field that uses Hierarchical Select on a custom content type. We would like to use node import to mass load new nodes from a csv file into the site. We want each node imported to retain its term hierarchy structure from the csv file. We see that the node import by default will let a person set a hierarchy specific delimiter but that only functions if you are using core taxonomy and not content taxonomy as a cck field. So essentially my question is the same as briand44's, has there been any work on a content taxonomy hierarchical select widget for node import or does anyone have suggestions or tips on how to begin writing one?

Attached is an updated version of #39 to support using TIDs in addition to term names. This was a quick edit, and the code isn't pretty, but it has worked in the few tests I have run. The code assumes that if the tag you entered was numeric then you meant for it to be treated as a TID. Let me know how it goes.

This may be useful for people (like me) who need support for hierarchical imports: I imported my vocabulary using the taxonomy_csv module, exported it via Taxonomy Manager to grab TIDs, and wrote myself a script to replace terms with their TIDs before I use Node Import. Kind of a hack, but it works.

Patch attached for #68 applies to dev

Alright I found a few issues with your code ...

Problem I found:

1. TID's are returned as an object so you can't actually reference $term[0] as you did. It kept crashing on me until I made the changes in the patch. I also had to shift some of the code around to make it work better.
2. There was a check in the code to see if the item did not have parents. I'm not sure why this was in the code but in my case it prevented the selection of terms lower down in the content taxonomy.

Something else worth mentioning is the following case...

If you have taxonomy built like the following:

1. Fruit
1. Carrots
2. Orange
1. Carrots

And you put the term Carrot into your uploaded CSV file it will pick the first term it finds (as it's hard-coded to use the first 0 array item. Ooops this is probably not good. We need a better mechanism to actually specify the parent path better...

Like if you put Fruit>>Carrot this tells content_taxonomy to use that path instead? Does anyone else know how to make changes in the code to parse that properly?

Wow!

That's really awesome @rconstantine I'll have to give it a try a little later.

I have to chuckle b/c I was just about to report the same issue with the number problem. Although my 'solution' was to do the following:

$termfound = null
is_numeric(value) -> $termfound = grab value from taxonomy by id
if (empty termfound) termfound = grab value from taxonomy name

This way if it thinks it's a number and fails to find the result then it will try again using the name... the only problem I see with your solution is that it may not be clear to people to put the # in front (dunno).

Thanks for your awesome work!

Okay that's a pretty fair approach and will work for sure. I was originally going to say that reversing the order (string check then number) could be effective. Then I realised that all these calls are damn expensive so it's probably better to approach it from your side.

Nice work btw! I'll be able to test this in a little while.

PS - Freetagging wouldn't have this issue but perhaps checkboxes (although I haven't used it)

subscribe

Oh man, thank you so much Drupal community!!!!

subscribe

wow! subscribe

rconstantine: can I import taxonomy to HS field using your plugin not having any terms in voc ?
this is what I really looking for..

this is what i need to import for example:

Online Communities >> Social Networks, Internet & Telecom >> Email & Messaging, Internet & Telecom >> Web Apps & Online Tools

so it is 3 multiples with hierarchy. I have empty voc by now..

ok, i understand. so i can confirm that this option is not added. when I tried with the example categories and emtpy vocabulary, I got info from line 221 of your file:

  if (empty($foundterms)) {
    node_import_input_error(t("The term %value was not found at the expected level in your taxonomy hierarchy. Please check your import file and try again.", array('%value' => $top)));
    return NULL;
  }

My categories look like this:

Online Communities >> Social Networks, Internet & Telecom >> Email & Messaging, Internet & Telecom >> Web Apps & Online Tools

How can I fill categories in vocabulary while import instead of before ?

#72 works perfectly with Autocomplete (Freetagging), Checkboxes/Radios, hs_Content Taxonomy

It appears, from #84 & #85, that this patch requires the vocabulary exist prior to import. As modules already exist for importing vocabularies (e.g., Taxonomy XML and Taxonomy CSV) I would propose that functionality does not need to be included in this patch.

I can also confirm, along with #86, that this patch works. Changing status to Reviewed & Tested.

I have noticed bug.
When I do import of numerical values (1,2,3,4) for CCK Taxonomy CheckBox they do not saved!

Node Import cannot define tid of the term

[cck:field_room_count:value] => Array
        (
            [0] => 3
        )

    [field_room_count] => Array
        (
            [0] => Array
                (
                    [value] => 3
                )

            [value] => Array
                (
                )

        )

So, will be it fix?

I can confirm that #72 works great for hierarchy but the problem is when you have multiple hierarchy, like

"Online Communities >> Social Networks, Internet & Telecom >> Email & Messaging, Internet & Telecom >> Web Apps & Online Tools"

In this case it imports only the last - Internet & Telecom >> Web Apps & Online Tools.

I have option in step 5 - Multiple values are separated by: - for this field but it seems not to work.

Can any of you confirm that or its just something wrong with my configuration.

THX

erwanpia, thanks!
#39 work for me.

Taisei, please don't change the issue status prematurely. See #72 and #89, specifically.

Hi, all.

I've found wrong order of functions calls from node_import/node_import.inc:726

In my case it is:

drupal_alter: book
drupal_alter: taxonomy
drupal_alter: user
drupal_alter: content_taxonomy
drupal_alter: date
drupal_alter: location_cck
drupal_alter: location
drupal_alter: content
drupal_alter: nodereference
drupal_alter: optionwidgets
drupal_alter: userreference
drupal_alter: node

I see that 'node_import/supported/cck/content.inc' prepares values and changes fieldname from 'cck:FIELDNAME:value' to 'FIELDNAME' format, that expected by 'node_import/supported/cck/content_taxonomy.inc' file.

Construction 'foreach (node_import_cck_fields($type, 'content_taxonomy') as $fieldname => $fieldinfo) {' in line 49 of this file return just FIELDNAME without prefix and suffix.

Dumps of 'values' argument made after construction:
'if (($node_type = node_import_type_is_node($type)) !== FALSE) {'
of file 'node_import/supported/cck/content.inc:81'

and before closing bracket of this 'if' shows this transformations.

Maybe this is a key of wrong behaviour?
Who is know which correct ordering logic should be here?
I see a hook on content.inc should be called before other CCK fields handlers, but where is the correct identifiers for that?

You can see similar manipulation in 'node_import/supported/location/location_cck.inc:80'
// Get location_cck fields for this node type
foreach (node_import_cck_fields($type, 'location_cck') as $field => $settings) {..
//dsm($values);
//$settings = content_fields('field_location', $node_type);
// $values['locations_cck'] = array();
$cck_field = 'location_cck:' . $field;

Somewhere lies the error, or it is here, or in 'content_taxonomy.inc', 'userreference.inc', 'filefield.inc'... Who can say where here is TRUTH?

I've found solution: UPDATE system SET weight = -1 WHERE name='content' ( http://drupal.org/node/110238 )

Now I have such order:

drupal_alter: content
drupal_alter: book
drupal_alter: taxonomy
drupal_alter: user
drupal_alter: content_taxonomy
drupal_alter: date
drupal_alter: location_cck
drupal_alter: location
drupal_alter: nodereference
drupal_alter: optionwidgets
drupal_alter: userreference
drupal_alter: node

and 'content_taxonomy.inc' see all it's values (I don't see any errors). However values were not stored.

I've tried such widgets type for CCK field:
HS - doesn't save any values.
Tree - doesn't save any values.
Multiselect - all values were saved correctly.

subscribing, want to check this out later

While my attempt #97 when I've used 'Multiselect' widget I've got several terms in the imported field but without hierarchy.

I see on the page 'admin/content/node_import/add' "(step 5 of 8)" that module is proposing me to set 'Multiple values separator'.
But I use Hierarchy Vocabulary and want to import terms as an hierarchy.

I saw to the code and found 'content_taxonomy.inc' doesn't declare any parameters for imported fields ('has_multiple', 'has_hierarchy' and so one...) as opposed to 'taxonomy.inc'

It's a bug or here is unknown to me dependence between 'content_taxonomy.inc' and 'taxonomy.inc'?

I've found that I shouldn't set 'Multiple values separator' for this field to '>>', so 'content_taxonomy.inc' parse it itself (if we use 'content_taxonomy_hs' widget).

But in this case I've catch errors with Term import. I'll see what's wrong.

The .inc file from #72 by rconstantine added to the latest 6.x-1.x-dev (2010-Jul-11) enabled importing a content_taxonomy field for me.

I've added ability for HS fields to create new terms if it was not found in vocabulary.
Functionality in #72 only fills field from vocabularies values for 'content_taxonomy_hs' widget type.

This patch should be applied to 'node_import' module with #72 file in 'support/cck/'.

To my previous post (#89- http://drupal.org/node/434572#comment-3581688 )

This is not a bug!
This is limitations of node_import (taxonomy, content_taxonomy).
If I have a term, which name is a number, I must write his tid in my import file (not a name!).

So, in my case, all works greate!

It seems to me that, as mentioned in #86, #87 and #88 by rconstantine and kbk, the importing of taxonomy should be done outside of this. Yet the patch, as is, imports taxonomy terms if the widget is not one of the ones defined, which falls under "other widgets". So if you have a standard content_taxonomy_select widget, all new terms are imported.

I'm adding a patch to #72's version that first checks whether free tagging is turned on before saving new terms. This filters out all new imports unless the taxonomy specifies it accepts new terms added by node submission.

Ooops, downloaded and saved patch to windows system, which added the ".txt" to the file. Here's the correct one.

Actual Linux file for the patch:

#39 ok with 6.x-1.0-rc4, though i should probably try later patches

Patch from #72 is working perfectly for me with the 6.x-1.x-dev version of Node import.

Would be great if someone could commit it and release 6.x-1.0-rc5 or even a final 6.x-1.0.

#72 works great for me too.

Many thanks +1

Joe

sorry kbk!

I used #39 to solve my problem. It works good. But I see there is a new patches - #72 etc. So how can I install new patches (they are not .inc like #105) to use in future?

Yeah, finally the final 6.x-1.0 version of this great module has been released!

Unfortunately, the .inc file to support content_taxonomy did not do it into the release. You can add it manually like this:

[root@www modules]# cd node_import/supported/cck
[root@www supported]# wget http://drupal.org/files/issues/content_taxonomy.txt
[root@www supported]# mv content_taxonomy.txt content_taxonomy.inc
[root@www supported]# cd -

Patch of #72 has been committed to 6.x-1.x. Will be included in 6.x-1.1 release.

Excellent - 6.x-1.1 is actually the first version that works completely perfect for me out of the box.