taxonomy terms are NOW imported

giorgio79 - April 18, 2009 - 20:58
Project:Node import
Version:6.x-1.x-dev
Component:Code
Category:bug report
Priority:normal
Assigned:Unassigned
Status:closed
Description

Hello,

I have free tagging and a hierarchical vocab for a content type, and when I do content import the taxonomies do not get imported, even though it detected fine the columns in the csv file.

Any ideas?

Are you succeeding with importing terms for your node?

#1

GreyHawk - April 19, 2009 - 03:47

Note: same issue when doing product-type imports.

No problems doing catalog vocabulary imports, tho.

#2

giorgio79 - April 19, 2009 - 04:24

This issue may be related as well

http://drupal.org/node/437834

#3

giorgio79 - April 19, 2009 - 05:33

I tried to investigate this.

I notice that even on preview, the taxonomy free tagging terms and the hierarchical taxonomy is empty.

Here is a part of the output of one

[taxonomy] => Array
        (
            [1] =>
            [tags] => Array
                (
                    [3] =>
                    [2] =>
                )

        )

    [op] => Preview
)

#4

giorgio79 - April 20, 2009 - 15:33

I just tried the sample csv, but that did not work as well for taxonomy...

Any tips are appreciated :)

#5

ludo1960 - April 20, 2009 - 17:47

How are defining the Taxonomy in your CSV file ? Print one line here

#6

giorgio79 - April 20, 2009 - 18:17

Thanks, it is very simple, I have a free tagging, and a hierarchical taxonomy and here they are for example:

TAG_CSV_COLUMN
these,are,my,tags

I am separating them wtih commas, which are offered as the default separators for tags in Node Import. ludo1960 can you tell me if you can import tags and/or terms during importing of nodes?

#7

ludo1960 - April 20, 2009 - 18:33

Try separating your tags with || double pipe like these||are||my||tags let me know how it goes

#8

kilrizzy - April 20, 2009 - 18:47

I am trying to import my catalog terms for products and have had no luck importing. I keep getting the errors:

warning: mb_strlen() expects parameter 1 to be string, array given in /websites/tilegallerypa/html/includes/unicode.inc on line 404.

I have tried a few different methods - "cat1"||"cat2", 1||2, cat1||cat2

Any thoughts?

#9

ludo1960 - April 20, 2009 - 18:51

Try cat1||cat2|| no commas

#10

giorgio79 - April 20, 2009 - 19:02

Thanks Ludo, just tried like you say

these||are||my||tags

even with quotes like

"these||are||my||tags"

specifying double pipe || as the separator, but no luck.
Everything else works fine, from title, body to CCK text fields except taxonomy

Regarding kilrizzy's comment, it may be a different issue as I am not getting any error messages. Also I am having these issue via a taxonomy enabled content type import.

#11

kilrizzy - April 20, 2009 - 18:59

Still no luck that way. It seems to be splitting the data though, I get two of the errors for every array entry.

To be clear, cat1 represents the machine readable term name correct?

#12

ludo1960 - April 20, 2009 - 18:59

Sorry that didn't work, JFYI I tried with a Vocab "City" and created terms "North" and "South" I was able to import terms into "South" and "North" seperately but not together, I got the same error as you when I tried.

#13

giorgio79 - April 20, 2009 - 19:08

Ludo are you referring to an import to ubercart product, or to a content type?

#14

ludo1960 - April 20, 2009 - 19:11

Content type

#15

Robrecht Jacques - April 21, 2009 - 15:43

There are problems with hierarchical vocabularies. As all vocabularies are hierarchical by default on Drupal 6, this means all.

I'm working on it right now.

#16

ludo1960 - April 21, 2009 - 16:13

Great!

#17

Robrecht Jacques - April 21, 2009 - 17:45
Status:active» fixed

Fixed in CVS. I'll release a -rc5 with this fix later today. Any of the formats above should work btw.

#18

giorgio79 - April 23, 2009 - 18:50

Thanks Jacques, I look forward to this.

Please let me know if you need any help.

I was trying looking at the code and understanding it hoping to fix it myself but it is massive :P

#19

Robrecht Jacques - April 22, 2009 - 20:45

@giorgio79 : you could test the -dev release of April 22th.

Things to do in the next couple of days (-rc5 ETA is April 25th):
- node reference fixes (working on now)
- date (including date_repeat) fixes
- book fixes
- get_csv fixes
- Fatal error: Only variables can be passed by reference in ...node_import.inc on line 594
- ...

#20

ludo1960 - April 23, 2009 - 11:54

Hi Robrecht,

Anything I can do to help?

Thanxs

#21

giorgio79 - April 23, 2009 - 18:50
Status:fixed» needs work

Thanks Jacques, I just tested it but I am still getting empty tags for my free tagging taxonomy import.

I also noticed I cannot set default values for CCK fields, like CCK text field :)

#22

cleanthes - April 25, 2009 - 18:19

Just wanted to say, I have tried the -dev version and Parents are now correctly imported. Thanks! Looking forward to -rc5

#23

giorgio79 - April 25, 2009 - 18:56

cleanthes have you tried importing a flat, free tagging terms that have no parents?

#24

cleanthes - April 26, 2009 - 06:35

Giorgio: No, but if you want to provide me with a test file I'd be happy to try it for you

#25

voidfiles - April 26, 2009 - 19:39
Version:6.x-1.0-rc4» 6.x-1.x-dev

Hi,

I have been following this bug, and I wanted to share my problems. I have tried to import vocabulary terms with hierarchy >> and I am reciving errors.

Like this

* user warning: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ') td, term_hierarchy th, term_synonym ts WHERE td.vid = 1 AND th.parent = 10 AND' at line 1 query: SELECT td.tid FROM term_data) td, term_hierarchy th, term_synonym ts WHERE td.vid = 1 AND th.parent = 10 AND td.tid = th.tid AND td.tid = ts.tid AND LOWER(ts.name) = 'jets - american products & pentair' in /home/.furbag/viresh_hwp/ubercart.tastestalkr.com/modules/node_import/supported/taxonomy.inc on line 407.
* Input error: is not allowed for Catalog (not a term or synonym in Catalog).
* Input error: is not allowed for Catalog (not a term or synonym in Catalog).

and this is what I see in the dump

values = Array
(
[created] =>
[node_import_build_mode] => 1
[title] => American Products Luxury Jet Tool
[body] => Tools American Products - Luxury Jet and Diverter Jets
[type] => product
[format] => 2
[cck:field_image_cache:data] => Array
(
)

[cck:field_image_cache:fid] => Array
(
[0] => 2
)

[cck:field_image_cache:list] => Array
(
[0] => 1
)

[model] => AP45271100
[list_price] => 0
[cost] => 4.1600
[sell_price] => 8.3200
[shippable] =>
[weight] => 1
[weight_units] => lb
[length_units] => in
[length] =>
[width] =>
[height] =>
[pkg_qty] =>
[default_qty] => 1
[ordering] => 0
[uid] => 1
[revision] => 0
[log] => Imported with node_import.
[sticky] => 0
[status] => 1
[promote] => 1
[path] =>
[comment] => 2
[field_image_cache] => Array
(
[0] => Array
(
[fid] => 2
[list] => 1
[data] =>
)

)

[name] => admin
[taxonomy] => Array
(
[1] => Array
(
)

)

[op] => Preview
)

I am also attaching the csv I am trying to import

AttachmentSize
prod_import_test.txt 11.78 KB

#26

hanjek - June 22, 2009 - 08:42
Title:taxonomy terms are not imported» taxonomy terms are NOW imported
Component:Miscellaneous» Code
Status:needs work» fixed

For people who import UTF-8 based taxonomy with multiple terms, you need to alter the include->unicode.inc :

<?php
function drupal_strlen($text) {
  global
$multibyte;
  if (
$multibyte == UNICODE_MULTIBYTE) {
    if (
is_array($text)) {
      return
mb_strlen($text[0]);
    } else {
      return
mb_strlen($text);
    }
  } else {
   
// Do not count UTF-8 continuation bytes.
   
return strlen(preg_replace("/[\x80-\xBF]/", '', $text));
  }
}
?>

#27

giorgio79 - June 23, 2009 - 04:35
Status:fixed» needs review

#28

GreyHawk - June 24, 2009 - 10:59

voidfiles -- just revisiting this, and saw your message. I know what part of your import problem is: the Category column is incorrect.

For example, for parent "Spa Parts Catalog >> Pumps >> Aqua-Flo", the keyword you want to apply to that line is aqua flo circ pump" -- you need to change the Parent category of "Category_DataPort_Id" to look like this: ">> Aqua-Flo"

You'd have to sort your list so that any terms which are not already in the appropriate vocabulary but are necessary for assigning as parents are uploaded first, I think, to help avoid problems.

So, you'd go from this:

"Category_DataPort_Id", "Customer_Price", "Keywords", "List_Price", "Large_Image", "Long_Description", "Part_Number", ...

"Spa Parts Catalog >> Pumps >> Aqua-Flo","0.8800","aqua flo circ pump","0","","Pipe Plug O-Ring (#011) #9","AF92200120",

To this:

">> Aqua-Flo","0.8800","aqua flo circ pump","0","","Pipe Plug O-Ring (#011) #9","AF92200120", ...

That would put keyword "aqua flo circ pump" as a child underneath parent term "Aqua-Flo" -- assuming I'm reading the data correctly.

#29

giorgio79 - June 24, 2009 - 11:48

@hanjek I notice your code requires core modification, which wont go through in Drupal, so we should look for sg we can change in node import.

I tried it anyway but no luck, my tags (several of them separated with a comma) did not import, not even showed on the preview.

#30

fitzed - June 29, 2009 - 22:40

Awesome! hanjek's unicode.inc fix above worked perfectly. I'll simply revert to the original include file after my import is complete.

Thanks!

#31

GreyHawk - July 2, 2009 - 03:55

@giorgio79 -- are you using the .dev release? Also, are you importing multiple taxonomies as separate values (i.e., separated by comma) or as multiple values in the same cell (separated by a | or whatever the symbol is)?

I found that the .dev release resolved a lot of issues I was having with taxonomies, particularly hierarchies, but I also had to get an example file that worked so that I could figure out the correct way to format my data.

See the response I gave above for an example of at least how I imported a hierarchy, and perhaps attach a small sample of your data...

#32

giorgio79 - July 3, 2009 - 20:20
Status:needs review» fixed

Hereby I would like to thank all you guys for the kind advice and help.

Finally I had some time to dedicate to getting to the bottom of it.

Basically it turned out that during preview my tags and CCK fields showed as empty and I assumed this is a problem and stopped.

Now I ignored this and went ahead with the import, and tags imported fine! Hurrah! :)

Thanks again,

Cheers,
G

#33

giorgio79 - July 5, 2009 - 05:26

One more thing I found is that I had to disable active tags for my vocabulary as it prevented tags from importing.

#34

coloryan - July 5, 2009 - 07:17

For some reason, I need to turn off the hierarchical select module to import taxonomies. No uninstall it completely, just turn off the module.

#35

System Message - July 19, 2009 - 07:20
Status:fixed» closed

Automatically closed -- issue fixed for 2 weeks with no activity.

#36

diego.paroni - July 21, 2009 - 13:59

I confirm #34, if you have installed the module Taxonomy Super Select, you have to disable the Taxonomy Super Select checkboxes in the vocabulary, otherwise the node import generate "warning: Invalid argument supplied for foreach() in ...\modules\taxonomy\taxonomy.module on line 70." and the taxonomy terms are no correct.
Thanx Coloryan!

#38

maaak - September 24, 2009 - 10:40

Hi all,
Been trying this for a good few days ;) maybe longer could be weeks
Thankyou to all that have contributed, with the result, i have at last seen a glimmer of product import working !
Got as far as pasting in hanjek 's code for people who import UTF-8 based taxonomy with multiple terms yet i am still getting the catalog error when importing products.
Please look at the sample .csv and advise as to what i am doing incorrectly ............

* Input error: is not allowed for CATALOG (not a term or synonym in CATALOG).
* CATALOG field is required.

.m

AttachmentSize
ageing_and_effects-all.zip 5.77 KB
node_import-node_product-5.txt 48.37 KB

#39

maaak - September 24, 2009 - 10:44

.....and here is the rejected .csv.................

AttachmentSize
ageing_and_effects-all-reject.zip 6.28 KB

#40

maaak - September 24, 2009 - 12:48

Oops.............
had two versions of uc_powertools loaded !
deleted the offending and Voila ! or as greywolf would say awooooooooooo!
.m

 
 

Drupal is a registered trademark of Dries Buytaert.