I've done some tests with importing data and came to a problem with german umlauts and special chars (äöuÄÖÜß):
Several of the node-titles to be imported contain umlauts, and since the import-csv-file is utf8, most work quite fine. But in places where the first character of the node-title to be is an umlaut, something goes wrong and the new node gets a '?' char instead of the umlaut there.
Fortunately I don't have too many titles starting with umlauts, but it's an annoying problem ...

any hints what could be going wrong here?

CommentFileSizeAuthor
#2 node_import-specialchars-1.patch1.68 KBray007

Comments

ray007’s picture

ray007 commented

I think I tracked the error to fgetcsv() already returning false data.
Seems we need to use a custom cvs-lineparsing routine, maybe get one from the comment at http://de.php.net/manual/en/function.fgetcsv.php#75332 would do it?

Will test that and report back ...

ray007’s picture

ray007 commented
Status: Active » Needs review
StatusFileSize
new node_import-specialchars-1.patch1.68 KB

All right, the attached patch gives node_import its own fgetcsv() replacement function that makes things work.
Should probably be a drupal function and not only a node_import module function, but better here for now.
Maybe we should also replace the old {} string index operators with [] access, but as I said: it works ;-)

Comments?

tanoshimi’s picture

tanoshimi commented

I've just noticed I had the same problem when importing a CSV which had an apostrophe ' (ASCII code 39) in the body. My database is UTF-8, so I don't quite understand why this came out as a �, but I'm hoping that your patch might fix my problem too. Is there any progress with this getting reviewed or added to the release?

summit’s picture

summit commented

Hi,

Above solution is not working for me. When Node-import sees a special character, it breaks the sentence and the end of the sentence is not imported anymore.
Please assist, thanks in advance!

greetings,
Martijn

ray007’s picture

ray007 commented

I guess you did make sure your input is valid utf-8 ?

summit’s picture

summit commented

Hi,

I made my Node-import list in Excel.
Then I saved as TXT (TAB-delimited) *.txt.
THis file I import then in Node-import.
Should I set an option in Excel?

greetings,
Martijn

summit’s picture

summit commented

Hi,
It worked indeed with valid utf-8 input. Am I correct that the " issue stated in http://drupal.org/node/168922 is also solved with this patch?

greetings,
Martijn

summit’s picture

summit commented

Hi,
It worked indeed with valid utf-8 input. Am I correct that the " issue stated in http://drupal.org/node/168922 is also solved with this patch?

greetings,
Martijn

yang_yi_cn’s picture

yang_yi_cn commented

I think an alternative way is save the .csv file using OpenOffice, which adds double quotes ("") to every fields.

freddyhardest’s picture

freddyhardest commented

OpenOffice resolved the problem. Thank you people!

Robrecht Jacques’s picture

Robrecht Jacques commented
Status: Needs review » Fixed

Fixed. Will be included in next release of node_import (5.x-1.5). Thanks!

Anonymous’s picture

Anonymous (not verified) commented
Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.