Hi,

I set Feeds up to import data from a CVS file. I mapped everything out and set it to never refresh, etc. The settings were fine and the csv file was fine. For some reason, the importing never ended and every time I refreshed the content listing page it grew larger and larger. I only had 600 something items in my CSV and I ended up with over 5000 items before i closed the page.

Any ideas?

Comments

eatsleepdev’s picture

eatsleepdev commented
Status: Active » Closed (fixed)

I figured out the problem. The csv I was using needed to be converted to utf8. Works great now

martinbutt’s picture

martinbutt commented
Priority: Critical » Normal
Status: Closed (fixed) » Active

Certain characters break the import and put it in to an infinite loop. Using this *nix command will help you locate them:

iconv -f ascii -t utf-8 file-to-import.csv

An error message from the system would be useful.

davidinnes’s picture

davidinnes commented
Title: Feeds CSV import infinite loop » Feeds CSV import infinite loop on non-UTF (high ascii) characters
Priority: Normal » Major

The priority of a node-generating infinite loop bug should be marked somewhere between major and critical.

Testing with MartinButt's "iconv -f ascii -t utf-8 file-to-import.csv" procedure I was able to generate errors in three out of four common spreadsheet programs (Mac Excel, Mac OpenOffice, Tables, Numbers) by typing the text "würst" in a single cell and then saving or exporting it to a CSV file using each program's default settings. (The Tables default is ASCII-7 so it worked by degrading the data, which accommodates the Feeds bug but, well, degrades data!) "Smart" quotes and dashes break things too.

It's very nice that Feeds import has a Delete Items tab -- based on testing it can delete 10,000 nodes as quickly as it can 500. Which is impressive if and only if one overlooks that one was attempting to import only 100.

Possible fixes:
- Post clear documentation on the Import page
- Fail with a warning as soon as a high-bit characters is encountered
- Ignore high-bit characters in the stream such that they're passed along and stored as expected
- Convert high-bit characters in the stream to low-ASCII (not a good solution)
- Convert high-bit characters in the to numerical (e.g. #0164) HTML character entity equivalents.
- Do... pretty much anything except run a node-generating endless loop.

twistor’s picture

twistor commented
Issue summary: View changes
Status: Active » Closed (outdated)