I am testing a large CSV import locally in MAMP. The file is 27MB and contains ~8000 rows with about 18 columns, each mapped to a CCK field (mostly text, one nodereference, a taxonomy, and an imagefield). I was getting PHP memory exhausted errors with memory_limit set to 128MB. When I set memory_limit to 768MB, the import was not exhausting memory but was going for several minutes, but looking at the content listing showed that it had stopped at importing only the first 50 nodes so I had to clone the importer/delete the old one to get it to stop.

I saw in the Feeds README that feeds_node_batch_size is set to 50 by default. It seems I can only get it to import more than 50 nodes if I set feeds_node_batch_size much higher in $conf in settings.php, and increase PHP's max_execution_time. Anyone know why Feeds is not importing more? I've imported much smaller CSVs with Feeds before without an issue. Thanks in advance.

Comments

czigor’s picture

czigor commented
Version: 6.x-1.0-beta11 » 6.x-1.0-beta13

This is still an issue with beta13. My importer has a file upload, a csv parser and a node processor. No Feeds tamper or anything fancy. My file consists of 377 lines. After exactly 50 lines the import hangs with a gibberish and long error message. Fortunately, when I restart the import, it starts from where it stopped.

zephirus’s picture

zephirus commented
Issue summary: View changes

Add the following line to sites/default/settings.php:

$conf['feeds_process_limit'] = 200;

You can change the number to what you need :)

twistor’s picture

twistor commented
Status: Active » Closed (outdated)