Mapping fields chokes on import file with more than 63 columns [#365600]

I'm attempting to map only a few fields from a csv file with almost 100 columns.
There are only 5 rows however (as a test), so it's a very small file - 8K.
The problem is php is choking when I click the map tab:

Fatal error: Maximum execution time of 60 seconds exceeded in /Applications/MAMP/htdocs/import/includes/common.inc on line 866

Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 43192860 bytes) in Unknown on line 0

I updated my php memory limit to 512M and upped the execution time to 60 secs. But maybe this is not the problem?
I'm not sure what "Unknown on line 0" refers to. Any ideas what could be the matter?

Comment	File	Size	Author
#4	feedapi_mapper_avoid_infinite_recursion.patch	1.42 KB	aron novak
#2	dummy_data_64cols.txt	1.6 KB	techninja

Comments

Comment #1

alex_b commented 23 February 2009 at 19:09

Status:

Active

» Postponed (maintainer needs more info)

Could you post an (anonymized) version of the CSV file you're processing?

Comment #2

techninja commented 15 March 2009 at 03:05

Status	File	Size
new	dummy_data_64cols.txt	1.6 KB

I have the same problem, and so far as I can tell it dies after trying to handle the 64th column.. everything including #63 work perfectly, then blam! 64 breaks it.
This is while trying to map the csv columns to CCK fields through feedapi mapper (/node/[feed-nid]/map)

Says it's out of memory (no matter how much you give it), and the dblog spits out

implode() [<a href='function.implode'>function.implode</a>]: Invalid arguments passed in [site_dir]/sites/all/modules/feedapi_mapper/feedapi_mapper.module on line 407

Not quite sure what's going on... in the mean time, I'll be putting a workaround that truncates incoming data to those first 63 cols (as I don't need any after that in my current case) though I'd imagine others wouldn't be so fortunate.

The attached file is just two rows and header for 64 cols. Remove #64 and you can suddenly map them.

Comment #3

techninja commented 17 March 2009 at 05:48

Well, much as I've proven this to be the case, it looks like it's not the CSV parser. I converted my 70+column CSV to XML, and ran it through the SimplePIE parser off into feedapi and it died on TWO completely different servers. Eating up as much memory as it could and then crapping out trying to allocate some incredibly tiney amount (~ 30 to 70 bytes).

After some careful pruning and trimming, I was able to get my xml to FINALLY get to the map page by, you guessed it, cutting the number of flat individual xml nodes to less than 64.

So.. I'd just like our faithful csv_parser maintainer to see this off in it's new life, to be passed to the feedapi issue queue (assuming there's not already a dupe issue there waiting...)