Out of memory error between pages 1 & 2 of upload wizard
| Project: | Node import |
| Version: | 5.x-1.6 |
| Component: | User interface |
| Category: | bug report |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | active (needs more info) |
Error received: "Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 79 bytes) in /includes/common.inc on line 2231"
Error raised on a Drupal 5.7 installation in otherwise good working order. Able to import other files using NI 5.x-1.6.
Under Node Import 5.x-1.5 the error-raising data file would timeout/run out of memory **but during import process**. This time the error came before I could even complete the form. (Sorry, can't recall exact error under NI 5.x-1.5. If you really want me to downgrade and regenerate, I can try, request in a comment on this issue).
I believe the file would crash because it had overly long lines (cf. http://drupal.org/node/235433). I was using the file to test if that would upload as fix was to come in NI 5.x-1.6
#1
Could you attach to this issue a sample CSV file that would cause this crash as described in this issue?
If you can't produce such a sample CSV file because your data is private and you don't want to share it globally, contact me using my contact form so you can send me a sample privately.
#2
CSV file attached.
I don't think it's a "Mac has weird line terminators" issue-- this is letter 'c' and letters a, b, & d load fine (they were made using exact same process). Sometimes I've had files be too large and that caused problems with node import, but usually chunking them down into 100-200K fixes that. Also, no problems with the Chinese characters in the other files. (However, with tens of thousands of entries, I haven't checked them all by hand.)
I appreciate any help you can offer. Let me know whatever I can do to pitch in.
#3
I've been working with this file more.
1) There are in-field double-quotes in some of the definitions. I thought those might be an issue, but I don't think so-- I converted them to single-quotes and re-ran the file, still got the same problem.
2) I'm now going to try processing this file 50-100 lines at a time and see if I can find offending records. I'll update when I find something, but it may take a while.
Thanks again for such a phenomenally useful module!
#4
Turns out that in at least some of the files, the first line with field titles has a "Mac" line ending. Copy a line ending from within the body (generated by PHP set to use unix line endings) and paste it over the line ending of the first line and fixed! Not all of my offending data is fixed by this, but much of it is.
The line endings were an artifact from how I process the stuff using my Mac-- php to generate the data and manual cut and paste to put the header in.