I have memory limit = 128M
I get a "Fatal error: Allowed memory size exhausted" when i try to import from a folder that has contents larger than that.
It matters not if I try to import only one image from that folder or all of them. The error is the same. It does not even start the batch.
If the folder is smaller than the memory limit then everything is smooth.
Could this be a problem of the Batch API?

Comments

David Lesieur’s picture

Title: Can't import from folders that are bigger than the drupal memory limit » Add option to skip the import form
Version: 6.x-1.6 » 6.x-1.x-dev
Component: Miscellaneous » Code
Status: Active » Needs work
StatusFileSize
new9.5 KB

While Imagefield Import already supports Batch API, the initial form that's created to select images and start the import does too much, making the module unusable for large imports. With large imports, the form either require too much time building itself, require too much memory (as reported above), or is simply unusable (try checking 4600 images for import; the form has a master checkbox for that, but the browser will have a hard time finishing the job).

This patch does the following:

  • Add a new option to skip the import form. With this option enabled, one is only shown a confirmation form for importing all images in the import path. Upon confirmation, a batched import is launched.
  • Move the file_check_location() call to the batch operations.

As a bonus, the patch also supports proper file path processing when the FileField Paths module is enabled.

Marking this as needs work because I have not tested all cases yet. I'm also unsure about what to do with the taxonomy during batch imports.

wulff’s picture

Assigned: Unassigned » wulff
Status: Needs work » Active

It seems like we have been working in parallel on this problem.

I have tried to incorporate most of your ideas in the latest dev release.

The current version can now import directories of images which used to result in out of memory errors, and the additional workflow (sans selection form) seems to work as intended with taxonomies and extra cck fields.

I will do some more testing tomorrow.

wulff’s picture

Status: Active » Fixed

I have just tested the latest version of the code by importing ~7500 images from a directory tree. Worked without any problems with PHP memory_limit set to 128M.

Thanks for your input!

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.