at about line 519 of apachesolr_attachments.admin.inc, the code reads:
$data .= file_get_contents($filepath);
If the file specified by $filepath is large (in my case, a couple of .zip files and one .flv file), the code terminates hard with a PHP out of memory error, blocking all further cron processing. There needs to be a test against the size of $filepath and the max memory left to PHP before file_get_contents is called. If this would lead to an out of memory condition, then the file should be skipped and the event logged.
Another possibility is to read the file in chunks and add to the POST rather than attempting to read the file file file_get_contents.
| Comment | File | Size | Author |
|---|---|---|---|
| #6 | Out_of_memory-1706542.patch | 3.21 KB | retorque |
| #4 | apachesolr_attachments-Max-file-size-1706542.patch | 2.75 KB | retorque |
Comments
Comment #1
nick_vhAre you willing to create a patch for this?
Comment #2
sterndata commentedI'd create a patch, but I need a little help.
I've patched my copy to
$fsize=filesize($filepath);
if ($fsize >= 268435456) {
$data .= 'file too large - '.$filepath;
watchdog('Apache Solr Attachments', 'File is too large:f %filepath', array('%filepath' => $filepath), WATCHDOG_ERROR);
}
else {
$data .= file_get_contents($filepath);
}
I need a programmatic way to determine how much memory is available for PHP. You can see the number I plugged in above, which I pulled from the PHP error message. What function can tell me how much is available for allocation?
Comment #3
jhedstromMarking as a duplicate of #1251308: File Size Limit.
Comment #4
retorque commentedI know this was closed as a duplicate of #1251308: File Size Limit, but that issue did not include a backport to Drupal 6. The approach in that issue is likely better, and it might be worth a follow-up to try to backport it if the Drupal 7 module architecture isn't dramatically different, but I feel that this simple fix needs a patch with configuration in case others are using it because there is no fix for Drupal 6 yet.
If the attached patch is considered an appropriate approach, and the module maintainers would like me to clean it up and add the apachesolr_attachments_max_filesize variable to the .install file, I would be happy to do so. If the maintainers are interested in a backport of the fix in #1251308: File Size Limit, I would be willing to look at it when I can get some time, but I can't guarantee that will happen soon.
Comment #5
nick_vhAlways happy to see contributions! So please go ahead :)
Comment #6
retorque commentedI was a bit optimistic in my previous post. 4 months to find a few minutes to make a tiny change...
The patch attached adds hook_update_n() to set the apachesolr_attachments_max_filesize variable. It also sets the default max file size to 40 mb to match the Drupal 7 implementation.