The help text and feedback messages regarding file upload size all claim the maximum size is set for PHP with PHP's upload_max_filesize parameter, but in fact, if upload_max_filesize is more than half of the PHP post_max_size parameter, file_upload_max_size() silently limits upload_max_filesize to only half of post_max_size.

I have 2 problems with this:
1. The only way to discover this is to read the code in file.inc -- usually only after wasting all kinds of time looking in all the places Drupal tells you to look (e.g. php.ini, settings.php, .htaccess, etc.).
2. The programmer who wrote this was just being lazy, as is evidence by the preceding comment which says:

// sanity check- a single upload should not be more than 50% the size limit of the total post

That's a lame sanity check. I should be able to upload a posting where only a few bytes are content and the rest is attached file without being limited by half the max size for the file.

This "mis-design" is in 4.7 and HEAD.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

ChrisKennedy’s picture

Agreed, this was added by http://drupal.org/node/72545 and acknowledged to be arbitrary.

jmcclelland’s picture

Version: 4.7.x-dev » 5.1

http://drupal.org/node/72545

Seems to be closed without this issue being addressed. It's also discussed here:

http://drupal.org/node/100373#comment-228710

As far as I can tell, this is still a bug in 5.1.

I think it should do a sanity check at 90% of max post size, not 50%. Or, at a minimum, the error message should report what it is doing.

cog.rusty’s picture

Version: 5.1 » 6.x-dev
Priority: Minor » Normal

Yes, this assumes too much for a sanity check.

Also, using a percentage for image vs (images+text) and even making it 50% does not seem so sane to me. It is fairly typical for Images to take up most of the size. It would be more reasonable to *subtract* 0.0001M (50-100 Unicode characters) from the post size. Or just -1M or 0.

If the intention was to allow uploading two almost-max-size images in a post, this is pretty arbitrary as well.

cog.rusty’s picture

Since I am not a php developer, I am also curious to know whether attaching multiple files to a node is a single POST operation or not.

Also, In addition to my objection to using a percentage

according to http://www.php.net/ini.core
- a sanity check should also include php memory limit
- there may be a way to check for an error and report it

RobRoy’s picture

Check this issue I posted a while back for upload module silently failing: http://drupal.org/node/30520.

We *definitely* need to check ini_get settins and provide an accurate error. Silently failing is just so lame.

ChrisKennedy’s picture

FileSize
752 bytes

Here's a patch to remove the 50% limitation.

ChrisKennedy’s picture

Status: Active » Needs review
drewish’s picture

Status: Needs review » Reviewed & tested by the community

Patch works as described. It's also the correct behavior. We're being way too conservative with the /2. It'd always annoyed me that core got this wrong.

Gábor Hojtsy’s picture

Status: Reviewed & tested by the community » Needs review

Hm, I am trying to understand what this limit is trying to prevent at the first place. I see pwolanin said this when he created the original patch:

My chopped off post above tried to say that I picked an arbitray cutoff of 50% of post_max_size as the upper limit for the size of a single upload. FYI, the PHP defualt values are post_max_size = "8M" and upload_max_filesize = "2M" (corresponding to 25%). However, a value of 80% or 90% might be fine too- please review and provide feedback so this can get finalized.

How should these relate? I don't see.

cog.rusty’s picture

The only restrictions mentioned are:

http://gr2.php.net/ini.core

Sets max size of post data allowed. This setting also affects file upload. To upload large files, this value must be larger than upload_max_filesize.

If memory limit is enabled by your configure script, memory_limit also affects file uploading. Generally speaking, memory_limit should be larger than post_max_size.

When an integer is used, the value is measured in bytes. You may also use shorthand notation as described in this FAQ.

If the size of post data is greater than post_max_size, the $_POST and $_FILES superglobals are empty. This can be tracked in various ways, e.g. by passing the $_GET variable to the script processing the data, i.e.
, and then checking if $_GET['processed'] is set.

I assume that post_max_size refers to a POST operation.

That patch was supposed to be a sanity check. I am unsure whether it prevents or reports any error, for example when you upload 3 files, each one 50% (is that a single POST operation).

My feeling is that "-1MB" would be sane enough, but error reporting would be even better.

cog.rusty’s picture

The sanity check also confuses users who have set both to 8M and then look at /admin/settings/uploads and see: "Your PHP settings limit the maximum file size per upload to 4 MB."

When I first saw that I was not sure what I needed to do. I thought I was all set for 8MB.

drewish’s picture

as rusty.cog points out the current formula is very confusing and incorrect. the post_max_size setting affects the total amount of data posted including the file. the upload_max_filesize setting limits the size of each individual file.

we can't provide an exact value of the maximum upload size because there are many factors that play into it. if more than one file is being submitted then each one must be less than upload_max_filesize, and the total size of the files and other data that's being POSTed must be less than the post_max_size.

i'll provide a few examples using the following settings:

post_max_size = 3M
upload_max_filesize = 1M

you could submit:
a ~1mb node body and ~1mb file.
a 0.1mb node body and two ~1mb files

you could not submit:
a 1.5mb file (upload_max_filesize - would report an error on the file)
a 0.5mb file and a 1.5mb file (upload_max_filesize - would report an error on the second file)
a 3.5mb node body (post_max_size - which would discard the entire POST)
four 1mb files (post_max_size - which would discard the entire POST)

now these numbers aren't exact because there's some encoding overhead in the POST that can fudge it.

drewish’s picture

Status: Needs review » Reviewed & tested by the community

#6 still applies and IMHO is the correct way to treat the upload limit.

Gábor Hojtsy’s picture

Version: 6.x-dev » 5.x-dev
Status: Reviewed & tested by the community » Patch (to be ported)

Agreed that this size restriction was arbitrary. Committed #6 to Drupal 6-dev. Needs to be backported.

Bart Jansens’s picture

Status: Patch (to be ported) » Needs review
FileSize
760 bytes

backport

drumm’s picture

Status: Needs review » Fixed

Committed to 5.x.

pwolanin’s picture

note - see my quote in #9 - it went in as 50% since no one reviewed it and suggested otherwise...

The main bug being fixed in that issue was that no information about the PHP limit was being shown in Drupal.

drewish’s picture

pwolanin, it sounds like you're looking for: http://drupal.org/node/30520

Anonymous’s picture

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.

doc2@drupalfr.org’s picture

How come this is closed?

EDIT for #21: Precisions:

I would have liked to upload files as large as DVDs, for archival purposes, as strange as this might sound. And I can't.
My config is:

- No particular restriction in .htaccess nor settings.php in my drupal directory regarding memory, post and upload limits. Time limits where pushed to about 50 minutes.

- php.ini :
upload : 5000 MB
post : 10000 MB
memory : 512 MB

With a 872 Mo file, upload is OK, but no more.

I've already read related:
#104220: file.inc: file_upload_max_size() silently cuts size in half
#72545: upload sizes limits are not validated
#30520: file_save_upload() not notifying user when PHP upload limit is exceeded.

But no luck!

EDIT for #22: Thanks, great topic! I think I found a workaround. Follow the link of that post.

drewish’s picture

doc2@drupalfr.org, it's pretty obvious why it's closed if you read the comments. the patch was committed to D6 and D5... there's nothing more to do so the issue gets closed.

cog.rusty’s picture

Uploading very large files is a very good request, but I am afraid it is unrelated to this issue, which was about fixing a problem. There are applications out there which slice files and upload them in chunks, or other methods (third party services, ftp solutions etc). See also this forum thread:

Upload very large files
http://drupal.org/node/129138

You could start a new issue with a feature request, but experience says that the way things are done in Drupal, this kind of new functionality would start as a contributed module if someone got interested. So, the best chance is to somehow stir interest in that direction.

cog.rusty’s picture

A small comment to #20: A reason that those php.ini settings didn't work may be that, according to what http://php.net/ini.core says about post_max_size, "Generally speaking, memory_limit should be larger than post_max_size", which of course is unrealistic.

doc2@drupalfr.org’s picture

Yes it is unrealistic!

My memory_limit was 256 MB and is now 512 (2 GB of physical RAM on our server, still 1 GB free), but I've still been able to upload files up to about 700 to 800 MB respectively. This throws troubles on relations between my specs and possibilities according to those PHP ini.core rules... We just set 5 GB to upload_max and 10 GB to post_max for not being as less restricted as possible.

Moreover, our server (OS X dual G5) uses about 70 GB of virtual memory, but I guess files can't be "cached" there. I was wondering if having 4 to 6 GB of server RAM and almost as much dedicated to my scripts in memory_limit would change anything, if not completely unrealistic!

Anyways, I think I found a workaround for my videos.

christefano’s picture

I've been wondering about this for ages and thought it was a PHP configuration problem. Thanks for fixing this, folks.

yogitha’s picture

Hi
I am using IMAGE(5.x-1.x-dev) module in my community portal site .I am able to upload images of size 400KB and i am unable to upload images of size above 500kB.

I changed php.ini file in my server
post_max_size = 32M
upload_max_filesize = 32M

and in settings.php file
ini_set('memory_limit','32M');
ini_set('post_max_size', '32M');
ini_set('upload_max_filesize', '32M');

and in Administrator>>Site configuration >> Image >>Maximum upload size: 4000KB

but even i am unable to upload images of bigger sizes. Do anyone have idea on this issue.I willl be thankful if anyone come forward with your suggestions.

cog.rusty’s picture

This issue is not for support, but anyways... First of all, upload a phpinfo file to your Drupal directory and see if these have any effect or not (see http://drupal.org/node/59680)

Remove the settings.php settings to keep things simple. The last 2 ini_set() lines don't work there anyway, because they are variables of type PHP_INI_PERDIR (see http://php.net/manual/en/ini.php).

Try a php.ini with:

memory_limit = 48M
post_max_size = 32M
upload_max_filesize = 31M

If this was a local php.ini file and doesn't work, it probably means that you can't use one on that host, so delete those to keep it simple and try the .htaccess file:

php_value memory_limit 48M
php_value post_max_size 32M
php_value upload_max_filesize 31M

Don't do both.

yogitha’s picture

Hi cog.rusty

I Remove the settings.php settings and i run cron.php. Now its working fine.Thanks a lot :-) .

thomasmurphy’s picture

Hi, I'm trying to establish what the relationship is between post_max_size and upload_max_filesize is in 6.x core. Is there still an arbitrary 50% ratio or has it been removed? This documentation page assumes that it is still in place - http://drupal.org/node/97193. Apolgies for posting on this old thread but it seemed like the best place to start.

thomasmurphy’s picture

I just sucessfully uploaded a 302MB file with a post_max_size of 384MB so it's must not be an issue anymore.

drewish’s picture

thomasmurphy, see #12.

thomasmurphy’s picture

Thanks, drewish, but I don't see how this affects my question. I'm trying to verify if there is still an arbitrary 50% effective limit on upload_max_filesize enforced by the upload module, rather than by the post_max_size in php.ini or .htaccess. This documentation page claims that it is,

http://drupal.org/node/97193

and yet I've just verified, to myself at least, that this isn't the case. I'm trying to discuss this issue with other people to see if the documentation needs to be corrected. This ended up being important to me because I needed a site administrator to be able to upload a single file of ~300MB, but I didn't want to have to make the post max size limit ~600MB. It turned out that I didn't, but according to the above link, I did. Sorry if I'm misunderstanding, but this is now I see it. cheers,Tom.

drewish’s picture

ah yeah the 50% limit was removed. so that doc was probably written for 4.7 and it's out of date for later versions. there is no hard limit but the comments in #12 describe PHP's role.