I'm about to launch a major mod database for a game that is about to be released, and I have this database set up around File Framework. I'm not too worried about downloads taking up much memory, because my repository's location is using lighttpd, but I am a bit concerned about memory usage when users are uploading files. If a user is uploading a 20 meg file, will that entire file have to be built up in my server's memory before being put into the repository? I'm just worried about what I should put my max upload size limitation to be in comparison with how much memory I will need to be paying for on my server each month.

Comments

miglius’s picture

miglius commented

No, the file content is not put in the memory.

Webbstre’s picture

Webbstre commented
Priority: Normal » Critical

Well, it's been a while since I originally posted this, but my site is live and people are uploading files. Here is the strange thing: Every time someone uploads even a TEENY TINY file the system memory appears to jump a couple hundred megs of memory. You can see my statistics here:

http://www.runicgamesfansite.com/mrtg/

The bottom graph is the system's memory, and most of the sudden 200-300 meg spikes appear to correlate to when people have uploaded a file. I'm running the latest dev versions of this and bitcache, but still there is no change. I saw someone say they managed to fix what appears to be the same problem several months ago, but as far as I can tell all they did was change the order of their installation.

What should I do?

miglius’s picture

miglius commented

From the graph is hard to view the correlation between peaks and file uploads. Is it a server memory or a php memory usage?

Webbstre’s picture

Webbstre commented

Sorry, I should have taken a copy of the graph at the time I posted it. At the time there were some very obvious correlations between the system memory usage shooting up for about 5 minutes any time someone uploaded a small file, but now the memory is up all the time. I'm doing more and more testing these days, but I suspect that various statistic logging, and cache refreshing was part of the problem, increasingly higher amounts of traffic is another part of the problem, and the final part is that simply something is going screwy with my Drupal installation in general and I'm having trouble pinpointing it.

I'll update it again when I can find and confirm what the problem is. I know it's not your job or anything, but if you (or anyone really) have any tips for finding resource hogs or other bizarre problems I would welcome them ^_^;

Yezid’s picture

Yezid commented

Hi,
sometimes (once a week) i have on my web bug report:
"The application has reached reserved amount of server's system resources"
What should I do? (short overview of my site's parameters is OK)

Webbstre’s picture

Webbstre commented

Oh wow, I forgot to ever update this. I ended up discovering that bad caching was the source of my sudden memory spikes. It just coincided with the file updates, so it looked like it was this module's fault. It wasn't.

johanneshahn’s picture

johanneshahn commented
Status: Active » Closed (fixed)

seems to be fixed