Someone correct me if I'm wrong - but the fact that the image module stores all images, and all resized copies, in the same directory makes the image module completely unusable. Say for instance that a person has galleries of thousands of images (and I do) using the image module with the gallery stores all the images in a single directory. Having thousands, or 10's of thousands of files in a single directory not only severely impacts performance of a server, but makes working with that directory difficult. With 10's of thousands of files in a single directory working with gzip for back up or even performing a simple list command on that directory is anywhere from impossible depending on a persons allowances on a server to very difficult. I have always used a separate gallery installation along side my Drupal site - and I would love to be able to incorporate my image galleries in to Drupal, but with this fatal flaw I just don't see any way to do it. I was excited to see the addition of the mass import function, which works great - but having all the images in the same directory with no way to import in to subdirectories just makes no sense to me.
Am I missing something? Is there a way to utilize subdirectories in the file structure and I am just not aware of it? Do the module developers not realize the enormous problem of keeping all the images in a single directory? --I'd be happy for someone to enlighten me if I am missing something obvious here...and I hope I am. The image module has come a long, long way recently and I think the Drupal community is finally ready to have a full fledged gallery solution inside Drupal - but only if this problem is solved.
Comments
Comment #1
scott falconer commentedI'm also interested in more information about this issue. As it is right now we're only at a few thousand images, but I can see it becoming a possible problem. Anyone else run into this?
Comment #2
scott falconer commentedI did a bit more digging, and it looks like it's fully dependent on which file system/OS you are using.
As for linux based systems:
"The 2.6 kernels fixed it completely, it keeps the directory as tree structure so you can have one dir with a billion files, and it will still find a single file quickly (but "ls" and other programs that scan the whole directiory will still take a long time of course!)"
Someone correct me if I've found faulty info here...
Comment #3
dsnydernc commentedI run 2.6.9-023stab041.3-enterprise and for me, yes the files are found quickly when pointing straight to them - that however doesn't alleviate the other issues. I manage my own dedicated server so I for instance can zip that directory and other functions without issue (other than it taking forever) but I'd guess 99% of folks running Drupal with mass image galleries will not be able to work with these directories at all - hitting resource restrictions etc. on shared servers among other problems.
It seems to me the most logical solution to this is when using the import function, having the import module recognize and maintain current directory structure. For instance, if importing '/files/import/directory 1' the import module would then place those files in '/files/images/directory 1' respecting the current directory tree.
Comment #4
ben soo commentedStoring images and other BLOB data as files on disk mightn't be the most scalable solution: if one goes multi-server then one has to implement some kind of network file share as well as a database server. i dunno, perhaps it's still more reliable and faster than storing BLOBs in the database?
Comment #5
drewish commentedi'm going to mark this as a duplicate of http://drupal.org/node/103793 because it's got a patch