Howdy.

A real issue comes into play with the caching methodology in the nodes directory.

I have a customer with a site that has ~650,000 articles. His nodes directory currently has 88,000+files and you can watch apache (via strace) struggling to read the directory. Apache gets very unhappy when there are more than 3-4k files in a directory (and realistically, having more than 1024 is bad news).

I propose a structured directory layout based upon the first 3 characters of the node;

Article node ID: 99002

/node/990/9/9/0/99002.html

Never allow the node directory structure to get more than 3-4 levels deep.

If you have a node ID of, say: 650283

/node/650/2/8/3/650283.html

This will make it nearly impossible to ever break the 1024 file-limit that should always be imposed in a hosting environment. Note that many shared hosting providers look for accounts with >1024 files-per-directory and happily suspend them.

Comments

andreiashu’s picture

andreiashu commented
Status: Active » Closed (duplicate)

Your two examples seem to not follow the same technique: in first example you use the first 3 chars of the node in the second you use all of them...
Anyway, this is a duplicate of #410730: System limits: Number of files in a single directory