Hi,
I propose the following:
Add a cache layer to cache_get and cache_set which doesn't use the database but stores the data on the disk using var_export(). To get the data back again one now just runs include on the cache file, which contains a function which returns the cached variable. Why would we want this?
Caching huge amounts of data over mysql is inefficient. Passing a big, 2d array to serialize/unserialize is inefficient, too. And when the data from var_export written into a file generates a file which is 1MB big.
Some numbers of some D6 code. As the logic of cache_get hasn't changed from D6 to D7, this is applicable for HEAD, too. The file cache is hacked together, but shows the point.
without caching, computing the 2d array from a huge sql select statement takes about 4 minutes, 18 seconds.
with the current cache, table 'cache' it takes about 3 minutes. With a file cache present, under the same conditions as the previous runs, it takes *** seconds to complete. The need for such an API is imho definitly given, if not in core, then in contrib/.
There are several ways one could go on implementing this:
a)
One file per variable, so we don't load other, potential *big* variables in memory just because we're serving one page.
b)
One file for all 'file' variables, every cache variable is returned by a function.
c)
like the current approach for db-caches, different tables:
$type = 'file': b)-like file for all file variables except:
$type = 'file_varname': a new file, a) like.
I'd be willing to implement this cache layer, if there is chance that it goes into core. Another interesting thing would be benchmarking it against the db cache and look which one is faster. Any opinions on this?

Comments

CorniI’s picture

CorniI commented

ok I forgot the benchmark value for the file cache, it's about 2 seconds, for following requests it decreases to about 0,7 seconds. All tests were done with xcache enabled, php 5.2.4-2ubuntu5.5

chx’s picture

chx commented

Good idea but the security implications... you are writing php code and that can be a security weakness... I am wondering what could be done.

chx’s picture

chx commented

To make this secure, serialize can not be avoided. You simply can't write PHP and read it back into the application.

catch’s picture

catch
he/him
Primary language English
commented

subscribing.

I think the locale cache is a candidate for this (currently loads the entire locale cache on each request). Shame about the serialize/unserialize but don't see a way around this.

catch’s picture

catch
he/him
Primary language English
commented
Issue tags: +Performance

and tagging.

CorniI’s picture

CorniI commented

just as a note, a proof-of-concept implementation showed about 50% savings(implementing an early pagecache for anonymous viditors) - as additional option for the server admins who understand the security implications (etc...) it would probably reduce the load of the server. There are problems, though, as var_export doesn't allow stdClass-objects to be exported in a way you could read them back without parsing their php.
About security:
Yes, you're writing arbitrary text to a file you later either read back and eval or include directly (The include path php.ini option could be cricumvented by the former). The problems here arise if the file permissions are not proper set _and_ you live in a multi-user environment. The attack vector would be to modify the php by hand, and instead of the FTP's user rights [or the accounts holder unix account] you'd need privileges of the webserver. Here we face another problem - for images/whatever, we already *have* webserver-writeable files - just not php files. If you're able to write to them, i'm pretty sure you've already won, as you can overwrite images with images containing XSS, overwrite possible downloads of the site with trojan ones, etc.
From my perspective, the clearly awesome performace gain here overweights the security problems i outlined. If I missed something security related, this may look different - chx, what do you think?
And, probably, still with serialize this would be interesting to benchmark against the database cache...

chx’s picture

chx commented

Crell, the problem is the unknown. If you have a minor file upload bug, where you can upload anything and you can upload PHP code to be executed then that minor upload bug suddenly became the end of the world. This is why the security team do not allow all those modules that try to autoupdate by simply writing modules.

killes@www.drop.org’s picture

killes@www.drop.org commented

Does this make sense when there's memcache?

catch’s picture

catch
he/him
Primary language English
commented

It wouldn't make sense to use it if you can install memcache, but it makes sense for people on hosting where they can't install it. Also seems like the only alternative caching mechanism we could support in core, except for maybe APC's user caching.

killes@www.drop.org’s picture

killes@www.drop.org commented
Priority: Critical » Normal

I'd recommend to create infrastructure in core that is needed to implement this in contrib.
If everything is there, I recommend to mark this as "fixed".

Changing priority, no way this is critical.

moshe weitzman’s picture

moshe weitzman commented

Dries has already stated a preference to have a second cache implementation in core so we can test the swapping logic and demonstrate our features. So I do think there is support for this in core.

catch’s picture

catch
he/him
Primary language English
commented
Version: 7.x-dev » 8.x-dev
ogi’s picture

ogi commented

subscribe

casey’s picture

casey commented

http://drupal.org/project/filecache

dawehner’s picture

dawehner
Primary language German
commented
Status: Active » Fixed

We have 

  cache.backend.php:
    class: Drupal\Core\Cache\PhpBackendFactory
    arguments: ['@cache_tags.invalidator.checksum']

which does exactly that.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.