PHP vs. JSON read performance deathmatch

Nice tests, thanks.

There are a lot of things to take into account, first of all: loading PHP files doesn't do any file I/O if you are actually using APC with apc.stat = 0, while the JSON file loading will do a lot of them. This can make a huge difference if you have a slow FS (memory consumption VS. velocity always has been the real fight).

Another stuff, if you store a copy of the file into cache, JSON gets serialized (caching...) so if you fetch the JSON, you unserialize the string, then decode it; If you store a serialized PHP array, you just unserialize it and can use it directly! But this won't be true anymore if you store directly a JSON string and not a serialized PHP string (but if I'm not mistaken, the actual cache API does serialize when storing).

Fetch with (cache backend|database) storage would be an interesting thing to do. If I understood well the debate, you don't need to re-read real files each hit because configuration is supposed to be cached (then not really read from files when in production).

The detail on APC memory is interesting news for me, thanks for that.

The rest is pretty much known:

http://www.phpdevblog.net/2009/11/serialize-vs-var-export-vs-json-encode...

+ follow-up http://www.phpdevblog.net/2009/11/serialize-vs-var-export-vs-json-encode...

I did some further tests, see the file attached. No caching involved or anything.

Tested on PHP 5.2 (linux gentoo):

pounard@guinevere /var/www/d7-core $ php -f test.php 
Testing 20000 iterations, array size is 10000 items.
json_encode(): 10.8694529533
json_decode(): 80.2525508404
serialize():   63.9363901615
unserialize(): 72.3440270424

And PHP 5.3 (linux ubuntu):

pounard@blaster:~
[Fri Jun 24, 16:56] $ php -f /tmp/test.php
json_encode(): 12.629450082779
json_decode(): 97.817018985748
serialize():   71.509289979935
unserialize(): 91.29324889183

The test is biased because it testes only with int and not strings (actually got the test from one of Sun's URLs and modified it). But it shows that unserialize() is (a bit) more faster than json_decode() (really pretty much nothing).

Configuration should be stable on a production environment, and read operations are more important than write operations IMHO. If the site rebuild often its configuration then we can assume that there is a serious problem behind.

One more note: the PHP 5.2 box's hardware is really faster than the 5.3 one's, that explains why numbers are higher on PHP 5.2 where we could expect the opposite.

EDIT: Did the same tests with array filled with random strings, and got a really huge difference: unserialize() two times faster than json_decode(). The code remains biased because it doesn't test with a hierarchical array though.

PHP 5.2:

pounard@guinevere /var/www/d7-core $ php -f test-strings.php
Testing 20000 iterations, array size is 5000 items.
json_encode(): 47.4064810276
json_decode(): 91.0086319447
serialize():   33.496325016
unserialize(): 42.2469871044

PHP 5.3:

pounard@blaster:~
[Fri Jun 24, 17:27] $ php -f /tmp/test-strings.php
Testing 20000 iterations, array size is 5000 items.
json_encode(): 63.624093055725
json_decode(): 109.14283394814
serialize():   42.207105875015
unserialize(): 52.044392108917

This can serve as a good base for performance tests for the final version if we implements the idea that sun submitted which is here: http://groups.drupal.org/node/157379#comment-528739. Allowing multiple formats defined in different parsers.

@#7 If I could +1 one your post, I'd definitely do it.

For most of the level 1 (file) storage I agree with catch. CPU usage is not the bottleneck, as it's not a critical path. Memory usage is. On the memory usage front, json beats included PHP for the reasons catch explains in #0.

@#8 Crell, yes I agree, as catch said in #7 memory will be critical mainly at rebuild time if the operation happens when a lot of other cache are being rebuilt.

I'm wondering if there's any other path for reducing memory usage while rebuilding cache globally? When the cache rebuilding become a problem there is maybe an overall problem with cache usage, which seems to be twisted.

I now that PHP include memory cost cannot be lowered here, but if 2MB alone was a problem, nobody would use Views. I don't want to minimize this memory usage, don't get me wrong, but it seems that catch's original testing assumed that 500 configuration files is a standard, but I really don't see when a site will have more than 40 or 50 of them (which seems really too much by the way: if you go into the path of one file per module, probably worth the shot to combine small ones together per packages, better for file loading, but also better for reducing amount of cache_get(), cache_set() or level 2 database retrieval queries).

There is not expected to be any cache_get() calls, at all. The Level 2 active store is not using the cache system. I think we're all on the same page that level 2 should be pluggable, though, so very-high-traffic sites can switch to Mongo or Redis or some other data store more appropriate for key/value-blob than SQL.

There's a lot we don't know yet about D8's internal usage patterns, and I fully expect them to change radically once we start adding more changes to the system. I think the take-away here is that we need something that will scale decently on rebuild, and JSON does. That gives us flexibility later. Trying to put all memory optimization in the config format is not a good approach; rather, while rebuilding other parts of the system we should keep memory usage in mind, since we're going to be mucking with most of Drupal anyway during D8 (again).

JSON scales better than PHP does, memory-wise. That gives us more flexibility and wiggle room. That's enough for me on the performance front.

@#11 Pretty much what I said except for the part where you defend JSON. It appears that we mainly agree.

Note that it should be "the level two store is not currently using the cache system". We haven't really discussed this at all that i recall.

Good point, eval() is slow. But whatever formatting is chosen, I hope you will never eval() PHP but include it directly. The good performance of PHP is when it's being included "normally".

It's not the speed that's the issue, it's the memory usage. include() for PHP has a very bad impact on memory usage.

On my limited testing, as soon as you are working on non trivial data structures, nothing beats serialize/unserialize. [not even igbinary_serialize/igbinary_unserialize, which is interesting].

json_encode/decode being faster is only true for trivial data structure, which probably means that it has a smaller fixed cost.

How trivial a data structure are we talking about? I expect *most* configuration objects will not be too outlandish, with a few outliers that are quite complex, eg, Views. But I don't know what the balance point will be there.

Given that we've chosen a file format I am closing this issue unless someone can come up with a good reason not to.