X Autoload has some bold statements about performance on the module page.
It is bad style to boast with numbers that are not backed up.

We should run some benchmarks to compare with the Symfony loader.
This is what I just did.

Comments

donquixote’s picture

X Autoload uses (or, *should* use, argh) an APC cache, with a performance comparable to that of other class loaders (Symfony).

However, it really outperforms those other loaders on systems without APC.
It is designed to scale much better with a high number of namespaces.

Here is a "Preview" benchmark result from a local version of xautoload.
The public version currently has the APC cache disabled, so it behaves mostly equivalent to the "xautoload, no apc".

loadClass() 1 x 300

We register 300 namespaces of test modules (in addition to those already installed), and load one class for each.

classloader + apc: 19 ms
xautoload + apc: 20 ms (*)
xautoload + apc + is_file(): 22 ms
xautoload, no apc: 27 ms
classloader, no apc: 82 ms

Repeated findFile() 10 x 300

We register 300 namespaces of test modules (in addition to those already installed), and call 10x findFile() for a class that is already loaded (one for each of the 300 test modules).

classloader + apc: 10 ms
xautoload + apc: 9 ms (*)
xautoload + apc + is_file(): 14 ms
xautoload, no apc: 62 ms
classloader, no apc: 620 ms

(*) The relevant code is almost identical in the cases where APC is enabled.
We assume the difference is just random, and may be different on other machines.

donquixote’s picture

Note: Again, this is a preview. I am going to upload something to allow others to reproduce, until then it is just my word.

You can have a look at Autoload benchmark in my sandbox, which is what I started with. But it is not the same.

donquixote’s picture

Here is an updated benchmark, finally using publically available code: xautoload-7.x-3.x, and the latest autoload_benchmark.

Benchmark results: 300 namespaces

(avg microseconds per single operation)
We register 300 test namespaces into each loader, and see how well they perform.

findFile.before loadClass findFile.after loadClass.nested loadClass.nonexisting
classloader.bare
171.748
239.519
188.962
292.520
427.918
xautoload.loader.bare
22.624
83.832
22.730
95.367
35.783
xautoload.loader.apc
6.239
71.482
6.926
78.984
7.081
xautoload.loader.apc_aggressive
3.725
66.670
3.309
68.565
6.799
classloader.apc_aggressive
4.097
62.350
4.093
66.451
7.168

Benchmark results: 100 namespaces

(avg microseconds per single operation)
We register 100 test namespaces into each loader, and see how well they perform.

findFile.before loadClass findFile.after loadClass.nested loadClass.nonexisting
classloader.bare
67.423
129.843
70.244
140.452
140.297
xautoload.loader.bare
22.366
95.904
24.617
102.150
38.993
xautoload.loader.apc
6.310
64.445
6.137
66.745
6.604
xautoload.loader.apc_aggressive
3.443
60.809
3.304
62.752
6.640
classloader.apc_aggressive
3.604
58.651
3.827
65.148
7.308

Legend

classloader.bare: Symfony\Component\ClassLoader\UniversalClassLoader->loadClass() / ->findFile()
classloader.apc_aggressive: Symfony\Component\ClassLoader\ApcUniversalClassLoader->loadClass() / ->findFile()
xautoload.bare: xautoload_ClassLoader_NoCache->loadClass() / ->findFile()
xautoload.apc: xautoload_ClassLoader_ApcCache->loadClass() / ->findFile()
xautoload.apc_aggressive: xautoload_ClassLoader_NoCache->loadClass() / ->findFile()

The "aggressive" just means that we skip the file_exists() if APC cache has a result, meaning that we potentially crash if that file has moved.

donquixote’s picture

Issue summary: View changes
Status: Active » Closed (fixed)

I think the information on the module page is balanced enough now.