On this page
Apache Bench (ab)
General documentation about ab can be found at Apache Bench (ab)
Begin by performing a
git pull
in order to ensure your source tree is up-to-date.
Then, execute the following from the command line before applying your new patch. This will establish a baseline from which to judge performance improvements or decreases.
ab -c1 -n500 http://head.localhost/
The -c command specifies the number of concurrent requests and -n specifies the total number of page requests. In 99 out of 100 cases it is not necessary to specify a higher value than 1 for the concurrency. Increasing c will only put more stress on your server. This is useful if you want a general stress test, but it will falsify your results if you only want to test the impact of a particular patch on a particular Drupal page.
This will produce output similar to:
This is ApacheBench, Version 2.0.41-dev <$Revision: 1.121.2.12 $> apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient).....done
Server Software: Apache/2.0.55
Server Hostname: localhost
Server Port: 80
Document Path: /head/index.php
Document Length: 16668 bytes
Concurrency Level: 1
Time taken for tests: 36.695602 seconds
Complete requests: 100
Failed requests: 0
Write errors: 0
Total transferred: 1718500 bytes
HTML transferred: 1666800 bytes
Requests per second: 2.73 [#/sec] (mean)
Time per request: 366.956 [ms] (mean)
Time per request: 366.956 [ms] (mean, across all concurrent requests)
Transfer rate: 45.73 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 316 366 29.7 362 608
Waiting: 298 346 29.3 342 588
Total: 316 366 29.7 362 608
Percentage of the requests served within a certain time (ms)
50% 362
66% 367
75% 370
80% 373
90% 384
95% 390
98% 462
99% 608
100% 608 (longest request)
There are two important parts of this result:
1) Requests per second.
This is a good value to judge the overall performance impact. Higher is better.
2) The Connection Times table
The first row of the table specifies the headers for the columns. We are mostly interested in the second (mean
) and third ([+/-sd]
) column of the last row (Total
). The mean or average tells us how long a page request took on average (in milliseconds). Here, lower is better. The value is actually not very different from the Time per request
value; it's just that the computation is done "the other way around".
What makes this value interesting is the fact that ab
provides the standard deviation of the value in the third column. The standard deviation is a means to judge how accurate a particular test result is. If the standard deviation is large, then this is an indicator that something wasn't working as it should, for example, the webserver was under high load or there was a network lag. If this is the case, you will usually see that in the table at the bottom (percentage of the requests served within a certain time) the last value is much larger than the first one. In this case you should investigate your experimental setup and try to reduce the standard deviation.
Now, apply your patch. Here we're showing that you can retrieve the patch file with curl
and apply it with patch
curl -LO http://drupal.org/files/issues/patch_file.patch
git apply -v patch_file.patch
Now re-execute the command above, and compare the results:
This is ApacheBench, Version 2.0.41-dev <$Revision: 1.121.2.12 $> apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient).....done
Server Software: Apache/2.0.55
Server Hostname: localhost
Server Port: 80
Document Path: /head/index.php
Document Length: 17100 bytes
Concurrency Level: 1
Time taken for tests: 37.903596 seconds
Complete requests: 100
Failed requests: 0
Write errors: 0
Total transferred: 1761700 bytes
HTML transferred: 1710000 bytes
<strong>Requests per second: 2.64 [#/sec] (mean)
Time per request: 379.036 [ms] (mean)
Time per request: 379.036 [ms] (mean, across all concurrent requests)
Transfer rate: 45.38 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 318 378 35.5 376 690
Waiting: 301 358 35.4 355 672
Total: 318 378 35.5 376 690
Percentage of the requests served within a certain time (ms)
50% 376
66% 381
75% 385
80% 388
90% 396
95% 401
98% 436
99% 690
100% 690 (longest request)
Interpreting results
Ideally, we now have two sets of values from our tests: The mean of the time taken per request and its standard deviation.
Without patch:
366 +- 29.7 ms
With patch:
378 +- 35.5ms
We can see that the second value is higher by 12ms and we might draw the conclusion that the patch makes Drupal slower. However, there is also the standard deviation (which is rather large at almost 10% of the result). If you add the standard deviation of the first result and subtract the standard deviation from the second you will get:
Without patch:
396 ms
With patch:
342 ms
The order is now reversed.
This means the two results are within their respective standard deviations. This means that the test wasn't able to deliver a decisive result. In such a case you might try to increase the number of requests that ab
does.
Ideally, your results should be several standard deviations apart to allow for a clear statement on the performance impact of a patch.
Help improve this page
You can:
- Log in, click Edit, and edit this page
- Log in, click Discuss, update the Page status value, and suggest an improvement
- Log in and create a Documentation issue with your suggestion