Benchmarking Memcache

Feature
Create a standard set of benchmark tests defined inside a document 'benchmark.txt' that profiles memcache performance.

Background - Follow-up: http://drupal.org/node/133410
In this post I outline a process of generation of a random set of URLs and then running 'siege' to profile the performance of memcache. In this test I used the devel_generate module to created a generic site (nodes, comments, taxos...) and then ran 'siege' against it. My recommendation is that we create a standard set of performance tests, using ab (apache benchmark), siege and devel_generate to profile the performance of memcache.

Supporting Posts
HOWTO: Benchmark Drupal code - http://drupal.org/node/79237
Server Performance Benchmarks - http://drupal.org/node/43478
Generation of URLs - http://drupal.org/node/48894
jmeter - http://drupal.org/node/44442

Benefit
- Allows for comparison of caching strategies under different types of website profiles
- Creates a focus for a performance discussion
- Enables developers to profile their installations before and after implementation of memcache

Assumptions
- Benchmarking is not reality: Performance benchmarking tools only represent a approximation of a real website. We should expect an organic process of refinement of any set of benchmarking tools as we attempt to get closer to real world websites
- No tools are perfect: Any tools we use for benchmarking are going to have pros and cons. Ab (apache benchmark) and siege are widely used and accessible.

Future Directions
- use 'installation profiles' to create a standard set of sites with different module configurations