Provide performance information on testing bots
aclight - December 5, 2008 - 20:34
| Project: | Project Issue File Review |
| Version: | 6.x-2.x-dev |
| Component: | Code |
| Category: | feature request |
| Priority: | normal |
| Assigned: | boombatower |
| Status: | closed |
Jump to:
Description
I think it would be quite useful to provide additional information on the past performance of tested patches for each slave server. The information probably wouldn't need to be made publicly available on t.d.o. Useful information might include the average time it took to go through each step of the process (installing, checking out, running tests), as well as information on the outcome of tests run on the individual server.
I suspect that there is a lot of room for improvements in performance, but without some solid numbers it's difficult to see if any changes made have improved performance or not.
#1
This would be quite useful, not only for benchmarking, but also to make test results available as fast as possible. If there are 4 idle testing clients, the master could pick the fastest one to run the tests.
From the logs, there appears to be quite a performance difference. The time to complete a full test run (from the recent t.d.o logs):
#4 : 7 minutes
#5 : 32 minutes
#7 : 17 minutes
#8 : 7 minutes
#9 : 25 minutes
#10 : 30 minutes
Of course, this information isn't entirely accurate because some patches include additional tests that could affect performance - but if we keep averages over the recent X number of runs, that should give quite a good idea of the performance. Don't simply count an average over all test runs, otherwise the older testing client could get punished when performance improvements are made to the testing code.
ps: i tried to avoid the word slave as in some parts of the world, people might get upset if we talk about making our slaves run faster and punishing them ;)
#2
lol at slave comment.
I'll look into this, I should have more time in a week.
#3
Marked duplicate: #411002: Simpletest showing the time impact of one patch
#4
#5
2.x provides detailed logs with timestamps, once we get things going lets look into enhancing this further where needed.
#6
Automatically closed -- issue fixed for 2 weeks with no activity.