I think we have way too much testbot confirmation.
Why don't we rip out the whole step-by-step thing and replace it with a simple configurable contrib patch test? That would prove that a patch could be applied and a test could be run.
The whole step-by-step confirmation system has been really unreliable, with unnecessary failures. Let's get rid of it.
| Comment | File | Size | Author |
|---|---|---|---|
| #5 | pifr_add-shortcut-confirmation-1411974-5.patch | 2.47 KB | jthorson |
| #4 | pifr_add-shortcut-confirmation-1411974-4.patch | 2.42 KB | jthorson |
| #1 | pifr_shorten-confirmation-time_1411974-1.patch | 950 bytes | jthorson |
Comments
Comment #1
jthorson commentedThe failures are not really due to the architecture ... but admittedly, I manually shortcut the last simpletest test any time I re-confirm a testbot.
Boombatower did point out that http://qa.drupal.org/incombatibility shows a case where shortcutting would hide a problems during a new testbot setup, but I think in the 'daily re-confirming' case, where the testbot environment itself has already been certified, running the full test suite during the last stage is a bit of overkill.
The attached patch shortcuts the full test suite on the last stage ... but again, we probably don't want this for 'initial' confirmation tests - perhaps we should add a toggle setting which let's us go between a 'short' confirmation (i.e. automated daily confirmation tests) and a 'full' confirmation (i.e. testbot environment changes, new testbot creation) setting per testbot.
Comment #2
jthorson commentedComment #3
jthorson commentedBeen using this all night on scratchtestbot, and it works beautifully.
Suggest that we add the admin setting before committing, though.
Comment #4
jthorson commentedAdded admin settings.
Comment #5
jthorson commentedOoops ... guess server admin settings should only go on the server. :)
Comment #6
jthorson commentedTested on qa.scratch, and committed to 6.x-2.x (1231dd8).
Comment #8
jthorson commentedMay need more attention ... appears to be causing issues in 6.x-2.8-rc3.
Comment #9
jthorson commentedRetested, and the issue must have been introduced in a later patch.