The patch attached adds two new options to run-tests.sh
The following command will run the StateSystemUpgradePathTest 100 times and if a failure occurs it will exit immediately.
./core/scripts/run-tests.sh --die-on-fail --repeat 100 --class "Drupal\system\Tests\Upgrade\StateSystemUpgradePathTest"
You can then change your settings.php to use simpletest db suffix to debug the test failure.
| Comment | File | Size | Author |
|---|---|---|---|
| #18 | 1825592-18.patch | 4.57 KB | pietmarcus |
| #15 | 1825592.drupal8.test-random-fail.15.patch | 4.72 KB | alexpott |
| #15 | 14-15-interdiff.txt | 1.59 KB | alexpott |
| #14 | drupal8.test-random-fail.14.patch | 3.33 KB | sun |
| #13 | drupal8.test-random-fail.13.patch | 2.38 KB | sun |
Comments
Comment #1
alexpottAnd now adding patch :)
Comment #2
berdir+1, this is really helpful to track these nasty random failures down.
Maybe we can move the exit a bit down after checking and printing the error log? Not sure if it's necessary, as we likely already know what the error is when using this, just not under which circumstances it happens.
Comment #3
alexpottRerolled...
The reason I put the exit where I did is because the simpletest_script_cleanup() routine removes the db and files as well as printing the log and we want to keep these.
Comment #4
alexpottAnd here by the patch... Cheers @swentel
Comment #5
swentel commentedAlright, let's get this in, this is so freaking handy
Comment #6
webchickThat seems incredibly handy! So handy, that I'd love to see an option for "Die on fail" exposed at admin/config/development/testing/settings, too. :)
Switching to a task rather than a feature so it doesn't get gummed up on thresholds.
Seems like at least the extra params (possibly the UI change too; it's not like this is a very user-facing form) should be eligible for backport.
Comment #7
xjmI think this is out of scope here, maybe a followup issue?
Comment #8
xjm#4: 1825592-3.simpletest-die-on-fail-and-repeat.patch queued for re-testing.
Comment #9
alexpottI'm inclined to agree with xjm that adding the option to the admin interface is out of scope. Especially since once set in the admin interface it'll be set for all test runs which is unlikely to be the desired outcome. So we'll maybe we'll need to add it to the re-run tests page... or something like that.
Patch attached improves the run-tests.sh message by providing the db prefix and files directory to make it easier to update your settings to debug the fail.
Comment #10
webchickHm. I guess we can defer it. It's very odd though to me for run-tests.sh and simpletest.module to be out of sync regarding the features they provide. :\ We're adding a new CMI value here, it seems natural to expose a UI for it.
Comment #11
xjmThere's already very different featuresets in
run-tests.shversus the module UI, so I've filed #1893930: Expose a setting to die on failure in the SimpleTest UIComment #12
webchickSorry, this fell off my radar. This has already been very field-tested in debugging several major issues so it's good to go.
Committed and pushed to 8.x, moving back to 7.x for backport.
Comment #13
sunThanks, this is a useful addition.
Marked #1893930: Expose a setting to die on failure in the SimpleTest UI as won't fix, with a short explanation for why.
Attached patch cleans up the committed changes.
Comment #14
sunSorry, public vs. protected.
Comment #15
alexpottReally like the changes in #14 - whilst trying to debug random upgrade failures I wanted to use the --verbose flag as well but discovered that --die-on-fail and --verbose are mutually exclusive... the patch attached fixes that.
Comment #16
sunComment #17
webchickCommitted to 8.x. Will push in a few mins.
Back to 7.x.
Comment #18
pietmarcus commentedI had to dust off the patch a bit, but I have backported it to drupal 7. Please review.
Comment #20
pietmarcus commentedTests ran successfully this time. Please review.
Comment #25
mustanggb commentedWould be super handy to have these in D7, anyone able to review?