Last updated September 7, 2011.
Since November 2008, patches against the development version of Drupal core (also known as Drupal HEAD) are tested automatically against a list of criteria by http://qa.drupal.org.
Why automated patch testing?
More people submit patches than test them, and more people review patches than commit them (there are only usually two core maintainers), so core committers always operate with a backlog of patches. This works very well in terms of keeping the code quality in Drupal high, but it makes the workload of those reviewing and testing patches quite high.
With the introduction of a testing framework into the Drupal 7 release cycle, much of the work of testing to see if anything is broken has been reduced: automated tests catch a very high percentage of regressions before the patch makes it into core. However, running all tests on a patch takes several minutes, and you may not discover that a patch breaks until 25 minutes into running tests.
Additionally, many patches to Drupal change APIs, might touch many different lines of code, so it's very easy for a patch to go 'stale': the patch might fail to apply, break automated tests, or otherwise cease to work. Additionally, many patches are uploaded and set to "code needs review" which don't apply properly, contain PHP syntax errors, or break tests.
qa.drupal.org
To ensure that the development version of Drupal HEAD remains as stable as possible, and to reduce the workload for patch reviewers and core committers, the testing.drupal.org platform downloads, applies and runs automated tests on every relevant patch against HEAD.
Patches are tested if:
- The issue is set to "Code Needs Review" or " Reviewed and Tested By the Community" (for more information on status settings, see status settings for an issue)
- The patch file has a .patch or .diff extension
- The patch has not already failed testing; you can reset manually if you think
something went wrong
Patches are skipped if the file name (before the .patch or .diff extension) ends with something like -d5 or _d4. For example somefile-d9.patch and somefile_d2.patch is skipped.
Patches are queued for testing when submitted, and also retested every 24 hours on a rotating basis (provided they previously passed all tests, and still meet the criteria listed above) to ensure they continue to work with changes to core. Under normal circumstances, test results should be returned to for any testable patches submitted within one hour.
Patches that pass
If the patch passes testing, a green bar will appear under the file download link to confirm this, showing the number of tests run, and a link to the full results.
Patches that fail
Patches may fail for the following reasons:
- The patch fails to apply. This may be due to changes to core, or it may be rolled incorrectly (see http://drupal.org/patch/create for instructions)
- Drupal HEAD fails to install with the patch applied
- The patch contains a PHP syntax error
- The patch causes a failure or exceptions in Drupal's suite of automated tests
In this case, a red bar will appear with a summary of the test results, a link to the full results, and a link to send for re-testing. Additionally if the patch is the most recent upload to the issue, the issue will be marked "needs work".
Patches that are being re-tested
If a patch has been submitted for re-testing, then a yellow bar will appear indicating that the patch is currently queued for re-testing.
Frequently asked questions
What if my patch depends on another patch?
At the moment, the issue should be marked 'postponed', we may add a new status or flag for this later on.
What if I don't want my patch tested automatically?
You can bypass automated testing by using a .txt extension for your patch file.