Daily test runs
Rok Žlender - March 27, 2008 - 21:33
| Project: | Test driven development infrastructure |
| Component: | Code |
| Category: | task |
| Priority: | normal |
| Assigned: | cyberswat |
| Status: | closed |
Description
It would be cool to have daily automated test runs of latest Drupal version dev and stable. This would mean that every day we would get report if latest HEAD or latest version of 6 branch is still passing tests.
#1
The current code base is slanted towards checking out the same version of the simpletest module as what is being tested. It performs a second checkout after getting the base version of Drupal it is testing and appears to run tests against this checkout. Is it desirable to follow this approach or should the simpletest modules be run in a stable version of Drupal that then interacts with the temporary test instance?
#2
Waiting on some info from hunmonk to make sure I didn't break pift processing ... This patch allows for full testing of specific versions in addition to the existing patch testing. It also reports on testing results and reports complete failures. Once I speak with hunmonk I may make modifications before signing off on this but don't want to loose the work.
#3
Hasn't been run through coder yet but here is what I believe to be a working version
#4
Since #308974: Test all RTBC and CNR issues on rotation is working I think that alleviates the need for this as patches shouldn't be committed if they make tests fail.
#5
In practice, patches are sometimes committed, even if they cause tests to fail. Sometimes the tests only fail in certain environments (PHP version, MySQL version, Windows/Linux, $clean_urls, $db_prefix etc.). It would be useful if there was a number of machines with different configurations running tests on HEAD daily or weekly. Of course it would be optimal if all patches were run on all these machines prior to being committed, but a more modest solution that just pulls from CVS would be useful.
Examples of issues that are only present in some environments:
#6
#5: The plan is to wait for testing results before committing, but that relies on all this stuff working. (very close)
As for machines with different OS/db engines, that will come later. Right now we just want to get all the features that are necessary working and stable. We are running into a few quirks.
Then eventually I see running a patch through multiple OSs and db engines and comparing results.
#7
Automatically closed -- issue fixed for two weeks with no activity.