Use a separate test database, transactional/rollback and easily load fixtures for test case data
RobRoy - September 29, 2007 - 04:28
| Project: | SimpleTest |
| Component: | Code |
| Category: | feature request |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | closed |
Description
We're going to be doing a lot with SimpleTest over the next few months and taking some things from Rails I'd like to work on using a separate test database that takes a snapshot of the current database (or loads some other fixture data), creates a transaction and then rolls it back before the next test (when running more than one test). We've already patched some smaller things but I'd like to do more and test all this stuff before submitting a patch.
If anyone has any suggestions/feedback/etc., please post them here.
#1
I really love this idea. If your tests are buggy, it's possible to totally fubar your database. :P And while there is a very clear sentence in the README that says NOT to use this on a production database, we all know how many people read READMEs. :P
#2
I hate posting unpolished patches, but here is a patch that we're using in some heavy development and it works great for us. I wish I had more time to totally clean up SimpleTest, but I don't. Hopefully someone can run with this. It's got a ton of fixes and some new features including a new variable 'simpletest_base_url' which if set to 'http://d.local' will use that for running tests from the admin panel (although I just use command line to run tests now, much nicer).
What I do for this idea of "fixtures" is create a couple aliases on my local box:
alias dfixtures='mysqldump -u root dbname > ~/d.test.sql; mysql -u root test < ~/d.test.sql'
alias dtest='/apache2/php/bin/php ~/Sites/dev/chipin/d/manager/trunk/htdocs/sites/all/modules/d/tests/run_d_tests.php d.local'
Then run dfixtures, and then dtest which runs a PHP file like this:
<?php
// $Id$
/**
* @file
* Run all unit tests for my custom modules.
*
* @todo Convert from this ghetto-ness to drupal.sh from HEAD in /scripts/ dir.
*/
// Kill script name.
array_shift($_SERVER['argv']);
// First argument should be the host for which sites/<host>/settings.php should be used.
$_SERVER['HTTP_HOST'] = array_shift($_SERVER['argv']);
$tests = array_shift($_SERVER['argv']);
if (!isset($tests)) {
$tests = array(
'MyModuleTest',
'MyModuleTest2',
);
}
else {
$tests = explode(',', $tests);
}
chdir(realpath(dirname(__FILE__) . '/../../../../../'));
require_once './includes/bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);
// If not in 'safe mode', increase the maximum execution time.
if (!ini_get('safe_mode')) {
set_time_limit(360);
}
simpletest_run_tests($tests);
?>
#3
Maybe we could include this into http://drupal.org/node/210679 .
#4
This sounds like something that could be useful for SimpleTest. If so we need to affirm it as a community and update this patch.
#5
Here's a snippet (untested, but something pretty similar is working for another project) that will install a profile.
<?php
// Create test database. Probably error-handle this so if it fails, resort to table prefixes instead.
db_query("CREATE DATABASE simpletest_". $this->randomName());
// Modify $db_url so you can connect to two databases.
global $db_url;
$db_url['default'] = $db_url;
// Parse out db username password etc.
$db_url['test'] = $test_db_url;
db_set_active('test');
// Install a given profile.
include_once './includes/install.inc';
drupal_install_system();
$module_list = drupal_verify_profile($profile, 'en');
drupal_install_modules($module_list);
// Do any preliminary stuff; setup content types, set variables, etc.
// Run the tests.
// Return to Drupal.
db_set_active();
?>
#6
Note that we want to use a prefix like simpletest_abcdefqw , aka randomname(8) so that if simpletest dies on us then we won't get a lot of angry MySQL errors the next run. Using a prefix makes it simpler, a separate database would be complicated for the users.
#7
This is easy stuff for Drush. I will add it to the new simpletest runner. But which profile to use?
This could be slower than just running a mysql statement to load up a DB but we'll try this first.
#8
Oops - I have to port Drush to D6. I'll do that real soon now.
#9
the profile you are running right now, it's available from settings.php isn't it? if not, then default or provide an admin ui.
#10
This is going to be somewhat tricky as you need to edit settings.php for subsequent Drupal boots to keep the new (random) prefix and then, when simpletest is done, we need to make sure it gets reset -- which is tricky if simpletest dies (can happen, we are testing). I think I will set CURLOPT_USERAGENT to "simpletest $db_prefix" and getting the handling of this written to settings.php at form time and adding something which indicates it's there. Something like:
<?phpif (substr($_SERVER['USER_AGENT'], 0, 11) == 'simpletest ')) {
$db_prefix = preg_replace('/[^A-Za-z0-9]/', '', substr($_SERVER['USER_AGENT'], 11));
}
$GLOBALS['simpletest_ua_handling'] = TRUE;
?>
should go to settings.php.
#11
Beware, this patch litters your database as there is no cleanup. Also it does not work -- but why? Add
<?phpif (substr($_SERVER['HTTP_USER_AGENT'], 0, 11) == 'simpletest ') {
$db_prefix = preg_replace('/[^A-Za-z0-9]/', '', substr($_SERVER['HTTP_USER_AGENT'], 11));
}
$GLOBALS['simpletest_ua_handling'] = TRUE;
?>
to settings.php
Edit: "Does not work" mean, the user created (always uid 3) does not get its roles. I still have to figure out why.
#12
Here we go. I have added a few more lines of code from install.php and made sure the useragent-based prefix is safe. It seems working.
#13
Oh, and the settings.php part:
<?phpif (preg_match('/simpletest\d+/', $_SERVER['HTTP_USER_AGENT'])) {
$db_prefix = $_SERVER['HTTP_USER_AGENT'];
}
$GLOBALS['simpletest_ua_handling'] = TRUE;
?>
we obviously need to work on code to make sure it's there.
#14
Reroll.
#15
Another reroll against HEAD. Also, this adds a safeguard before dropping tables.
#16
Looks great. Very nice first step.
The way you use the useragent to pass the db_prefix to allow for multiple different test sessions was very clever.
#17
Per test.
#18
Now with bonus whining.
#19
This appears to work great.
This patch means that the db will be flushed after each test method is called. Which is exactly what we have been needing.
This patch also works for multiple testing sessions running simultaneously.
Great work chx!
Note: leaving this for further testing, review, and comments.
#20
Patch works tables are created, used and removed after every test case. I have 2 problems with this patch:
1) page node creation tests I tried don't finish successfully probably this is the reason. I get this "Sorry, unrecognized username or password. Have you forgotten your password?" right after attempt to login simpletest user
2) building and destroying new tables before and after every 2 line unit test might be a bit of an overkill not sure what we can do about it.
#21
I've read through the patch and it looks GREAT.
+(500500)
#22
looks good, but haven't actually tested it yet.
Also, is there any security concern with this USER_AGENT method? What happens if I access a site remotely with a browser where I can set HTTP_USER_AGENT- can I get Drupal to run a new install with prefixed tables?
#23
@pwolanin: Good point. I know you can get an extension to do that in Firefox and I believe Konqueror lets you do it by default.
Probably need to generate some sort of key when SimpleTest is installed then pass that along with db_prefix to ensure it is coming from SimpleTest.
@Rok Žlender: Yea that could be an issue. One thought I had was make a different test case that unit tests extend since allot of the DrupalTestCase functionality is for function browser based tests.
If we create a new test case that could fix the issue.
#24
@boombatower - one suggestion for a key that can be checked in settings.php: pass (in the query string?) the hash of the drupal private key with the simpletest prefix. Or alternately (since I guess this is all running before bootstrap) on install write such a key private key into settings.php?
#25
This patch require a key to set the prefix in settings.php.
This will prevent the situation which pwolanin described.
I have tested and seems to work.
#26
@boombatower - something doesn't look right about that code - you are going to set
$GLOBALS["simpletest_ua_key"]every time settings.php is parsed?#27
@pwolanin: the key needs to be there when SimpleTest starts the chain...which we have no way of knowing before hand...and settings.php must have same key so it needs to be there.
Limits to that yes.
#28
Committed.
#29
I ci a small patch that fixes problems with clean urls. http://drupal.org/cvs?commit=107508
#30
Automatically closed -- issue fixed for two weeks with no activity.