run-tests.sh class and module options broken [#1936806]

Posted by alexpott on March 7, 2013 at 10:40pm

8 followers

Issue Summary

Since #1901670: [Change notice] Start using PHPUnit for unit tests the output of simpletest_script_get_test_list is expected to be something like:

array(
  'group' => array(
    'testclass1',
    'testclass2',
  ),
);

However if you use --module or --class then this function currently returns

array(
  'testclass1',
  'testclass2',
);

as it used too. This means that simpletest_script_execute_batch flattens the test_group array to an empty array and no tests are run.

So for example, without this patch the following command is a total fail...

$ php ./core/scripts/run-tests.sh --class "Drupal\book\Tests\BookTest"

Comments

#1

Posted by alexpott on March 7, 2013 at 10:46pm

Status:

active

» needs review

The patch attached is a comprise... basically instead of trying to determine the correct test group for --class and --module we just use 'class' and 'module' as the test group... this keeps us clear of the following code in simpletest_script_execute_batch()

  // Separate PHPUnit tests from simpletest.
  if (isset($test_groups['PHPUnit'])) {
    $phpunit_tests = $test_groups['PHPUnit'];
    unset($test_groups['PHPUnit']);
  }

whilst making this code work...

  // Flatten the simpletest tests into an array of classnames.
  $test_classes = array();
  foreach ($test_groups as $group) {
    $test_classes = array_merge($test_classes, array_values($group));
  }

The group key is only used by the PHPUnit additions to run_tests.sh so this change minimises the impact and just works.

Attachment	Size	Status	Test result	Operations
1936806.run-tests.1.patch	1.17 KB	Idle	PASSED: [[SimpleTest]]: [MySQL] 52,426 pass(es).	View details

#2

Posted by alexpott on March 7, 2013 at 11:45pm

The patch attached reverts the change to pass test lists around keyed by group eg...

array(
  'group' => array(
    'testclass1',
    'testclass2',
  ),
);

And instead detects is the class is a PHPUnit test using is_subclass_of(). This means you can mix classes like this...

$ php ./core/scripts/run-tests.sh --class "Drupal\Tests\Core\Cache\NullBackendTest,Drupal\search\Tests\SearchBlockTest"

Where one test is PHPUnit and one test is a WebTestBase :)

#3

Posted by alexpott on March 7, 2013 at 11:45pm

Now for the patch

Attachment	Size	Status	Test result	Operations
1936806.run-tests.2.patch	4.88 KB	Idle	PASSED: [[SimpleTest]]: [MySQL] 52,487 pass(es).	View details

#4

Posted by alexpott on March 7, 2013 at 11:46pm

+++ b/core/scripts/run-tests.shundefined
@@ -431,8 +418,8 @@ function simpletest_script_execute_batch($test_groups) {
@@ -459,7 +446,7 @@ function simpletest_script_run_phpunit($test_id, $phpunit_tests) {

@@ -459,7 +446,7 @@ function simpletest_script_run_phpunit($test_id, $phpunit_tests) {
         $summaries[$result['test_class']]['#exception']++;
         break;
       case 'debug':
-        $summary['#exception']++;
+        $summaries[$result['test_class']]['#debug']++;
         break;
     }

This patch also fixes this small bug with PHPUnit test summarisation.