Running Tests Through command-line

Last updated July 16, 2012. Created by stefgosselin on November 28, 2009.
Edited by dstol, larowlan, Chaulky, TR. Log in to edit this page.

Command-line test execution - Linux

This section is for the those that have the desire and/or the need to run the test-suite by command-line. This is often needed when one wants to integrate 3rd-party continuous-build tools, for example. I am hoping this page will collect info and tips on the manual aspects of testing, when one is doing it small-scale and only wants to test one or a few modules.

For those looking for info on the GUI way of running Simpletest, head for the Simpletest tutorial; lower down in that tutorial and you will find some information for running tests through the Drupal user interface. The two important links for command-line commandos accessing Simpletest through Drupal interface are admin/config/development/testing/ and admin/config/development/testing/results/

Running the tests through the command line in Drupal is quite easy with the help of run-tests.sh, located in /scripts. One can easily get the test-suite running by going to the root of the site directory, as web-server user and typing php scripts/run-tests.sh.

If you don't feel like typing /scripts/run-tests.sh all the time, symlink the script to your ~/bin directory, assuming you have an existing bin directory that is in your path:

#Drupal 8
ln -s core/scripts/run-tests.sh ~/bin
# Drupal < 8
ln -s scripts/run-tests.sh ~/bin

Running the script with no arguments will bring-up the help page, with a list of the available options.

Here is the run-down, shamelessly ripped from run-scripts.sh, note Drupal 8 users need to add core/ to front of scripts:

www-data@dev:/var/www-drupal-8-head$ php scripts/run-tests.sh

Run Drupal tests from the shell.

Usage:        run-tests.sh [OPTIONS] <tests>
Example:      run-tests.sh Profile

All arguments are long options.

  --help      Print this page.

  --list      Display all available test groups.

  --clean     Cleans up database tables or directories from previous, failed,
              tests and then exits (no tests are run).

  --url       Immediately preceeds a URL to set the host and path. You will
              need this parameter if Drupal is in a subdirectory on your
              localhost and you have not set $base_url in settings.php.

  --php       The absolute path to the PHP executable. Usually not needed.

  --concurrency [num]

              Run tests in parallel, up to [num] tests at a time. This requires
              the Process Control Extension (PCNTL) to be compiled in PHP, not
              supported under Windows.

  --all       Run all available tests.

  --class     Run tests identified by specific class names, instead of group names.

  --file      Run tests identified by specific file names, instead of group names.
              Specify the path and the extension (i.e. 'modules/user/user.test').

  --xml <path>

              If provided, test results will be written as xml files to this path

  --color     Output the results with color highlighting.

  --verbose   Output detailed assertion messages in addition to summary.

  <test1>[,<test2>[,<test3> ...]]

              One or more tests to be run. By default, these are interpreted
              as the names of test groups as shown at
              ?q=admin/config/development/testing.
              These group names typically correspond to module names like "User"
              or "Profile" or "System", but there is also a group "XML-RPC".
              If --class is specified then these are interpreted as the names of
              specific test classes whose test methods will be run. Tests must
              be separated by commas. Ignored if --all is specified.

To run this script you will normally invoke it from the root directory of your
Drupal installation as the webserver user (differs per configuration), or root:

sudo -u [wwwrun|www-data|etc] php ./scripts/run-tests.sh  --url http://example.com/ --all
sudo -u [wwwrun|www-data|etc] php ./scripts/run-tests.sh  --url http://example.com/ --class UploadTestCase

Command-line test execution with Drush - Linux

Drush comes with a few commands to aid in testing:

drush test-run mail : Runs the test-suite and mails the results to site email.
drush test-clean : Cleans out test suite tables and left-over files.
drush test drush : Test drush ( only seems to work for drush)

Please, read the code. This page just scratches the surface as the test-running script mentioned here can be extended and re-used at will.

Friendly advice:

Do make sure you run tests as web-server user.
If you are running tests on a CLI from remote shell access, the screen(1) utility can be handy for long running test sessions.
Xdebug extension may cause conflict with some tests.
Either disable Xdebug, or add xdebug.max_nesting_level=500 in the [xdebug] section of your php.ini

Command-line test execution - Windows

Windows users do not have a ready-made .bat file that would make things click and go. I assume it would be easy to either make the .bat file or install cygwin and try and carry out the linux instructions. Windows developers are welcomed to update this section to best suit command-line testers using Windows.

Comments

"sh run-tests.sh" won't work

Posted by flaviovs on February 6, 2010 at 2:46pm

Notice that run-tests.sh is not a shell script, but a PHP script. So if you try to run it using $ sh ./run-tests.sh (as I did) all you'll get is a lot of errors.

You may also get errors if you run the command directly (e.g. $ ./run-tests.sh) because your current shell may try to feed the script to your shell interpreter. If this is the case with your shell (it is the case of the bash shell), then the symlink to your $HOME/bin directory won't work. Don't be trapped by the fact that the script has the executable bit set. The only sure way of running the script is by explicitly feeding it to your PHP interpreter e.g. $ php ./run-tests.sh. If you don't want to type the full path of the script everytime, your better choice is to create a new script on your $HOME/bin directory, which should look like this:

#/bin/sh
exec php /path/to/your/drupal/sites/all/modules/simpletest/run-tests.sh "$@"

Also, you'd better not to rely on the name of this script not changing on the near future (e.g. in your own automated scripts), as there's already a proposal to rename it (see #655178: Rename run-tests.sh to run-tests.php or the like), which makes a lot of sense BTW.

run-tests.sh with MAMP

Posted by jlstrecker on April 11, 2012 at 6:50pm

cp simpletest/run-tests.sh YOUR_SITE_ROOT/scripts/
cd YOUR_SITE_ROOT
php scripts/run-tests.sh --php /Applications/MAMP/bin/php/php5.3.6/bin/php --list

Alias

Posted by alexweber on April 21, 2012 at 3:51pm

In order to make this task easier on a Mac you can create an alias as follows:

In your ~/.profile:

alias simpletest='php scripts/run-tests.sh --php /Applications/MAMP/bin/php/php5.3.6/bin/php'

If you are getting "command not found" errors make sure you reload your .profile:

source ~/.profile

It's important that you omit the "--list" option from jlstrecker's example so that you can specify your own options. From any Drupal site root:

# List all tests
simpletest --list

# Run a specific test by specifying class name
simpletest CacheClearCase

d8 syntax

Posted by boran on August 28, 2012 at 9:21pm

Not that the D8 syntax has changed some what, you example would now be:
"simpletest Cache" would run all cache tests.

One test:
simpletest --file core/modules/node/lib/Drupal/node/Tests/NodeTypeInitialLanguageTest.php

Also the D8 path is core/scripts/run-tests.sh