It would be nice if the simpletest module would provide hooks that allows contrib modules to be informed about the progress of a test run.

I'm thinking about three hooks:

  1. hook_testrun_started()
  2. hook_test_finished($results)
  3. hook_testrun_finished()

Using these hooks contrib modules could implement cool stuff to make testing fun: trigger lava lamps, trigger built-in laptop LEDs, etc.

CommentFileSizeAuthor
#20 simpletest-407294.patch2.55 KBAnonymous (not verified)
#17 simpletest-407294.patch2.53 KBAnonymous (not verified)
#5 simpletest-407294.patch2.53 KBAnonymous (not verified)
#3 simpletest.api_.php.txt790 bytesAnonymous (not verified)
#2 simpletest-407294.patch1.42 KBAnonymous (not verified)

Comments

damien tournoud’s picture

damien tournoud commented

;)

Why not?

Anonymous’s picture

Anonymous (not verified) commented
StatusFileSize
new simpletest-407294.patch1.42 KB

A patch with a basic/initial implementation.

Anonymous’s picture

Anonymous (not verified) commented
StatusFileSize
new simpletest.api_.php.txt790 bytes

API documentation that goes with the patch.

boombatower’s picture

boombatower commented

Nice examples :)

Please include the API doc in the patch, so that everything is testing by the bot and all in a single patch file.

Anonymous’s picture

Anonymous (not verified) commented
StatusFileSize
new simpletest-407294.patch2.53 KB
Anonymous’s picture

Anonymous (not verified) commented
Status: Active » Needs review

I'm marking this as "needs review" because I think that's what I'm supposed to do...

dries’s picture

dries commented

This looks OK to me, but I wonder if our terminology is right. Do we consistently call testruns testruns and tests tests? (A test run seems to be a test group.)

Where and how does one obtain a Lava lamp that integrates with the test framework? :-)

boombatower’s picture

boombatower commented

Well we run tests by test case (class), which may contain several testMethods(), but all under the same title. We usually consider a test case a single test. If that holds then I think the terminology is fine.

As for the testrun...I usually refer to that as a test suite, but I am usually talking about the whole suite being run (t.d.o). I think testsuite sounds nice and is more clear then testrun. Still not sure it is the answer.

Anonymous’s picture

Anonymous (not verified) commented

I used the term "testrun" because the page admin/build/testing mentions the word "run" three times. The words "suite" and "group" do not occur on that page. In fact, both these terms might be confused with "all tests of a specific module".

Similarly, the UI contains the words "test" multiple times in the descriptions of tests/test cases. The term "test case" is nowhere to be found.

PS: To keep this issue on-topic, I'll create a blog post with some ideas how to use these new hooks, fun stuff that some agile teams have done and links to online stores where you can buy the necessary gadgets.

boombatower’s picture

boombatower commented

The UI has almost no text whatsoever and uses the work run as a verb. "Select the tests you would like to run, and click Run tests.

I agree that "suite" makes you think of all the tests (as I stated)...so I'm not sure.

testrun just seems awkward, but at lack of something better...

Status: Needs review » Needs work

The last submitted patch failed testing.

berdir’s picture

berdir
Primary language German
Location Switzerland
commented

Nice, http://drupal.org/project/code_coverage could for example use those.

Anonymous’s picture

Anonymous (not verified) commented
Status: Needs work » Needs review

Why does the patch fail testing with "Failed: Failed to install HEAD."? The patch hasn't changed and I can still use it locally on an up-to-date HEAD.

I'm guessing (hoping?) it was a temporary glitch in the Matrix and I'm marking this issue again as "needs review" to trigger another attempt.

Anonymous’s picture

Anonymous (not verified) commented

In the meantime I have also committed a new contrib module that uses these new hooks to execute shell commands: http://drupal.org/project/testlistener

I use it together with the dellledctl utility to turn on/off the built-in LEDs of my laptop.

berdir’s picture

berdir
Primary language German
Location Switzerland
commented

The test bot had to be changed, all tests failed with that message for a short time. Try to re-upload the patch.

boombatower’s picture

boombatower commented

Due to a change in the core installer which requires a change to the bot since Drupal has no install script in core.

Anonymous’s picture

Anonymous (not verified) commented
StatusFileSize
new simpletest-407294.patch2.53 KB
dries’s picture

dries commented

If we run a group of tests in between the hooks, I recommend that we rename the hook to 'test_group_start' and 'test_group_end'.

dries’s picture

dries commented
Status: Needs review » Needs work
Anonymous’s picture

Anonymous (not verified) commented
Status: Needs work » Needs review
StatusFileSize
new simpletest-407294.patch2.55 KB

I've recreated the patch with "test group".

Anonymous’s picture

Anonymous (not verified) commented

There seems to be some dust settling on this issue.
Is there anything I can do to move this patch forward?

dries’s picture

dries commented
Status: Needs review » Fixed

Committed to CVS HEAD. Feel free to ship me a Java-lamp. ;-)

yched’s picture

yched commented

[J|L]ava lamp ? ;-)

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.