Problem

  • Maintainers and developers are confused about varying positive test assertion counts for patches passing tests.

Goal

  • Ensure that assertion counts can be trusted.

Details

  • Running the same test suite on the same machine can yield a difference of hundreds assertions.
  • The difference is caused by various tests that use rand(1, 10) for their initial setup (which means that test assertions are exponentially increasing for each additional item).

Solution

  • Remove all calls to rand() from all tests.
Files: 
CommentFileSizeAuthor
#11 d7-1560028.diff2.03 KBthehong
PASSED: [[SimpleTest]]: [MySQL] 39,240 pass(es).
[ View ]
#9 drupal8.tests-rand.9.patch1.91 KBsun
PASSED: [[SimpleTest]]: [MySQL] 37,043 pass(es).
[ View ]
#4 1560028.patch1.82 KBbleen18
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1560028_0.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]
#1 1560028.patch1.82 KBbleen18
FAILED: [[SimpleTest]]: [MySQL] 36,670 pass(es), 9 fail(s), and 9 exception(s).
[ View ]

Comments

Status:Active» Needs review
StatusFileSize
new1.82 KB
FAILED: [[SimpleTest]]: [MySQL] 36,670 pass(es), 9 fail(s), and 9 exception(s).
[ View ]

Incidentally, we should not remove *all* calls to rand() (or mt_rand()) from simpletests as most of them are used to set a random weight, languagecode, chr, etc... We only need to remove those calls that case an inconsistent number of assertions.

In fact, I only found three places where rand() is used in such a way that there are random assertions.

Component:other» simpletest.module
Status:Needs review» Reviewed & tested by the community

Awesome, thanks!

Status:Reviewed & tested by the community» Needs work

The last submitted patch, 1560028.patch, failed testing.

StatusFileSize
new1.82 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1560028_0.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]

this should fix that errr

Status:Needs work» Needs review

here testbot .. here boy

Status:Needs review» Reviewed & tested by the community

Issue tags:-Testing system

#4: 1560028.patch queued for re-testing.

Status:Reviewed & tested by the community» Needs work
Issue tags:+Testing system

The last submitted patch, 1560028.patch, failed testing.

Status:Needs work» Reviewed & tested by the community
StatusFileSize
new1.91 KB
PASSED: [[SimpleTest]]: [MySQL] 37,043 pass(es).
[ View ]

Version:8.x-dev» 7.x-dev
Status:Reviewed & tested by the community» Patch (to be ported)

Committed to 8.x. Thanks!

Status:Patch (to be ported)» Needs review
StatusFileSize
new2.03 KB
PASSED: [[SimpleTest]]: [MySQL] 39,240 pass(es).
[ View ]

Status:Needs review» Reviewed & tested by the community

Thanks @thehong! :)

Status:Reviewed & tested by the community» Fixed

Automatically closed -- issue fixed for 2 weeks with no activity.