Problem

  • Maintainers and developers are confused about varying positive test assertion counts for patches passing tests.

Goal

  • Ensure that assertion counts can be trusted.

Details

  • Running the same test suite on the same machine can yield a difference of hundreds assertions.
  • The difference is caused by various tests that use rand(1, 10) for their initial setup (which means that test assertions are exponentially increasing for each additional item).

Solution

  • Remove all calls to rand() from all tests.
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

bleen’s picture

Status: Active » Needs review
FileSize
1.82 KB

Incidentally, we should not remove *all* calls to rand() (or mt_rand()) from simpletests as most of them are used to set a random weight, languagecode, chr, etc... We only need to remove those calls that case an inconsistent number of assertions.

In fact, I only found three places where rand() is used in such a way that there are random assertions.

sun’s picture

Component: other » simpletest.module
Status: Needs review » Reviewed & tested by the community

Awesome, thanks!

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 1560028.patch, failed testing.

bleen’s picture

FileSize
1.82 KB

this should fix that errr

bleen’s picture

Status: Needs work » Needs review

here testbot .. here boy

sun’s picture

Status: Needs review » Reviewed & tested by the community
sun’s picture

Issue tags: -Testing system

#4: 1560028.patch queued for re-testing.

Status: Reviewed & tested by the community » Needs work
Issue tags: +Testing system

The last submitted patch, 1560028.patch, failed testing.

sun’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
1.91 KB
Dries’s picture

Version: 8.x-dev » 7.x-dev
Status: Reviewed & tested by the community » Patch (to be ported)

Committed to 8.x. Thanks!

thehong’s picture

Status: Patch (to be ported) » Needs review
FileSize
2.03 KB
sun’s picture

Status: Needs review » Reviewed & tested by the community

Thanks @thehong! :)

David_Rothstein’s picture

Status: Reviewed & tested by the community » Fixed

Automatically closed -- issue fixed for 2 weeks with no activity.