Download & Extend
Issues

Remove all calls to rand() from tests

Posted by sun on May 3, 2012 at 11:08pm
Project:Drupal core
Version:7.x-dev
Component:simpletest.module
Category:task
Priority:normal
Assigned:Unassigned
Status:closed (fixed)
Issue tags:Testing system

Issue Summary

Problem

  • Maintainers and developers are confused about varying positive test assertion counts for patches passing tests.

Goal

  • Ensure that assertion counts can be trusted.

Details

  • Running the same test suite on the same machine can yield a difference of hundreds assertions.
  • The difference is caused by various tests that use rand(1, 10) for their initial setup (which means that test assertions are exponentially increasing for each additional item).

Solution

  • Remove all calls to rand() from all tests.
Login or register to post comments

Comments

#1

Posted by bleen18 on June 2, 2012 at 3:42pm
Status:active» needs review

Incidentally, we should not remove *all* calls to rand() (or mt_rand()) from simpletests as most of them are used to set a random weight, languagecode, chr, etc... We only need to remove those calls that case an inconsistent number of assertions.

In fact, I only found three places where rand() is used in such a way that there are random assertions.

AttachmentSizeStatusTest resultOperations
1560028.patch1.82 KBIdleFAILED: [[SimpleTest]]: [MySQL] 36,670 pass(es), 9 fail(s), and 9 exception(s).View details

#2

Posted by sun on June 2, 2012 at 3:45pm
Component:other» simpletest.module
Status:needs review» reviewed & tested by the community

Awesome, thanks!

#3

Posted by System Message on June 2, 2012 at 4:07pm
Status:reviewed & tested by the community» needs work

The last submitted patch, 1560028.patch, failed testing.

#4

Posted by bleen18 on June 2, 2012 at 4:29pm

this should fix that errr

AttachmentSizeStatusTest resultOperations
1560028.patch1.82 KBIdleFAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1560028_0.patch. Unable to apply patch. See the log in the details link for more information.View details

#5

Posted by bleen18 on June 2, 2012 at 4:29pm
Status:needs work» needs review

here testbot .. here boy

#6

Posted by sun on July 1, 2012 at 11:29pm
Status:needs review» reviewed & tested by the community

#7

Posted by sun on July 1, 2012 at 11:29pm

#4: 1560028.patch queued for re-testing.

#8

Posted by System Message on July 1, 2012 at 11:31pm
Status:reviewed & tested by the community» needs work

The last submitted patch, 1560028.patch, failed testing.

#9

Posted by sun on July 2, 2012 at 12:04am
Status:needs work» reviewed & tested by the community
AttachmentSizeStatusTest resultOperations
drupal8.tests-rand.9.patch1.91 KBIdlePASSED: [[SimpleTest]]: [MySQL] 37,043 pass(es).View details

#10

Posted by Dries on July 4, 2012 at 2:00pm
Version:8.x-dev» 7.x-dev
Status:reviewed & tested by the community» patch (to be ported)

Committed to 8.x. Thanks!

#11

Posted by thehong on July 5, 2012 at 12:28am
Status:patch (to be ported)» needs review
AttachmentSizeStatusTest resultOperations
d7-1560028.diff2.03 KBIdlePASSED: [[SimpleTest]]: [MySQL] 39,240 pass(es).View details

#12

Posted by sun on July 5, 2012 at 12:37am
Status:needs review» reviewed & tested by the community

Thanks @thehong! :)

#13

Posted by David_Rothstein on July 9, 2012 at 3:59am
Status:reviewed & tested by the community» fixed

Committed to 7.x - thanks! http://drupalcode.org/project/drupal.git/commit/e0961c9

#14

Posted by System Message on July 23, 2012 at 4:01am
Status:fixed» closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.