0 tests passing is not green

Posted by webchick on May 13, 2011 at 7:09pm

4 followers

Issue Summary

This is happening on the D8 branch right now:

Nope.

It's misleading if you're not paying attention because it *looks* like everything is hunky-dory and it's not.
http://drupal.org/node/999004#comment-4457962 was the culprit this time.

Comments

#1

Posted by boombatower on May 13, 2011 at 7:51pm

Something we will need to think about since projects can use bot to check for syntax and non-applicable patches without the need for tests.

#2

Posted by catch on May 16, 2011 at 9:11am

webchick beat me to it, subscribing.

#3

Posted by webchick on June 12, 2011 at 7:40am

Could we maybe count the number of tests we had going in, and if the number of passes + fails + exceptions < total, throw some sort of visual indication of that?

#4

Posted by boombatower on June 13, 2011 at 2:32am

Seems reasonable, since one shouldn't have a testcase with nothing in it...or makes little sense.

#5

Posted by rfay on June 17, 2011 at 2:58am

Just saw this happen in testing... and it was the case where the testbot times out or is otherwise reset. I think we might handle that case wrong.

#6

Posted by rfay on June 18, 2011 at 12:11am

In studying this, I saw one where we were getting the "mysql server has gone away" and that was the cause.

I increased the max_allowed_puppet in my.cnf in puppet to 32MB from 16MB. We'll see if it has an impact.

#7

Posted by rfay on June 20, 2011 at 4:12pm

Please post a link to the test any time this happens.

It looks to me like the primary problem is in simpletest, PIFR or the testbot configuration. It seems that when there are serious exceptions or database problems, the testbot bails and this is the result. This was happening in #1192680: Testbot fills up /tmpfs partition and was resolved when the exceptions were resolved.

#8

Posted by rfay on June 26, 2011 at 3:32pm

There are actually a fair number of cases where testing is enabled (on a per-project basis) but any given branch may not have tests. I wonder what the right response is in this case? We've now seen

1. Inappropriate "0 tests pass" green, where the testbot failed in one way or another to do the right thing.
2. Branches that have no tests.

Any other cases?