I have been thinking about this for some time and believe I have finally come up with a simple (in concept) solution. Instead of stopping at the first failing environment, the system should stop within "groups" defined by the plugin associated with the environment.

Meaning if I have 3 databases being tested with the SimpleTest plugin, coder review, and performance testing on two machines configurations the following would occur.

All 3 databases would only run if each passes, meaning once one fails those left will not. Coder review will always run since it is only in one environment. Performance will run on at least one machine configuration.

CommentFileSizeAuthor
#5 663456-sudo-grouping.patch19.85 KBboombatower
#2 663456-sudo-grouping.patch2.34 KBboombatower

Comments

boombatower’s picture

boombatower commented
Assigned: Unassigned » boombatower

pifr_server_test_result() - logic needs to be changed to follow.

messages/e-mails need to be updated to allow for multiple environments being the source of the error.

boombatower’s picture

boombatower commented
Status: Active » Needs review
StatusFileSize
new 663456-sudo-grouping.patch2.34 KB
boombatower’s picture

boombatower commented

This will obviously require the summary messages and such to change along with this. I am thinking to solve a number issues that would be noticed down the road with more environments and #632212: Report # of passes even on success, make the summary message plugin centric.

For example:

FAILED: [[SimpleTest]]: [MySQL] 4000 pass(es), [Postgres] pass(es), 4 fail(s), 0 exception(s); [[Coder]]: 17 minor(s); [[Performance]]: +14 seconds.
boombatower’s picture

boombatower commented

Locations that reference pifr_server_result_summary_message() which will need to be updated:

./server/pifr_server.result.inc:107:function pifr_server_result_summary_message(array $test) {
./server/pifr_server.event.inc:79:      '%result_summary' => pifr_server_result_summary_message($result_summary),
./server/pifr_server.xmlrpc.inc:414:        'message' => pifr_server_result_summary_message($result),
./server/pifr_server.test.inc:537:    '#value' => '<span id="pifr-result-summary-message">' . pifr_server_result_summary_message($summary_result) . '</span>',
boombatower’s picture

boombatower commented
StatusFileSize
new 663456-sudo-grouping.patch19.85 KB
boombatower’s picture

boombatower commented
Status: Needs review » Fixed

Committed.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.