before we can address #325481: Mark issues as CNW when most recent patch has failed testing we have to be absolutely sure that the test results coming back from testing.drupal.org are accurate. this will require some human verification of PIFR's automated test results.

i think a decent approach is for me to post a publicly editable page on groups.drupal.org that has links to files that have been tested by the testbed, and people can take a look at the patch and compare it to the automated result, then leave some feedback regarding it's accuracy.

i'll work this page up tomorrow, after we have built up some more patches, and provide a link here.

Comments

Amazon’s picture

Amazon commented

We will need CVS checkout versions to compare the results of patch testing.

boombatower’s picture

boombatower commented
Status: Active » Postponed (maintainer needs more info)

Can you add the ability to display links back to the testbed? very simple format.

For example:
http://drupal.org/node/200185#comment-717546

One way that is supported:
http://testing.drupal.org/pifr/file/drupal.org/rename_truncate_utf8.patch

or
http://testing.drupal.org/pifr/file/1/rename_truncate_utf8.patch

or by node

http://testing.drupal.org/pifr/node/drupal.org/200185
http://testing.drupal.org/pifr/node/1/200185

catch’s picture

catch
he/him
Primary language English
commented

I'll help with this. Linking back to results in the interim would be very handy for tracking, and it'll get people used to the idea that it's running again too (without the need to reset incorrect status changes).

hunmonk’s picture

hunmonk commented
Assigned: hunmonk » Unassigned
Status: Postponed (maintainer needs more info) » Active

@boombatower: you can place the link information right in the result data that you send back to pift. i think this would be best, as pifr will always know what link needs to be visited to view the results.

hunmonk’s picture

hunmonk commented

If we want to get a human review going, here's a wiki page: http://groups.drupal.org/node/16209

That's a list of links to issues/comments that have test results returned and displaying on drupal.org, and my suggested instructions are at the top of the post

hunmonk’s picture

hunmonk commented
Status: Active » Fixed

catch did a marvelous job of manually verifying about 50 of the automatically tested patches listed over at http://groups.drupal.org/node/16209 and found no false positives or negatives!

i'm satisfied enough to say this step is complete

Anonymous’s picture

Anonymous (not verified) commented
Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.