Download & Extend
Project Issue File Review » Issues

Update test summary messages to take environments into account and provide better detail

Posted by boombatower on February 24, 2009 at 2:03am
Project:Project Issue File Review
Version:6.x-2.x-dev
Component:Miscellaneous
Category:task
Priority:normal
Assigned:boombatower
Status:closed (fixed)

Issue Summary

--

Login or register to post comments

Comments

#1

Posted by boombatower on February 24, 2009 at 2:05am

Some possible messages:

Test passed on 2 of 5 environments.

Test fail on MySQL 5.0 INNO (9,000 pass, 4 fail, 1 exception).

Test passed on all environments (10,000 pass).

#2

Posted by boombatower on February 24, 2009 at 2:08am

The result codes we have to deal with are:

<?php
define('PIFR_SERVER_TEST_RESULT_SETUP', 1);
define('PIFR_SERVER_TEST_RESULT_FETCH', 2);
define('PIFR_SERVER_TEST_RESULT_CVS', 3);
define('PIFR_SERVER_TEST_RESULT_PATCH', 4);
define('PIFR_SERVER_TEST_RESULT_APPLY', 5);
define('PIFR_SERVER_TEST_RESULT_SYNTAX', 6);
define('PIFR_SERVER_TEST_RESULT_FORMAT', 7);
define('PIFR_SERVER_TEST_RESULT_INSTALL', 8);
define('PIFR_SERVER_TEST_RESULT_FAIL', 9);
define('PIFR_SERVER_TEST_RESULT_PASS', 10);
?>

#3

Posted by boombatower on February 24, 2009 at 2:26am
Status:active» postponed (maintainer needs more info)

My current thoughts for the messages. When viewing the testing.drupal.org page for the test results a link will display next to the message that when clicked will provide more details and possible solutions/courses of action.

The first code deals server setup and should never report to d.o, only during initial server configuration. The next two are issues with d.o infrastructure.

  • Setup environment: @reason. (failed to clear checkout directory, failed to create checkout directory, failed to create review directory, failed to find symbolic link needed to invoke checkout, or failed to clear checkout database)
  • Fetch test file: @reason. (failed to retrieve file from project client or failed to save file data)
  • Repository checkout: @reason. (failed to checkout Drupal core, failed to create modules directory, or failed to checkout module)
  • Invalid patch format.
  • Unable to apply patch.
  • Invalid PHP syntax in @filename.
  • format not implemented
  • Failed to install Drupal.
  • Run tests: @reason. (failed to set SimpleTest variable in checkout database, failed to enable SimpleTest, or failed to run tests)
  • Failed on @environment, with: @pass pass, @fail fail, and @exception exception. (MySQL 5.0 ISAM, PostgreSQL 8.3, SQLite, MySQL 5.1 ISAM, or MySQL 5.0 INNO)
  • Passed on all environments, with @pass pass.

#4

Posted by boombatower on February 24, 2009 at 2:27am
Title:Update test summary messages to take environments into account» Update test summary messages to take environments into account and provide better detail

#5

Posted by boombatower on February 25, 2009 at 2:54am
Status:postponed (maintainer needs more info)» fixed

Implemented.

<?php
switch ($code) {
    case PIFR_SERVER_TEST_RESULT_SETUP:
      return 'Setup environment: @reason.';
    case PIFR_SERVER_TEST_RESULT_FETCH:
      return 'Fetch test file: @reason.';
    case PIFR_SERVER_TEST_RESULT_CVS:
      return 'Repository checkout: @reason.';
    case PIFR_SERVER_TEST_RESULT_PATCH:
      return 'Invalid patch format.';
    case PIFR_SERVER_TEST_RESULT_APPLY:
      return 'Unable to apply patch.';
    case PIFR_SERVER_TEST_RESULT_SYNTAX:
      return 'Invalid PHP syntax in @filename.';
    case PIFR_SERVER_TEST_RESULT_FORMAT:
      return '-- not implemented --';
    case PIFR_SERVER_TEST_RESULT_INSTALL:
      return 'Failed to install Drupal on @environment.';
    case PIFR_SERVER_TEST_RESULT_TEST:
      return 'Run tests on @environment: @reason.';
    case PIFR_SERVER_TEST_RESULT_FAIL:
      return 'Failed on @environment, with: @pass pass, @fail fail, and @exception exception';
    case PIFR_SERVER_TEST_RESULT_PASS:
      return 'Passed on all environments.';
    default:
      return 'Unknown';
  }
?>

#6

Posted by System Message on March 11, 2009 at 3:00am
Status:fixed» closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

#7

Posted by JohnAlbin on May 16, 2009 at 10:58pm
Status:closed (fixed)» active

I hate to quibble about help text, but "Failed to install Drupal on @environment." is not much better than "Failed to install HEAD".

I still think "Drupal installer failed to run after patch applied." gives a better sense that the problem lies with the patch. It's too easy to think the above error messages mean there was a testbot error (not related to the patch) that caused the testbot to fail. Another example of developer misunderstanding "Failed to install HEAD": #415710: Favicon.ico defaults to 'type="image/x-icon"'

testbot is from venus, developers are from mars. ;-) Once we understand each other, the love will flow! /me already loves the testbot, btw. Thanks for all your work, Jimmy!

#8

Posted by boombatower on May 20, 2009 at 4:50am
Status:active» closed (fixed)

Since the testbot in 2.x is not specific to patches, it runs tests on clean installs as well, putting the word patch in the message seems a bit too specific. I would vote for adding more detail like reasons for the issue and how to correct (as I planned to add once this goes stable) in #400438: Add help links on test view page.

So lets focus discussion over there.

nobody click here