I'm building a Drupal 6 module to automate most of the testing plan. It might run both local or remotelly. It includes some configuration options to avoid messing the live git-dev site. The tests are divided currently by my own rude test matrix, untill I finish and publish a full matrix with imports from eliza411 and webchick (matrix will be updated in #1028272: Define matrix for Git client testing.

Comments

webchick’s picture

webchick
she/they
Primary language English
Location Vancouver 🇨🇦
commented

I've created two test users on both drupal.org and git-dev.drupal.org for this purpose: "git-peon" (with absolutely no permissions) and "git-user" with the "Git user" role. ilo has the passwords. These should stay present and working even through db syncs now.

We decided not to do git-admin for now, because that's an edge case that we can get working after we open for testing.

webchick’s picture

webchick
she/they
Primary language English
Location Vancouver 🇨🇦
commented

Also, in terms of the critical path, I think this is what we need to test:

NEW USER, PASSWORD AUTH
(browser)
- Create new user account
- Go to /node/add and verify that "Project" and "Project release" are not avaialble.
- Go to user profile, verify that "Projects" tab does not appear.
- Edit user account, select the "I agree" checkbox.
- Verify that "Proejcts" tab now appears.
- Go to /node/add and verify that "Project" (and *not* "Project release") are available.
- Click the "Project" link to create a new project.
- Note that "sandbox" should be force-selected and the "Project shortname" field should NOT appear.
- Create a sandbox project.

(shell)
- git clone the project
- verify you are prompted for your password. enter it.
- add a file
- commit it
- push it

(browser)
- Verify that push shows up at /commitlog
- Verify that a "Maintainers" block shows up on your project page

VETTED USER, SSH AUTH
(browser)
- Log in as vetted user.
- Go to /node/add. Should not be able to see either "Project" or "Project release".
- Go to user profile, Projects tab. Should not see link to create a new project.
- Edit user profile, check "I agree" checkbox
- Go back to Projects tab on user profile. *Should* see a link to create a new project.
- Click SSH keys tab, add a SSH key.
- Go to /node/add. Should see both "Project" and "Project release".
- Click in to a new project. You should have the option of either full project or sandbox project.
- Create a full project (fill out shortname)

(shell)
- git clone the project
- verify you are NOT prompted for a password
- add a file
- commit it
- push it

(browser)
- Verify that push shows up at /commitlog
- Verify that a "Maintainers" block shows up on your project page

EMAIL ADDRESS FUN
(shell)
- Set your .gitconfig git.email field to something different than your d.o email.
- Commit/push a change to your project.
(browser)
- Go to /commitlog and verify that it's attributed to whatever your git.name is (unverified)

(shell)
- Set your .gitconfig git.email field to your primary d.o email.
- Commit/push a change to your project.
(browser)
- Go to /commitlog and verify that it's attributed to you.

(browser)
- Go to drupal.org, user profile, edit, multiple emails and add a new email address and verify it.
(shell)
- Set your .gitconfig git.email field to this new email
- Commit/push a change to your project
- Go to /commitlog and verify that it's attributed to you.

If that was working, that'd be enough to get us going. Then of course there's all the various sub-branches of that (vetted user with password access and a sandbox, new user with ssh access, etc.)

ilo’s picture

ilo commented

Currently site is down for maintenance, so I've started to build the testcase (is in somehow very bartik dependant) and working on the git client class. I can't go further with web tests as long as I need to verify the site content, but I'm not pushing the team to get this done, I do have a lot of things to do in the mean time the site is back.

The remote location can be set in the simpletest settings page. Leaving empty will test locally, when a remote url is set requests are sent to that url. Forget using database for remote settings, but the rest of simpletest will almost work. HTTP auth settings are still honored by the script, but the git-dev testcase class will setup remote location and authentication settings on the testcase setUp() function.

Module includes two testcases:
- remote testcase to handle a remote location url (and doing $base_url and user agent replacements for remote tests).
- git-dev testcase (extending remote testcase with git-dev.drupal.org default settings).

This is an example of a remote testcase for drupal.org:

// $Id$

/**
 * @file
 * Sample Test case for drupal.org.
 *
 * Just perform some checks in drupal.org
 */
class SampleDrupalOrgTestCase extends RemoteWebTestCase {

  /**
   * Implementation of getInfo().
   */
  public static function getInfo() {
    return array(
      'name' => 'Sample drupal.org testcase',
      'description' => 'Just do various checks in drupal.org',
      'group' => 'Git Dev',
    );
  }

  /**
   * Can't be more simple.
   */
  function setUp() {
    $parent_callback = 'parent::setUp';
    if (version_compare(PHP_VERSION, '5.3.0', '<')) {
      $parent_callback = array($this, 'parent::setUp');
    }
    call_user_func_array($parent_callback, array());

    // Ignore settings and force remote url to be drupal.org
    $this->simpletest_http_location = 'http://drupal.org';
  }

  /**
   * Verify a user login.
   */
  function testFrontpage() {
    // Verify frontpage links
    $this->drupalGet('');
    $this->assertLink(t('Code commits'), 0, t('CVS link found.'));
    $this->assertLink(t('Issue comments'), 0, t('Issues link found.'));

    // try to login with a valid user (change this).
    $account = new StdClass();
    $account->name     = 'PUT A VALID USER NAME';
    $account->pass_raw = 'PUT A VALID PASSWORD';
    // Bartik theme fails with drupalLogin because of 'Log out' link. Need a
    // custom login in remotewebtestcase.test.
    $this->drupalBartikLogin($account);

    // Verify links at front for auth users.
    $this->drupalGet('');
    $this->assertLink(t('Your Dashboard'), 0, t('Dashboard link found.'));
    $this->assertLinkByHref('/user/' . $this->loggedInUser->uid . '/dashboard', 0, t('Dashboard link found.'));
  }

}

From here almost anyone used to see simpletest code can write additional tests, and I expect this help, for sure. I'll be pushing to github later with a more detailled README, once I get the git client class working.

ilo’s picture

ilo commented

Getting the work done on: https://github.com/ilo/sitetesting if anyone wants to review or test.

eliza411’s picture

eliza411 commented
Title: Automated tests module » [Meta] Automated tests module

I'm changing the title to indicate that this is a meta issue. While ilo works on the matrix above, we'll be adding high priority test issues, such as #1031852: Create tests for existence of a repository after project creation

eliza411’s picture

eliza411 commented
Component: Code » Tests

Also changing component.

eliza411’s picture

eliza411 commented

Also see #1028272: Define matrix for Git client testing for the spreadsheet matrix.

eliza411’s picture

eliza411 commented
Status: Active » Postponed
Issue tags: +git phase 2 leftovers

postponing

eliza411’s picture

eliza411 commented
Assigned: ilo » Unassigned

Unassigning ... working tests is something the project can really use, so assign this to yourself if you can work on it.