Running DrupalCI Locally

Running a DrupalCI Testbot can be an effective way to rapidly iterate and debug a test that is failing on the testbots, but not in your local environment.  It will allow you to have insight into exactly what is happening during the build and execution of your tests so you can compare it to your local to see what might be causing the discrepancies.

The DrupalCI testrunner has a large list of dependencies and assumptions required to run it, and as such is only supported in a virtualized environment that already has all of those dependencies and assumptions accounted for.

How to use this guide:

1. First, ensure that you have all of the pre-requisite requirements for running a local testbot
2. Once your environment is ready, proceed to Initializing the Test Environment
3. Next you'll be able to Run tests locally
4. And once you've run tests Learn how to access results locally
5. Check out the Debugging Tips to build a minimal set of containers, force build all locally, or clean up containers.

Requirements

• Vagrant: - Create and configure lightweight, reproducible, and portable development environments.
• VirtualBox:
  Provides virtual machines so you can run other operating systems (Linux) in your host operating system (OS X). Note that it's recommended to use VirtualBox 5.2 at present as 6.0 seems to cause some odd and tricky to debug problems.
• VBGuest Plugin: Ensure that you have the VBGuest plugin installed. 
  

  vagrant plugin install vagrant-vbguest

• At least 4GB of free memory, and between 10-20 GB of disk space

Initialization - (OS X/Unix/Windows)

These instructions were tested on OS X.
Note: These instructions will probably work on other Windows, OS X, Linux, and unix-like systems. If you notice any differences please feel free to add them here.

Start a command line and run these commands:

1. This will copy the production testbot code to your local machine.
   

   $ git clone --branch production http://git.drupal.org/project/drupalci_testbot.git
   $ cd drupalci_testbot

   Note: Make sure when you checkout the repository that the line endings are kept intact. Otherwise you will have problems executing scripts. For doing development work, you should check out the dev branch

2. The following vagrant command will create your vagrant box for the first time. It can take up to 30 minutes or so (depending on your connection speed), and will download a vagrant box file of the testbot.
   

   $ vagrant up 
   Bringing machine 'default' up with 'virtualbox' provider...
   ==> default: Box 'drupalci/testbot' could not be found. Attempting to find and install...
       default: Box Provider: virtualbox
       default: Box Version: >= 0
   ==> default: Loading metadata for box 'https://s3-us-west-2.amazonaws.com/drupalci-vagrant/vagrant/json/drupalci/testbot.json'
       default: URL: https://s3-us-west-2.amazonaws.com/drupalci-vagrant/vagrant/json/drupalci/testbot.json
   ==> default: Adding box 'drupalci/testbot' (v0.13.0) for provider: virtualbox
       default: Downloading: https://s3-us-west-2.amazonaws.com/drupalci-vagrant/vagrant/boxes/drupalci/testbot/0.13.0/drupalci_testbot_jessie-vboxiso.box
   

3. When it's done you'll see:
   

   [...]
   ==> testbot: Box started, run vagrant halt to stop.
   ==> testbot:
   ==> testbot: To access the box and run tests, run:
   ==> testbot: - vagrant ssh 

4. Enter the following:
   

   $ vagrant ssh 

   You should now see a prompt:

   testbot@drupalci:~$ 

   Note: If you do not see this, you may need to add git's bin folder containing ssh to your systems PATH variable. 

5. Change to the testbot directory and initalize with Composer:

   $ cd testrunner
   $ composer install
   

You now have a running local testbot. If you wish to shut it down, but preserve it (freeing up memory, but not disk) you can use

$ vagrant halt

If you want to start over, and free up disk and memory, you can 

$ vagrant halt
$ vagrant destroy

The vagrant VM is configured to use 4GB memory, which is appropriate for a computer with 16GB of memory, but you may wish to have more memory available, or enable more cpu's if your local machine supports it.. You can do this by editing the following lines in the Vagrantfile:

config.vm.provider "virtualbox" do |v|
    v.memory = 8192
    v.cpus = 4
  end

Then you’d need to restart the vagrant box with ‘vagrant halt’ and ‘vagrant up’ again. 

$ vagrant halt
$ vagrant up

Important folder structures on the vagrant box:

When you ssh into the vagrant machine, you will start in the testbot's home directory: /home/testbot

In order to interact with the testbot code, you'll need to change directory into testrunner/.

$ cd ~/testrunner

Workflows

There are two main local development workflows: Patch-based and branch-based.

What they have in common is that you can use the build file from an existing test run and repeat the same steps.

Using an existing build definition

You can click through a testbot report from drupal.org, and eventually click 'View results on dispatcher'.

From there you'll see a report, including a cardboard box icon labeled 'Build Artifacts.'

The build artifacts are all the files generated during the testbot build. We are most interested in one with a name like build.jenkins-drupal_patches-56356.yml. The number will very likely be different, but it will start with build. and end with .yml. You might need to click through a second artifacts link to get to the file.

Now you can copy the URL to this file, and use wget or another tool to get the YAML file into your local testbot directory.

For instance, here's how it looks when we get that file:

$ cd ~/testrunner
$ wget https://dispatcher.drupalci.org/job/drupal_patches/56693/artifact/jenkins-drupal_patches-56693/artifacts/build.jenkins-drupal_patches-56693.yml

Now we can run a build based on that file:

$ /home/testbot/testrunner/drupalci run build.jenkins-drupal_patches-56693.yml

We'll then see the local testbot perform the exact same steps as the one on the remote server. This can take a lot of time, so if you started it, you can hit control-c to stop it.

Patch-based Drupal development with local DrupalCI testbot

Now that you know how to reproduce a build locally, you can modify the build file to add your own patch.

If we open the YAML build file we downloaded in the last step, we'll see a section like this:

      fetch:
        files:
          -
            from: 'https://www.drupal.org/files/issues/2018-04-28/2905007_23.patch'
            to: .
      patch:
        patches:
          -
            from: 2905007_23.patch
            to: .

This tells the fetch task to get the patch over HTTP from drupal.org and place it in the project directory. Then it tells the patch task to apply the patch to the same directory.

So if we have patches accessible over HTTP, it's straightforward what to do: Substitute your own location in fetch's from: parameter, and then change the file name in the patch task. An easy way to do this might be to place your patch file in a github gist, or other relatively easy file location.

However, maybe you don't want to put your patch on an HTTP server. In that case, you can just place a patch file wherever you want within the drupalci_testbot project directory. That's because there's a special value we can add to the patch path: TESTRUNNER_SRC.

Let's imagine a patch file in the root directory of drupalci_testbot. We'd modify our build like this:

#      We don't need a fetch step.
#      fetch:
      patch:
        patches:
          -
            from: TESTRUNNER_SRC/our_work.patch
            to: .

Note that if your patch is for a contrib project as opposed to core, your patches entry will need to instruct the test runner to put the patch into the project's repo directory rather than core's. For example:

      patch:
        patches:
          -
            from: TESTRUNNER_SRC/my_patch.patch
            to: PROJECTVCSREPO_DIR

So now we can use a workflow:

• Work locally.
• Generate a patch in the same directory as the testbot.
• $ /home/testbot/testrunner/drupalci run our_build.yml
• See the testbot results eventually.
• Modify locally, repeat as needed.

Feature branch based development with local DrupalCI testbot

In addition to the patch-based workflow we can also use the git feature branch workflow.

In this workflow we do the following:

• Modify Vagrantfile to map our local Drupal repository to a virtual machine directory.
• Restart the vagrant box.
• Set up our build file to use the replicate task to copy our desired feature branch into the testbot for testing.

Modify Vagrantfile

First we need to modify the Vagrantfile to map our local directory to the vagrant box.

drupalci_testbot/Vagrantfile contains a section that looks like this:

config.vm.synced_folder ".", "/home/testbot/testrunner"

This tells vagrant to map the local directory (the dot) to home/testbot/testrunner/ within the vagrant box.

We can add another line, like this:

config.vm.synced_folder "/Users/you/projects/drupal_core/drupal_8", "/home/testbot/drupalcheckout"

This tells vagrant to map your Drupal 8 repo to the drupalcheckout/ directory.

We now have to restart the vagrant box, so it uses this new configuration:

$ vagrant halt
$ vagrant up

We can then check if it worked:

$ vagrant ssh
$ ls
drupalcheckout  testrunner

So now we have our repo represented in the virtual box, but we still need to tell the testbot what to do with it.

We'll change our YAML build file, like this:

      replicate:
        exclude: {  }
        local_dir: '/home/testbot/drupalcheckout'
        git_branch: 'featurebranch'
        git_commit_hash: ''
#      checkout_core:
#        repositories:
#          -
#            repo: 'git://git.drupal.org/project/drupal.git'
#            branch: 8.6.x

You'll see that we can comment out or remove the checkout_core task.

We also need to add a replicate task. We tell it to use the drupalcheckout/ directory as a local repository to copy into the testbot workspace. We also tell it which branch to use. This can be any branch you wish to use... Here we've just named it 'featurebranch.'

Now we have all our pieces in place, we can run the test, and it will use the feature branch of our local repo.

In order for this to work, you have to commit your changes locally to your feature branch. Then run the testbot. It will copy the feature branch and perform the build.

Testing with a local copy of a contrib project

Similar to using a local copy of core, you can modify the build file to checkout a contrib project from a local copy. For example:

  codebase:
    assemble_codebase:
      checkout_core_d7:
        repositories:
          -
            repo: 'https://git.drupalcode.org/project/drupal.git'
            branch: 7.x
      checkout:
        repositories:
          -
            repo: '/home/testbot/seckit7checkout'
            branch: 'fix_d7_tests'
            # commit-hash: ''
            checkout-dir: 'sites/all/modules/seckit'
      fetch:
        files: {  }
      patch:
        patches: {  }
      update_build: {  }
      update_dependencies_d7: {  }
    project-name: seckit
    project-type: module
    project-subdir: ''

See the code for the different build steps available.

Troubleshooting

If you get any mount errors when starting the VM like the message below:

/sbin/mount.vboxsf: mounting failed with the error: No such device

Solution: Install the VBGuest Plugin