A suggestion for easing maintenance and making it easier for people to donate sever resources:

Let's create a docker container with a standardized and pre-configured tesbot.

Probably this means normalizing pifr to run on Ubuntu 12.04, though possibly we could use Debian 7 if they have the necessary kernel backport and other packages available.

There is already some discussion of using Docker at https://groups.drupal.org/node/388908

Comments

ricardoamaro’s picture

I'm grabbing this.

A couple of questions:
1 - Do we want the container with the normal clean lamp stack or something more elaborated?
2 - what are the expectations for the usage and deployment?

Thank you

ricardoamaro’s picture

Assigned: Unassigned » ricardoamaro
Issue summary: View changes
pwolanin’s picture

So according to this, the standard env is Debian 7: https://drupal.org/node/1748618

the next page talks about puppet manifests, but I'm not sure it's up to date.

Probably need some input from rfay or jthorson about deploying on 12.04, or whether we need Debian 7, and also their approach for different PHP versions, etc.

jthorson’s picture

Currently, if using the drupaltestbot-puppet project, then Debian 7 is the standard testbot environment. That project also contains a fully functional vagrant file for turning up a new local bot.

However, in order to align better with the rest of the infrastructure team, I think I would probably advocate for an eventual move to CentOS instead of Ubuntu (at least, I got the impression that CentOS was the unofficial standard when I asked around at BADCamp). I recognize that something like Ubuntu might be easier at first; but in the long run, I'd like to help reduce the number of differences between the testing infrastructure and the rest of the infrastructure supporting drupal.org.

Creating a docker image would definitely help us get a head start on the initiative I hinted at in https://groups.drupal.org/node/388908 ... so I strongly encourage anyone who's interested in pursuing this; and would like to offer whatever help and support I can. I will (unfortunately) not be available much over the Christmas break, and am starting in a new position at work on January 2nd; but if I'm not getting back to you, please be both patient, and persistent. :)

jthorson’s picture

Project: Drupal.org infrastructure » Drupal.org Testbots
Component: VMs » Miscellaneous

This item was discussed on the infrastructure working group call on January 15th ... where it was agreed we would move it over to the testbot queue for now.

pwolanin’s picture

@jthorson - if the goal is to get more people contributing to testbot maintenance, I'm not sure aligning with the infra team (or OSUOSL?) choice of CentOS necessarily makes sense if it's Docker support for Ubuntu is much more robust?

jthorson’s picture

As of Docker 0.7, it no longer requires a patched kernel ... which means it is supported on all standard Linux distros:

http://blog.docker.io/2013/11/docker-0-7-docker-now-runs-on-any-linux-di...

RedHat was heavily involved in the storage driver work that made this possible, so I feel fairly confident that we should be okay with CentOS.

That said, if it doesn't work out, it doesn't work out, and we go with a different distribution. However, for the web server and jenkins components at least, it makes sense to at least try to align with the similar components already supported by the infrastructure team.

nick_schuch’s picture

Hi everyone! Over the past few days I have been working on a Docker based Drupal testbot.

The guys in #drupal-infrastrucutre pointed me to this ticket.

Wanted to post it in here to get some feedback: http://youtu.be/v3ue8bqH7YI

Ill be cleaning up the code and publishing it in the next few days (going away for Australia day holidays).

dsnopek’s picture

@nick_schuch: Looks cool! Please post a link here when you share the code somewhere. :-)

nick_schuch’s picture

Ok, here we go!

Docker container - Used for spinning up the environment.

https://github.com/nickschuch/docker-drupal-testbot

Phing based Mink install script - Used for installing Drupal and enabling the Testing module.

https://github.com/nickschuch/phing-drupal-install

Want to see it in action?

http://107.170.22.175:8080/job/drupal-test-head/38/console

Note: Test builds take 3.5 to 4hrs. Im not sure if this is a hardware limitation (if we are using beefy testbots for the real world stuff) or if there is some tweaking I can perform.

I plan to make this a puppet manifest.

nick_schuch’s picture

Here we have 3 x parrallel test runs with a smaller subset of test groups for each:

http://107.170.22.175:8080/job/drupal-test-head/132/console
http://107.170.22.175:8080/job/drupal-test-head/131/console
http://107.170.22.175:8080/job/drupal-test-head/130/console

Which brings the tests down to 2hr 38min. Im still assuming we are running on some rocking hardware to get the times we are now.

jthorson’s picture

Current hardware is 8 cores, 8G of RAM, running 8 tests concurrently. Also, mysql is running in memory on /tmpfs, which helps speed things up considerably.

I'm hoping to get around to looking at this tonight ... but thanks a TON for your work on this!

nick_schuch’s picture

No worries at all! Ping me (or leave a tell) in IRC if you want to have a chat.

jthorson’s picture

Looks like the docker spinup took about a minute to get everything built and ready for testing on your first link ... was that the initial build, or would that be after the docker layers were cached?

nick_schuch’s picture

Yep, it took roughly 1 min to spin up the container and install Drupal 8 with Mink.

Breakdown:
- Container spin up: 36sec (creation of database and starting processes)
- Install Drupal 8 with Mink and enable Simpletest module: 20sec.
- Running the tests: The rest.... :)

ricardoamaro’s picture

Assigned: ricardoamaro » Unassigned
ricardoamaro’s picture

I've unassigned myself from this since you guys have more time and have done a lot of work already.
Kudos for that! :)

I will still help on this during Drupaldevdays with @jthorson

jthorson’s picture

I did a bunch of work playing with this yesterday ... mostly focused on sorting out what you can and can't do with docker, exploring the 'layers' concept and experimenting with a few different configurations (such as pulling the PHP install and configuration into a different layer, re-using the same Docker base build and layering one of three PHP layers on top depending on the desired test environment).

I'm also experimenting with an all-in-one build (using supervisord) versus individual segmented containers for each service (mysql, apache, filestore).

What I'd like to try next is to mount an external read-only volume containing the drupal core codebase at /var/www, but have /var/www/sites come from within the container (i.e. using the AUFS filesystem). Does anyone know if this is possible? If not directly, I was thinking of trying to fake things out with a symlink to another directory outside of the /var/www path (and thus in the container), but have read of others having problems getting Docker to follow the symlinks.

As for the "why", this would give me a base container which could be used for contrib testing, without having to worry about checking out (or possibly even installing) Drupal core.

And for anyone else wanting to play with this ... my ultimate first goal would be to get a container that can run the PIFR testbot build ... if we could get Docker running PIFR testbots as containers, it would greatly simplify the complexity involved in enabling testing of multiple PHP versions on our existing testing infrastructure. (Instead of actively playing with php-fpm, or swapping out apache configs on the testbot, we could simply pass the php version as a parameter and use it to select the appropriate base docker container!)

nick_schuch’s picture

Sounds like your making a tonne of progress.

I think an all in one build is will the answer for us, since we will be burning down the environment afterwards.

Im not sure I follow the /var/www vs /var/www/sites question. Maybe a little more information? I don't follow how the current testing works so might be missing some information there.

Did you have any success with the code I provided above? It runs the full test suite.

jthorson’s picture

The directory question was related to an idea around removing the git clone and install steps from a test run, to try and squeeze some extra efficiency out of a contrib test run ... if we could mount a read-only external volume containing the Drupal core codebase, it could be re-used across all docker test runner containers. However, the sites directory would have to belong to the container itself, as the site directory contents would vary between test runners. Thus, we need the ability to have a container owned sub-directory within an external volume from the host ... as writing directly to the external volume would otherwise be persistent.

This should be possible using a bind-mount, mounting a directory from outside of the /var/www external volume (i.e. from inside the container) onto the /var/www/sites directory ... and because it's a true mount, it should reside within the AUFS file system. That said, I still need to test the theory to verify it.

nick_schuch’s picture

Ok, I understand now. You could always download and install Drupal in the Dockerfile instead.

That would mean:
- We are rebuilding the docker image often thus verifying it can be build.
- It's not mounted on a volume that all the other hosts would have to have available.

These suggestions are only for the module testing. I don't think we will be able to get away with it in core.

jthorson’s picture

Agreed.

However, another area this would be helpful is with the current PIFR testbot approach.

If we could have a single re-useable file system that contained the 'testbot' build, and mounted it read-only, that might help simplify with overall testbot maintenance; in addition to preventing leakage between the site-under-test and testbot itself. Turning up a new testbot would consist of starting a new container with it's own database ... The current testbot checks out another copy of core in the /sites/default/files directory; which could be contained entirely within the container. In this model, it applies for both core and contrib.

Why bother with the current PIFR bots, when I've been stressing our need to get away from PIFR? Because choosing between one of three containers as a means of swapping between php 5.3, 5.4, or 5.5 is much easier (and less work) than the other options we've looked at (including php-fpm and/or multiple apache vhosts).

jthorson’s picture

I was also toying with the idea of having a core commit trigger that causes a new 'data-only' container to be built, containing only the drupal checkout, and tagged with the git commit id. There are a number of redundant tasks performed by the existing testbots which we could get away with doing only once. For example, PHP linting (syntax checking) of the D8 codebase takes multiple minutes on the current testbot; whereas we could perform this once at commit time, and then lint only those files modified by a particular patch when running the test. Also, the desire to be able to support parallel processing requires that we can generate a list of available tests that we can distribute to the parallel test runners; something which only needs to be done once per commit version.

This 'data only' container model would also give us the flexibility to execute a test against any particular core commit, using the commit id tags to select which container snapshot we use as the codebase ... being able to re-test patches against any point in the codebase history is something we are not able to do today.

nick_schuch’s picture

Sorry when I said "I don't think we will be able to get away with it in core." I only meant that in regards to installing the site in the container prior to running the bot for module test runs.

We could have multiple containers for each type of tests eg. Core vs Modules. As long as we approached them in the correct manner it would not be hard eg. build a container for module testing based off the core one.

drumm’s picture

nick_schuch’s picture

@jthorson,

Is there anything I can do to help with this one?

jthorson’s picture

nick_schuch:

I'd love to be able to get a docker environment running todays 'PIFR' code ... this would ease qa.d.o integration (though isn't really the end game).

The other thing I'd love to do is figure out some way to mount the /var/www directory as an external volume containing the D8 codebase, but have the /var/www/sites directory belong to the container's AUFS file system. This way, we could re-use a data-only container containing the D8 codebase, mounted as a read-only volume, as a re-useable checkout (thus removing the git checkout step). I spoke to folks in the #docker IRC channel, and they suggested having the /var/www/sites contents exist in a different directory outside of the /var/www external volume, but bind-mount it to the /var/www/sites location.

If you want to experiment, that was going to be my next step (currently stalled while I enable PHP 5.4 testing and RTBC re-testing).

nick_schuch’s picture

No worries, I will work on that!

nick_schuch’s picture

Just want to touch base. I will be working on this over the weekend (the next 2 days). Will update this issue after that.

ricardoamaro’s picture

This finally on advanced stage at git.drupal.org:sandbox/jthorson/2190713.git / github.com:ricardoamaro/modernizing_testbot__dockerfiles.git
so we can mark this milestone fixed.
We now need to integrate D.O and Jenkins with the docker project.

ricardoamaro’s picture

Assigned: Unassigned » ricardoamaro
ricardoamaro’s picture

Status: Active » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.