Custom Docker Images in DrupalCI: Case Study

Intro

As explained in "Customizing DrupalCI Testing for Projects # Executing Custom Shell Commands", the host_command can be used to:

... execute the shell commands on the host itself, outside of the testing environment. Potentially useful if you need to restart one of the docker containers after installing new PHP extensions on it.

Apart from managing existing containers, another possibility when using this directive would be to start additional services that you need to consume during your project's testing, e.g. a Java-based validator.

The following example use case would show how this is possible as well as provide a blueprint for converting an application into an HTTP-accessible service, that can subsequently be accessed and used from PHP code.

The use case: iCal validation

iCal validation was needed during testing the ics_field module, locally as well as via the DrupalCI. The most reliable validators for the iCal format are either available online, or available as applications that cannot be directly used as PHP dependencies.

Initial research concluded that the best option would be using the https://github.com/faph/icalendar-validator

For inter-container communication, the main idea was to somehow expose a terminal-based application via HTTP. The most prominent solution turned out to be https://github.com/msoap/shell2http

So the idea presents the following challenges:

1. Obtain a docker image that combines msoap/shell2http with the application we want to expose via HTTP
2. Configure Drupal CI to run the respective image prior to running the tests
3. Configure Drupal CI so that the dockerized test runner has access to the HTTP-exposed application
4. Accessing the HTTP-exposed application from PHP code

What follows are the details of handling all these points.

Building and publishing a custom docker image

The application we need, a Java iCal validator for the terminal is available as a docker image. We are going to use that image as our base.

FROM faph/icalendar-validator

msoap/shell2http, the application we need to expose the validator via HTTP is also available as a docker image, which means that we can COPY the needed binary from that image, in our Dockerfile.

COPY --from=msoap/shell2http /app/shell2http /app/shell2http

Finally, specifically for the application we want to expose, we need to do some stdin/stdout redirection, we can handle that using a shell wrapper that we can also generate during image build. That wrapper will then be passed as a parameter to the shell2http application, which we will define as the image's entrypoint.

RUN echo 'cat - > /tmp/file.ics && /usr/bin/java -jar /app/app.jar /tmp/file.ics' > /app/validate_shortcut && chmod +x /app/validate_shortcut

ENTRYPOINT ["/app/shell2http"]
CMD ["-cgi", "-show-errors", "/validate", "/app/validate_shortcut"]

This custom built image is going to deliver containers that listen on http://localhost:8080/validate for a POST request delivering an iCal file as text. This knowledge is needed when writing the respective PHP testing code.

Publishing the image to a public registry is required to have that image be available to Drupal CI.

This particular image was published on Docker Hub, at https://hub.docker.com/repository/docker/stefanospetrakis/icalvalidator2...

Configuring Drupal CI for running an additional docker container

We need to edit drupalci.yml for our project and use the host_command directive in order to start our container.

build:
  assessment:
    ...
    testing:
      host_command:
        commands:
      	  - "docker run -d --name icalvalidator --rm stefanospetrakis/icalvalidator2http"

Configure Drupal CI so that the dockerized test runner has access to the HTTP-exposed application

This is the trickiest part, since it depends on the bridge networks concept of Docker. Ultimately, we need something like this:

build:
  assessment:
    ...
    testing:
      host_command:
    	commands:
       	  ...
          - "docker network connect --alias icalvalidator drupalci_nw icalvalidator"

The --alias option is very important, as this is the DNS name that we can use to access the running container from inside the test runner container.

Accessing the HTTP-exposed application from PHP

Once everything is in place, our PHP testing code can access the Java application over HTTP by using the defined alias name.

$icalValidationUrl = 'http://icalvalidator:8080/validate';

$httpClient = new Client();
$postArray = [
  'body' => $icsString, // This is a string containing an iCal
];
$response = $httpClient->post($icalValidationUrl, $postArray);
$responseText = strip_tags($response->getBody()->getContents());
$found = strpos($responseText, 'Validation successful');

// Testing can now the result of the validation