GitLab CI

What is GitLab CI?

GitLab CI is the CI/CD pipeline feature offered by GitLab, and in use on our self hosted instance at git.drupalcode.org. It is an automated testing system for validating changes to code and ensuring code quality. GitLab's website offers extensive documentation on how to use GitLab CI.

If you are a project maintainer, GitLab CI is a tool you can enable to run automated tests whenever you make changes to your code, or whenever a contributor submits a merge request. These tests include compatibility tests with different versions of Drupal Core, different environments (php, sql, etc), as well as any test coverage you have written for your own project. For more details about writing tests, please refer to the Automated Testing guide. 

This page will get you started with using GitLab CI. 

How do I get started with GitLab CI? 

GitLab CI is now enabled for every project on Drupal.org.

For now, you can use both DrupalCI and GitLab CI in parallel, but once you have GitLab CI working we recommend disabling your DrupalCI configuration to save resources. Eventually, DrupalCI will be fully deprecated and GitLab CI will be the only testing option. 

Your first step is to add a .gitlab-ci.yml file to the root of your project. 

.gitlab-ci.yml

GitLab CI uses .gitlab-ci.yml file in the root of the project to configure testing. If you are familiar with the configuration files for Travis CI, GitHub Actions, or even custom DrupalCI yml files, you will find this quite similar. If you are not familiar with using a .yml file to configure testing, don't worry. The Drupal Association and the community have provided everything you need in a template to make it easy to get started. 

By using the Drupal Association .gitlab-ci.yml template, your project will be configured to run the most commonly needed test scenarios for you. This template also 'includes' supplementary configuration which can be updated by the Drupal Association on an ongoing basis. This means we can update 'variables' keywords to keep your testing up to date with the latest versions of Drupal and supported environments, and you do not need to make any changes to stay up to date!

The template is configured to work by default with contrib modules for newer versions of Drupal (9 and above), so note that if you are adding this to your Drupal 7 contrib module, you will need to comment out one line and uncomment another. Everything is explained in the ".gitlab-ci.yml" file.

Using the Drupal Association .gitlab-ci.yml template

1. Create a .gitlab-ci.yml file in the root of your project. There are a few ways to do this. 
   • The easiest way to do this is through the GitLab UI
     1. From your project page, navigate to the source code on git.drupalcode.org
        

        

     2. Then use the repository UI (not the Web IDE) to add a new file
     3. Add a new file 
        

        

     4. Type the file name as .gitlab-ci.yml
     5. You will be presented with the option to select a template, select the Drupal Association's template.gitlab-ci.yml from the top of the list. 
        

        

   • Create the file manually in a git checkout.
     1. Checkout your project.
     2. Create a new file called .gitlab-ci.yml in the root of the project
     3. Copy in the contents of the template: https://git.drupalcode.org/project/gitlab_templates/-/blob/main/gitlab-c...
        • Note: You do not need to copy in any of the other 'include' files, GitLab will reference those directly from the template. 
     4. Commit and push your changes. 
2. Confirm that it works: 
   • When you first commit your .gitlab-ci.yml file the pipeline should be triggered automatically. 
     1. Navigate to the sidebar
     2. Click on 'Build'
     3. Click on 'Pipelines'
     4. Check the status of the most recent pipeline 
        

        

        Your pipeline might show success or failure, depending on whether the tests themselves passed. 
        You can click through the pipeline name to see more details, including a summary of all of the jobs that ran (and even click through to see the full console output of each): 

        

        And you can see a summary of the test results themselves: 

        

When will pipelines run?

The default configuration from the provided template will run jobs when: 

• There is a merge request event
• There is a commit tag
• When an upstream pipeline calls it
• There is a commit to the default branch
• If triggered manually from the 'Run pipelines' function
• If triggered manually from the Web IDE

workflow:
  rules:
  # These 3 rules from https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Workflows/MergeRequest-Pipelines.gitlab-ci.yml
    # Run on merge requests
    - if: $CI_MERGE_REQUEST_IID
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'
    # Run on tags
    - if: $CI_COMMIT_TAG
    # Run when called from an upstream pipeline https://docs.gitlab.com/ee/ci/pipelines/downstream_pipelines.html?tab=Multi-project+pipeline#use-rules-to-control-downstream-pipeline-jobs
    - if: $CI_PIPELINE_SOURCE == 'pipeline'
    - if: $CI_PIPELINE_SOURCE == 'parent-child'
    # Run on commits to the default branch
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
    # The last rule above blocks manual and scheduled pipelines on non-default branch. The rule below allows them:
    - if: $CI_PIPELINE_SOURCE == "schedule"
    # Run if triggered from Web using 'Run Pipelines'
    - if: $CI_PIPELINE_SOURCE == "web"
     # Run if triggered from WebIDE
    - if: $CI_PIPELINE_SOURCE == "webide"   

If you want to change this behavior you can override the 'workflow' keyword in the root .gitlab-ci.yml file. Note that if you do so, you will no longer be automatically updated when the Drupal Association updates the include include.drupalci.workflows.yml file. 

Scheduling pipelines

If you would like to schedule pipelines to run on automatically, separately from the triggers defined by the template above, you can do so with 'Scheduled Pipelines'. 

• Navigate to your project on Drupal Gitlab
• Navigate to Build -> Pipelines schedules in the sidebar.
• Click New Schedule button.

From here, you can set an interval pattern, such as every day, week, or month, or a custom pattern based on standard cron syntax. You can then choose a target branch or tag, and set any variable values. Examples of variables are below. Also see All variables.

1. _SHOW_ENVIRONMENT_VARIABLES=1Show all env variables in console log. Open disclosure section to view.
2. _TARGET_PHP=7.4
3. _TARGET_CORE=9.5.1 Drupal core version
4. _TARGET_DB_TYPE=pgsql

Gathering Results

There are multiple ways to gather GitLab CI results, depending on what data you need. 

• You can use the test result summaries from the pipeline, which will highlight test failures and summarize successes.
• You can download the test artifacts.
• You can view the full console output of any particular job by clicking through it. 

If you need additional debugging, you can also leverage GitLab CI features to capture additional logs from some of the services that are running, by setting CI_DEBUG_SERVICES variable to "true". You can read more about this feature on the GitLab documentation page here. We only recommend this while debugging, as the volume and size of artifacts can grow very rapidly with this feature enabled.

[Work in progress]

How does the .gitlab-ci.yml template work? 

[More detailed break down of the components of the template and how to override them] 

Variables, and when to override them

1. To see the variables and their values, view the job output of a job like `composer` or `phpunit`. Open the disclosure triangle that says Show environment variables.
2. Variable descriptions are visible in the include file.

Overriding parts of the template

Here are a few helpful examples :

1. Use a particular version of PHP and Drupal core. See Akamai
2. Customize PHPCS with own phpcs.xml file. See Akamai
3. Test with multiple versions of Drupal core (aka matrix testing). See KeyCDN as an example. Issue.
   1. OPT_IN_TEST_PREVIOUS_MAJOR=1
   2. OPT_IN_TEST_PREVIOUS_MINOR=1
   3. OPT_IN_TEST_NEXT_MINOR=1
   4. OPT_IN_TEST_NEXT_MAJOR=1
4. Test against the maximum supported PHP version.
   1. OPT_IN_TEST_MAX_PHP=1
5. You may apply a patch Drupal core or any dependency by adding it to you project's composer.json as per usual with cweagans/composer-patches project.
6. You may want to break up your PHPUnit suite into parallel runs by test suite, or by submodule. See the PHPUNIT_EXTRA environment variable for that. An example is Scheduler.
7. To use your own services (e.g Redis, memcache, etc), override the `services` key in the jobs you care about. See search_api_opensearch. You can also install code into the default web container if you need client extensions and so on. See MongoDB. The before_script and after_script keywords are not used by the template so feel free to use them, as MongoDB does.
8. If you are changing an ecosystem module like group or flag, you might want to issue downstream pipelines in those projects which test against your MR. See gitlab_templates for an example of this.
9. If you want to fail the pipeline when a linting job fails you can override theallow_failure like so:
    

   #
   # Linting jobs are passing so any issue that breaks them should fix them.
   #
   cspell:
     allow_failure: false
   phpcs:
     allow_failure: false
   phpstan:
     allow_failure: false

If the part of the template that you are overriding uses in-template references, you don't need to replicate them in your overrides, you can just use the !reference notation. ie: !reference [ .setup-webserver ]

Updating dependencies when testing against future releases

When testing against future releases, dependencies may not be compatible with the target release, so will need help to enable the test.

Learn about using Lenient support and Composer patches integration on the documentation site.

Package versions

The templates will favor the drupal/core-recommended package when installing Drupal, which does NOT include a composer.lock file. This is expected and is the recommended usage for most cases.

If your module relies on specific versions of certain packages from core you need to update your module's composer.json to declare this.

Note: to test against the same dependencies as GitLab CI locally, you need to run composer update in the Drupal root directory. This can be important when running PHPStan and PHPCS.

Common tasks

Some additional examples for solutions to common test needs. You can find more information on the dedicated documentation site here: Common tasks.

Publishing a documentation site

Projects can automatically publish a documentation site powered by Gitlab Pages. Learn how to do that for your module on the documentation page here: GitLab Pages.

Using your own .gitlab-ci.yml file

If you would prefer to use your own .gitlab-ci.yml file, or to modify the template, you are certainly welcome to do so. GitLab provides extensive documentation for GitLab CI, or you can join the #GitLab channel in Drupal Slack.

Running GitLab CI tests locally

Detailed information on how to run GitLab CI tests locally can be found in the dedicated documentation site here Test locally

FAQ

When will DrupalCI be deprecated and GitLab become the only option?

The Drupal CI infrastructure will be disabled on July 1st, 2024, with some functionality already deprecated.

Are there any limitations to GitLab CI?

GitLab CI cannot test patch files. This means that any contributions need to be made in the form of a merge request in order to be tested. 

How can I contribute improvements to the template?

Thank you for your interest in contribution! If you want to contribute improvements, you can open an issue on the GitLab Templates project. 

How does GitLab CI differ from DrupalCI?

If you have already used DrupalCI for your project, you know that Drupal.org does a lot of the configuration work for you. Drupal.org provides a UI where you can select environments and versions of core, and configure when tests run against those environments (on commit, on issues, etc). 

In GitLab CI, this configuration model is much more open ended. You can control every aspect of your CI configuration using a gitlab-ci.yml file. This is a powerful tool that can give you much more control of your project testing, but can also be overwhelming if you are not a CI expert. 

The Drupal Association team together with community volunteers has built some tools to make this transition easier.

Known problems

When using parallel matrix, variables are not passed to the different jobs if the pipeline is triggered manually.

If a ".gitlab-ci.yml" file defines a parallel matrix to split a job into separate streams based on the value of a variable, then this does not work if you trigger the pipeline manually. In this scenario, the specified values are not passed to the separate jobs. The other automatic ways where the pipeline is triggered (ie: merge request) are working as expected. GitLab issue #330013.

More documentation

A dedicated documentation site exists that contains more detailed information about the individual jobs, options available, etc. Visit the site GitLab Templates documentation for more information.