Config Auto Export

Introduction

Config Auto Export detects changes made to the Drupal configuration system and exports those changes into a directory of your choosing. In addition, it can send a POST request (webhook) with configurable parameters to an HTTP service you define, so that an external system can react to those configuration changes.

Consider a production environment that is deployed fully automatically and locked down for security, where none of the directories are writable except the public and private file storage areas. The configuration sync directory often lives outside those writable areas, which makes it easy to keep track of every configuration change on a live site. That configuration is typically also kept in version control and will be overwritten on the next deployment.

This is where Config Auto Export helps. If you still allow selected users to make configuration changes on a live site (for example to blocks, webforms, or views), you usually want those changes to flow downstream into version control as well. This module exports each change into a temporary directory and fires an HTTP request to a service you configure. That service is intentionally out of scope for this module, but you might, for example, configure a CI tool to listen for that request, collect the exported configuration changes, push them into version control, and notify developers so they can review and merge them into a main branch.

The result is a full circle between Drupal's configuration management, continuous deployment, and the reality of clients who need to make configuration changes on live sites.

Configuration

The settings form is available at /admin/config/development/config_auto_export (route config_auto_export.settings) and requires the administer site configuration permission.

All settings below are stored in the config_auto_export.settings configuration object. Their default values come from the module's config/install directory.

Webhook parameters and headers as YAML

Both webhook_params and webhook_headers are entered as YAML in the settings form. Each is a simple map of key/value pairs. The parameters are sent as the POST body of the webhook request, and the headers are attached as HTTP headers to that same request. This lets you pass authentication tokens, target branches, or any other data your receiving service expects.

How it works

Config Auto Export subscribes to the Drupal configuration save, rename, and delete events (ConfigSubscriber). When a relevant change occurs, a dedicated service (config_auto_export.service) is responsible for exporting the changed configuration into the configured directory and, when appropriate, dispatching the webhook.

The dispatch behavior depends on your settings:

• With autorun enabled and delay set to 0, changes are exported and the webhook fires immediately.
• With autorun enabled and delay greater than 0, the export is deferred and triggered by the next feasible cron run after the delay period. Delayed exports therefore require cron to be running.
• With autorun disabled, changes accumulate and you trigger the webhook manually.

A manual trigger form is available at /admin/config/development/config_auto_export/trigger (route config_auto_export.trigger). It requires the custom trigger config_auto_export permission, which is marked as restricted access. The form confirms with the prompt "Trigger webhook for exported config changes?" and processes the exported configuration changes accumulated since the last run.

A Drush command set is also provided (under src/Drush) so that the same export and trigger operations can be scripted from the command line.

Webhook and CI/CD examples

The following community-contributed examples show how the external service that consumes the webhook can be implemented. They are illustrative and specific to GitLab, but the webhook itself is generic and can target any HTTP service, such as another CI system or a custom endpoint.

Example A: LakeDrops GitLab pipeline trigger

Contributed by the maintainer (jurgenhaas), this approach triggers a GitLab pipeline directly.

• Webhook URL form: https://gitlab.example.com/api/v4/projects/PROJECT_ID/trigger/pipeline, where PROJECT_ID is the numeric GitLab project ID of the website's repository.
• webhook_params (YAML):

ref: "main"
token: "PIPELINE TRIGGER TOKEN FROM GITLAB"
variables[CAE]: "--extra-vars=collect_config=true --extra-vars=config_path=[config directory] --extra-vars=export_path=[export directory]"

The triggered pipeline then:

• verifies that the export directory (containing the changed configuration from the Drupal site) exists;
• checks out a new, uniquely named branch of the website repository (for example cc-);
• moves the export directory into a temporary directory for processing;
• applies the changes from the export directory to the git repository for regular configuration, translations, and config splits, including handling of deleted configuration entities;
• checks git status to confirm whether there were really any changes;
• if so, commits the changes, creates a merge request into the main branch, and optionally auto-merges it while skipping CI;
• performs cleanup.

Example B: Standalone .gitlab-ci.yml

Contributed by charginghawk (comment #6), this standalone pipeline is based on the LakeDrops approach but runs from its own repository. It uses the ruby:3.1 image, which is one of GitLab's default runner images.

collect-config:
  stage: deploy
  image: ruby:3.1
  before_script:
    - git config --global user.email $GITLAB_USER_EMAIL
    - git config --global user.name "${GITLAB_USER_NAME}"
    - |
      apt-get update >/dev/null
      apt-get -y --no-install-recommends install rsync openssh-client >/dev/null
  script:
    # Set up SSH identity with artifact repo access.
    - eval $(ssh-agent -s)
    - cat "$SERVER_PRIVATE_KEY" | ssh-add - > /dev/null
    - mkdir -p ~/.ssh
    - chmod 0700 ~/.ssh
    - echo -e "HOST *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
    # Copy exported config from the server.
    - rsync -r -e ssh user@site.com:/tmp/cae/ /tmp/cc-${CI_JOB_ID}
    - git clone https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.com/company/site/project.git
    - cd project
    - git checkout -b cc-${CI_JOB_ID} HEAD
    - |
      function collectDir() {
        CURRENT=$(pwd)
        mkdir -p $1
        cd $1
        find . -type d -exec mkdir -p "$1/{}" \;
        find . -type f -name '.*' -delete -exec sh -c "echo {}|sed -r 's!./.!$2/!g'|xargs rm" \;
        find . -type f -exec cp "$1/{}" "$2/{}" \;
        cd "${CURRENT}"
        rm -rf $1
      }
    - CURRENT=$(pwd)
    - collectDir "/tmp/cc-${CI_JOB_ID}/config_split" "${CURRENT}/config/override"
    - collectDir "/tmp/cc-${CI_JOB_ID}/language" "${CURRENT}/config/default/sync/language"
    - collectDir "/tmp/cc-${CI_JOB_ID}" "${CURRENT}/config/sync"
    - git status > /tmp/gitstatus.log
    - EC=0
    - grep "nothing to commit, working tree clean" /tmp/gitstatus.log || EC=$?
    - if [[ $EC -eq 0 ]]; then exit 0; fi
    - git add config/*
    - git commit -am "Config change on server."
    - git remote set-url --push origin "https://oauth2:$ACCESS_TOKEN@gitlab.com/company/site/project.git"
    - git push -o ci.skip -o merge_request.create -o merge_request.target=main origin cc-${CI_JOB_ID}
  cache: {}
  only:
    refs:
      - triggers

Known limitations

When using Config Split in "Collection" storage mode, Config Auto Export can only associate a configuration entity with a split whose exported configuration file already exists in that split's storage. Splits that are configured but have never been exported are not detected. This is a known constraint related to finding the split for a given configuration entity.