Introduction

AI Migration imports unstructured web content into your site's content types, taxonomy, media, and paragraphs. This module is ideal for large-scale migrations where content is complex and data dumps unreliable or unavailable, such as sites for

  • Government
  • Higher education
  • Large corporations

Goals

  • To make it easier to parse fairly unstructured content from web pages or documents and transform it into structured content in Drupal. "Fill the fields, not bloat the body."
  • To make it easy to migrate slices of content into a Drupal instance to support phased migration of legacy sites into Drupal sites.
  • Spend less time tuning migrations and more time migrating.

Requirements

This module requires:

  • drupal/ai:^1.2
  • drupal/document_loader_html_processor:^2.0 (brings in document_loader and html_processor)
  • drupal/jsonapi
  • drupal/migrate:^6.0
  • drupal/migrate_plus:^6.0
  • drupal/migrate_tools:^6.1
  • drupal/serialization
  • drupal/schemata:^1.1
  • drupal/schemata_json_schema

We recommend using these models with AI Migration:

  • OpenAI GPT-4.1-mini
  • OpenAI GPT-5-nano
  • Gemini 2.5 Flash with Google Vertex
  • Gemini Flash-lite Latest with Gemini

We are currently addressing compatibility issues with Anthropic Claude.

Installation & Configuration

  1. Install the module as you would any other module.
  2. Configure an AI API provider at /admin/config/ai/providers.
  3. Set the default AI API provider for Chat and Chat with Structured Response at /admin/config/ai/settings.
  4. Create a Drupal content type with fields that you want to populate with migrated content.
  5. Create a migration yml file in your custom module.
  6. Execute the migration to migrate content.

HTML Processing

The html_processor: key in a migration YAML controls how raw HTML is cleaned before it is sent to the AI provider. It is processed by the document_loader_html_processor module.

ai:
  html_processor:
    # Extract a specific region of the page by CSS selector.
    container: [ 'article', '#main-content' ]
    # Strip matched patterns before sanitizing (optional).
    strip_regex:
      - |-
        /]*>.*?<\/script>/is
    # Symfony HtmlSanitizer options (nested under 'sanitizer').
    sanitizer:
      allowStaticElements: true
      allowRelativeLinks: true
      dropAttribute:
        - [ 'style', [ '*' ] ]

Supported top-level keys: container, strip_regex, remove_ads, base_url, sanitizer, output_full_document, minify. See the document_loader_html_processor README for the full options reference.

minify: true collapses whitespace to reduce AI token count. Only add it when the processed HTML goes directly to the AI provider — minified HTML breaks Markdown converters and other downstream processors that depend on whitespace structure.

Caching

The module creates a cache bin to store a key hashed with SHA-256 from a combinations of AI responses keyed by the following:

  • prompt
  • schema
  • provider
  • model

This way, identical queries to the same model will not incur additional costs from the provider. Keep in mind that clearing all caches also clears out the AI Migration cache.

HostnameFilter Service of the Drupal AI module

To ensure successful migrations we recommend setting the HostnameFilter service to full trust mode. This will avoid additional sanitization of the JSON response from the AI provider.

To enable full trust mode, edit your settings.php file and adding these lines:

// AI output filtering configuration.
$settings['ai_output'] = [
  // Full trust mode - bypass all filtering.
  // Default: FALSE
  'full_trust_mode' => TRUE,
];

Note: Full trust mode is recommended only for local or testing environments.

Submodule: 'AI Migration Example'

This submodules provides a sample migrations that demonstrate AI Migration functionality. The submodule is a good starting point for creating your own migrations.

Simple migration

  • modules/ai_migration_example/simple_content_migration.yml
  • Creates a content type called 'Simple Content Migration' that is populated on import
  • Targets CivicActions' accessibility website: https://accessibility.civicactions.com/
  • The end result is blog entry with taxonomy terms for each page imported

Complex migration

  • modules/ai_migration_example/complex_content_migration.yml
  • Creates a content type called 'Complex Content Migration' that is populated on import
  • Imports a structurally complex page with the following dependencies:
    • Media references (complex_content_migration_media.yml)
    • Paragraph entity-reference revisions (complex_content_migration_editions.yml)
  • Can be run with --execute-dependencies to migrate the referenced entities
  • Requires the paragraphs module
  • Targets two book entry pages copied from Open Library (The Martian Chronicles and Anne of Green Gables)

Similar module

Developers and Contributors

This is an official AI Initiative project. We welcome contributions from AI Partners as well as from the wider community.

See CONTRIBUTING.md for details on how to set up a local development environment and contribute.

More Documentation

See our additional documentation for more details.

Supporting organizations: 
provides time and technical support

Project information

Releases