Reinforcement Learning (or A/B & Multivariate Testing)

Reinforcement Learning (RL) is an A/B and multivariate testing framework for Drupal where every visitor click is treated as human feedback (RLHF-style). Each page view is a trial, each conversion is a reward, and the algorithm continuously shifts traffic to whichever variant is winning. No fixed test horizons. No manual winner picking. No third-party SaaS.

RL is part of the DXPR marketing CMS stack and ships in DXPR CMS.

What you can A/B test with RL

• RL: A/B Test Views Content (rl_sorting): the order of items in any Drupal View
• RL: A/B Test Page Titles (rl_page_title, bundled): page titles for nodes, View pages, and any controller
• RL: A/B Test Menu Links (rl_menu_link, bundled): labels in any menu link
• DXPR Builder integration: variant slots inside builder blocks

Why RL instead of fixed-horizon A/B testing?

Traditional A/B tests run for a fixed window (say two weeks) and split traffic 50/50 the whole time, even when one variant is obviously losing. RL turns the experiment into a feedback loop: every click adjusts the model, traffic shifts toward the leader as soon as evidence emerges, and the test never has to "end". You can run dozens or thousands of variants simultaneously (true multivariate testing), and a newly added variant is in play on the next render with no manual setup.

How it works

RL uses a multi-armed bandit (Thompson Sampling). Each variant has a reward distribution; on each render the algorithm samples from the distributions and picks the highest sample. Wins update the distribution toward higher rewards; losses update toward lower. The math: ThompsonCalculator.php.

Features

• Multivariate by default: 2 to thousands of variants, no manual configuration
• Real-time RLHF loop: visitor clicks update the model on every page
• Fast HTTP REST API: optimized JSON endpoint for tracking, decisions, and full rankings
• Admin reports: per-experiment performance, traffic, and confidence
• Service-based architecture: extensible decorators, custom variant selectors
• Data sovereignty: no cloud, no third-party SaaS, all data stays in your Drupal database
• GDPR-friendly tracking: only anonymous interaction counts, no user IDs or cookies

You need RL if

• You want to A/B or multivariate test any part of your site without third-party SaaS
• You want continuous optimization rather than fixed-horizon experiments
• You want to add or remove variants on the fly without restarting tests
• You want to sort lists, accordions, or FAQs by visitor engagement
• You want a core API you can call from any module, View, or block

Installation

composer require drupal/rl
drush en rl

Verify rl.php access

RL ships an .htaccess file that allows direct access to rl.php (same pattern as Drupal core's statistics.php). Test it:

curl -X POST -d "action=ping" http://example.com/modules/contrib/rl/rl.php

If the test fails:

• Apache: ensure .htaccess files are processed (AllowOverride All)
• Nginx: copy the rewrite rules from .htaccess to your server config
• Security modules: whitelist /modules/contrib/rl/rl.php

If server policies prevent direct PHP file access, consumer modules must implement their own tracking controller or proxy.

Drush command reference

AI coding assistant integration

RL ships a built-in Agent Skills file that teaches AI coding assistants how to manage A/B tests through natural language. Compatible with Claude Code, Codex CLI, Gemini CLI, GitHub Copilot, Cursor, and any tool supporting the standard.

After installing the module, run drush rl:setup-ai to enable AI assistant support. Your assistant will then respond to prompts like:

• "List all running A/B tests"
• "Analyze the hero_cta_test experiment"
• "Create a new A/B test for the homepage banner"
• "What's the conversion rate for variant B?"

API

// Get the experiment manager
$experiment_manager = \Drupal::service('rl.experiment_manager');

// Record a trial (content shown)
$experiment_manager->recordTurn('my-experiment', 'variant-a');

// Record a reward (user clicked)
$experiment_manager->recordReward('my-experiment', 'variant-a');

// Get scores for the variants currently in play
$scores = $experiment_manager->getThompsonScores('my-experiment', NULL, ['variant-a', 'variant-b']);

// Pick a winner
$ts_calculator = \Drupal::service('rl.ts_calculator');
$best_arm = $ts_calculator->selectBestArm($scores);

JavaScript API

Attach the rl/api library to get Drupal.rl on the page:

// Pick a single winner.
Drupal.rl.decide('hero_cta', ['v0', 'v1', 'v2']).then(function (armId) {
  showVariant(armId);
});

// Get a full ranking (for sorting lists, accordions, FAQs).
Drupal.rl.rank('faq_sort', ['t0', 't1', 't2', 't3']).then(function (sorted) {
  // sorted = ['t2', 't0', 't3', 't1'], all arms, best first
});

// Record impressions and conversions.
Drupal.rl.turn('hero_cta', 'v0');
Drupal.rl.reward('hero_cta', 'v0');

All four methods feed a shared 500 ms batch window, so every A/B test on the page rides one POST to rl.php. decide() returns a single winner; rank() returns the full sorted arm list for use cases like reordering accordion items by visitor engagement. See the README for the HTTP wire format and server-side patterns.

FAQ

Does RL store my A/B test's variants?

RL stores performance data (trial counts and reward rates) for each variant, but the authoritative list of which variants are active is owned by your module, not RL. This means adding or removing a variant takes effect on the next render with no second save step required.

Different consumer modules keep the live variant list in different places:

• rl_sorting: the content returned by a View
• rl_page_title: fields on a content entity
• rl_menu_link: labels on a menu link
• DXPR Builder: slots inside a block component

On each call your module passes its current list (getThompsonScores($id, NULL, $arms) in PHP or Drupal.rl.decide(id, arms) in JS) and RL matches it against the stored stats to pick a winner.

When do I pick a winner and end an A/B test?

You pick a winner only when you choose to. RL has no fixed horizon and no significance gate to wait out; it continuously shifts traffic toward whichever variant is winning and keeps adapting as evidence changes. You can lock in a confident winner or leave the experiment running indefinitely.

Two patterns, depending on what you are testing:

• Converging tests: a better page title, a clearer checkout button, a stronger hero image. Once the report shows a confident winner, lock it in and move on.
• Evergreen experiments: blog post lists, banner ads that fade as returning visitors tune them out, seasonal calls to action. Leave them running. RL follows the winner as it shifts.

Thompson Sampling preserves exploration, so a losing arm always retains a small chance of selection. Traffic only truly stops when the experiment owner removes the variant.

How many variants can I run at once?

There is no hard limit. Thompson Sampling scales to thousands of arms per experiment, so you can run anything from a classic two-variant A/B test to true multivariate testing with dozens or hundreds of alternatives. The practical limit is how many variants you can create and maintain.

Does RL work with page caching?

Yes, RL is fully compatible with page caching. For server-side decisions the cache lifetime determines how often the algorithm re-evaluates. For client-side decisions using the Drupal.rl JavaScript API, the page can be fully cached (including behind Varnish or Fastly) and the variant swap happens in JavaScript after load.

DXPR Builder uses the client-side pattern to stay compatible with aggressive edge caching.

Does RL set cookies or track users?

No. RL tracks only anonymous interaction counts: impressions (trials) and conversions (rewards). It does not set cookies, store user IDs, fingerprint browsers, or create visitor profiles. This makes RL GDPR-friendly by design, with no consent banner required for the testing itself.

How is RL different from Google Optimize or other SaaS A/B testing tools?

RL runs entirely inside your Drupal installation with no external JavaScript snippet, no third-party account, and no monthly fee. All experiment data stays in your own database. Unlike fixed-horizon tools that split traffic 50/50 for a set duration, RL uses Thompson Sampling to shift traffic toward the winner continuously.

Related modules

• RL: A/B Test Views Content (rl_sorting): A/B test the order of any Drupal View
• Analyze: content analysis and quality scoring for Drupal
• AI Content Strategy: AI-driven content strategy recommendations

Resources

• Multi-armed bandit problem (Wikipedia)
• Free micro-course on Thompson Sampling
• Thompson's original paper (JSTOR)
• Finite-time analysis of the multi-armed bandit problem