Overview

In many cases, it's useful to allow users to define patterns or large chunks of text that contain programmatically derived values. For example, from email messages addressed to a given user, or URL path aliases containing the title of a given node. Both examples require bits of data that vary each time the text is generated -- node titles, user ids, and so on. Rather than forcing users to embed ugly snippets of PHP, or creating elaborate and bizarre UIs for configuring the patterns via the browser, it's most useful to give users a set of 'placeholder' tokens to place in their text.

For Drupal 7 the base functionality of listing available tokens and replacing them with their dynamic values is part of core. Core however does not provide a UI for browsing available tokens or tokens for Field API field values. This functionality continues to live in the contrib token module. This documentation covers both the core API and the additional tools provided by the token module.

Drupal core provides a shared API for exposing and using placeholder tokens and their appropriate replacement values. This allows for any module to provide placeholder tokens for strings without having to reinvent the wheel. It also ensures consistency in the syntax used for tokens making the system as a whole easier for end users.

There are three main components to the token system:

  1. Providing a list of placeholder tokens your module is responsible for.
  2. Providing values that can be substituted for placeholders that your module is responsible for.
  3. Replacing tokens in a given string with the appropriate value.

In addition it's also helpful to provide:

  1. A UI for end users to browse available tokens.
  2. Validation for form elements that contain tokens to ensure the tokens entered are correctly formatted.

Declaring Tokens - hook_token_info()

The first of two steps required to add new tokens to Drupal 7 is implementing hook_token_info() in order to give Drupal a list of the placeholder tokens that your module provides. Placeholders can be grouped together by creating a new token type and placing new tokens under that grouping or by adding new tokens to an existing token type provided by another module like the node module.

Token types can be defined as global meaning their value can be calculated without any additional context, or as 'needs-data' tokens that require additional information about the context in which they are being used in order to calculate their value. For example, a [node:title] token needs to know which node is being referenced, whereas a [date:now] token doesn't need any additional reference data in order to calculate the current date.

Module's implementing hook_token_info() can use the 'needs-data' key for a token type in order to declare that tokens of this type require additional contextual data for processing. For example, the node type tokens can set 'needs-data' => array('node') and require that a $node object be provided when calculating the value for those tokens.

Example:

/**
 * Implements hook_token_info().
 */
function databasics_token_info() {
  $info = array();

  $info['types'] = array(
    'databasics-totals' => array(
      'name' => t('Databasics totals'),
      'description' => t('Global databasics tokens.'),
    ),
    // [databasics-page:]
    'databasics-page' => array(
      'name' => t('Databasics'),
      'description' => t('Tokens for databasics page counts.'),
      'needs-data' => array('databasics_record'),
    ),
  );

  $info['tokens'] = array(
    'databasics-totals' => array(
      // [databasics-totals:count]
      'count' => array(
        'name' => t('Total page views'),
        'description' => t('Total page views for entire site.'),
      ),
    ),
    // Page specific tokens.
    'databasics-page' => array(
      // Add a token for the view count.
      // [databasics-page:view-count]
      'view-count' => array(
        'name' => t('View count'),
        'description' => t('Number of times the page has been viewed by the current user.'),
      ),
      // [databasics-page:last-viewed]
      'last-viewed' => array(
        'name' => t('Last viewed'),
        'description' => t('Date the page was last viewed.'),
      ),
    ),
  );

  // [node:view-count], [node:view-count:last-viewed]
  $info['tokens']['node'] = array(
    'view-count' => array(
      'name' => t('View count'),
      'description' => t('Number of times the current node has been viewed by the current user.'),
      'type' => 'databasics-page', 
    ),
  );

  return $info;
}

See hook_token_info(), or the examples for developers project for detailed code examples.

Providing Replacement Values - hook_tokens()

Now that we've declared the list of tokens that our module provides we need to provide the corresponding replacement values for those tokens. This is done by implementing hook_tokens(). When processing a string that contains placeholder tokens hook_tokens() is called once for each type of token encountered and is passed an array containing all the tokens of that type that need values.

For example take this string:

Hello [current-user:name], Welcome to the page titled [node:title] created on [node:created].

This string contains 3 tokens, 1 user type token and 2 node type tokens. In this example hook_tokens() would be called to retrieve values once for the user token, and then again for both node tokens.

Depending on the type of token for which values are being requested hook_tokens() may receive additional contextual data in the $data parameter. The need for this additional data is declared in hook_token_info() and the values provided there will map to the keys used in the $data array provided to hook_tokens() implementations.

In this example the tokens require knowledge of which user is currently viewing the site and which node they are viewing. The $data parameter would contain either the $user object or the $node object required to calculate the appropriate values.

Implementations of hook_tokens() return an array of replacement values keyed by the original token string.

Example:

/**
 * Implements hook_tokens().
 */
function databasics_tokens($type, $tokens, array $data = array(), array $options = array()) {
  $replacements = array();

  // Replacement values for tokens that can be calculated without any additional
  // data.
  if ($type == 'databasics-totals') {
    foreach ($tokens as $name => $original) {
      switch ($name) {
        case 'count':
          $count = db_query('SELECT SUM(view_count) FROM {databasics}')->fetchField();
          $replacements[$original] = $count;
          break;
      }
    }
  }

  // Replacement values for tokens that require additional contextual data.
  if ($type == 'databasics-page' && !empty($data['databasics_record'])) {
    $record = $data['databasics_record'];
    foreach ($tokens as $name => $original) {
      switch ($name) {
        case 'view-count':
          $replacements[$original] = $record->view_count;
          break;

        case 'last-viewed':
          $replacements[$original] = $record->last_viewed;
          break;
      }
    }

  }
  // An array of replacement values keyed by original token string.
  return $replacements;
}

For more detailed code examples see the hook_tokens() documentation, or the examples for developers project.

Replacing Tokens In A String

Given a string with placeholder tokens developers can use the token_replace() function to process the string and replace all tokens with their dynamic values. When called the token_replace() function will scan the string for any tokens and then invoke hook_tokens() in order to retrieve the appropriate values so that it can perform the placeholder substitution.

The token_replace() function can be provided with additional contextual data via the $data argument which is then passed on to implementations of hook_tokens() so they can know for example who the current user is and what node they are viewing. It is the responsibility of the code calling the token_replace() function to know which token types are valid for the given string and thus what additional contextual data may need to be provided.

Example:

$node = node_load(10);
$message = token_replace($unprocessed_message, array('node' => $node));

Providing A UI For Browsing Tokens

The token module provides a couple of different ways to include a user interface element that will allow end users to browse a list of tokens available for use when writing text and filling in the form fields that allow for tokens. The UI however isn't provided automatically and any module that allows for tokens needs to decide where, and when to include the token browser. It's not required, but highly recommended as it makes the process of knowing which tokens can be used much simpler.

The preferred method for displaying the token browsing UI is via the theme_token_tree_link() function provided by the token module. Using this method will place a link on the page that when clicked will open a modal window displaying a tree like interface for browsing available tokens. This method is preferred because the list of tokens can get quite large and displaying the entire token tree and the JavaScript required to provide the collapsible interface can slow down loading of pages. Since the token tree is likely not used the majority of time we prefer to defer loading until someone clicks the links requesting the list of tokens and then load it via and ajax call.

By default only global tokens are listed. If you would like to display additional token types in the list you'll need to explicitly declare them. This ensures that only the tokens that are relevant in the context of the current string are displayed and prevents users from seeing tokens that might not be available to them. Knowing which token types are available when processing the string associated with the display of the token browser is the responsibility of the developer implementing the form.

Example:

$form['token_help'] = array(
  '#theme' => 'token_tree_link',
  '#token_types' => array('user', 'node'),
);

Validating Form Fields With Tokens

Another helpful tool provided by the token module is the token_element_validate() function. This function can be added to any form field that allows tokens and will ensure that only valid tokens are provided. Checking entered tokens against the provided list of token types to prevent usage of for example a [user:] token in a context that does not support them. It also ensures that tokens are properly formatted.

Just like the token browsing interface above, it's the responsibility of the developer to know which token types are valid in the context of the form elements they are adding this validator to.

Example form element with token_element_validate():

$form['databasics_message'] = array(
  '#type' => 'textarea',
  '#title' => t('Message'),
  '#default_value' => variable_get('databasics_message', ''),
  '#element_validate' => array('token_element_validate'),
  '#token_types' => array('user', 'node'),
);