This is steps of how to reveal this issue:
1,Install latest drupal,
2,Install token module
3,Install compound_token module
4,Create a test node which node type is "Article".

5,Change the code in compound_token.module from:

if (!module_exists('token')) {

to:

if (true) {

6,Run test code:

/**
 * Implements hook_menu().
 */
function test_menu(){
  $items['test'] = array(
    'title' => 'Test',
    'page callback'    => 'test_test_page',
    'access arguments' => array('access content'),
    'type' => MENU_CALLBACK,
  );
  return $items;
}
function test_test_page(){
$output = "";
$output .= '123456:';
$node = node_load(1);
$output .= token_replace('[node:nid|user:mail][node:nid][node:title]', array('node' => $node));
return $output;
}

7,Then we will get error message:
Notice: Array to string conversion in token_replace() (line 102 of D:\xampp\htdocs\drupaldev\includes\token.inc).

8,And the result output is:
123456:[node:nid|user:mail]ArrayArray

After a lot debug, i find that hook_tokens_alter() was called before hook_tokens, it is very strange. Maybe compound_token module is the first module that use hook_tokens_alter. The hook_tokens_alter has been called twice,one is before hook_tokens, one is after it.

I have examined my code, and do not find any logic error. My purpose of compound_token module is, after run hook_tokens, If there are still some tokens that not be replaced, If these token following the rule provide by compound_token, then this module replace these tokens.

CommentFileSizeAuthor
#2 hook-tokens-alter-run-before-hook-tokens-1541642.patch1.23 KBg089h515r806

Comments

g089h515r806’s picture

g089h515r806 commented

After I move the code:

    // Allow other modules to alter the replacements.
    $context = array(
      'type' => $type,
      'tokens' => $tokens,
      'data' => $data,
      'options' => $options,
    );

from function token_generate to function token_replace,Then the code of token_replace is:

function token_replace($text, array $data = array(), array $options = array()) {
  $text_tokens = token_scan($text);
  if (empty($text_tokens)) {
    return $text;
  }

  $replacements = array();
  foreach ($text_tokens as $type => $tokens) {
    $replacements += token_generate($type, $tokens, $data, $options);

    // Allow other modules to alter the replacements.
    $context = array(
      'type' => $type,
      'tokens' => $tokens,
      'data' => $data,
      'options' => $options,
    );

  drupal_alter('tokens', $replacements, $context);
    if (!empty($options['clear'])) {
      $replacements += array_fill_keys($tokens, '');
    }
  }

  // Optionally alter the list of replacement values.
  if (!empty($options['callback']) && function_exists($options['callback'])) {
    $function = $options['callback'];
    $function($replacements, $data, $options);
  }

  $tokens = array_keys($replacements);
  $values = array_values($replacements);

  return str_replace($tokens, $values, $text);
}

After this i get the correct result.

g089h515r806’s picture

g089h515r806 commented
Status: Active » Needs review
StatusFileSize
new hook-tokens-alter-run-before-hook-tokens-1541642.patch1.23 KB

Here is my code, this is the first patch that i build for drupal core.

dave reid’s picture

dave reid
he/him
Primary language English
Location Nebraska USA
commented
Category: bug » support
Status: Needs review » Closed (works as designed)

Since this problem only happens with your compound_tokens module installed, I'm inclined to say it's a problem caused by the code in compound_tokens. I strongly believe the problem is that you're calling token_replace() from inside your hook_tokens_alter() function, which then is going to cause another invocation of hook_tokens_alter(). You need to ensure that your token_replace() call will not be recursive.

g089h515r806’s picture

g089h515r806 commented
Category: support » bug
Status: Closed (works as designed) » Needs review

The issue still exist even i remove the code of compound token part:

<?php

/**
 * @file
 * Support compound token which contains several sub token. Support normal field tokens.
 */

/**
 * Implementation of hook_tokens_alter().
 */
function compound_token_tokens_alter(&$replacements, &$context) {
  $type = $context['type'];
  $tokens = $context['tokens'];
  $data = $context['data'];
  $options = $context['options'];
  //Already replaced by other module.
  foreach ($tokens as $key => $token) {
    if (isset($replacements[$token])) {
      unset($tokens[$key]);
    }
  }
  /*
  //Support compound tokens.
  $compound_tokens = array();
  foreach ($tokens as $key => $token) {
    if (strpos($token, '|')) {
      $compound_tokens[$token] = $token;
    }
  }
  
  if (!empty($compound_tokens)) {
  //if (FALSE) {  
    foreach ($compound_tokens as $key => $compound_token) {
      $compound_data = $data;
      $compound_token = strtr($compound_token, array('|' => ']|['));

      $sub_tokens = explode('|', $compound_token);

      foreach ($sub_tokens as $delta => $sub_token) {
        $sub_replacement = token_replace($sub_token, $compound_data, $options);

        if (isset($sub_tokens[$delta+1])) {
          $next_text_tokens = token_scan($sub_tokens[$delta+1]);
          $next_token_type = '';
          foreach ($next_text_tokens as $next_type => $next_text_token) {
            $next_token_type = $next_type;
            break;
          }
          $entity_info = entity_get_info($next_token_type);
          if (empty($entity_info)) {
            break;
          }

          $entitys = entity_load($next_token_type, array($sub_replacement));
          if (isset($entitys[$sub_replacement])) {
            $entity = $entitys[$sub_replacement];
            $compound_data[$next_token_type] = $entity;
          }
          else{
            break;
          }
        }
        else{
          $replacements[$key]  = $sub_replacement;
        }
      }
    }
  }
*/
  // Support field/property tokens
  foreach ($tokens as $key => $token) {
    $token_args = explode(':', $key);

    if (count($token_args) ==4) {

      $field_name = $token_args[0];
      $langcode = $token_args[1];
      $delta = $token_args[2];
      $column = $token_args[3];
      $entity = isset($data[$type]) ? $data[$type] : NULL;
      //Support default language.
      if ($langcode == 'default') {
        global $language;
        if (isset($entity->{$field_name}[$language->language])) {
          $langcode = $language->language;
        }
        elseif (isset($entity->{$field_name}['und'])) {
          $langcode = 'und';
        }
      }
      if (isset($entity->{$field_name}[$langcode][$delta][$column])) {
        $replacements[$token]  = $entity->{$field_name}[$langcode][$delta][$column];
      }
    }
    if (TRUE) {
      // Conflict with token module.
      if (count($token_args) ==1) {
        $property = $token_args[0];

        $entity = isset($data[$type]) ? $data[$type] : NULL;

        if (isset($entity->{$property}) && is_string($entity->{$property})) {
          $replacements[$token]  = $entity->{$property};
        }
      }
    }
  }
}
g089h515r806’s picture

g089h515r806 commented

Dave Reid, Thanks for you review my code.
There is no token_replace in hook_tokens-alter now. Issue still exist, actuall speaking, I have put a lot of print debug in my code yesterday.And find it run twice,one before hook_tokens, one after hook_tokens. I put a print debug after "$replacements += token_generate($type, $tokens, $data, $options);", and found it only run one time. That is why i think this is a bug of drupal core instead of a bug of compoumd_token or token module.

For short code that you read:


function compound_token_tokens_alter(&$replacements, &$context) {
  $type = $context['type'];
  $tokens = $context['tokens'];
  $data = $context['data'];
  $options = $context['options'];
  //Already replaced by other module.
  foreach ($tokens as $key => $token) {
    if (isset($replacements[$token])) {
      unset($tokens[$key]);
    }
  }

  // Support field/property tokens
  foreach ($tokens as $key => $token) {
    $token_args = explode(':', $key);

    if (count($token_args) ==4) {

      $field_name = $token_args[0];
      $langcode = $token_args[1];
      $delta = $token_args[2];
      $column = $token_args[3];
      $entity = isset($data[$type]) ? $data[$type] : NULL;
      //Support default language.
      if ($langcode == 'default') {
        global $language;
        if (isset($entity->{$field_name}[$language->language])) {
          $langcode = $language->language;
        }
        elseif (isset($entity->{$field_name}['und'])) {
          $langcode = 'und';
        }
      }
      if (isset($entity->{$field_name}[$langcode][$delta][$column])) {
        $replacements[$token]  = $entity->{$field_name}[$langcode][$delta][$column];
      }
    }
    if (TRUE) {
      // Conflict with token module.
      if (count($token_args) ==1) {
        $property = $token_args[0];

        $entity = isset($data[$type]) ? $data[$type] : NULL;

        if (isset($entity->{$property}) && is_string($entity->{$property})) {
          $replacements[$token]  = $entity->{$property};
        }
      }
    }
  }
}
g089h515r806’s picture

g089h515r806 commented

When i found compound_token module conflict with token module, usually I need to ask myself if there is a bug in my code first, then if there is a bug in token module, after that maybe i could suspect if there is a bug in drupal core.
I put a lot of print debug code in my code and in token.inc, i put print debug code in drupal core this is because i want to research the token system of drupal core.
When i found this issue, I read the debug message, I found that it is caused by Drupal core.

This is my first patch to drupal core, before i submit this patch, I have test a lot method, such as use 

if(!moduel_exist('token'))

to prevent this issue.
Use :

function compound_token_module_implements_alter(&$implementations, $hook) {
  if ($hook == 'tokens_alter' && array_key_exists('compound_token', $implementations)) {
    $group = $implementations['compound_token'];
    unset($implementations['compound_token']);
    $implementations['compound_token'] = $group;
  }
}

To let my code run at the last place.

After a lot of these effort, I think this is a bug in drupal core. I am not sure, because maybe i make a mistake in other place.

g089h515r806’s picture

g089h515r806 commented
Issue summary: View changes

typo error fixed

Revathi Manohar’s picture

Revathi Manohar commented

Hi,
i tested that issue in drupal 7 and i got same isuue after the patch is applied i got answer. i tested to print other token also working fine.output is like 123456:[node:nid|user:mail]8sprint token 8 is my nid,sprint token is node title
Thank you

robindh’s picture

robindh commented
Status: Needs review » Reviewed & tested by the community

#2 works like a charm!

I was trying to alter the node summary token, but could not get it to work. After a lot of debugging, I finally came to the conclusion that the alter hook went off after hook_tokens. Then I found this issue!

Marking as RTBC, since patch applies cleanly and fixes the issue.

poker10’s picture

poker10 commented
Category: Bug report » Support request
Status: Reviewed & tested by the community » Postponed (maintainer needs more info)

Thanks for the detailed report and patch.

I have gone through steps to reproduce and was not able to simulate it without using compound_token module. I have used vanilla D7.91 with token module installed.

1. Created a new node
2. Run the test code (see issue summary)

$node = node_load(1);
$output = token_replace('[node:nid|user:mail][node:nid][node:title]', array('node' => $node));

The output was:

[node:nid|user:mail]1tes test

And no warning message was present. This leads me to the fact that the root cause of this problem should be the compound_token module. If that is the case, it needs to be fixed there.

"$replacements += token_generate($type, $tokens, $data, $options);", and found it only run one time.

This is correct - token_generate() is run separately for each token type. And because the string you have presented ([node:nid|user:mail][node:nid][node:title]) contains only NODE type tokens (see how it is scanned by token_scan()), this is correct behavior. If you would use another user token separately (so that the token_scan() will discover that USER type token) then the function will run two times. Maybe in this case you will get the correct results without the warning.

Something like this:

$node = node_load(1);
$account = user_load($node->uid);
$output = token_replace('[node:nid|user:mail][node:nid][node:title][user:uid]', array('node' => $node, 'user' => $account));

But I have not experimented with the compound_token module now, because the module seems a bit outdated.

g089h515r806’s picture

g089h515r806 commented

My expectation:
1,run hook_tokens first
2,then run hook_tokens_alter

When I write compound_token module, I find the actual is:
1, compound_token_tokens_alter run first (even i remove the code "token_replace() in compound_token_tokens_alter").
2, then run token_tokens

I fix it in a dirty way at last, compound_token is compatible with drupal core and token module.

Steps to reproduce the issue:
Maybe you can write a custom module, that is used to change some token provide by token module, you implement:

mycustom_tokens_alter(){
...

//your logic code run before token_tokens, your code is useless, it is very strange.
}

hook_tokens_alter is not a commonly used hook, after several years later people use it again and find the same issue.

Status: Postponed (maintainer needs more info) » Closed (outdated)

Automatically closed because Drupal 7 security and bugfix support has ended as of 5 January 2025. If the issue verifiably applies to later versions, please reopen with details and update the version.