Need to improve the implementations of hook_token_values

If several modules make the same token keys available via the hook_token_values (as do Text and Number in CCK) then, when the token module calls module_invoke_all() to call all the hook implementations, it creates an array of values for that replicated key in its output array. This occurs through the call to array_merge_recursive() that it makes to put all the returned token key/replacement string arrays together.

Since this is supposed to be an array of string tokens and their replacement strings, this causes problems.

The problem I was seeing can be replicated by:

1. • installing token, http://drupal.org/project/token
   • installing pathauto, http://drupal.org/project/pathauto
   • installing filefield, http://drupal.org/project/filefield
   • installing filefield_paths, http://drupal.org/project/filefield_paths
   • enabling all CCK component modules such as number, text, nodereference ...
2. Creating a new content type that has one field of type File
3. When configuring that field, in the File name cleanup settings group within the Path settings group, select Cleanup using pathauto
4. When you create content of that type you will be able to but you'll receive a heap of warnings of type warning: mb_eregi_replace() expects parameter 3 to be string, array given which are generated by a function in pathauto which reasonably expects the output of the replacement value from the token module to be a string rather than an array.

The approach some of the CCK plugin modules have been taking is to prefix the token keys they provide with their module name to make them unique. If CCK followed that approach it would mean a few changes to the includes/content_token.inc file along the lines of:

From:

  function text_token_values($type, $object = NULL) {
    if ($type == 'field') {
      $item = $object[0];

      $tokens['raw']  = $item['value'];
      $tokens['formatted'] = isset($item['view']) ? $item['view'] : '';
      return $tokens;
    }
  }

to

  function text_token_values($type, $object = NULL) {
    if ($type == 'field') {
      $item = $object[0];

      $tokens['text-raw']  = $item['value'];
      $tokens['text-formatted'] = isset($item['view']) ? $item['view'] : '';
      return $tokens;
    }
  }

I've made these changes in my code and it seems to work. I haven't tried any more complicated testing that this test case though and I don't really know of other ways to test token functionality (very new to Drupal etc etc).

I've been using the latest official release for all modules.

Guessing I should have set it back to active after providing more info