We tried to import nodes in a loop using drush, but the token cache was breaking the ANT output. The titles for successive nodes were the same.

The problem was that when insert more than one node (in a loop), the title for all subsequent nodes was built as the same title as the first node for all subsequent nodes

It was caused by _auto_nodetitle_patternprocessor invoking token replacement without flushing the token cache, although the token values were being built from $node, meaning they change at each new node being created

function _auto_nodetitle_patternprocessor($output, $node) {
  if (module_exists('token')) {
    token_get_values('global', NULL, TRUE); // Flush cache in case the node changed
    $output = token_replace($output, 'node', $node);
  }
...
}

on a normal page cycle, only one node gets created at once, so there is no need to flush the cache because it is only used once anyway, but in a mass operation, the node changes every time, so the tokens values have to be flushed. Hence the addition of token_get_values('global', NULL, TRUE); right before the token_replace() call.

Comments

agileware’s picture

Status: Active » Patch (to be ported)
StatusFileSize
new2.62 KB

I have also come across this problem when running a node import that calls auto_nodetitle_set_title() to set the node title. Whe multiple nodes are imported in a row the titles for all nodes are the same as the first one that is imported.

I have made a patch for the 5.x-1.11 version of the token module, which is here - http://drupal.org/node/262360#comment-1288822

I have attached a patch this problem for the 5.x-1.2 version of auto_nodetitle but it won't work until the patch I made for the token module is applied.

fago’s picture

Status: Patch (to be ported) » Closed (works as designed)

I don't think it makes much sense to clear the token cache each time when ant operates - as this would just result in not using it. However the token caching approach obviously fails for new nodes, which is odd. So I think this is somehow "by design" (as token caches that way), but node-import could work around it by clearing the token cache before importing a node.

takinola’s picture

There's a simple workaround using the Rules (or Workflow-ng) module

See http://drupal.org/node/360359#comment-1460918

niklp’s picture

You want me to install Rules to get round a problem in token?? I don't think so...

agileware’s picture

Status: Closed (works as designed) » Patch (to be ported)

@fago,

The way this works is that auto node titles calls token_replace() which in turn calls token_get_value().

token_get_value() uses a static variable of tokens and takes in a $flush parameter to determine whether or not to reset the static variable. So it's not really a cache that you can clear from another module....
Unless of course you make token_replace have the same $flush variable and pass it to token_get_values. Then other modules (such as this one) can call token_replace with the $flush variable.

That is what this patch (and the token one I linked to in #1) achieve.

Also, as you say it doesn't make sense to clear the cache every time but this patch doesn't do that. The base functionality of the module after this patch would be the same. It just gives the opportunity to flush the cache if desired.

It isn't anything major and won't break any existing sites that call these functions so I don't see why this couldn't be done. You really shouldn't have to install another module to achieve this.

agileware’s picture

Status: Patch (to be ported) » Needs review
StatusFileSize
new2.36 KB

Here is a D6 version of the patch.

It works for 6.x-1.1 and the current dev version (2009-Mar-27).

fago’s picture

Status: Needs review » Closed (works as designed)

I know how the token cache works - if you flush it everytime you use token you bypass it.
You can already manually flush the cache by just calling respective token function.

So I see this is odd when mass-importing nodes, but I see no other way to fix this case by manually clearing the token cache after/before each import.

agileware’s picture

Because if you mass import nodes and call auto_nodetitle_set_title() for each one as you go without flushing the cache then you get the title of the first node for every node after.

Why would you want that to happen?

agileware’s picture

And if there is no other way then why can't we do it this way?

niklp’s picture

That's exactly the problem we had in the first place. Can't we just apply a fix? :)

tylerwalts’s picture

I just came across this thread while searching for a solution to the very same problem. I have a module that is used to create a new content type, then populate it using content pre-existing in the site as nodes. I have a function which contains a for loop for every node that I want to import, such as this:

// Query source data
$ret = db_query($query_string);
// For every source record, create a new node
while ($row = db_fetch_object($ret)) {
        $node = new StdClass();
        $node->type = 'my_type';
        // assign node owner 
        $node->uid = $row->uid;
        // assign some custom cck field values from source data
        $node->field_myfield[0]['value'] = $row->myfieldsource;
        // ...
        // Do NOT assign the title, as we have the content type configured to Auto-Title based on token
        // Save the node
        node_save($node);
    }

The result is that everything works except all the titles are the same = the very first node title to be auto-processed. :( Would be nice if this worked out of the box, without patch. I just applied the patch, but it did not work (just to verify, I just put the .patch file in the auto_nodetitle module folder and run 'patch < auto_nodetitle_token_flush_D6.patch', right?) Even if I get this to work in the morning after a second look, I'm concerned that next time I do a 'drush update' then it will over-write the patch.

This instance is based on Drupal 6.x

Many thanks,

- Tyler

agileware’s picture

A couple of things might be causing you problems.

For one, make sure you have also applied the token patch, which is here - http://drupal.org/node/262360#comment-1583428

Secondly, I don't let node save do the auto nodetitling.

In my code I have:

auto_nodetitle_set_title($node, TRUE);

before the node_save.

Note that this is with drupal 5. I'm not sure if the auto_nodetitle_set_title function has changed at all in it's usage in drupal 6.

bshensky’s picture

Justin:

My fresh install of auto_nodetitles for D6 shows no second parameter for auto_nodetitle_set_title.

I'm with with folks who deliberately call token_get_values('global', NULL, TRUE) when performing batch inserts. There's a good reason I have auto_nodetitles enabled in batch mode - we have a combination of UI and batch inserts, so we need it enabled at all times, even if it means we have to program around the discrepancy in batch mode.

-brian

agileware’s picture

@bshensky:
If you are referring to the flush parameter in the patch, the reason it is not there is because this patch has not been committed yet. You will have to apply the patch in comment #6 until this is committed.

If you want to help get this committed test the patch and let us know if it fixes your problem.

agileware’s picture

Status: Closed (works as designed) » Needs review

The patch in the token module that this patch relied on has now been committed ( see http://drupal.org/node/262360#comment-2090972 ) so I am reopening this.

I have tested the patches in #1 & #6 on the most recent dev versions and they still apply cleanly.

fago’s picture

Status: Needs review » Closed (works as designed)

However still my comments in #7 apply.

cyberwolf’s picture

Subscribing.

agileware’s picture

There is no point subscribing as this will not be committed.
See #16 & #7.

I still don't know why this can't be committed as there are no negative side affects and the required patch in token was committed so if you want this feature post here your reasons and if enough people want it maybe something can be done.