What I've done is add the pathauto module to a live site with ~30,000 nodes on it. I started running some bulk updates, and they worked fine, adding aliases for non-aliased nodes, but after a while, it broke. I think what happened is that a user did something, there was an auto-alias created, and since then, none of the older nodes will get bulk-updated.

The bulk update *seems* to work, as in it returns:

* The configuration options have been saved.
* Bulk update of nodes completed, xxx aliases generated.

...with xxx being the number of nodes, typically 3 or 4 less than I've set the bulk update limit to - again, that seems fine. But checking the url_alias table, no actual new aliases are added.

I've tried looking for some sort of internal value for "last aliased node", but I can't see one. The reason I'm assuming it works like this is because even if I delete all the user-generated aliases from the url_alias table and then run the bulk update, it does the same thing - returns like it worked, but doesn't add any actual aliases.

Anyone got any suggestions on this one? It's driving me batty.

CommentFileSizeAuthor
#24 pathauto.inc__1.patch843 bytesantgiant
#16 pathauto.module_15.patch844 bytesantgiant

Comments

greggles’s picture

I recently changed that code, so it's not surprising that it doesn't work as expected.

It doesn't work on a system of "last aliased" nodes - it just selects a limited number each time of un-aliased nodes. In my tests it worked fine. I'll try to test this some more, but if you could try it out on a smaller test site under a variety of conditions that would be really helpful. It might lead to some specific settings that need to be a certain way to get this behavior.

greggles’s picture

I recently changed that code, so it's not surprising that it doesn't work as expected.

It doesn't work on a system of "last aliased" nodes - it just selects a limited number each time of un-aliased nodes. In my tests it worked fine. I'll try to test this some more, but if you could try it out on a smaller test site under a variety of conditions that would be really helpful. It might lead to some specific settings that need to be a certain way to get this behavior.

themselves’s picture

Hrm.

I've tested it on a test setup, and I can't get it to replicate this issue either. Damn, eh.

So on the problem machine, what I've done is stuck a watchdog line in to _pathauto_set_alias to see what was actually going on. Here's what i did...

  // Skip replacing the current alias with an identical alias
  if ($oldalias != $dst) {
    path_set_alias($src, $dst, $pid, 1, 10);
        watchdog("pathauto", "$src - $dst - $pid");
    if ($verbose and user_access('create url aliases')) {
      if ($pid) {
...

Just so it can dump the variables out. And this is what's landing in the watchdog...

pathauto 29/05/2007 - 15:04 node/14133 - baby/no-name-ye-821 - superuser
pathauto 29/05/2007 - 15:04 node/14171 - baby/no-name-ye-821 - superuser
pathauto 29/05/2007 - 15:04 node/14174 - baby/no-name-ye-821 - superuser
pathauto 29/05/2007 - 15:04 node/14175 - baby/no-name-ye-821 - superuser
pathauto 29/05/2007 - 15:04 node/14177 - baby/no-name-ye-821 - superuser
pathauto 29/05/2007 - 15:04 node/14185 - baby/no-name-ye-821 - superuser
pathauto 29/05/2007 - 15:04 node/14189 - baby/no-name-ye-821 - superuser
pathauto 29/05/2007 - 15:04 node/14193 - baby/no-name-ye-821 - superuser
pathauto 29/05/2007 - 15:04 node/14195 - baby/no-name-ye-821 - superuser
pathauto 29/05/2007 - 15:04 node/14209 - baby/no-name-ye-821 - superuser
pathauto 29/05/2007 - 15:04 node/14220 - baby/no-name-ye-821 - superuser
pathauto 29/05/2007 - 15:04 node/14234 - baby/no-name-ye-821 - superuser
pathauto 29/05/2007 - 15:04 node/14260 - baby/no-name-ye-821 - superuser
pathauto 29/05/2007 - 15:04 node/14268 - baby/no-name-ye-821 - superuser
pathauto 29/05/2007 - 15:04 node/14269 - baby/no-name-ye-821 - superuser
pathauto 29/05/2007 - 15:04 node/14286 - baby/no-name-ye-821 - superuser
pathauto 29/05/2007 - 15:04 node/14297 - baby/no-name-ye-821 - superuser
pathauto 29/05/2007 - 15:04 node/14309 - baby/no-name-ye-821 - superuser
pathauto 29/05/2007 - 15:04 node/14326 - baby/no-name-ye-821 - superuser
pathauto 29/05/2007 - 15:04 node/14332 - baby/no-name-ye-821 - superuser
pathauto 29/05/2007 - 15:04 node/14333 - baby/no-name-ye-821 - superuser
pathauto 29/05/2007 - 15:04 node/14337 - baby/no-name-ye-821 - superuser

It seems to be stuck! It's repeating $dst. Looking in the DB after this runs, node/14133 is the only one set in the url_alias table, the rest are gone - clearly the core path_set_alias module is working correctly and preventing duplicate aliases.

This looks likely to be the problem. Any ideas how it could be getting stuck like this?

greggles’s picture

Status: Active » Postponed (maintainer needs more info)

I'm not really sure what could be causing this so I'm marking it active (needs more info) since we can't repeat it. What are your update actions, alias and component max length, and the node pattern for this type?

themselves’s picture

Solved it!

It was the transliteration setting. I had it on since I started, and it clearly worked fine for the first 9,000 odd nodes, but then it got stuck. This is probably still a bug, as in it shouldn't be doing this, but yeah... at least now you know where to look ;)

themselves’s picture

For the sake of completion, the actual node title was 'No Name Yet!', and yeah, there's a lot of them in the DB with that title - hence why it was up to 823 at the time.

It looks like the bug is going to be some weird character translation thing. Previous nodes with this title were transliterating to (as an example):

baby/no-name-yet-236

However, according to the log, this one it was getting stuck on was transliterating to:

baby/no-name-ye-821

Snipping the final 't'? Weird, eh.

greggles’s picture

Title: Pathauto bulk update issue - stops working if users add a new node. » Pathauto bulk update stops working with lots of similar titles and/or transliteration

Well, this is certainly an interesting case ;)

Unfortunately it's not specific enough yet for me to be willing to spend time on it so I'm leaving it at "needs more info". If you can get a set of steps to repeat it quickly and consistently then that will go a long ways towards finding the solution. Otherwise we can just wait until someone else runs into this.

themselves’s picture

Uh oh. It's happening again.

This is with the transliteration *off* now. It's getting stuck on:

pathauto 30/05/2007 - 11:00 node/7842 - baby/no-name-yet-811 superuser
pathauto 30/05/2007 - 11:00 node/7847 - baby/no-name-yet-811 superuser
pathauto 30/05/2007 - 11:00 node/7863 - baby/no-name-yet-811 superuser
pathauto 30/05/2007 - 11:00 node/7870 - baby/no-name-yet-811 superuser
pathauto 30/05/2007 - 11:00 node/7876 - baby/no-name-yet-811 superuser

Again, it seems to be clipping off the last character of the string? It should be no-name-yet!-811. I'm gonna dive in to the module and see if I can dump out some debug info, try and work this out, but any inkling you may have as to where I should start looking would be greatly appreciated.

themselves’s picture

Ok, I think I've narrowed this one down for us. Here's the section of code...

  if (_pathauto_alias_exists($alias, $src)) {
    for ($i = 0; _pathauto_alias_exists($alias . $separator . $i, $src); $i++) {
    }
    // Make room for the sequence number
    $alias = drupal_substr($alias, 0, $maxlength - 1 -($i/10 + 1));
    $alias = $alias . $separator . $i;
  }

It's the drupal_substr line. It seems to be snipping off the last character for inserts that are failing, and I'm not exactly sure why it's necessary. If I comment it out, the alias adds just fine. Is there any special reason for why this line is there?

greggles’s picture

As the comment says, it does this to ensure that there is enough space for the sequence number. The sequence number should ensure that the alias is unique. And then pathauto uses the drupal_substr function to reduce the length of the alias by the length of the sequence number. This way if someone has a really large number of nodes that have the same title (like you ;)) then when they have an alias like 'asdf-1234567890' they won't have to worry about it overflowing the length of the dest column in the database or being longer than their max component length.

The question to me is why it keeps trying with 811 even though (presumably) that alias already exists. Why does _pathauto_alias_exists return what seems to be the wrong result... ?

themselves’s picture

Ok, I've solved it. It actually is a problem with that line.

What it's doing is subtracting the *value* of the incrementor from $maxlength, not the length of the digit as a string, thereby shortening the alias well before it actually needs to be and making a general nuisance of itself. I would suppose the line should probably be...

$alias = drupal_substr($alias, 0, $maxlength - 1 -(strlen($i) + 1));

I've tested this change on my setup and it seems to work fine. Your thoughts?

greggles’s picture

Status: Postponed (maintainer needs more info) » Fixed

I think this should be fixed now in the .dev build. I used a slight modification since -1 and +1 will balance each other out - right?

Anonymous’s picture

Status: Fixed » Closed (fixed)
antgiant’s picture

Status: Closed (fixed) » Active

Sorry to reopen but the problem is not fixed, even in head.

Here is a description of the problem

Original Code

  // If the alias already exists, generate a new variant
  $separator = variable_get('pathauto_separator', '_');
  if (_pathauto_alias_exists($alias, $src)) {
    for ($i = 0; _pathauto_alias_exists($alias.$separator.$i, $src); $i++) {
    }
    // Make room for the sequence number
    $alias = drupal_substr($alias, 0, $maxlength - 1 -($i/10 + 1));
    $alias = $alias.$separator.$i;
  }

Correct Code

  // If the alias already exists, generate a new variant
  $separator = variable_get('pathauto_separator', '_');
  if (_pathauto_alias_exists($alias, $src)) {
    for ($i = 0; _pathauto_alias_exists(drupal_substr($alias, 0, $maxlength - 1 -($i/10 + 1)).$separator.$i, $src); $i++) {
    }
    // Make room for the sequence number
    $alias = drupal_substr($alias, 0, $maxlength - 1 -($i/10 + 1));
    $alias = $alias.$separator.$i;
  }

The Important part

    for ($i = 0; _pathauto_alias_exists($alias.$separator.$i, $src); $i++) {

    for ($i = 0; _pathauto_alias_exists(drupal_substr($alias, 0, $maxlength - 1 -($i/10 + 1)).$separator.$i, $src); $i++) {

The problem is that the for loop does not respect the maxlenth parameter. This creates a situation were very long urls pass the for loop but fail the actual insertion. It is agravated by the way it fails causes pathauto to claim it did not fail.

Hope that helps.

greggles’s picture

@antglant - Can you create this as an actual patch file? code in issues is hard to follow.

See http://drupal.org/patch/create for details.

antgiant’s picture

Status: Active » Needs review
StatusFileSize
new844 bytes

See Attached

greggles’s picture

@antglant - I just got around to looking at this - sorry for the delay.

Your patch is against the 5.x-1.2 branch of pathauto. This issue is about 5.x-2.x. Can you repeat the problem there? If so then can you update your patch against the 5.x-2.x branch?

Also, please let me know the exact steps to repeat the problem so that

greggles’s picture

Status: Needs review » Needs work
antgiant’s picture

I'd be happy to create a patch against the 5.x-2.x branch. However, I can't figure out how to get to it via CVS. So if you can tell me where to find it in CVS I'll make the patch.

As for duplicating the problem.
It occurs when there are three or more pages with a title that is the identical under the maxlenth parameter.

Consider this simplified case

function pathauto_create_alias($alias, $src, $separator, $maxlength) {

  ...

  // If the alias already exists, generate a new variant
  if (_pathauto_alias_exists($alias, $src)) {
    for ($i = 0; _pathauto_alias_exists($alias.$separator.$i, $src); $i++) {
    }
    // Make room for the sequence number
    $alias = drupal_substr($alias, 0, $maxlength - 1 -($i/10 + 1));
    $alias = $alias.$separator.$i;
  }

  ...

}

pathauto_create_alias("A Page", 1, "", 1);
//This creates path A

pathauto_create_alias("A Second Page", 2, "", 1);
//The for loop thinks this is creating "A Second Page0"
//"A0" is actually created

pathauto_create_alias("A Third Page", 3, "", 1);
//The for loop thinks this is creating "A Third Page0"
//"A0" is what it actually tries to create.  However,
//"A0" already exists so there is a silent failure.  
//Pathauto interpets the silent failure as success.

greggles’s picture

Ok, here is what I found:

0) basically any pattern will work for this
1) Set "Maximum alias length:" to 1
2) create 3 new nodes with the same title

Expected results:
3rd node gets the alias -1

Actual results:
3rd nodes gets the alias -0 while the second node loses its alias

Is that the problem you describe? If not, please use this format for describing the steps to repeat.

I look forward to the patch - thanks.

antgiant’s picture

I need instructions on how to download 5.2 from CVS to create a patch.




A few minor updates to your description

0) basically any pattern will work for this as long as it is longer than "Maximum alias length:"
1) create 3 new nodes with the same title

Expected results:
3rd node gets the alias -1

Actual results:
3rd node fails to get any alias.

antgiant’s picture

I suppose I should be extra clear.

I created my patch against the CVS head avaliable at http://cvs.drupal.org/viewvc.py/drupal/contributions/modules/pathauto/. Apparently 5.2 is located elsewhere. Please let me know where so that I can create the patch.

greggles’s picture

It depends on which cvs client you use - in most you select the "BRANCH" to be "DRUPAL-5--2"

on the command line:

cvs -d :pserver:anonymous:anonymous@cvs.drupal.org:/cvs/drupal-contrib co -d pathauto -r DRUPAL-5--2 contributions/modules/pathauto

antgiant’s picture

Status: Needs work » Needs review
StatusFileSize
new843 bytes
greggles’s picture

Status: Needs review » Fixed

I've applied this to the 5.x-2 branch. Thanks, antgiant!

Anonymous’s picture

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.