Hi,
I've been having a problem where I cant create more aliases than the amount set in pathauto_max_bulk_update. Doing some digging in the code, I [think] I've found a problem with the SQL in the node_pathauto_bulkupdate() function defined in pathauto_node.inc.
The line reads like this:
$query = "SELECT nid, type, title, uid, created, src, dst, vid FROM {node} LEFT JOIN {url_alias} ON CONCAT('node/', CAST(nid AS CHAR)) = src WHERE src IS NULL ". $type_where;
The problem is that the CAST(nid AS CHAR) part of the SQL is truncating the node id to 1 character, so the two tables never link and the same rows are returned every time the bulk update is run.
Eg: If the nodes id is 1234, the above SQL essentially comes out joining on 'node/1' = 'node/1234'.
By changing the type to VARCHAR(10) I have managed to get my bulk updates to work as they should.
Comment | File | Size | Author |
---|---|---|---|
#26 | 319070-pathauto-concat-fixes.patch | 6.43 KB | Dave Reid |
#23 | 319070-pathauto-use-native-concat.patch | 3.5 KB | Dave Reid |
#20 | 319070-pathauto-cross-db-concat.patch | 6.9 KB | Dave Reid |
#17 | 319070-pathauto-cross-db-concat-D7.patch | 6.58 KB | Dave Reid |
#15 | pathauto-319070-6-1.x.patch | 4.1 KB | tuffnatty |
Comments
Comment #1
gregglesI'm not sure exactly what change you're asking for - could you provide a patch http://drupal.org/patch ?
Comment #2
DrDeth CreditAttribution: DrDeth commentedSure thing.
I've attached a patch for the file.
Hope this helps.
Comment #3
gregglesThis seems fine to me. Which database do you use?
I assume you use PostgreSQL and I'm hesitant to fix this since this has basically no reviews and the original code was reviewed pretty heavily in #212327: cast the concatenated join value to be able to use the url_alias index.
Comment #4
Dave ReidMarked #328178: Postgresql 7.4 Bulk Update as a duplicate of this issue.
Comment #5
Josh Waihi CreditAttribution: Josh Waihi commentedfind attached a fix. Have fixed the error in the patch from #373841: Use update action for all existing aliases, not just the current existing alias
Comment #6
Dave ReidTagging all the bulk alias issues for #713238: RFC: Pathauto Bulk module.
Comment #7
JoelKleier CreditAttribution: JoelKleier commentedI can verify this is an issue for PostgreSQL db's -- we fixed our install with the attached patch, but the patch in #5 looks like it may be more db agnostic.
Comment #8
Josh Waihi CreditAttribution: Josh Waihi commentedI'm RTBC'ing #5 on the bases that this problem is a postgres problem and that the changes fix postgres and don't break mysql.
Comment #9
Dave ReidDoes this patch need to apply to the pathauto_taxonomy.inc and pathauto_user.inc? They seem to have the same things going on:
If so, we'll need an updated patch.
Comment #11
Josh Waihi CreditAttribution: Josh Waihi commentedyes.
Comment #12
progga CreditAttribution: progga commented#11 has worked fine for me on Postgres 8.3. I have only used it for creating url aliases for nodes. Thanks for the patch.
Comment #14
jamespharaoh CreditAttribution: jamespharaoh commentedCasting to char(99) should work in both databases. Patch against HEAD attached.
Comment #15
tuffnatty CreditAttribution: tuffnatty commentedRebuilt the last patch against HEAD. Also changed CHAR(99) to CHAR(20) although I don't know if it will give us any significant speedup.
Comment #16
AlexisWilke CreditAttribution: AlexisWilke commentedtuffnatty,
In PostgreSQL, the fastest is TEXT, not CHAR(<whatnot>);. It is specified in the docs, check out the first Tip on that page:
http://www.postgresql.org/docs/8.0/interactive/datatype-character.html
It would be good to always use TEXT and not bother with CHAR() at all. It has generated many problems in Core (aggregator URL too long and watchdog strings too long for two that I recall.)
Thank you.
Alexis
Comment #17
Dave ReidHow about a function that will return the proper concatenation depending on the database type being used? This seems like a much better and future-proof method.
Comment #18
Dave ReidI've tested #17 with success using MySQL 5.1.37 and PostgreSQL 8.4.4. Still needs testing with lower versions of PostgreSQL and also with SQLite in D7.
Comment #19
Dave ReidTested and it works great with sqlite with Drupal 7. Still could use someone testing on PostgreSQL below 8.3. Please report back your results/success.
Comment #20
Dave ReidSince I've tested this on all possible database types for D7, I'm committing #18 to this branch. The following patch still needs testing as per #19.
http://drupal.org/cvs?commit=377544
Comment #21
Damien Tournoud CreditAttribution: Damien Tournoud commentedThat doesn't look correct. On MS SQL, '+' is both the concatenation operator and the addition operator, and doesn't do automatic casting. You need an explicit cast here if you want to be able to use non-text columns in the concat.
That's one of the rational behind *not* using || in Drupal 7, see #809698: Refrain from using the || operator for concatenation.
Comment #22
Damien Tournoud CreditAttribution: Damien Tournoud commentedLet's get back to the old code.
Our own PostgreSQL's CONCAT() function has been fixed, and now properly accept any non array type as arguments. This abstraction is thus completely unecessary, and actually breaks Drupal on SQL Server (because using || *requires* a cast, while our CONCAT() function doesn't).
Comment #23
Dave ReidAgreed, especially since we removed support for feed aliases, so now we never need to concat more than two strings.
Comment #24
Dave ReidSo we can remove this in D7, but we still have to do something about this in the D6 versions, as this is affecting critical functionality.
Comment #25
Dave ReidCommitted #23 to 7.x-1.x: http://drupalcode.org/project/pathauto.git/commit/45784f0
Comment #26
Dave ReidHere's the patch for 6.x-2.x.
Comment #27
Dave ReidIn #26 we removed the support for MSSQL for 6.x since I don't think I can support it. I tested it with both MySQL and PostgreSQL.
Committed to 6.x-2.x: http://drupalcode.org/project/pathauto.git/commit/4aff481
Comment #28
Dave ReidCommitted to 6.x-1.x as well as fixed a small bug in the committed code with the forum aliases in 6.x-2.x
http://drupalcode.org/project/pathauto.git/commit/7af542f
http://drupalcode.org/project/pathauto.git/commit/084dbd4