Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
It would be great to add some drush commands for bulk updating and deletion.
Comment | File | Size | Author |
---|---|---|---|
#68 | after-apply-patch.png | 67.45 KB | amit.drupal |
#62 | pathauto-add-drush-support-867578-62.patch | 4.29 KB | D2ev |
Comments
Comment #1
msonnabaum CreditAttribution: msonnabaum commentedHere's a command I wrote a while back for pathauto 6.x-1.x which may help move things forward:
https://gist.github.com/940710
It could use a little cleanup and I think there's a bug with the reporting, but it worked great when I used it after a big migration. I stole the drush_backend_invoke() technique from migrate module, which works quite well to avoid memory limits.
Comment #2
Dave ReidIs there docs on how Drush can run a batch API process? Because that's what we use now.
Comment #3
msonnabaum CreditAttribution: msonnabaum commentedWe have our own implementation of batch, which is used for update.inc:
http://drupalcode.org/project/drush.git/blob/HEAD:/commands/core/drupal/...
It should accomplish the same as my example above.
Comment #4
Dave ReidComment #5
delta CreditAttribution: delta commentedHello,
i use this implementation actually on my project, for the moment i tested it just with 1500 alias max (node,user)
it implements two command :
pathauto-bulk-update all OR (node,taxonomy_term,user,other_modules...)
pathauto-delete-alias all OR (node,taxonomy_term,user,node/,other/default/path/,..)
the code contains enough comment and it's directly inspired from the actual pathauto code, actually this code use the batch from the pathauto code.
Comment #7
delta CreditAttribution: delta commentedGit diff it's maybe better
Comment #9
colanNote for later: Once this gets finalized, we need to update the documentation page.
Comment #10
colanI found some problems. Here are the changes with this new patch:
Things seem to work, but recreating the nodes on my site takes hours, so will report back if there are problems. Deleting and creating everything else works fine.
EDIT: Turns out it only took 20 minutes with no problems. From the UI, it took several hours.
Comment #11
Alex Andrascu CreditAttribution: Alex Andrascu commentedGo go go :). Get it in .dev. Need this badly !
Comment #12
colan@Alex: This process would go much faster if you helped test the patch, and/or provide a code review, and then report back. Comments like that really don't help speed things along.
Comment #13
Alex Andrascu CreditAttribution: Alex Andrascu commentedOK @colan.
You'll be happy to know then, that patch applied successfuly against 7.x-1.x-dev and works as intended with drush version 4.5.
Don't get so dramatic :)
Comment #14
colanThere's something missing here. Translations (with Entity Translation) aren't being updated, even though this works perfectly well from the UI. See #1155134: Integrate pathauto bulk generation for details. I'll try to figure out what's going on.
@Alex: Dave Reid has ranted about getting spam in the past. I'm just trying to keep you on his good side. ;)
Comment #15
Dave Reid1. I think it would probably be better to present the user with a list of options for bulk generation or deletion (like the drush cache-clear command) that also works with an argument if you provide it. Also like the drush cache-clear command I'd also not worry about multiple types, but an all or only one single at a time. Should make this a bit easier.
2. Don't need to add pathauto.drush.inc to pathauto.inc as that file doesn't contain any classes for the D7 class registry. Let's back out that change.
Comment #16
Dave ReidComment #17
Dave ReidThat said, I definitely would love to see this feature land. Can we also add unit tests for Drush commands provided by a module at all? Anyone know?
Comment #18
colanHere is a list of things that I can get done in the immediate future:
Comment #19
Dave ReidI'd rather the create command keep with the current logic of the bulk alias batch operation and only work on unaliased items only and not also attempt to update existing content with aliases as well. We can consider that a separate feature change and improvement to the bulk generation after this patch lands.
Comment #20
Dave ReidIn fact we already have an issue for that: #1003490: Allow users to re-alias entities with the Bulk update page
Comment #21
colanUntil a few minutes ago, I thought the bulk update from the UI was using that setting, but it's not. So yes, I agree.
Here's a patch that:
The problem with the second one was that only one module's settings were being kept for each type. So Entity Translation's (ET's) override settings for node were being clobbered by those from node itself. I've changed it so that settings for each type are only being recorded once. If they already exist, they don't get set again. This works because ET's settings get added to the settings array before node's. (I'll admit I don't really understand why. ET's weight (11) is higher than Pathauto's (1) which contains the settings for node.)
I changed:
...to:
If I understand Dave's comments correctly, we still need to add the following:
Comment #22
colanComment #23
colanComment #24
delta CreditAttribution: delta commentedThank you for your help, we need this feature, but i don't know how to make a patch in a good (drupal) way. so thank you for your help. I'm gone a test the latest patch and let you know.
Thx
Comment #25
Dave ReidHere's my effort to get this working as 'pathauto-aliases-generate' which also uses the same options as the tab in the pathauto UI, offers the user the ability to select a type, or provide it via the command. I've left off the delete callback as I'd really like to have #1359306: Bulk delete "Users" should not delete aliases for user/register, user/login or user/password land first, then add support for bulk delete via the batch operations.
Comment #26
delta CreditAttribution: delta commentedYour code seems to be very clean as if we not pass the parameters, he add a list of choices of types of alias pathauto are going to recreate. We need to merge the last patch from colan #21
with #25. Are you agree with that ? and do you want to add a choices of types too for the delete command ?
Comment #27
_redfog CreditAttribution: _redfog commentedHello everyone.
I applied the path new_drush_commands_to_create_and_delete_url_aliases-867578-21.patch and 867578-pathauto-drush-generate.patch but independently.
For the first one, I launch the command:
drush pathauto-aliases-create
and for the second one, I tried this:
drush pathauto-aliases-gerenate
Commands are executed, but end up quickly. I checked some url, nothing happened. Someone else tried ?
Pauthauto version I use: 7.x-1.1
Comment #28
colanComment #29
DuaelFrAdded delete support on the David's patch basis because of a personal and urgent need.
Use with caution.
Comment #30
make77 CreditAttribution: make77 commentedI updated patch given in #29 as it didn't work with pathauto 7.x-1.2.
Comment #31
make77 CreditAttribution: make77 commentedComment #32
perkeJust tested the patch in #30 on bulk delete and update on ~25k nodes and it works perfectly.
Only issue was that job was failing due to exceeding maximum execution time of 120 seconds but that's related to my particular setup, so I had to run the command 3 times to get everything updated.
Thanks
Comment #33
Mac_Weber CreditAttribution: Mac_Weber commentedadding an argument to limit the number of nodes to be processed would fix the issue related on #32
Comment #34
MiroslavBanov CreditAttribution: MiroslavBanov commented#30: pathauto-add-drush-support-867578-30.patch queued for re-testing.
Comment #35
SGhosh CreditAttribution: SGhosh commentedPatch at #30 tested. Deleted and generated 72 aliases at one go. Works awesome!
Comment #36
Mac_Weber CreditAttribution: Mac_Weber commentedIt seems good but I think it would need a bit more work to address scenarios on comment #32, maybe as stated on #33
Comment #37
Dave ReidI'm surprised the drush batch processing doesn't automatically handle that for us? Is there something we're not doing right in invoking the batch?
Comment #38
pwaterz CreditAttribution: pwaterz commentedI dont think the command should print out output each time it updates. Writing to stdout can slow down the command. I think it should be a percentage update.
Comment #39
emilyf CreditAttribution: emilyf commentedI tried patch #30 on 7.x-1.2 and it applied cleanly but then when I issued either command
drush pathauto-aliases-generate
drush pathauto-aliases-create
I get:
Fatal error: Cannot redeclare redirect_drush_command() (previously declared in /sites/all/modules/redirect/redirect.drush.inc:13) in /sites/all/modules/redirect1/redirect.drush.inc on line 24
Drush command terminated abnormally due to an unrecoverable error. [error]
EDIT: That error was just my own stupidity with a redirect issue. THE PATCH APPLIES AND WORKS. Running drush pathauto-aliases-generate then gives me the option to choose what I want to generate.
Comment #40
emilyf CreditAttribution: emilyf commentedAnyone have ideas on why it might be extremely slow?
Just doing about 20 nodes took more than 20 minutes...
drush pathauto-aliases-generate
Enter a number to choose which entity types to bulk generate aliases.
[0] : Cancel
[1] : all
[2] : node
[3] : taxonomy_term
[4] : user
2
Updated alias for node 14893. [ok]
Updated alias for node 14918. [ok]
Updated alias for node 14943. [ok]
Updated alias for node 14968. [ok]
Updated alias for node 14993. [ok]
Updated alias for node 15018. [ok]
Updated alias for node 15043. [ok]
Updated alias for node 15068. [ok]
Updated alias for node 15093. [ok]
Updated alias for node 15118. [ok]
Updated alias for node 15143. [ok]
Updated alias for node 15168. [ok]
Updated alias for node 15193. [ok]
Updated alias for node 15218. [ok]
Updated alias for node 15243. [ok]
Updated alias for node 28918. [ok]
Updated alias for node 28943. [ok]
Updated alias for node 28968. [ok]
Updated alias for node 28993. [ok]
Comment #41
DuaelFr@emilyf : This article contains some performance tips it is a bit outdated but it could help you.
Comment #42
saurabh.arya CreditAttribution: saurabh.arya commenteddrush pathauto-aliases-generate is not working for me.I have large number of node,any other solution?
Comment #43
Owen Barton CreditAttribution: Owen Barton commentedI think this looks good - confirmed the Drush batch usage is correct. Attached patch just fixes a doxygen typo.
The parent php process is still subject to php timeout constraints (the batch backend calls are still useful in that they limit excess memory usage). In theory we could set an unlimited timeout in the callback, but I think this is a bad idea - if this is being run on cron and something goes wrong (e.g. infinite loop somewhere) it will kill the server - best to leave that decision to the site/sysadmin. This can be changed in any systemwide php.ini, or by the user via Drush drush.ini/DRUSH_INI, or via $drush_php_override/PHP_OPTIONS.
In terms of the script output, I agree that might be improved, but I don't think is a performance factor (writing a line of terminal output is minute relative to loading a node), so would suggest it is better to do in a followup patch.
Comment #44
thermador CreditAttribution: thermador commentedApplied the patch, got it working, but regardless of my selection:
... I always get "No New URL aliases to generate."
Do I really have to *delete* all of the old URL aliases before I generate new ones?
Can't I just *update* the aliases to a new replacement pattern (specified in: /admin/config/search/path/patterns) without deleting the old aliases?
I don't want to break all my links, and the redirect module don't seem to pick up on the fact that the aliases are changing (even though it is supposed to, I have the box checked "Automatically create redirects when URL aliases are changed."), so deleting an alias == broken link.
I think I'm probably just going to have to update my URL aliases 50 at a time from admin/content/.
Pathauto is used by over half a million sites, one of the most widely used modules (it should be in the core) and yet there still isn't any simple upgrade path from Drupal 6 -> Drupal 7...
Comment #45
jlporter CreditAttribution: jlporter commentedPatch from #43 works for me. It's beyond time to merge this so it will get used and further user testing can be done. Low risk to release as only drush users will be affected. I was shocked this didn't exist already.
Comment #46
annya CreditAttribution: annya commented#43 works great for me. I update this issue, cause we really should to implement drush integration in pathauto.
Comment #47
kenorb CreditAttribution: kenorb commentedTested and it works fine.
Testing path alias change via:
Works.
However these seems to inconsistent (capital and lower-case names):
Comment #48
DamienMcKennaI suggest including this in the 1.3 release.
Comment #50
AnybodyThanks a lot, this patch works just fine and is quite important. Is it possible to get this into the next release soon?
Comment #51
bewilled CreditAttribution: bewilled commentedIn regards to the performance issue mentioned in #40 the following query shows how to generate url aliases at a database level for nodes in Postgresql:
replacing %your_path% and '%your_content_type% with their respective value.
What this basically does is select the titles from the node table, trim off all non alphanumeric characters(except spaces), then replace all spaces with the '-' char. If there happens to be a two titles that are equal, then it appends a number that increases as many times as there are duplicates to the end of the url.
Use at your own risk.
Comment #52
doitDave CreditAttribution: doitDave commentedSubscribing, can #43 please make it into 1.3, and more important: Can 1.3 pls make it soon into the public? ;) (Any plans on this?)
Comment #53
DamienMcKenna@doitDave: We need a) code review on several pending issues, possibly a few needing feedback, b) a maintainer / new co-maintainer who has the time to deal with it.
Comment #54
Media Crumb CreditAttribution: Media Crumb commentedDid this ever go anywhere?
Comment #55
DamienMcKenna@Media Crumb: The issue is still at RTBC, therefore it has not proceeded anywhere in the past two years.
Comment #56
Media Crumb CreditAttribution: Media Crumb commentedCrazy,
I've been using it for a while now and have had not real issues with it. Seems a shame not to move it closer to a commit.
Comment #57
doitDave CreditAttribution: doitDave commented@damien #53 sorry, somehow I did not receive the "updated" flag but found it updated accidentally today.
Is there still something to be done? Can I assist?
Comment #58
DamienMcKenna@doitDave: If you think you have the time to contribute to this module, going through the issue queue and reviewing patches would be the best step forwards, then send Dave a message offering to comaintain the module.
Comment #59
Dave ReidI think the default bootstrap is fine?
Is this really needed?
Trailing space here.
The $form variable is not always defined by the time it gets here and may cause a PHP notice.
Comment #60
Dave ReidComment #61
amad16 CreditAttribution: amad16 commentedHello i take it that this patch is not included in version 7.x-1.3
Comment #62
D2ev CreditAttribution: D2ev commentedUpdated patch with issues addressed in comment#59.
Comment #63
kenorb CreditAttribution: kenorb commentedComment #64
Chris CharltonBump. :)
Comment #65
joelpittet@chris charlton if you've tried the patch and it does what it says in the issue summary, feel free to set the issue status to "Reviewed and tested by the community" aka RTBC. That will help the maintainer out and they may commit it from there.
Comment #66
Anybody#62
Comment #67
fubarhouse CreditAttribution: fubarhouse commentedI've just tested #62 and I love it - any chance we can get this pushed to release?
Comment #68
amit.drupal CreditAttribution: amit.drupal as a volunteer and at gai Technologies Pvt Ltd for gai Technologies Pvt Ltd commentedTested #62 and it works fine.
After Apply Patch image upload.
Comment #69
joewhitsittTested #62 with taxonomy terms. Worked as expected.
Comment #70
Chris CharltonWe're up to four verified RTBCs. :)
Comment #71
MustangGB CreditAttribution: MustangGB commentedAny maintainers around?
Comment #72
Dries ArnoldsTried the patch in #62 and it works as expected.
Comment #73
vasikei would prefer another approach here that the one is in the UI currently.
Where i could have a "full" (force) update of the existing path aliases which togheter with Redirect will make no harm to a production env.
Also it would be nice if there is also a away to have this update per entity bundle.
For example i want to force the update of the current aliases for "Product display" content type only.
Comment #74
Chris Charlton@vasike, would two phases of this feature make sense? This ticket has been open since 2010, and the patch above is a good start.
We could openanother ticket for the additional features you're suggesting.ADDITIONAL FEATURE REQUEST TICKET OPENED HERE: #2911862: Enhance pathauto bulk Drush commands
Comment #75
Chris Charlton(anyone know why a D8 ticket #2717721: Drush commands for bulk alias updates is referenced in this D7 ticket?)
Comment #76
Chris CharltonBump? (RTBC)
Comment #77
Ronino CreditAttribution: Ronino as a volunteer commented#62 works great!