Hello pathauto users & developers.

I am creating this issue to get the ball rolling on pathauto for Drupal 7. I plan to try my hand at a patch against 6.x-2.x-dev that will get pathauto working.

My Strategy:

My general strategy is going to mimic the efforts going on over in Views land. I have copied the list from the Converting 6.x modules documentation (http://drupal.org/node/224333) into a word doc and will be going down the list and ticking off each api change as I make it. If someone has an idea of how to best share this list, please let me know.

Since I will be generally tackling the porting process in order of the core commits, some patches will likely contain some code where I have fixed one thing but not another. For example, I just started tackling db conversion tonight and have converted some queries to dynamic queries that SELECT from the '{term_node}' table. Obviously, this table no longer exists but for the sake of my sanity I'd rather get the "A completely new database API has been added" issue solved before I worry about other api changes. In the future just the table may change for this example query, or maybe the query will change altogether.

Having said that, I am a relatively new to the patching/upgrading game so if this is totally the wrong way to go about it, please let me know the more standard alternatives.

Any new code comments I make will have '@te-brian:' appended so we can re-word or clean them up as needed later.

Getting Started

I plan to spend the next two days working on this as much as possible (learning more about pathauto than I ever wanted to in the process). I'll post a patch with my progress at that point and people are more than welcome to jump on board and tackle a specific issue.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

te-brian’s picture

In implementing the change from hook_user('delete') to hook_user_cancel() I need some advice/clarification.

How do we want to handle the different methods?

Here is my initial thought:

  • 'user_cancel_block': Don't remove aliases
  • 'user_cancel_block_unpublish': Don't remove aliases
  • 'user_cancel_reassign': Change aliases to something appropriate for anon user?
  • 'user_cancel_delete': Remove Aliases
greggles’s picture

I think user_cancel_reassign should delete the alias. Otherwise, looks good to me.

te-brian’s picture

Alright, good news and bad news.

Good News:

I automatically aliased my first Drupal 7 node tonight :)

I have made all the updates required for the code to at least run. I changed some settings in the form and saved them successfully. I created a node and the node title and date were aliased correctly.

There is much testing to do (especially for taxonomy related stuff) and some code cleanup and then I'll post a patch.

Bad News:

To be honest I think a little more code rewrite is in order. After converting the old token.module approach to the new core token functions, I think the whole "placeholders" approach is overly-verbose and outdated now that we have functions like token_replace and token_scan. The other token functions are pretty nice to use also. Another factor is that the "type" of token is now in the token itself (ex. "[node:title]") and there are core functions for parsing the tokens. I think the code could be trimmed down quite a bit due to the new token API.

So, before I really go crazy I want to make sure Greggles and co. are OK with me trying out some more sweeping changes. I'm happy to code up some big changes that don't get committed, I'm having fun with thew new drupal 7 API anyhow, but I also don't want to needlessly waste other people's time on reviews and such.

greggles’s picture

Great progress! Congrats.

Sure, please go for sweeping changes in that area. It makes a ton of sense. We can also change that in Drupal 7 core right now, so if you bump up against limitations of the token module...voice them here and I'll do my best to help.

te-brian’s picture

Sounds good.

Yeah, I've already found a few bugs in token related code (mostly in node.tokens.inc) and posted a few patches. Overall a pretty clean system. I'll be glad to use it in future apps, especially in a lot of the notification email settings forms that I write.

te-brian’s picture

Implementation Questions:

The old token system used the "-raw" versions of tokens to differentiate between sanitized and non-sanitized replacements. In the new system we have the $options array and the 'sanitize' option.

1. What default sanitization mode do we want?

2. How do we want users to be able to control sanitization options? Checkboxes per pattern? Per Token? A Global Option? Or, do we show a set of pseudo-tokens that mimic the "-raw" behavior?

- On this topic, since the token system deals with sanitation at a more global level (you determine sanitization for the whole replacement text, not per token), the easiest course of action is to default to unsanitized tokens, since that is what is recommended to the user in previous versions of pathauto.

greggles’s picture

Yeah, we can just always use the unsanitized tokens. Users should not see any options related to sanitized/not, and pathauto should choose the not-sanitized always.

Also, I created #609810: Create "token" component in core issue queue so we can better track token issues in core.

te-brian’s picture

Status: Needs review » Active
FileSize
96.82 KB

Alrighty, we have our first patch folks.

Summary

Its pretty big, and for that I apologize. Many lines have been changed just to account for changes in coding standards (such as string concatenation). If the coding standards changes are making it difficult to review, please let me know and I can re-roll without them and we can do that later.

Here is a quick, incomplete list of the major changes:

  1. Updated .info syntax.
  2. Updated .install with new default patterns.
  3. pathauto.admin.inc: Updated to account for change to core token handling. Also remove "raw" references.
  4. pathauto.inc: In "src" becomes "source", "dst" become "alias". pathauto_create_alias() changed to account for new tokens api. Uses a $data key similar to how token_replace() uses it. pathauto_get_placeholders not needed anymore.
  5. pathauto.js: updated to follow new javascript standards.
  6. pathauto.module: New hook implementations for core token system. New nodeapi, user, taxonomy hook implementations.
  7. node.inc, taxonomy.inc, user.inc: Updated to use new token style.

This patch should apply to the latest pathauto HEAD properly. I've been testing it in a clean install of the Drupal HEAD with only the devel module installed. Basic node creation, term creation, and user creation, as well as updates and deletes seem to create aliases properly. There are definitely some bugs left (and there are also bugs in dependent systems such as node.tokens.inc and taxonomy.tokens.inc that I've submitted patches for). Please grab this, read it, test it, and let me know what needs fixing, rewriting ect.

EDIT:
There are some bugs in core code that will stop pathauto from working. I have created patches to address some of these:
#609090: Invalid call to node_get_types() in node.tokens.inc - Committed!
#609118: node.tokens.inc: $node->body issues - Committed
#610022: Taxonomy Tokens: Fix old queries

One note, this new implementation may have an effect on modules that implement pathauto hooks. The way I've recoded it, its much more dependent on the core token system (which cleans it up a bit). What this really means is that modules will not be able to supply arbitrary placeholders and replacements, they will need to properly implement the token hooks for the appropriate object types (node, term, vocabulary, user, etc.). I think this is the preferred way to move forward, but I could be wrong.

Known TODOs

  1. Finalize new comments, permission descriptions, and other text.
  2. Fully test all types of aliasing (especially taxonomy related).
  3. Finish porting Forum module related stuff. I simply am not that familiar with the forum module. -- Thanks David
  4. Use node_load_multiple anywhere where there is still a loop with node_loads. - Only one spot left, not critical
  5. Do we need to implement hook_menu_link_insert(), update(), delete() ?
  6. Any Triggers and Actions related functionality.
  7. Update API.txt language and convert that to a pathauto.api.php as per the new recommendation.
  8. Chained Tokens: Deal with chained tokens where they are needed (ex. [node:author:name]).
  9. Upgrade Path.

Wishlist

  1. Use Batch API for bulk updates and do away with the # of aliases per batch restrictions.?

Wew. If your still reading, thanks for the patience. Due to the amount of ... ahem .. "working" hours I've spent on this, I should say that this patch is sponsored by Terra Eclipse. My co-workers have been kind enough to leave me alone in my Royksopp-fed-pathauto-frenzy.

greggles’s picture

Great news and great work!

  • Finish porting Forum module related stuff. I simply am not that familiar with the forum module.

I know a little about that based on the work I've done here. Hopefully we can get it close.

  • Use node_load_multiple anywhere where there is still a loop with node_loads.

The only plaace I know of is bulk generate.

  • Do we need to implement hook_menu_link_insert(), update(), delete() ?

Maybe, but let's leave that for later.

  • Any Triggers and Actions related functionality.

Getting these to work should be a part of this issue, but not enhancing them.

  • Update API.txt language and convert that to a pathauto.api.php as per the new recommendation.

Yes.

  • Chained Tokens: Deal with chained tokens where they are needed (ex. [node:author:name]).

What does token need to do to handle those specifically? I thought they would "just work."

  • Upgrade Path.

Which could also be a separate issue. I'd like to get this out soon so people can start testing. Then we can work on upgrade path.

  • Use Batch API for bulk updates and do away with the # of aliases per batch restrictions.?

Let's keep this separate, so it can easily be backported if we end up doing it.

te-brian’s picture

  • Chained Tokens: Deal with chained tokens where they are needed (ex. [node:author:name]).

What does token need to do to handle those specifically? I thought they would "just work."

I should have been a little more clear. You are right in that the replacement will work just fine. The part that is not implemented is adding them to the fieldset on the admin settings form. For example, for nodes we would need to loop through the user tokens in addition to the node tokens and add them to the set of "tokens" in the $settings array in the form of [node:author:name], [node:author:mail], etc.

What is the general plan? I would guess that once we work out any big issues that remain, you'll create a 7.x-dev that we can do future development on. From your initial review, what needs to be fixed to get us there?

greggles’s picture

Ah, for the UI? There's been a lot of discussion on this and I'm not sure that there's a great solution yet. My feeling is that we should do two things:

  1. Provide help text under each set of boxes that provides a list of the most popular relevant tokens for that area.
  2. Provide a link in there that goes to a page with all of the available tokens on the site.

It may be easier to say the second part than do it... ;)

Regarding branches - yes, I plan to make a DRUPAL-6--2 branch soon and then commit this code to HEAD and we'll make HEAD the 7.x code. While I don't like it too much, there are a lot of people using 6.x-2.x in production and we can't leave them stranded.

te-brian’s picture

I like your thoughts on the token listing. A full fieldset per "pattern item" is probably overkill for the average user. They just need node:title, user:name and a few others. Too bad we don't have metrics on all the patterns people use :) I'll try to work up a version of the patch that displays it in this new way and we can see how it feels. The tricky thing is that not all tokens are available for all items due to the $data objects that are available to be passed in, but it would be easy enough to make a page callback that displays all the tokens for one "type" (or two types in the case of "term and "vocabulary").

Anonymous’s picture

Subscribe

te-brian’s picture

Uh oh, now that I know people are looking at this, I better dust it off.

I have found a few more token bugs that I need to post some patches for. Once those and the others I posted get in core (in one form or another) then I can get back into full swing finishing up with the TODOs listed in #8.

If anyone has experience writing test cases, Dries has asked for some more token tests in #609090: Invalid call to node_get_types() in node.tokens.inc.

Jeff Burnz’s picture

Subscriben - ok point taken (below), my reason for subscribing is to help improve the UI for Pathauto in D7. I know I can at help out in this area and will be testing for usability and accessibility of this module and hopefully will even have time for some patches.

greggles’s picture

Joshua mentioned that this patch doesn't apply - I had applied several patches to pathauto for 6.x-2.x around the time you were rolling this, so it is likely those will need to be merged in here.

Also, anyone who subscribes to this issue without finding something constructive to say in their comment will get a -1 next to their name in the karma-book I keep in my brain :p I realize it's a module lots of people care about, but really there is a way to fix this and it's not to post a subscribe comment.

greggles’s picture

This is a straight upgrade and not a redesign. There are other issues in the queue for UX improvements (see this list).

te-brian’s picture

FileSize
88.62 KB

Here is a re-roll with some other small fixes.

  • Cleaned up some white-space issues that I left behind.
  • Removed the "@te-brian" from my comments (that style doesn't seem to be the standard)
  • Replaced/removed references to taxonomy_term_path(). Was removed in #320331: Turn custom_url_rewrite_inbound and custom_url_rewrite_outbound into hooks.
  • Fixed all the user token replacements. They were passing $user->uid instead of $user in the data array.
  • In pathauto_create_alias, the call to token_replace was passing the language string, not an object with a 'language' property.
  • Fixed an error in the bulk delete query (incorrect LIKE syntax)

This patch should apply to HEAD properly.

Don't Forget:
Until they get committed, or an alternative is committed, you'll need to apply these patches for pathauto to work:

David_Rothstein’s picture

I had a bit of time at work today dedicated to helping with this, and expect to have at least a bit more either tomorrow or Monday. @te-brian, are you continuing to work on it also? I want to see if there's something I can help out with that doesn't overlap.

In the meantime, I've posted reviews and updated patches for #609118: node.tokens.inc: $node->body issues and #610022: Taxonomy Tokens: Fix old queries, so hopefully those can get in soon.

I've also spent a fair amount of time looking through the patch above and just trying to get up to speed. It looks like it's very much on track to me. The only things that I noticed while reviewing were pretty minor, but listing them here:

+  if (($data = $result->fetchObject()) !== FALSE) {

Seems like if ($data = $result->fetchObject()) would work just as well here?

+Drupal.behaviors.pathauto = {

+  attach: function(context, settings) {

+    (function ($) {

+      if ($("#edit-pathauto-perform-alias", context).size() && $("#edit-pathauto-perform-alias", context).attr("checked")) {
...............
+    })(jQuery);

I am not even remotely good at jQuery so this comment may be totally wrong :) But I'm pretty sure that the function ($) and (jQuery) stuff goes outside of the Drupal.behaviors part rather than inside. It also seems like this is a case where jQuery.once() would be used - something like $('#edit-pathauto-perform-alias', context).once('pathauto', function () { and then checking $(this).size() and $(this).attr() inside that.

function pathauto_admin_settings()
function pathauto_admin_delete()

Do to (very) recent changes in D7, these form definition functions now need to at least take a $form parameter as input and then make sure to add on to that rather than setting $form = array() internally.

te-brian’s picture

@David: Yeah, I plan to work on this as issues come to my attention. We are spending some time at work playing with Drupal 7 to stay ahead of the curve and try to give back a little. I'll work up a patch to address the above issues (although I'm not sure about the javascript one, to be honest). I'd love some help with this, especially with general testing. I imagine we will find a few more token related bugs and the sooner we can get those patches filed, the better. The area that I could use the most help with is forum-related code. As you'll notice if you apply the patch, I have just commented out the entire forum code block from the taxonomy include. Thanks for the review and help. I'm excited to get this to a dev-branchable state so that others can begin to use and test it.

Edit - @David I will not likely be working on this tomorrow, so feel free to touch any part of the code. I can then get up to speed on any modifications you make to the patch.

te-brian’s picture

FileSize
88.62 KB

This patch fixes the 'catpath' and 'termpath' aliases. The token replacements were correct, but the slashes were incorrectly being removed later in the process.

David_Rothstein’s picture

FileSize
93.97 KB

OK, here is a new patch:

1. Fixed the issues listed in #19.
2. In doing so, found that the JavaScript didn't work at all in D7 (for a variety of reasons), so fixed that - should be working now.
3. Also found that there were a few issues with the node form (URL alias checkbox not staying checked when the node was reedited, URL aliases not always getting saved correctly when the node was reedited, etc). Most of these were due to changes in the structure of $node in D7 and what is contained within it at various points. Those should be fixed now.
4. Fixed some broken URLs.

New (relatively minor) issue to point out:
1. When creating a new node, the summary info in the vertical tabs says "no alias" - Pathauto should ideally change that to at least read something like "automatically generated" so it's telling the truth...

I started looking at the forum stuff, but unfortunately did not have enough time to actually make it work. I'll hopefully have a bit more time on Monday to help with this, though, and can look at it then if it hasn't fixed itself by that point :)

te-brian’s picture

@David thanks for your work on this. I'm very happy to have another set of eyes on this one, especially someone who is much more up to date on Drupal 7 than I am. Depending on where the two of us get early this week, we should probably come up with a new TODO list based off the original one in #8 and any new findings. That way we can coordinate between each-other and any new contributors.

David_Rothstein’s picture

Status: Active » Needs review
FileSize
8.55 KB
99.28 KB

Here's an updated patch:

  1. This should get the taxonomy and forums both working correctly (at least based on my light testing).
  2. It also makes it so that the module's SimpleTests will pass in D7.

I probably don't have much more time to give to this, but it seems like it's pretty far along. In terms of the remaining TODOs, I think most of the TODOs in #8 are still relevant (with the exception of the forum module stuff), as well as the minor issue I mentioned in #22. I don't know which issues on that list should still be considered critical, though.

te-brian’s picture

Thanks again @David_Rothtein, I'll try to make some time to test everything you've done and look into the remaining TODOs.

One general question for you. From an implementation point of view, did you think my approach was the right way to go. Its slightly more limiting than the Drupal 6 approach in that A) Only tokens can be used, not arbitrarily supplied placeholders and B) the available tokens are determined by which objects we pass to the create_alias function in the $data argument, which other modules will not really be able to modify. Can you think of a case where, for example, people need to pass a node object in on a taxonomy alias?

I think the way we are doing it best utilizes the core token system without rewriting all of Pathauto, but I was just curious to get another Drupalista's point of view.

Also, whats you're take on my use of the 'internal helper' functions on the hook_node, hook_user, and hook_taxonomy implementations? This is how I often do it in-house, but I don't know if it is a popular pattern in contrib.

As I try to contribute more back to core and contrib, I'd like to make sure my coding practices are in line with accepted standards and patterns.

Thanks

greggles’s picture

A) Only tokens can be used, not arbitrarily supplied placeholders

I'll admit I haven't looked at the code, but...does this mean that people can't do something like "blog/[title-raw]" ?

David_Rothstein’s picture

@te-brian, regarding your first question, I'm not sure I understand the problem (but then again, I'm not that familiar with exactly how Pathauto worked in D6).... As far as I can see, in D6 the module also hardcodes the object that gets passed along for token generation - in that case, it gets passed to the pathauto_get_placeholders() function. And in D7, it's possible for modules to affect the list of allowed tokens by using hook_token_info_alter() and hook_tokens() to define their own additions to the list, which it looks to me like Pathauto will pick up. So is there a more specific scenario that you are worried about? (I haven't looked that carefully, but I don't see any major differences in the UI, and things like blog/[node:title] are certainly still possible.)

Regarding the internal helper functions, certainly from my point of view the way you did it is perfectly fine! Overall, the code looks very good to me.

te-brian’s picture

@#26 - blog/[node:title], as David said, is certainly possible. I was just referencing the fact that people will need to use the token hooks to define their replacements, instead of doing so via pathauto-specific hooks. And with all the alter-ability of tokens, I think there is plenty of power.

@#27 - You are right, I think I am over-thinking things. It is nerve racking working on a top 10 module :)

On with the show. I'll stop nay-saying my own work :)

franz’s picture

subscribe!

FiNeX’s picture

Status: Active » Needs review

Subscribe... and thanks for your work!!! :-)

Dave Reid’s picture

Issue tags: +D7CX

Adding tag...

BenK’s picture

Subscribing...

timcosgrove’s picture

Subscribing. Thank you for your work on this, but: I can't get any of the patches to apply to any version of pathauto 6.x-2.x-dev. Is it possible the patches need to be rerolled based on changes to 6.x-2.x ?

Thanks

te-brian’s picture

@tim.cosgrove quite possibly, I'll try to make time to check this out. Been super busy lately though.

kvit’s picture

Subscribing & waiting for 7.x release. Thank you for your work

outlying’s picture

I encounted some problems using the patch (maybe conflict of versions of pathauto). Could somebody post here not the patch, but a package of the files already patched (or the whole module already patched and repacked)? PLEASE!!!

greggles’s picture

The hunk failures were caused by #647812: Update to code required for Path_Redirect compatability which got applied after this was created.

This is now committed http://drupal.org/cvs?commit=326580 I created a Pathauto DRUPAL-6--2 branch from HEAD just before committing it in case we want to do anything with that, but I think it will most likely sit dormant.

The simpletests didn't work for me, but I think that's more about my environment than actual problems with the tests.

I'm going to leave this issue open for now so that we can try to address some of te-brian's items from comment #8 (with the exception of those in #9 that we shouldn't test).

I'll create a release tomorrow or so - I'm waiting on advice from dww about how to do it so that all issues marked 6.x-2.x are changed to 7.x-1.x.

greggles’s picture

WHOOPS: I forgot the most important part of my comment!

A huge thank you to te-brian and David_Rothstein and all the other participants who provided feedback. Great work on this, everyone!

arhak’s picture

@#11 greggles

Ah, for the UI? There's been a lot of discussion on this and I'm not sure that there's a great solution yet. My feeling is that we should do two things:

  1. Provide help text under each set of boxes that provides a list of the most popular relevant tokens for that area.
  2. Provide a link in there that goes to a page with all of the available tokens on the site.
    It may be easier to say the second part than do it... ;)

- would you provide links to discussions?
- the one I know #514990: Add a UI for browsing tokens

@12 te-brian

The tricky thing is that not all tokens are available for all items due to the $data objects that are available to be passed in, but it would be easy enough to make a page callback that displays all the tokens for one "type" (or two types in the case of "term and "vocabulary").

- the separated page sounds like a good first approach
- but
-- it will be madness unless using kind of advanced_help popup to allow the user to see both, the textfield where he/she is gonna enter the path and the available tokens at once
-- would the separated page be explicitly titled to clarify what does that list applies for? (e.g. "Available tokens for: taxonomy term path") since the module might select every available node/term tokens and filter out some of them knowing they won't apply (e.g. tokens providing URLs or huge data, like [node:body], [site:url], [site:login-url])

@#15 Jeff Burnz

my reason for subscribing is to help improve the UI for Pathauto in D7. I know I can at help out in this area and will be testing for usability and accessibility of this module

- a fieldset into vertical tabs (intentionally) stops being collapsible
- can you suggest another UI approach?

arhak’s picture

the first approach I toke for token UI was:
- select the tokens according to the $object (e.g. comment)
- prepare a table of: Token / Name / Description
-- chaining options are only displayed one level deep, like [comment:created:*], [comment:node:*], [comment:parent:*]
-- the chaining fragment being shown in bold i.e. [comment:<b>node:*</b>]
-- the chaining description being prepended with "chains:", e.g. "chains: The node comment was posted to."
- immediately bellow continue with another table of: Token chains / Name / Description
-- chaining options are displayed like ":node:title" if they have no more chaining options
-- a nested chain is displayed like ":node:author:*" with its description being prepended with a "chains:" prefix
-- chains of chains continue in the same table
-- more than one leve deep chains are displayed like: *:author:created:* with description "chains: The date the user account was created."
- until now there is the problem of listing chains multiple time for different chain suffix
-- i.e. node:created:* chains to a date, its chaining variants are *:created:short
-- but node:changed:* chains to a date as well, and its chaining variants are *:changed:short

tokens_UI.jpg (the snapshot attached to #514990-20)

PS: this first approach can be experienced with comment_subject-7.x-2.0-alpha1 (only fetching one nested level of chains)

Dave Reid’s picture

@arhak: See #273104-11: Using "progressive disclosure" for alias patterns for the UI direction I'd like to take Pathauto for D7 in that'd also help control that the user see relevant tokens and not the giant list of them all.

arhak’s picture

in that'd also help control that the user see relevant tokens and not the giant list

that was discussed in #514990 but didn't see any conclusion

See #273104-11 for the UI direction I'd like to take Pathauto for D7

I'm talking about what would you like into the "Token browser stuff goes here" spot
for instance, comparing to proposals at #514990 what would you prefer?

where should I follow the topic?
- in this issue?
- don't know if the title of #273104 suits for this "token browser" stuff as well
- issue #514990 seems stuck and not necessary owned by pathauto
- should we open a new pathauto issue for "token browser"?

I'm interested on knowing what would be an approved UI approach
I'm thinking preferably with no fancy JavaScript stuff, since it will be needed to degrade anyway, so I would prefer to know how the degraded version would look like, and from there what enhancements can get

Dave Reid’s picture

Now that the module is ported, can we mark this as fixed and open new issues for all the follow-ups so they're easier to track?

Status: Needs review » Needs work

The last submitted patch, pathauto-606062-24.patch, failed testing.

te-brian’s picture

Thanks again to everyone helping with this, and to greggles for letting me run with it in the beginning. Sorry for leaving it so dormant lately. We've been very busy at working launching sites lately and I haven't had the contrib time that I'd hoped for.

Dave Reid’s picture

@te-brian: Can you please help create new issues for all the follow-ups that you identified?

Dave Reid’s picture

Status: Needs work » Fixed

And since we released 7.x-1.0-alpha1, I'm marking this as fixed. Also see #43 and #46. :)

Status: Fixed » Closed (fixed)
Issue tags: -D7CX

Automatically closed -- issue fixed for 2 weeks with no activity.