We've encountered several problems with the current token UI which uses the treeTable jQuery plugin:
- It is not accessible basically at all
- It is pretty bad performance-wise.
- It has to load all possible tokens in order to render it (and we're running into problems with recursive tokens).
- It would not be appropriate to move the current UI into Drupal core. It should be.

Once we get past Token 7.x-1.0 I want to seriously consider how we can improve this. Ideas:
- Use jQuery UI Accordion for the token type groupings rather than including them into the table - this should help improve performance since we're adding viewer rows that have to be processed.
- Use AJAX when a user wants to view nested tokens (get deeper in the token tree) - then we will only have to load the first level of tokens by default. Getting further will require relatively small request each time.

Proposed resolution

  • Create token 2.0 branch with dependency on Drupal Core ^11.3
  • Create an HTMX route to pass token as argument to
  • Refactor how default recursion limit is handled to only return top level tokens initially
  • Using HTMX buttons on parent tokens that have children, we can dynamically pull in the child elements via HTMX callback
  • The returned html should also account for child elements that include children and incorporate same HTMX button logic so we're always dynamically requesting only whats needed for each level.
  • The HTMX button should be replaced with simple button without HTMX attributes upon callback return so we can still expand/collapse whats already in the DOM but eliminate additional requests.
  • Need to ensure drupalBehaviors are attached in the returned html which is why 11.3 is required as htmx library in that version apparently already accounts for this.

Issues / Considerations

  • The token table HTMX button click functions on the groups in a dialog set focus back to the first button at top. Need to see if there's an HTMX solution for this. Fixed using htmx onAfterSwap event in token.js
  • There is currently not an HTMX way of replacing the ajax dialog. See https://www.drupal.org/project/documentation/issues/3546790
  • The cache token entries could be improved by perhaps checking for existing token in current type cache object vs. automatically creating a new cache record every depth.

Issue fork token-1229568

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

dave reid’s picture

Will be experimenting with http://jsfiddle.net/jGwKn/1/

Bojhan’s picture

subscribe

Noyz’s picture

subscribe

c4rl’s picture

[subscribe]

Some sort of autocomplete-driven browser seems like another possibility, but it could be inflexible and may not be appropriate for novice use since it requires needing to know some component of the machine name before typing.

dave reid’s picture

dave reid’s picture

alan d.’s picture

We are going one step further, completely removing the themed tree and rendering these in a popup. While the server settings / resources are massively high, speed is an issue and this reduced the load time by about a 1 second. The token table by itself is 478KB.

7wonders’s picture

Is there anything that I can do in the meantime to fix unresponsive jquery issues when the token tree gets too big? I just installed the rpx/janrain engage module and it completely killed some pages with token options. Im assuming its related to this issue?

jhodgdon’s picture

Just a note that we are trying to get a token browser in core for Drupal 8 on this issue:
#514990: Add a UI for browsing tokens

zterry95’s picture

if there is some configaration to let us to decide which way we display the token list ( token_tree or token_tree_link),
it is will be great!

I am really experiecing a tough time to change all the token list to ajax by coding...

Thanks for your attention.

Sk8erPeter’s picture

@zterry95: I definitely favour the idea! Unfortunately by default, Token outputs huge nested tables without letting the administrator decide whether he/she wants to risk a memory exceeding: #1836264: Huge nested table trees displayed in field settings lead to "allowed memory size exhausted" errors - display should be AJAXified.
For some reason, unfortunately Dave Reid said he "just fundamentally disagrees" with changing Token's default behavior, and using Token tweaks instead: http://drupal.org/node/1842738#comment-6741198. I'm curious about the approach, but I didn't get an answer.

elijah lynn’s picture

Issue summary: View changes
Related issues: +#514990: Add a UI for browsing tokens
elijah lynn’s picture

This, this, this...

Use AJAX when a user wants to view nested tokens (get deeper in the token tree) - then we will only have to load the first level of tokens by default. Getting further will require relatively small request each time.

tkoleary’s picture

Use AJAX when a user wants to view nested tokens (get deeper in the token tree) - then we will only have to load the first level of tokens by default. Getting further will require relatively small request each time.

Sounds like this: https://www.drupal.org/project/hierarchical_select

alison’s picture

Hi all, did you end up anywhere with this? I'm super grateful for the improvement in the token module whenever that was (couple years ago?) to switching to a token-browser-dialog instead of when the page with a form is rendered, but, it's extremely frustrating waiting for the token browser to load, and sometimes it simply doesn't work in some of my VMs -- I could go on, but I know I'm preaching to the choir :) Anyway, just curious if any of you got anywhere, even if just locally (or on some other related issue that simply didn't get linked to from this one), on this issue. Thank you!

darvanen’s picture

Version: 7.x-1.x-dev » 8.x-1.x-dev

Had a short chat with @Berdir today about this, summary:

  1. Starting point should probably be finding a well maintained JS library that is accessible and supports AJAX - if that can be found then this might be fairly easy to solve
  2. One potential issue is that libraries are not allowed to be committed which we will probably face if updating the library used here. We would need to update how we go about including it.
  3. A quick search brought up a few options that have not yet been fully researched (in no particular order):

    That's as far as we've got.

darvanen’s picture

Re #16.2 I propose including whatever we choose in the module's composer.json file and put installation instructions for non-composer users on the project page.

That won't work for JS libraries, it'll need to be an example composer.json file like File Entity Browser does, and/or plain installation instructions.

damienmckenna’s picture

jstree might work reasonably well and supports JSON, which could be useful with the community's move to using JSON for more things. It does seem to tick a lot of boxes.

damienmckenna’s picture

As identified by apmsooner, v4 of jsTree is removing the jQuery dependency:
https://github.com/vakata/jstree/issues/1904

darvanen’s picture

That sounds promising :)

berdir’s picture

That was 2017 and v4 is not yet released, last release in 2020. not very promising?

damienmckenna’s picture

The maintainer's last comment was in February of this year:

I am struggling with time for jstree - I did nudge the repo a bit in 2023, so I expect to be able to find more and more time for jstree, but it is not as much as I want it. v4 lacks all the plugins of v3 and is not yet thoroughly tested, but it looks promising - I hope to reenable plugins one by one in the upcoming weeks / months.

berdir’s picture

I know, but we're talking about something that could also get into core. The chances of getting a library maintained by single person who barely has time for it (which is perfectly understandable) into core seem _very_ slim :)

rkoller’s picture

rkoller’s picture

Issue tags: +Usability, +Drupal CMS a11y
kim.pepper’s picture

Issue tags: +Singapore2024
apmsooner’s picture

Assigned: Unassigned » apmsooner
apmsooner’s picture

Issue summary: View changes
Issue tags: +HTMX initiative

berdir’s picture

This should not include general coding standards cleanup, removal of legacy hooks and so on, makes it way harder to keep two branches in sync and review this.

apmsooner’s picture

@berdir,

I was just knocking it out cause it has to be in a new version anyway because of the core 11.3 dependency and it was pretty out of date and hard to follow without being properly typed and such.

apmsooner’s picture

Issue summary: View changes
apmsooner’s picture

Issue summary: View changes

apmsooner changed the visibility of the branch 1229568-token-ui-2.0 to hidden.

apmsooner’s picture

Assigned: apmsooner » Unassigned