We've encountered several problems with the current token UI which uses the treeTable jQuery plugin:
- It is not accessible basically at all
- It is pretty bad performance-wise.
- It has to load all possible tokens in order to render it (and we're running into problems with recursive tokens).
- It would not be appropriate to move the current UI into Drupal core. It should be.
Once we get past Token 7.x-1.0 I want to seriously consider how we can improve this. Ideas:
- Use jQuery UI Accordion for the token type groupings rather than including them into the table - this should help improve performance since we're adding viewer rows that have to be processed.
- Use AJAX when a user wants to view nested tokens (get deeper in the token tree) - then we will only have to load the first level of tokens by default. Getting further will require relatively small request each time.
Proposed resolution
- Create token 2.0 branch with dependency on Drupal Core ^11.3
- Create an HTMX route to pass token as argument to
- Refactor how default recursion limit is handled to only return top level tokens initially
- Using HTMX buttons on parent tokens that have children, we can dynamically pull in the child elements via HTMX callback
- The returned html should also account for child elements that include children and incorporate same HTMX button logic so we're always dynamically requesting only whats needed for each level.
- The HTMX button should be replaced with simple button without HTMX attributes upon callback return so we can still expand/collapse whats already in the DOM but eliminate additional requests.
- Need to ensure drupalBehaviors are attached in the returned html which is why 11.3 is required as htmx library in that version apparently already accounts for this.
Issues / Considerations
The token table HTMX button click functions on the groups in a dialog set focus back to the first button at top. Need to see if there's an HTMX solution for this.Fixed using htmx onAfterSwap event in token.js- There is currently not an HTMX way of replacing the ajax dialog. See https://www.drupal.org/project/documentation/issues/3546790
- The cache token entries could be improved by perhaps checking for existing token in current type cache object vs. automatically creating a new cache record every depth.
Issue fork token-1229568
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #1
dave reidWill be experimenting with http://jsfiddle.net/jGwKn/1/
Comment #2
Bojhan commentedsubscribe
Comment #3
Noyz commentedsubscribe
Comment #4
c4rl commented[subscribe]
Some sort of autocomplete-driven browser seems like another possibility, but it could be inflexible and may not be appropriate for novice use since it requires needing to know some component of the machine name before typing.
Comment #5
dave reidMerged in #1233720: It would be helpful to have examples in the token descriptions..
Comment #6
dave reidCross-linking #1334456: Enable ajax on-demand loading of sub-trees in the Token help
Comment #7
alan d. commentedWe are going one step further, completely removing the themed tree and rendering these in a popup. While the server settings / resources are massively high, speed is an issue and this reduced the load time by about a 1 second. The token table by itself is 478KB.
Comment #8
7wonders commentedIs there anything that I can do in the meantime to fix unresponsive jquery issues when the token tree gets too big? I just installed the rpx/janrain engage module and it completely killed some pages with token options. Im assuming its related to this issue?
Comment #9
jhodgdonJust a note that we are trying to get a token browser in core for Drupal 8 on this issue:
#514990: Add a UI for browsing tokens
Comment #10
zterry95 commentedif there is some configaration to let us to decide which way we display the token list ( token_tree or token_tree_link),
it is will be great!
I am really experiecing a tough time to change all the token list to ajax by coding...
Thanks for your attention.
Comment #11
Sk8erPeter commented@zterry95: I definitely favour the idea! Unfortunately by default, Token outputs huge nested tables without letting the administrator decide whether he/she wants to risk a memory exceeding: #1836264: Huge nested table trees displayed in field settings lead to "allowed memory size exhausted" errors - display should be AJAXified.
For some reason, unfortunately Dave Reid said he "just fundamentally disagrees" with changing Token's default behavior, and using Token tweaks instead: http://drupal.org/node/1842738#comment-6741198. I'm curious about the approach, but I didn't get an answer.
Comment #12
elijah lynnComment #13
elijah lynnThis, this, this...
Comment #14
tkoleary commentedSounds like this: https://www.drupal.org/project/hierarchical_select
Comment #15
alisonHi all, did you end up anywhere with this? I'm super grateful for the improvement in the token module whenever that was (couple years ago?) to switching to a token-browser-dialog instead of when the page with a form is rendered, but, it's extremely frustrating waiting for the token browser to load, and sometimes it simply doesn't work in some of my VMs -- I could go on, but I know I'm preaching to the choir :) Anyway, just curious if any of you got anywhere, even if just locally (or on some other related issue that simply didn't get linked to from this one), on this issue. Thank you!
Comment #16
darvanenHad a short chat with @Berdir today about this, summary:
That's as far as we've got.
Comment #17
darvanenRe #16.2 I propose including whatever we choose in the module's composer.json file and put installation instructions for non-composer users on the project page.That won't work for JS libraries, it'll need to be an example composer.json file like File Entity Browser does, and/or plain installation instructions.
Comment #18
damienmckennajstree might work reasonably well and supports JSON, which could be useful with the community's move to using JSON for more things. It does seem to tick a lot of boxes.
Comment #19
damienmckennaAs identified by apmsooner, v4 of jsTree is removing the jQuery dependency:
https://github.com/vakata/jstree/issues/1904
Comment #20
darvanenThat sounds promising :)
Comment #21
berdirThat was 2017 and v4 is not yet released, last release in 2020. not very promising?
Comment #22
damienmckennaThe maintainer's last comment was in February of this year:
Comment #23
berdirI know, but we're talking about something that could also get into core. The chances of getting a library maintained by single person who barely has time for it (which is perfectly understandable) into core seem _very_ slim :)
Comment #24
rkollerComment #25
rkollerComment #26
kim.pepperComment #27
apmsooner commentedComment #28
apmsooner commentedComment #30
berdirThis should not include general coding standards cleanup, removal of legacy hooks and so on, makes it way harder to keep two branches in sync and review this.
Comment #31
apmsooner commented@berdir,
I was just knocking it out cause it has to be in a new version anyway because of the core 11.3 dependency and it was pretty out of date and hard to follow without being properly typed and such.
Comment #32
apmsooner commentedComment #33
apmsooner commentedComment #35
apmsooner commented