Enable ajax on-demand loading of sub-trees in the Token help

Interstingly I used to receive out of memory errors earlier which had disappeared with the previous two versions of the module but have now reappeared again. Have there been any significant changes made?

Autonode Title is useless due to this issue when entity references are being used on a node. I get a WSOD with a memory exhausted error. Until this issue is resolved token_tweaks works great.

I threw together something for ajax lazy-loading using the jsTree jquery plugin (http://www.jstree.com/) instead of tabletree. I went with jsTree instead of tabletree since my goal was to get something up and working as-simply and as-quick as possible while jsTree provided both 1) tree-like functionality and 2) json ajax leaf support.

Currently missing with this patch (that I can think of) is 1) No JS support and 2) Descriptions for each token

Given the above I wouldn't be sure if jsTree is suitable in the long run, but it is working great for me and eliminates all recursive/memory issues as well so hoping it can be of use to others and maybe help push the issue along.

Great job Sarva! I wasn't aware of jsTree and it does look very promising from a quick look at its feature list (+ it seems more polished from the screenshots available in its home page). Besides the fact that it already offers AJAX loading, it also includes a search functionality that I saw being suggested as a feature someplace, ...I think in #514990: Add a UI for browsing tokens.

Updating to reflect status.

Marked #1783878: Reduce token list memory usage as a duplicate.

It looks terrible that token-tree is rendered as table, suppose better to convert this to ul>li or dl>dt-dd

@andypost: why would it be a better solution? And how would this token tree look like a table if it wasn't rendered as a table? :) I think the table structure is justifiable: it's needed, because there's a "Name", "Token" and "Description" column. What needs to be changed is that tokens list should be displayed on-demand: #1842738: Provide admin setting to replace all uses of the core token browser with the new popup.

This is even more heavily needed after the 7.x-1.5 update. See #1954464: Memory Issue

It sounds like ajax loading has been added via that linked issue

Confirming the issue here. It's notable that when using filefield_paths this actually hosed a site- to the point that I couldn't load any pages with tokens at all. Token Tweaks is helping a bit- but not seeing the full list of tokens isn't a solution (in my humble opinion)... kind of defeats the purpose.

Perhaps a patch to disable the ajax/js on this page (or token lists) and simply list all the tokens is a temporary solution?

This issue is also breaking flags on my install, which is a large site with many, many fields. I can now only use the site with the token tweaks module. Upgrading to critical.

What needs to be done to fix this? I might try a start with some more direction.

Any progress here?
I'm still facing the issue.

UPDATE: I've installed patch #5 and seems to work well! Many thanks for this great job!

Closed a duplicate: #1954464: Memory Issue

The patch in #5 does not apply to either the latest -dev release or the latest stable release.

This is a simple reroll of #5 for the latest -dev release.

I've tested the patch a but it fails with the following 404 errors:

[Error] Failed to load resource: the server responded with a status of 404 (Not Found) (style.css, line 0)
[Error] Failed to load resource: the server responded with a status of 404 (Not Found) (d.png, line 0)

The problem still exists and is still highly critical because it kills messy IE. How can we proceed here?

Just posting this question here: Are there working patches out there on the various issues involving the token browser that allow replacing the trees with an AJAX browser or a link to a popup AJAX browser?

I've found 8 or so issues related to this and am just trying to make sure I'm not missing a "fix" out there that I haven't come across yet.

Rerolled.

I'm finding that with "token-n1334456-25.patch" I get a WSOD error in a pop-up token browser, and when the link to the pop-up token browser is opened in a new browser tab the base level "folders" of the tree are visible, but the toggle-downs and all functionality are absent (unable to click to view deeper level tokens and folders of even deeper level tokens).

This was the same on two different sites. Reverting back to the latest dev of token without any patches brought back the ability to see the functioning tree in a new browser tab but I'm still getting a WSOD with the ajax browser popup (probably something other than this issue). FYI, Token tweaks is set on level 4 so it's not an issue of restriction by that module (also tried it with level 2) and I'm pretty sure it's not a server memory error since it works in it's own browser tab.

Here are some other related errors I recorded from some other page load (can't remember now where that was), these were with the patch in #25:

• Notice: Undefined index: recursion_limit in token_tweaks_preprocess_token_tree() (line 20 of /.../sites/all/modules/token_tweaks/token_tweaks.module).
• Warning: file_get_contents(sites/all/modules/token/jquery.jstree.js): failed to open stream: No such file or directory in drupal_build_js_cache() (line 5038 of /.../includes/common.inc).

Token tweaks hasn't been updated since 2014-Jan-27, so I don't think it's that module.

Yes, when I rerolled it I forgot to include jquery.jstree.css and jquery.jstree.js.

Note that we're still missing the d.png and throbber.gif images, so the CSS needs to be updated and/or new images added.

The last patch doesn't work:
1. There's no way to select tokens or expand token tree
2. "Undefined index: recursion_limit" error appears

I have also the problem that it also can't load the whole site, because of the tokens which are too many. how we can solve this?

There where still some files missing (themes/default/*). I exported the repository from google code to Github. The jsTree Version (1.0rc3) is quiet old, but versions require at least jQuery 1.9. I tested with default drupal jQuery Version and with 1.7 provided by jQuery Update. It doesn't seem to be working with versions >= 1.8.

Screenshot of token widget with jstree enabled:

Treetable library depends on jQuery 1.6+. Is it reasonable to require it for one of the most popular Drupal modules (I mean Token)?

Patch in #31 does not work if token widget is loaded via ajax (link called "Browse available tokens."), because then the jstree script does not correctly detect the path to the jstree theme; in result the tree is not visible.

[please ignore this comment]Patch from #31 removes treeTable, but it is used in token browser.

Hi! I'm running into an issue when I go to /node/NID/devel/token because, I can only guess, it doesn't use the token browser, so it's trying to load everything? I'm not positive b/c I've never actually seen this page b/c I haven't gotten it to load successfully on any of my sites, local or in clouds.

I don't mean to hijack a thread, I just wanted to post a comment about it before creating a new ticket, b/c it felt to me like my problem fits with what y'all are discussing, but I wasn't sure.

[if I'm in the right place, here's more stuff -- if not, please lmk and I'll re-post elsewhere!]

I also tried token_tweaks, even though it isn't what I really want, but token_tweaks did not help with /node/NID/devel/token -- my helpful Pantheon support persons dug around the new relic data and indicated that this devel tokens page circumvents token_tweaks, so that's why token_tweaks didn't help -- specifically, they pointed to token_find_with_prefix as being the primary culprit of resource overload. In my local environments, I kept getting WSOD until I upped my memory limit to 1024M, at which point I got "uncaught exception" errors on unstyled white screens ("PDOException: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away:" and so on). The helpful Pantheon support persons suggested I implement mymodule_preprocess_token_tree (https://www.drupal.org/node/1058912#comment-4801902), and I intend to do that -- but, while there, I looked through #1058912, and comment #69 over there is how I ended up over here!

Again, if I'm in the wrong place, pls just say the word. Thank you!

My browser keeps crashing due to the huge token list. Any hopes that this can be solved at some point? It's been two years without any activity.

Edit/add: I just saw that my token list HTML file has about 17.5mb.

I haven't tried it yet, but there is a new module aimed at addressing this issue: https://www.drupal.org/project/fast_token_browser

I am the author of the Fast Token Browser module linked above. If you need a fix for D7, I recommend installing my module. I've been advised by one of the maintainers of Token that there are no plans for further feature additions on the D7 branch.

I'm hoping to port the module to D8 in the next few months, and ultimately contribute it as a patch to Token.

We had this issue, and using Fast Token Browser did seem to resolve the problem.