Scaleable/themeable taxonomy selection widget

great patch. this widget could certainly be prettier, especially for small vocabularies (i.e. radio buttons or checkboxes). themers will make that happen.

- is there any reason to keep _taxonomy_term_select(). You could call the theme function directly from taxonomy_form()
- if we do keep it, perhaps collect $tree there and then pass $tree to the theme function.

We have used this for some of our select lists for our other CMS (Back- End.org):
http://www.barelyfitz.com/projects/filterlist/

Might be worth considering bringing it over to Drupal.

There are other ways of approaching this too, but this seems to be a fairly easy way to deal with long lists.

There are some other javascript solutions that might be more applicable, but this is one of the few that is clearly GPL.

Mike

+1

I especially like the link mgifford provided: this would make it _very_ easy to select long taxonomy lists! (although that example only allows for single select, right?). This is not possible with current drupal, but providing the theming function someone could easily implement it. This would make using (and reusing) taxonomies much easier.

As for passing $tree or not. I actually prefer to have $vid only, not $tree as sometimes you don't need $tree at all.

+1, definitely.

Folks - when an issue is in 'needs review' status, we expect a *patch review* to be posted. Guidelines for a patch review are at http://drupal.org/patch/review. I encourage everyoen to reread that page. We specifically discourage a simple +1 or -1 in a patch review.

I will be enforcing these guidelines for a while, as the quality of patch reviews has decreased recently. If you have issues/questions, pelse email me directly. I'd rather not hijack this fine patch.

-moshe

Ok, let's re-review:

1. I think the functionality added is needed and can't be achieved in any other way. The theming of a taxonomy select form will add usability in both small vocabularies (eg replacing it with checkboxes if only a few terms) and large vocabularies (see the example above).

2. The code degrades to exactly the same functionality as is now if no theming function is defined.

3. I have tested the patch:
- it applies cleanly.
- the taxonomy form is shown as before.

4. The code is well documented. It confirms to the coding style.

5. I do not like moshe's suggestion to pass $tree: not all implementations of this would need $tree. If the themer needs $tree, he can easily call taxonomy_get_tree() like is done.

6. I notice that _taxonomy_term_select() is called on other places in taxonomy.module too. Can you use the theme function there too?

Not only does the patch work, I'll be looking at it to see if something similar can be done for book/outline handling. I started a patch that split the Parent combo into two that cascaded changes, but it does not degrade at all...I didn't consider theming as a way to do it.

I also need to use this widget, but with a different label, in another project...right now my only option is a mass of duplicated code.

+1 works as expected.
* I can now style voc X different from voc Y, wohoo!
* I really like this workflow: first let contribs "fight" for the best implementation, then in future evaluate the winners to go in core as default.

I hope this is still in time for 4.7.....

note: I tested the first patch, not the last.

I've tested the second patch. I didn't try any theming, but created a vocab, some terms and posted a forum topiuc without problems.

if somebody could add an option to display a part of the terms as optgroup terms this would be awesome for forum containers, btw.. But that is of course completely unrelated.

I wonder if this patch makes sense when you have the form API in mind ... Adrian?

Adrian told me we should let the form API patch land first.

I think this patch needs to wait for the form api to get in first, as the return value of taxonomy term select is changed, and the optimal way to accomplish this is different.

The form API went in, please revisit this patch as necessary.

This is not particularly hard with the new forms API...

I commited to HEAD, but I modified the themable function's name from "theme__taxonomy_term_select" to plain "theme_taxonomy_term_select".

The double underscore looks weird, and really the themable function is not a 'private' function (which is what a leading underscore is normally used for).

Shouldn't the theming take place when the form is being rendered, rather than when the form is being constructed. Looks like this patch takes the wrong approach. I don't understand why this is better, or why '#theme' => 'taxonomy_term_select' is not adequate. Please elaborate.

I can confirm that Robert has a case here. This function simply does not scale too well.

+1 on Rob's use case...same would hold for large numbers of free tags. He's got a great write up of why he needs it.

+1 from me too on Rob's issue. My module (tagnode) depends on Drupal permitting selection from a very large list of taxonomy terms. Plus, I would like to load large taxonomies into Drupal (e.g. MeSH or SNOMED CT)

+1 here, too. I'm migrating a number of sites away from awTags now, and the sizes of the resulting taxonomies involved are daunting. It's fine when entering, but once administration comes up...

+1 from me too, for a slightly different use case. We eend to "freeze" freetagging after approx 600 entries. This means that our taglist will still use autofill, yet not allow new tags to be posted, anymore.

I haven't tested this yet, but could you document this a bit better in the code? From a forms API's point of view this is the wrong way to do it, so I think it needs some additional code comments explaining why we decided to take this unconventional (dare I say awkward?) approach.

Taxonomy selection with checkboxes/radios in 4.7 can be produced with forms API, see http://drupal.org/node/84286.

Robert, I understand your idea - do not build control at all before theming. I think that this option would be very useful for large complicated vocabularies, but I think theme is not a place to introduce such functionality. Themes are for theming. Theming must be as easy as possible to use. Like {smarty}. I think it will be good to extend taxonomy module with special control overriding hook to be implemented in modules, not in themes. Is it possible?

As for checkboxes/radios -- they greatly increase taxonomy usability, but only for small vocabs -- not a situation your patch addresses. So this is different issue, I just commented here because I searched for taxonomy/checkboxes and found relevant only your year old activity, but later spotted other solution.

Whether or not theming is the right place for this, robert is right in *that* he is suggesting to make the *generation* taxonomy widget customizable (i.e. being able to change the widget which is generated) and to do that without obligatory creation of the term-hierarchy (since there could be really large taxonomies...!).

Since we see more and more jQuery & Ajax stuff getting into Drupal, this opens the door for really useful taxonomy-widgets, like a hierarchical dropdown-(multi)-select which branches out a large taxonomy on demnad via Ajax, perhaps combined with an autocomplete field (could be super-powerful while keeping the user interface lean and optimizing usability).

However, the latest patch (from nov. 05 i guess) is out of date and will not be able to applied to current 4.7 releases, so i attached a new version of the patch (should work).

Guys, you should really keep this idea going (don't know whether it's already commited for 5.0, or have to wait for 5.x)!

patching file modules/taxonomy/taxonomy.module
Hunk #1 FAILED at 364.
Hunk #2 FAILED at 533.
Hunk #3 succeeded at 663 (offset 107 lines).
Hunk #4 succeeded at 1195 with fuzz 1 (offset 150 lines).
2 out of 4 hunks FAILED -- saving rejects to file modules/taxonomy/taxonomy.module.rej

What about Drupal 5.2?

Does this work for 5.2?

taxonomy_get_tree() just slaughters your site if a vocab grows big. This is needed badly. Here's a super quick diff for 5.6. I had some local tax changes so it may need an offset.

This is an API change. Moving to 7.x. Doesn't apply anymore.

I have a vocabulary with 5000+ terms. It will grow to 100,000+ terms by the time it is done.

With a module like: http://drupal.org/project/hierarchical_select it is easy to select terms from a vocabulary of this magnitude. If I understand the patch - this module will be able to hook in sooner and be more efficient?

If so, I would like to voice my support for this patch.

If 100,000 terms seem like a lot - remember it is less than 10 terms per level x 5 levels.

Actually it's unrelated. A function that's themeable means that a theme can override it, not a module. Modules can't override themeable functions, but they can intercept in this particular case, because it's a form widget, and form widgets are defined by the Forms API, thus can be overridden through hook_form_alter().

subscribing

Also, this patch should include a themeable fix for taxonomy_form_all() which is hit on admin/content/node as with large taxonomies that page yields WSOD as I just found.

Could this last feature be implemented, in a way or anther, into previous versions of drupal? (5.x for example :)

There's an undocumented variable in taxonomy_term_form() for D6 which allows you to intercept the select list before the terms are loaded - only for parents on the admin form though: http://api.drupal.org/api/function/taxonomy_form_term/6

This menu module patch deals with a similar issue: http://drupal.org/node/191360

Reclassifying, seems like the grandfather issue to most of this, and since freetagging vocabularies can WSOD your site on quite trivial administration pages, it's a bug.

This is pretty much the same as UB Usability: menu parent choosers, but for a taxonomy hierarchy instead of a menu hierarchy.

And the answer can be the same: HS (Hierarchical Select)! *However*, HS doesn't solve taxonomy_get_tree()'s scalability problem. HS even uses it itself (although a slightly modified version). HS in itself *is* scalable.

Without having given deep thought about this, I think the key to making taxonomy more scalable, is by adding a depth column to the term_hierarchy table (because sometimes you need to retrieve part of a tree, but only from a certain level, without having the parent/all parents, and due to the lack of a depth column, you're pretty much forced to retrieve the *entire* tree using taxonomy_get_tree()).

@WL: I don't see how it's possible to add a depth column to the term_hierarchy table, as long as it's a multiple-parent hierarchy. Consider the following:

foo
--bar
----woo
tan
--woo

foo: depth 0
bar: depth 1
tan: depth 0
woo: depth... 1? 2? Both? Neither?

taxonomy_get_tree has defied all attempts at scalability, ever since the day it was committed to Drupal. IMHO, we will never get it to scale properly, until we remove multi-parent support from taxonomy.module. But every time we suggest removing it, many people complain that they use it and that they love it.

We often advertise the multi-parent taxonomy feature as one of Drupal's unique features. But the truth is, there's a reason why no other CMS has this. It's a data structure for which there simply is no known scalable traversal algorithm. Personally, I vote to remove multi-parent support, and to ignore the cries of those who use it. Then, we can make taxonomy_get_tree work efficiently.

#54, #55: I don't see why HS would ever need a full-fledged taxonomy tree to work.

If I'm not mistaken, it only needs to be able to retrieve children of a given item, and this is trivial from the {term_hierarchy} table.

Jaza: excellent point, that won't work indeed. I said I hadn't given it a lot of thought yet :) I'm not sure I wholeheartedly agree with your proposal though. I do know people that use the multiple parents feature. They aren't many, but they are there. It's used on http://rtbf.be/ for example, for classification of news articles.

Damien: for some operations, it's useful to be able to start from a certain depth. It's been several months since I last looked at HS' code that adds support for Taxonomy though, so I don't remember the details.

Haven't looked at them yet, but there's supposed to be a rewrite of taxonomy_get_tree in this module: http://drupal.org/project/leftandright and also some depth search efficiencies in http://drupal.org/project/taxonomy_depth

>I don't see how it's possible to add a depth column to the term_hierarchy table, as long as it's a multiple-parent hierarchy.
...
>woo: depth... 1? 2? Both? Neither?

We may store min and max depth for a term in two columns. They would be equal for a major of cases.

We often advertise the multi-parent taxonomy feature as one of Drupal's unique features. But the truth is, there's a reason why no other CMS has this. It's a data structure for which there simply is no known scalable traversal algorithm.

Jaza -- great point. I'd add that in my experience, using the multiple parent feature is nearly always a mistake. It makes management of the content more difficult. It results in unpredictable behavior from 3rd party modules. It makes development more hellish and bugprone than it needs to be. You know you are going down a wrong path when you have to configure the weights of an objects parents so the breadcrumbs work properly.

The correct solution is usually to use two vocabularies instead, and do some custom development to make the two play nice -- with resulting menus that may look like:

Foreign Policy (All content below filtered with vid 1->tid 1)
-Editorials ( vid 2->tid 10)
-Research reports (vid 2->tid 11)
Science and Technology (All content below filtered with vid 1->tid 2)
-Editorials (vid 2->tid 10)
-Research reports (vid 2-> tid 11)

Note that in this case (like most others where people blunder by going down the multiparent route), the terms "Editorials, research reports" are completely different ideas than their parent categories (foreign policy).

Its not hard to see why this is easier to work with than having to use some sort of convoluted voodoo to determine which parent category is supposed to show up. The parent category is dead clear, because it its vid identifier. The result is also easier from a user perspective. Instead of having to both select science and technology, and editorial, they have two drop downs.

So -- really -- I think removing multiparent support would be a good way from preventing people from using approaches they'll regret later. Maybe instead of multiparent support we should be supporting using taxonomies in this sort of fashion. Now if you'll excuse me, I'm going to hop on my segway and bore another audience with my lectures.

Jaza, I am unsure what problem you need an algorithm for, there are a number of algorithms to traverse a DAG, DFS and BFS being two common ones. For UI, I prefer http://drupal.org/node/191360 obviously. It would need minimal work to work with taxonomy too.

Nick Lewis: It can result in unexpected behavior because most modules haven't been developed with multiple parent support in mind.
Your point about figuring out which is the parent in case there are multiple nails down the whole problem (well, the whole problem minus the performance issues). That's where Drupal does things wrong. IMO, when multiple parents are used, the entire lineage should be saved (thus multiple terms). Which implies that "multiple select" really becomes "multiple lineages". This is one of the things Hierarchical Select solves completely, at least UI-wise.
It works with Taxonomy's current node relationship (the term_node table), but it might be better if Taxonomy stored relations to nodes "per lineage" instead of "per term": if you store "per term", you have to do several queries to figure out which lineages exist in a given set of terms. (See _hierarchical_select_dropbox_reconstruct_lineages_save_lineage_enabled() in hierarchical_select.module.) When you're not using multiple parents, then you simply get lineages with a single term.

Moving all usability issues to Drupal - component usability.

#63, I'm not sure this issue is only related to usability. It also addresses a WSOD, scalability and performance cases.

But in the first place it's a usability issue. It only results in scalability/performance issues in case of massive taxonomies.

seems like menu choosers will stay as is for a release.

Only feature requests and tasks can be assigned to D8 for now, because the Drupal 8 tree is not opened yet.

Fortunately, this is a feature request.

Downgrading all D8 criticals to major per http://drupal.org/node/45111

Actually this is a task because the current widget can bring down servers.

The final verdict of http://drupal.org/node/45111 resulted in http://drupal.org/node/1201874 (which prevents new features based on major tasks). Problem or not, this is an unlikely backport to Drupal 7. I'm putting this back in features, please recategorize again if this really should be something that should prevent other new features.

In relation to some of the points raised above;

Performance & multi-parent
I agree that having single parents makes life so much easier, but there are compelling arguments for it: #338959: Move multiple parents to related terms. I'm building a regional db module and the first 3 of 4 countries that I have looked at required this. So from nearly having a working beta, I'm having to refactor this out into a multi-parent hierarchy. (Sorry for the side rant, this was what I was searching for when I came across this thread)

Either way, there are a number of other issues related to the performance issues of the taxonomy, I think that this issue should remain about preemptive strikes in avoiding the actual rendering of the tree. For example, Taxiselect does a single multi-join query in an autocomplete and HS does a small number of simple queries. Both run in the ms even with taxonomies in the millions.

Theme replacement:
Great for experienced developers, but I pity the newbies. It is like saying, "Yep core is broken and the best minds can not really find a solution, but now you can fix it by overriding this yourself!"

Override selectors:
From my experience with taxonomic override selectors in D6, here are the negatives: (as maintainer of the TaxiSelect module)

a) Only one module can exist that supplies this functionality
b) It is an all or none approach from a vocabulary perspective.
c) All of the free tagging logic was transferred to the override, and none of the four modules that I know that implemented this in contrib actually done nothing differently from core

So wrapping the logic in the tag widget code in core would be a great start if this is kept for D8+, exposing the optional widgets available would be nice and allowing vocabulary based widget selectors. I created an initial stab at this in #589332: Changes to the taxonomy_override_selector settings two years ago. The patch will be very outdated now.

Marked #302790: Usability: Multiple select, Vocab term groupings (indymedia sites etc) as duplicate. See also:

• #12089: Better method of showing forum containers in form
• #211494: Usability: Unify select option formatting

The term selection widget is a definite usability pain point.

Marked #955560: Taxonomy functionality and usability to edit terms, associate tags with individual terms and a dynamic taxonomy term menu as duplicate.

Just unassigning issues that haven't been developed for a bit in the D8 queue.