Synonym collapsing in core

Needs review

amaaaazing. Tried with multiple-word terms etc, all seems very smooth.

A couple of issues:
1. patch has windows line breaks (easy to fix in Notepad++ or similar)
2. I think there should be a drupal_set_message() to inform the user that foo has been saved as bar because it has been defined as a synonym by the administrator.
3. It doesn't deal with 3: - is that by design?

I reckon 2 without 3 would be fine though to avoid unnecessary complexity in the code. I'm not sure how usable it is to second guess what people are typing for autocomplete (think MS Word).

There's an issue against D7 which attempted to do this (but iirc, wasn't as clean and was CNW). I've wanted this for a long time but fear it won't make it into D6 since it's a 'feature'. Any chance of modularising it once the task is complete?

forgot to change status.

And one more thing:

Synonym collapsing
Enable collapsing of synonyms for this vocabulary.

My synonyms are going to collapse??? I think we need to find another way to describe what's happening. I'd probably say something like "Alias synonyms to terms when saving freetagging forms" - although that's not much good either.

I also just came up on the duplicate issue:

core's synonyms allow you to:

1. add synonyms which are the same as synonyms in different terms
2. add synonyms which are the same as real terms in the same vocabulary.

1. Should possibly be disabled?? Or some kind of validation warning when you do it? (not necessarily for this task of course)
2. hmm

I can think of use-cases for the second.
Say I have a load of news items tagged with New Zealand, then New Zealand changes it's official name to Aoteroroa. I want my old items to stay in New Zealand, but any new ones to move to Aoteroroa, so I enable this setting and it happens by magic. There's probably use cases in the other direction as well though (like keeping both tags and using the synonyms to make them show up in searches.

#1, yeah I agree with this. Maybe it should do it by the weight of the term? ack but then most terms will have identical weights anyway :(

Or maybe if you try to save the form with the 'collapse synonyms' option checked on a vocabularly which has duplicate synonyms, it should stop and tell you off. That wouldn't prevent people doing adding dups afterwards, but it'd help, and would just be an extra bit of validation.

Ok, not sure what you want me to do. A major rewrite of the node module & taxonomy module to make terms change during preview? Setting a message "Term x has been changed to term y"? Rewrite the definition of synonyms so that synonyms can't be assigned to more than one term? Or just re-saving the patch with unix line endings? ;)

And I did add the AJAX stuff: all the patch's changes to taxonomy_autocomplete were addressing that. Was that not what you had in mind?

Maybe it would be better for you people to work out the details so that the time I commit into this is well spent. :)

-cwgordon7

Attached is a new patch that is pretty throughly tested. Hopefully this is what you envisioned.

As to how I think the taxonomy module should handle synonyms: the admin should be warned when attempting to add a synonym to a term that already has another term its a synonym of. Perhaps open up a separate issue for that.

Whoops, setting to cnr.

This is not just a new feature, but a database change without an upgrade path as well. Not to say that the feedback in #16 was not taken care of. New features and database changes go to Drupal 7.

fwiw, I've discussed with cwgordon about turning this into a contrib module for D6, so hopefully we can refine how it works there - and either keep it as a very useful contrib, or revisit it for core in D7 depending on how that works out.

catch: sounds very good!

I should note that we discussed cwgordon turning it into a contrib, not me ;) Although if he decides not to that might have to be my first module.

dman: I agree, synonyms—and this patch—do belong in core. However, since no new features go into drupal six, this is only applicable for Drupal seven core (and it should be there). Providing it as a contributed module for drupal 6, however, would maybe not be ideal, but would still be useful for people who use synonyms on their sites. It's not so hard to do as a contributed module: just some fancy hook_form_alter work on #autocomplete_path, plus some cool stuff with hook_link_alter, and that's it. The module's almost finished, and should be up sometime today or tomorrow. Fixes for the problems dman pointed out should appear in the contrib module; I'll post a revised patch in maybe a few days.

-cwgordon7

Update: Has been added as a contributed module at http://drupal.org/project/synonym_collapsing.

Removing "GHOP" from issue title so people feel free to come in and help / test :).

I think synonyms definitely do belong in core: perhaps a separate module from the taxonomy module? Taxonomy, after all, does not require synonyms, but synonyms are a nice-to-have core functionality for nearly all freetagging vocabularies with this patch. The answer is not to just remove synonyms from core because they're not used, but to ask why they're not used and improve them. My answer: (1) They didn't do anything, and (2) It's a little hard to understand at first: are taxonomy terms synonyms of each other? Are synonym relations two-way? (Etc.) Better documentation / descriptions would improve #2, and this patch would make a good start at #1

Setting this to feature request because, well, it's a feature request.

Just wanted to chime in and say that my shop uses synonyms. We had a legacy taxonomy (non-Drupal) with synonyms, and it was great to have that there in core. That said, it was a little disappointing to not have more uses for it (as mentioned above). This new feature begins to open up the possibilities of synonyms, making for some very interesting free-tagging! I can't remember if this is stock or something I've coded for my site, but synonyms on taxonomy/term pages is one place they should definitely be showing up.

+1 to the ideas here (except for pulling it out of core), and I've already downloaded the contrib module (just haven't been able to play with it yet). Now that synonyms are in the spotlight though, maybe new restrictions need to be imposed? Like-named terms shouldn't be allowed as synonyms, as an example (case insensitive). And certainly better documentation is always helpful!

Again, great work here!

I believe this should do (?).

+1 to put this patch in core in D6 or D7, but I don't like the last change:

$added_terms[strtolower($term)] = strtolower($term);

I have a taxonomy that the term "a" and "A" are different terms.

Ok, so let's decide something: are taxonomy terms going to be case-sensitive? That's really what #31 is about. Is it really useful for people to have different terms for different cases? Or does it just add more confusion?

So long as the auto-suggest matches in a case insensitive way, I'd say make it case sensitive. I agree that sometimes it might sound silly to be case sensitive, but you never know how people are going to use their Taxonomy. "Us"/"us" vs. "US" and any other acronym collisions would be a good example.

So then the patch in #13 is really the desired behavior.

Aren't we able to suggest all terms case-insensitive, but replace only 1:1 matches? This patch should not prevent users from adding new terms, but only replace synonym matches.

+1 aside from that, badly needed for #187480: Allow free tagging vocabulary in issue followups

Ideally the synonym collapsing should work just as if the term was a 'full' one - so it'd be good to follow whatever the autocomplete/new tag creation behaviour now. in regards to case-sensitivity. If it's case insensitive - then this patch should use drupal_strtolower though. Case sensitive tags are really an issue for non-free tagging vocabularies, which this doesn't impact on at all.

+1 this is a great feature and I know many websites that would benefit from it. I think synonyms have a very important role in many Drupal sites and this takes it to the next level. I'm installing the D6 module of this feature now.

Could case-sensitivity be an option on a per taxonomy basis. The websites I run wouldn't want case-sensitivity.

Needs re-rolling for dbtng. Since the EOL taxonomy sprint, where our synonym support showed itself to be useless for nearly all use cases (apart from this one of course), I'm thinking we should move synonyms into a contrib module for D7 - this would be a cool feature for such a module, which could also handle some other stuff with synonyms too.

#504078: Synonym collapsing in core marked as duplicate.

This is the patch from http://drupal.org/node/504078#comment-1751434.

I've tested this and it works. Committed.

Note that this patch only updates the autocomplete functionality and not the backend. Users can still enter synonyms and they will be added as new terms. We are still thinking about a proper way to handle this.

Erm. Why is the second LOWER() missing from this?

-      ->where("LOWER(t.name) LIKE LOWER(:last_string)", array(':last_string' => '%' . $last_string . '%'))
+      // Select rows that either match by term or synonym name.
+      ->condition(db_or()
+	    ->where("LOWER(t.name) LIKE :last_string", array(':last_string' => '%' . $tag_last . '%'))
+	    ->where("LOWER(ts.name) LIKE :last_string", array(':last_string' => '%' . $tag_last . '%'))
+      )

Also, this is making an already slow query orders of magnitude slower. We really need a better solution for this.

Note: the second lower is not needed on MySQL, but would break on PostgreSQL.

+  $tag_last = drupal_strtolower(array_pop($tags_typed));

I'd rather see the synonym query only run if we don't have a current match for a tag, this at least means only one slow query at a time.

@Xano: because collation rules between PHP and the database might (and are) different, you have to run both LOWER()s in the database engine.

That means the current logic for finding matches needs a total rewrite.

@catch: Currently we only use one query, which is just a tad slower than the original query. I'm not sure if we should only search for synonyms if the typed tag doesn't match any terms. Especially with larger vocabularies (and the increased possibility of terms with nearly similar names) this can cause unwanted behaviour.

Can someone please update the status of this issue?

If I remember correctly, support for synonyms has been dropped from D7, so how do we continue here?

@asb Drupal 7 allows you to add fields to taxonomy terms, so you can still add a synonym field just like Drupal 6. But because D7 core doesn't come with a Synonym field anymore this issue would have to be dealt with in contrib.

Thanks for the update.

If even synonym handling itself moves to contrib, this issue isn't relevant for the castrated taxonomy.module in D7 and up. So the 'component' tag must not be 'taxonomy.module', right?

Since not even D6 plus existing contrib modules support synonym collapsing (see 'Synonym Collapsing' issue queue), this also can not be a bug report against core anymore (#47).

As I understand it, this is now

• either a wishlist for a not yet existing contrib module that provides a synonym field for D7,
• or a feature request for D8 taxonomy.module to re-implement synonyms plus synonym collapsing in D8 core.

What will it be?

There's no reason for it not to be both.

The 7.x-1.x-dev version of synonyms module already provides a synonym field for taxonomy terms and also adds synonyms to the search index. It seems like synonym collapsing could easily be added into this module too.
http://drupal.org/project/synonyms

Since synonyms no longer exist as such in core, I think this is for contrib now.

Related/f'ups: #857630: Synonym collapsing doesn't work ('Synonyms'); #289891: Taxonomy doesn't autocomplete after the module is activated, #694796: Synonym collapsing does not collapse synonyms ('Synonym Collapsing'); #1259778: Contextual Filters stil have synonyms as an option (removal of support for taxonomy term synonyms in 'Views').

Subheadings and list formatting added, primary contact linked