This is a follow-up from http://drupal.org/node/171646, where I explained the background about locale tables being full of outdated garbage. We already solved most of my proposals for D6 in that issue, including removal of untranslated strings at the time of major-version upgrade (which should be repeated for D7 probably), and simple registering of string-history, VERSION-wise, resulting in reduced cache load.
Now I start this issue to implement the last of my proposals: UI allowing the admin to prune locale tables.
It's based on data already collected 'behind the scene' by the previously added code, and provides simply an overview of strings, divided into groups by Drupal version of last use, textgroup, and (un)translated status. Each item offers it's own 'delete' link (with the common 'Are you sure?' confirmation of course), so equipped with a matching help-text, the page allows admin to identify and prune old garbage easily. Additionally, the VERSION-history field is added to the Translate interface search-screen.
The purpose is both performance (reducing table sizes) and usability (keeping translation-work sane, especially by reducing irrelevant search results, and avoiding pointless dead-strings translation generally); the whole applies especially to long standing localized sites, with a few updates already done.
The patch is now against D6, as there's no D7 code yet - currently it works, but I created it just to have some basic version, my ideas written down, as a starting point for future D7 version. Attaching only for reference.
| Comment | File | Size | Author |
|---|---|---|---|
| #10 | 175798_locale_string_prune.patch | 9.6 KB | kscheirer |
| #1 | prune-screen5.jpg | 37.35 KB | JirkaRybka |
| locale-prune-screen.patch | 13.59 KB | JirkaRybka |
Show commandsStart within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
compare
Comments
Comment #1
JirkaRybka commentedAdding screenshot from a working instance (D6) on a test-install using real site's data. Might explain a bit more, if you prefer visual input. :-)
Comment #2
bas.hr commentedsubscribing
Comment #3
meba commentedsubscribing
Comment #4
nedjoComment #5
catchThis is great idea which we sadly missed for Drupal 7, moving up a version.
Comment #6
kscheirerThis really needs to get in soon. I'm not sure when we introduced VERSIONs to locale strings exactly - but when working with older sites its almost impossible to find the "correct" string to translate. The only way to tell which version a string is tagged with is by looking at the database rows directly.
This seems like a bug to me - 7's release is just going to make this problem worse - and as far as I know this issue isn't documented anywhere. We need this interface to properly use localized strings. I would say we need a backport to 6 as well.
Comment #7
kscheirerOk, pruning does need to go in, but its not critical by itself. Setting this issue back to where it was, and opened #993820: UI for locale string versions
Adding this information to the UI when searching for strings and the resulting table of results is absolutely critical - we can't hide that data from admins.
Comment #8
JirkaRybka commentedThe VERSION was introduced in 6.x (my patch) for the purpose of later pruning. My site had a bazillion of dead strings due to 5.x (I think) change of all t() placeholders, but not only... But since I came into the core development a bit late (code freeze), the maximum I managed was just recording VERSION, so that the pruning code (to be added later) already have the needed strings-history data. In fact, it probably only got in because it also helps performance by reducing locale cache size (currently the only real use for VERSION data).
I did a proof-of-concept pruning UI at the same time (initial post here), but it contains some MySQL-specific stuff, and was obviously too late for 6.x. I'm sure several people would suggest to release this as a contributed module, but I don't want to do that because:
1. If this was a separate module, no-one would ever care to find it and install it, so the root of the problem won't be really fixed.
2. If this was a separate module, it won't get considered as a patch for core again.
3. I'll be stuck maintaining it forever.
As much as I wanted to re-open this for 7.x and finish my work, my time was more and more limited, and my testing environment got incompatible with 7.x (being primarily a testing environment for production site changes, so upgrade wasn't desirable), locking me out of development somehow. Now with DBTNG and all the new stuff, getting up to speed again is beyond my free time possibilities.
So all I can do for now, is unassign myself (should've done long ago, only just forgot). Sorry for that.
Comment #9
kscheirerNo worries, that happens all the time. I would definitely appreciate a review on #993820: UI for locale string versions though, and then I'll see if I can get a patch for the pruning part too.
Comment #10
kscheirerPosting an interim patch, updating JirkaRybka's original patch to HEAD.
I have the main prune table displaying, but the confirm form doesn't work yet. Not sure how thats changed in D7. I probably won't have any more time for a bit, so anyone feel free to add to it :)
Comment #11
JirkaRybka commentedYou'll probably need to replace the cool but sadly just MySQL specific SQL "DELETE s FROM {locales_source} s LEFT JOIN {locales_target} ....." with something else, most probably a subquery like "DELETE FROM {locales_source} WHERE lid NOT IN(SELECT lid FROM .... JOIN ....)". I'm not sure how this is done in 7.x, and how it's going to perform, though.
Also I remeber adding an update function to 6.x, to delete all untranslated strings on 6.0 upgrade - untranslated strings are safe to remove, reducing the tables a bit in size. This needs to be repeated on 7.x (and every major version upgrade), not sure whether somebody already added it... (Original issue #171646: Performance: Locales pruning, where I learned my lesson about MySQL specific stuff as well, one of my very first ones for core.)
Just trying to provide random bits I remember from the initial patch time, like I said, I'm not really up-to-date now.
Comment #12
gábor hojtsyI think it is dangerous to do pruning based on this information, as it might remove important translations for say error messages, which are not displayed often, but would be pretty important to display in the right language when used. I'd suggest people use potx maybe cross-referenced with this information to make sure they actually remove strings which are even less likely to be needed. Not sure this should be done just based on the version info core has.
Comment #24
andypostComment #26
anybodyI think we fixed this in contrib with https://www.drupal.org/project/l10n_tools
Still it would be great to have this feature in core, if maintainers like it. We're open to improve and integrate the functionality. Until then simply use https://www.drupal.org/project/l10n_tools and help pushing things forward :)
Adding "Performance" tag like according to the issue summary.