There are several small things about locale caching, which this patch attempts to clean up:
Cache clearing:
- Inconsistent: While editing or deleting a string from admin UI, cache is rebuilt straight away by calling locale_refresh_cache(), while .po import and runtime-discovered strings just clear the cache and let the work to next page request.
- Incorrect: All the three cases, where cache is flushed (a .po import, new string discovered, and new string registered for current Drupal version) are affecting source (english) strings, so a full cache-rebuild is necessary. But however, the existing code only flushes a partial cache for current language, which means a hidden performance-bug:
When there are more languages set up, and a new string is added, only the cache for current language is flushed. If following page-requests are performed with another language, the cache for that language is still here and used, resulting in no rebuild, and looking-up the new string in database on every occurence - until a page request in the originally flushed language comes and triggers the rebuild at last. Note that this is just a small, untested problem, which I discovered while analyzing the code; still a bug IMO, though.
So, this patch changes all the places to consistently just flush cache, and WHOLE cache (using the cache_clear_all() capability for wildcard, meaning no additional query). This fixes the above, plus reduces actual cache rebuilds to only just one place inside locale() - allowing for further optimizations described below. It's also consistent with how the whole Drupal caching works, I believe.
Cache rebuilding - performance:
- The existing code re-builds cache for all languages together, which means unnecessary overhead in some cases. For example if admin edits strings, he gets full cache rebuild for all installed languages on each string-submit - only to be flushed by another submit half a minute later. There's clearly no need to re-build all languages until really requested. Also note that the rebuilds for all languages are concentrated in a single page request, which might be a problem if several languages are set up.
- There's unnecessary extra query executed: After the rebuild is done, locale() reads the recently-created data back via cache_get() instead of just passing it from locale_refresh_cache(). This means extra query (plus un-serializing a rather big array).
So, this patch changes the rebuild to be done for requested language only. This is only allowed by the first (consistency) part of this patch - now that we have the rebuild done only on a single place in locale(), we know exactly which language is necessary.
The benefits of this change are:
- We no more need to collect enabled languages (a call to language_list() executing it's foreach loop, let alone the query in case of it being the only use on a page).
- We do only what's needed, never re-build a cache that is never requested. Also the rebuilding overhead is better distributed between pages, making the time-per-page mostly independent of number of languages.
- We don't need to extract current language from the processed bulk of data, as there's only just one language now. So cache_get() (with it's extra query) is gone, and no other code needed instead.
- The remaining code of locale_refresh_cache() is only just one query and a simple loop. Since the function is now called from only one place (and that's by its nature after the cleanup), needs a new argument and return value, and doesn't serve the cache-flushing purpose anymore, I see no reason to keep it as a separate function at all. I just included the code directly into locale(), which allowed for further code simplification, building the array directly, without passing from a function and so on.
The only problem to consider is the locale_refresh_cache() function being listed in API documentation. I searched core, and after this patch there's no code using it anymore. If still needed, it may (and should, for the reasons mentioned above) be replaced by a simple cache_clear_all() call, so I don't see this as a problem.
If we want to keep the function for API-legacy/freeze purposes/reasons, then I can roll a patch with dummy code like this:
/*
* Dummy function - to be removed in Drupal 7
* ...(description here)....
*/
function locale_refresh_cache() {
cache_clear_all('locale:', 'cache', TRUE);
}That will keep the existing functionality, but I really doubt if this is necessary.
Otherwise this patch doesn't break anything, even cache flushing is not necessary for testing. I tested all I can think of...
If going to test this - just enable locale module, add a few languages, perhaps import/edit some strings and browse a bit... You should see no change and nothing broken. It's all only just about translated strings shown on pages, no change to the database.
| Comment | File | Size | Author |
|---|---|---|---|
| locale-cache-handling.patch | 4.8 KB | JirkaRybka |
Comments
Comment #1
JirkaRybka commentedOne more idea - if we want to be really strict about keeping locale_refresh_cache(), then we can leave it's existing code unchanged. It'll still work as expected, the only change is that optimized locale() no more needs it (so it probably needs to be marked as 'deprecated'). I can roll this into the patch easily too, if we decide this is the way to go.
Now, some review/feedback is really needed here ;)
Comment #2
gábor hojtsyI have read through your explanation here, and looked into the patch. So far these improvements look good and indeed bring consistency to how locale uses caching. I don't think we need to keep a function which is better kept in locale() for performance reasons. Thanks for looking into these issues. I hope to take some more time to look into this and test the patch soon.
Comment #3
gábor hojtsyLooked through this again, and all looks nice and a great consistency and accuracy improvement, so committed.
Please update the module update docs, so people know about locale_refresh_cache() replaced by cache_clear_all('locale:', 'cache', TRUE); Marking needs work for this reason, as usual. Mark fixed, when done.
Comment #4
JirkaRybka commentedI just added a comment at http://drupal.org/node/114774#comment-281060 for quick reference, having no permission to do more. So leaving this issue open until someone from documentation team consider how (and whether or not) to integrate this information better.
Comment #5
gábor hojtsyI think this is fine. Whenever I have the time to document, I'll take a stab at moving this to the node and also to document other locale changes which were left undocumented.
Comment #6
(not verified) commentedComment #7
Mumonkan commentedi am not sure if this is the right place to put this, but i just spent a frustrating hour of downtime on a production site, and felt i should post something.
during some heavy usage of the site, we saw our mysql instance fill up (800 connections!) all with one query, which we traced back to locale_refresh_cache() due to its telltale
LENGTH(s.source) < 75.i think i know what is going on. if you look at the code in drupal 5 for locale_refresh_cache(), you will see it is ripe for a race condition. namely, assuming the cache for locale has just been cleared, user A comes in (via locale() i assume) and wants locale cache rebuilt. this takes time (before the cache_set happens). if user B comes in wanting the same thing, a second attempt is made to rebuild the locale cache; and so on...
maybe this is why locale_refresh_cache was taken out of drupal 6 ???
is there any fix for drupal 5?? please help!
thanks,
-jon
Comment #8
JirkaRybka commentedI would say - open a new issue, and set this back to it's original status. I proposed this change merely to clean up code for consistency, and make the flow more straightforward (removing some unnecessary rebuilds in the progress, and winning a little performance gain), but I'm not aware of any considerations targetted directly to possible race conditions. I even don't know whether the 6.x code would be any better in your case.
Also this was a 6.x patch, so it should stay there for reference, unless going to be ported (which is not the case here).
Otherwise, I haven't anything to say to your problem, so leaving that to others :-)