Import module/theme translations when enabled (the first time)

Updated to post menu patch state.

Doh, install.txt cruft was left in the patch...

Patch applies cleanly against latest CVS.
I turned on "devel" module before applying the patch, and the patch didn't cause "devel"'s po to be imported.
I then fetched the latest version of "gsitemap" from CVS, activated that, and... it pulled in the translation. However, it turned out to look funny in the translation interface (see attachment) and it doesn't seem to find anything when searching for translated strings. I repeated for the image module, with the same result. This might be due to Drupal 6 incompatibilities though ('image' gave me several PHP errors, so that is certainly not Drupal 6 ready). I'll try again later with another clean install, trying to see how it behaves with "devel" (unless someone can give me a hint as to which modules should work with Drupal 6).

Fresco, thanks for testing! What languages do you have set up? That could be key for understanding the problem in your attached screenshot (although it is not related to automatic importing, it might be a bug in locale.inc). Did devel.module have files in a po/ subdolfer ending in your language name and a dot po? Like it.po or devel-module.it.po if you also have Italian set up? If not, then that is the answer why you have no translations imported. The code obviously only import translations for languages you have set up. Thanks for the feedback!

I have only set up Danish (da) (as can be seen in the screenshot). "Devel" module does have a da.po in its po folder, however I mentioned that it doesn't pull it in as you have described above that it shouldn't do that. So my note was mostly meant to confirm that the patch acts as described (at least in regard to already installed modules). :)

Also, what this patch is supposed to do, is something I've wanted for a very, very long time - of course I'll help getting it submitted before code freeze!

@gabor : Currently, the import of a .po file is an atomic operation.
We could probably rework this into something like :
- atomic step is 'process x lines of the .po file' (mutlistep batch operation)
- one batch set handles the import of one .po file, and you get a 'real' progress feedback on the import
- several .po files get handled in as many batch sets.
This would remove a few cases where the server 'works' for some time without the user getting any feedback.

I can work on this if you are interested - (i think we will have to provide something similar for .po export anyway, since _those_ can get massive)

I tried patching and doesn't apply to HEAD but HUNKS are succesful. I then tried enabling some modules with Czech language enabled and importing translations works.

I then tried disabling Comment module and enabling Forum module. Importing works even after confirm page, but i am not sure it imported translations for dependant modules too. Forum + Comment = 2 modules, but "Importing translations, remaining 0 of 1".

I think importing on language enabling is important too. Let's add "checkbox" Import translations on admin/settings/language/add if some translations are found during rendering this page?

Fresco: now I see. Yes, this patch is intended to import *new stuff*. This coupled with the install time translation import which is already included in Drupal 6 could ensure that all your installed modules have their translations imported.

yched: Translation packages for Drupal core will include small PO files, like in the packages meba and I were attaching to other importing related issues. I don't think there is a point in breaking up the import of one PO file, it could very well happen that a PO file gets imported out of a batch. (As I noted in the comment to the issue, we could import a PO file in one go without starting a batch if the module/theme change only involves one PO file).

Dries reviewed the patch for me and noted that we might not need the "magic" of knowing what modules exactly are we working with, but we could import all PO files to refresh the cache again. I noted that if we have 4 languages and 15 modules (which is really not unrealistic), we would get around 60 PO files (or more or less, depending on translation availability and whether a module's translation is in multiple files). So we would need to reimport all 60 files if we install a new module and import around 4 new files for the languages for the new module. So this is not really doable.

But Dries does have a point in that we will need to support updates. When people update from Drupal 5 to Drupal 6, they should be able to update translations of Drupal core and all modules and themes. We can look into whether the module needs an update, and also import translations along when the module is updated. What if only the translations are updated? What if update.php does not offer a module for update because it has no DB updates, but any of the PO files for your languages changed? How do we know that? Well, unfortunately we only know that if we have a list of PO files already imported with their path and md5 hash for example. So we know that the hash differs, and we need to reimport.

Dries though that is important to bring up, because it would have consequences in this patch too. Well, I think installing and enabling modules all at once is in itself a usability improvement. So if you need to confirm dependencies, you don't get a 'Configuration changes saved' message on top of the confirm form, and don't get confused that if settings already changed, why do you need to confirm... So changes are only done if a confirm is not required or otherwise after the confirm. Whether we get the list of new PO files by looking at the updated system table, getting files from the file system for all modules and then comparing with the already imported PO file list as opposed to getting the component list directly from system module might not look like any different. It could be different though in some edge cases, when you have updated translations on modules you are not touching on the modules form, but locale would still import them... So if we are not working from the list of modules actually installed on the request, we can have unintended side effects.

Anyway, having a list of already imported PO files with their filepath and md5 hash could be useful for Drupal and module/theme updates, but I don't think we should make use of that here (apart from updating that list). So I would rather include such a list and management code for it, once we have this change in, and we can work on the language addition and update code for automatic imports.

I am updating the patch to include the documentation pieces Dries noted to be missing, will attach later.

Meba: you probably experienced that you only got one file imported because comment module translations were imported already by the installer. Only *newly installed* modules get their PO files imported in this patch. If you had a module enabled previously and now you enable it again, the translation will not get imported.

Dries might say that this is a hidden dependency. What if I disable a module and clean out it's strings to help performance (suppose I do it somehow on the SQL level, there is no such possibility in Drupal unfortunately). Now when I enable the module again, translations will not get imported, and I really cannot do anything but import by hand. (If a list of imported PO files would be available in a table, I could remove the corresponding line and get the PO file imported again). This gets into very hackish territories now... So I am not sure we should support that. (But that table will eventually materialize soon, so we can support module updates. I don't think it has a place in this patch.

Updated patch against current 6.x-dev. **Same code**, but more documentation, as Dries requested. I tested both Drupal install time importing and module install time importing and all seem to work.

Now applies smoothly and works without a problem.

Dries reviewed the patch for me and noted that we might not need the "magic" of knowing what modules exactly are we working with, but we could import all PO files to refresh the cache again. I noted that if we have 4 languages and 15 modules (which is really not unrealistic), we would get around 60 PO files (or more or less, depending on translation availability and whether a module's translation is in multiple files). So we would need to reimport all 60 files if we install a new module and import around 4 new files for the languages for the new module. So this is not really doable.

Yep, it would make it slower to install a Drupal module, but is that a /big/ problem? If it takes 2 seconds to active rather than 0.3 seconds to activate, it might not be that big of a problem. If it takes 10 seconds to activate, it would be a problem, of course. Just thinking up loud.

Personally, I'd really love an 'Update my translations' button that I can hit to update/sync my translations. :) Might be a good idea to implement that first, so we know how long it would take to synchronize all the translations. This isn't a requirement for this patch to get committed, but it could be a useful exercise.

That said, this patch looks good. My only tiny concern left is that we do a /poor/ job communicating with the user. I might be a small but subtle improvement to tell the user that we automatically (un)loaded translations. It is the kind of message that might make a user go: "Oh, cool!" and it helps create transparency about what happens under the hood.

I've looked at the code twice now, and it looks ready. Feel free to commit this, Gabor.

Added a finished callback to the batch, so we set a message to the user informing about the translation files we imported. Committed the attached patch.

The follow up to import translation files when you add a new language is here: http://drupal.org/node/145671 Please review! (Warning! it is soooo simple :)