Make it possible to fetch .po files for dev modules/core

Yes, the module don't know at file to download. Since the translation only exist for release version and not the cvs version.

I'm saying that here: ftp://ftp.drupal.org/pub/drupal/files/translations/6.x/xmlsitemap/ don't have any dev file translation. The same goes to http://localize.drupal.org.

But you use the last translation version available, it may be not accurate for all cases, but if the patch add an option i'm fine with it.

I understand the use case and support this functionality. But I have some remarks on the code:
* Don't use a radio button for enable/disable. Use a checkbox instead.
* When the module is a HEAD release I expect (have not tested) $data['info']['version'] to be 'HEAD'. strpos($data['info']['version'],"HEAD") results in 0 (zero).
* update_get_available() is found in locale.module. No need to include files.
* Check your code style with Coder
* Add some comments to the code. Especially the processing of $projects_info, it is not obvious.

This is really a top problem that many more people will face. And we clearly do not have a solution for this yet. Looking at the patch, I have a few issues:

1. Update module is not required and might not be enabled. We either need to make l10n_update module dependent on update.module or make this feature dependent on update.module.

2. This feature will only work I believe if the module had a tagged release prior (or later?) to the dev version being run for the same major compatibility (6.x, 7.x, etc). Think of 7.x modules coming out now, and that we suggest everybody for Drupal 7 to use this module. Now those modules in many cases have no tagged release yet in any way. I'm not saying we need to solve all the worlds problems, just pointing out the limitation of this approach.

3. The patch has some whitespace issues.

Marking it needs work for (1) and (2) at least. Thanks for your efforts btw. I think this is one of the major problems we need to solve still..

Hi, I would find this functionality very useful !
For example I'm using Ubercart 6.x-2.x-dev which is essentially a bug-fix and optimization release of the stable 6.x-2.4 branch (which is 5 months older) and I'd like to have automatic translations.

Regarding the Gábor Hojtsy issues:
1) this feature should be dependent on update.module
2) this feature could have a complex logic for choosing which stable release to fetch translations from, but I think, for starting, it should include at least the following rules:
- given the module is a.b-c.d-dev (-dev suffix) version
- then search for a a.b-c.d'' (no suffix) version to fetch from, where d'' is the same of d (if d is numeric) or it is the max value among others (if d is "x")
- if nothing is found (because the module is a new dev minor branch), then search for a a.b-(c-1).d'' version, where d'' is the max value among others. Repeat until (c-1) = 0.
- if nothing is found yet (because the module is a new dev major branch), then search for a a.(b-1)-c''.d'' version, where c'' and d'' are the max values among others. Repeat until (b-1) = 0.
- not sure what to do else (when there are only a-1 versions)...

Just my 2 cents.

A definite side effect of importing translations for unmatching versions is that you'll end up with translations for stuff that is not anymore in the module. Code is refactored in big ways, so a previous translation might not be very applicable. I agree it will fill in some of the strings most probably though, so it is helpful. We should get people aware that we import stuff that will most probably contain unused strings too and try and point people towards string cleanup functionality (once built that is :). The relevant issue for that is at #161114: Add string cleanup feature for lingering unused strings and it does apply for cases when you update from one version of the module to another the same way, so its not just for when .po files imported are not for the exact version used.

@mandreato: I think your suggestion sounds good, maybe we'll need to implement some kind of metafile on the server side at the end to have an index of files related which would speed up looking at files for a release.

Yes: trying to adapt translations from other releases to dev could populate database with a lot of orphan strings and yes: those cleanup functionality into potx module could resolve the problem.

I think that both the issues should go in tandem: let the users gain the convenience of dev translations, but advice them that a cleanup process will be required. Maybe a warning message should pop up in the /admin/build/translate/update in the following manner:

Warning: there are nnn unused localization strings since last check. Press this button to delete them.

By the way: such a functionality should be implemented even without dev translation, because (as you suggested in the #161114 thread) orphan strings increase day after day, update after update... Just maybe it would be better to have it in Localization update module instead of Translation template extractor.

Yes, well, before localize.drupal.org / l10n_update, you importing orphan strings was basically by design, translation files were often outdated. Now with l10n_update, we currently enforce current string sets, which is the other extreme :) So just falling back on previous versions would basically recreate the previous situation, we are not going to be worse off :)

Unfortunately though there are basically two ways we can implement orphan string detection:

(A) We mark all strings orphan candidates and watch as they are used, mark them non-orphan. After some time (weeks?), we can tell which strings are *likely* orphan. This can maybe done in l10n_update. But of course its not too accurate, since some parts of the code will not be used in weeks. Critical error messages, etc.

(B) Translation template extractor has the tools to look at all the module source code you have for enabled modules, and tell strings which are sure to be in those modules. Then we can mark the rest orphan candidates, look at them for weeks again, and do like (A). This is due to modules using t() for non-code based data unfortunately. We tell them not to do it, but they do it.

So with (B) we can get to better results sooner, but without leaving it alone to observe the site for some time in (A) and (B), we'll potentially throw away data too easily.

Anyway, my point was that solving the -dev version problem (plus when the given version has no translation file yet) is just recreating part of how translation files were distributed before sometimes in outdated forms, so we are not becoming worse off. There is no pre-requisite I think to implement the string cleanup, we should go forward with this and not wait on that completed.

I see...
Maybe we can think to a (B) way with a sort of API which can be recalled from (A) (creating an optional dependency). But I'm going far beyond the imagination :-)

Subscribe. This is really needed.

I would prefer a more flexible solution where user has control over the modules to update and the names of the language files. This would rock!

@SkyWombat: if they are not included that is a different bug, please submit another issue. Thanks!

I find this feature really useful too. You see, I always use dev versions (where/if available) for almost all my modules - I stick with stable care though. ...so, subscribing.

...btw, shouldn't this go against 7.x first?

Yes, it should.

Subscribe

that's a significant issue. subscribing

Subscribe

subscribe

A little update to #5, to work with D7.2 and the l10n_update 7.x-1.beta2:

-          if (!$projects_info)
-              $projects_info = update_get_available();
+              $projects_info = update_get_available(true);

sub

subscribe

Tagging for Drupal 8 multilingual initiative. This is needed for #1191488: META: Integrate l10n_update functionality in core.

subscribe

subscribe

subscribe

subscribe

Subscribing.

Subscribing.... does anyone have a status update on where this stands?

There were several comments regarding code style and some update suggestions before the last patch. Also, not much feedback on testing results with the patch. That is my understanding of where it stands. Needs work in short.

Subscribe. This is really needed for d7.

For what it worth, I needed this as well and I think it's a good feature until dev gets its proper release tag in the localize server.

Attached is a reworked patch for 6.x. Sorry, I haven't moved to D7 yet, but since the codebase should be very similar, I expect it to be a straightforward port to 7.x

I took ideas from #5 but I found on it quite a bit of issues. Also it has the proper drupal coding style.

I added a description to the setting to be dependent on the update module (and make it a disabled setting if update is not enabled) and also added some logic to properly download the latest translation for the major version.

Say you have notifications-2.x-dev, the latest is 4.0-beta7, but it should really download 2.3, which is what's doing now with this patch.

Also make the logic dependent on update module regardless of the setting.

Oops, small bug, attached corrected.

worth noting it applies to the latest stable release for 6.x

Patch looks generally good. Testing feedback needed.

Ok good starting point.. shortcomings were the usual
Notice: Undefined variable: dev_disabled in l10n_update_admin_settings_form() (line 209 of /var/www/localhost/htdocs/d7.git/sites/all/modules/l10n_update/l10n_update.admin.inc).
also the inability to deal with version strings generated from git_deploy module (like 7.x-3.0-rc1.16-dev) and not handling core translations.. attached patch is based on #40 but applies to 7.x version. Fully auto-updating translations for 100+ git submodules, yeay! now to sleep :)

+++ b/l10n_update.project.incundefined
@@ -42,7 +42,34 @@ function l10n_update_build_projects() {
+      if (preg_match("/^[6-9].x-([0-9]+)\..*-dev$/", $data['info']['version'], $matches)) {

I think this excludes the possibility of matching for Drupal 10, although I think that for when we get there this module should have evolved enough.

+++ b/l10n_update.project.incundefined
@@ -42,7 +42,34 @@ function l10n_update_build_projects() {
+      unset($release);

Is unsetting really necessary?

Will try it on 6.x

if (preg_match("/^[6-9].x-([0-9]+)\..*-dev$/", $data['info']['version'], $matches)) {

I think this excludes the possibility of matching for Drupal 10

Yes, indeed. I was thinking about if to support arbitrary Drupal versions...
so [0-9]+\.x should replace [6-9].x ;)

unset($release);

Is unsetting really necessary?

Yes, else it will overwrite correct version info of non-dev modules because they neither match .*-dev nor HEAD and previous $release is still around..

The patch works correctly on several sites with dev releases.

I don't see the need for this feature being an option. As a site owner I want as many strings translated as possible. As a builder I use dev releases to fix bugs that can not be worked around. The only cost is extra time and effort to collect available translation version. I see no major downside to enabling this features at all times.

Now that the translation version can be different from the installed module version, the module version numbers on the Translation update status page have the wrong version. They list the translation version instead of the module version. E.g. Devel 7.x-1.2 (translation version) instead of Devel 7.x-1.x-dev (module version). The attached patch fixes this and also includes the regex change of #44.

Fixing status.

Patch #45 works very fine on two Drupal 7.9-websites (1x XAMPP on Win7 local and 1x hosted on Linux).

l10n_update is now very good for Drupal 7. Please take this patch in the module.

Thank you very much.

Ok now the basic functionality is there, on to touching up the lo0ks. :)
Correctly showing installed module versions (patch #45) was a good enhancement, thx Sutharsan.
Now the collapsible project entries on the update page didn't look right (wrong status descriptions). Reworked the logic a bit and tried to clarify project status tracking:

• 'ok' => 'uptodate'
• 'update' => 'updatable'
• + 'available'

The code all in all is not overly beautiful (=optimized to use fewer variable assignments) but not performance relevant anyways, also the CSS/icons could be improved, but hey it works good enough for now.
...so please re-review :)

ok here's a quick and dirty comparative screenshot of the changes made in #48.. some information was plain wrong or misleading before.. please check if it makes sense for you.

The Text "This feature needs the update status module to be enabled." in patch #45 is obsolete wrong in Drupal 6+.
Because the module "Update Status" is part of Drupal 6 and 7 as the "Update module" in core.

[Edit] obsolete wrong[/Edit]

@wusel, having it in core doesn't mean it has to be enabled. You can have it disabled.

@eMPee584 any chance you can backport your patch to drupal 6?

@hanoii:

ok, but in Drupal7 there is no module "Update Status". In Drupal 7 it is called "Update manager".
So at the first moment, nobody can understand that text, if he/she is using Drupal 7.

I think, the text should be corrected in the Drupal7-version.

@hanoii: tried to, no idea with which level of success^^.. try this patch for D6 (untested).

uuuhm sorry that was the wrong one.

Why should we have "Fallback to latest point release for development versions" as an option at all? Just turn it on always. Why or when will this not be used?

@eMPee584:

can we get the patch for the change in #49 for Drupal7.9?

Thanks in advance

@mkalbere, can you explain why you made fetching translations for dev releases an option?

[edit]: changed name of initial author.

@wusel: the screenshot in #49 is D7 with applied patch #48. The version in #55 is an untested D6 port of the same thing.
@sutharsan: well in #3, int asked for an option ^^
If you look at it, theWrongThing™ might indeed happen when the development version has substantial string changes from the last release - but the worst case here is a comparable small amount of unused translations in the database.
Would the LOCALE_IMPORT_OVERWRITE mode purge these orphaned translation strings?

Unfortunately unused strings are not purged, there is a request for that in the issue queue.
Strings in dev releases may not be stable, but at the same time it does take some time before new strings become translated. I expect that pollution of the translation database due to changing strings in dev releases will be minimal compared to string changes between point releases.

It is *very* hard to tell what is unusued and what is not, even if you also have potx module there to try and help tell you...

OK then. This is probably even less a problem since supposedly only development sites should use module dev releases (in theory).
Rerolled patch for both D6 and D7 - no more optional C;

o0Ops. missing )
trap: APC opcode cache with apc.stat=0 requires apache restart for every file change ^^

@eMPee584 I tried the patch for D6, it contains one but as $available[$lang] is an object on D6.

I see all modules as its latest version, there's no reference of them being -dev, is that fine? Will review it a bit as I think there's some other issues.

Fixed version for D6 of #63.

@hanoii: thx for testing. Whitespace is moderately screwed up in the patch you posted though, here's a fixed version. And please give more detail about your setup - are you using dev tarballs or git checkouts?

@eMPee584 git checkouts.

I use #63 Drupal 7 version, and works well! :)

@honoii
you might try need git_deploy with patch http://drupal.org/files/git_deploy-add-caching.patch ...
& thx for testing nevergone! C:

@eMPee584 why? Seems to be working without that module. When asking about git or dev tarballs did you mean what kind of dev modules I want to update translations on? If that's the case then they are mostly -dev. I do have this module checkout by git and very few others.

Why git deploy then?

...git_deploy just generates proper version tags for git checkouts. That means the update status (with patch from http://drupal.org/node/1343768) and translation localization update module can properly handle those git modules.

The 7.x version is now committed.

The 6.x version displays a wrong module version number on the translation update status page.
To reproduce: install webform 6.x-2.8, download translation. Download 6.x-2.x-dev, Refresh information. Optionally repeat for 6.x-3.10 and 3.x-dev versions. See #45
I don't have the time now to fix it myselves.

@sutharsan: can you please post a screen shot of the 6.x fail? don't have a working d6 on my hands right now..

The theme_l10n_update_project_status() uses $project->info but no such thing exists. $project only contains the data of the l10n_update_project table.

Webform 2.x-dev installed, before updating.

Webform 2.x-dev installed, after updating.

I have some second thoughts about the module version number. Looking at the direction the issue #1029554: Translations update feature user experience is heading, it is likely that the module version number will be removed from the translation update page. How relevant is the module version number if the task at hand is the translation. The translation version numbers are probably all we need to decide about updating.

I have some second thoughts about the module version number. Looking at the direction the issue #1029554: Translations update feature user experience is heading, it is likely that the module version number will be removed from the translation update page. How relevant is the module version number if the task at hand is the translation. The translation version numbers are probably all we need to decide about updating.

But isn't translation numbers dependent on module versions?

Translations have the same version numbers as the modules. But in between module releases translations are updated to. Is that what you mean with 'translation numbers'? Are the module numbers used or are they important to decide whether to update a the translations of all modules? The translation update status (green/yellow/gray + icon) is available to indicate the status.

Hiding available information (versions) seems not beneficial to me.. Makes debugging l10n state of a site harder.
mmh i'll try and get tested the D6 patch..

Bug sighted on 7.x dev version.

Using module views 3.x dev version; Localization update unable to fallback on the available .po file for views 3.1

@chriscalip, I can't reproduce this. When installing the current dev without prio views installation the 3.1 translation is downloaded. When upgrading from 3.0-rc3 to the current dev, the 3.1 translation is advised as the latest translation. Both tests were carried out with two languages simultaneously (Dutch and German).

@Sutharsan

did it with latest drupal 7 stable, views 3.x dev, spanish language pack.

Using the latest dev release of Localization Update module? Did you upgrade Views? If so, what was the previous version? I need to reconstruct the situation.

Unable to reproduce with Spanish.

I've committed #66 to the 6.x-1.x branch. So left I the module version number as it was (#79). No that I changed my opinion ;) but we should not mix this with this patch. Improving the interface is handled in #1029554: Translations update feature user experience. Closing the issue now.

@chriscalip, if you continue to have difficulties getting dev releases translated, feel free to re-open the issue or create a new one.

Is that in D7 branch? Seems not to work with me...

@SebCorbin,
If you have a problem, please create a new issue and include information to enable us to reproduce your problem. And yes, this is available in 7.x too.

Closing the issue again.