Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
This patch extends the l10n_packager module for packaging all projects/releases per language, works , on cron, has some admin ui, etc, etc..
Just in case, here's the full module,
https://devseed.svn.cvsdude.com/sandbox/drupal-6/l10n_tools/l10n_packager/
It is a remake of our old l10n_download module for l.d.o
Comment | File | Size | Author |
---|---|---|---|
#7 | update-translations.patch | 885 bytes | Gábor Hojtsy |
#6 | packager.patch | 29.93 KB | Gábor Hojtsy |
#4 | 594570_l10n_packager_3.patch | 27.63 KB | Jose Reyero |
#2 | l10n_packager_2.patch | 25.73 KB | Gábor Hojtsy |
l10n_packager.patch | 23.94 KB | Jose Reyero | |
Comments
Comment #1
Gábor HojtsyMarking as needs review. However, there was a pre-existing issue at #568994: Implement translation package export in l10n_packager which outlined some requirements, so not sure whether to mark that as duplicate or this as duplicate and ask to move the patch there.
Comment #2
Gábor HojtsyI've applied and looked at the patch. Tried to clean up the code (missing CVS ID tags, bad phpdoc, etc), and found some issues:
- not all L10N_PACKAGER constants are used, so I'm not clear on the purpose of them (I've even removed a pointless one from the top of the admin.inc)
- the language export is now only one of the packager tasks, so used more specific names there and moved to this new admin area
- localize.drupal.org needs a way to not run this on Drupal's cron but instead on its own cron from a specific machine. (this is explained in detail at #568994: Implement translation package export in l10n_packager), so I've added an option to disable running on cron; you've already had a way to disable cron by setting the interval to 0, but we'd need to be able to set the interval, but it will be applied to the non-Drupal-cron run
I've also fixed various smaller issues, but this still needs work:
- the purpose of all items is still not clear to me, needs more time to study
- the manual packager / mark to package feature's project selector is not consistent with the rest of the modules, and on the manual export, the release picker is a second form step (which has a patch suggestion to Ajaxize)
- all files are saved into the same directory without a further structure to help distribute the number of files (we have 33 languages already with 5000+ releases, so that is already 165.000 files, and we do plan to keep older release dumps there for some time, so this can grow pretty much; also we could grow quickly to 100 languages)
Looks generally good, but needs more cleanup.
Comment #3
Jose Reyero CreditAttribution: Jose Reyero commentedDone some improvements over the latest one:
- Export directory is now either relative to Drupal install or absolute, so this is more flexible and doesn't need to be inside site files anymore
- Added a configurable file path (and file name). There are some variables now to build a directory tree and to name the file.
Example (default): '%core/%project/%project-%release.%language.po'
Hey Gabor, if you think this looks good, can we just commit it and move on with smaller patches?
I know there are some more things we need to do, like keep track of which is the latest release for repackaging and think about #595644: Thinking about the release / translation workflow but I think in current state you could already do some manual packaging and see how the whole thing works...
Comment #4
Jose Reyero CreditAttribution: Jose Reyero commentedSorry, forgot to post the patch
Comment #5
Gábor HojtsyTagging.
Comment #6
Gábor HojtsyOk, worked a lot more on this patch:
- sticked the export to all-in-one format, since the file name template was forcing that anyway; also set it to use the compact format for export, since Drupal can import that and it takes way less disk space / bandwidth
- found a NOTICE issue in the compact export due to the above
- figured that your last change code was a bit overcomplicated; turns out the l10n_community code was in error of not always setting the uid_approved and time_approved; it was only set when an existing previous string was there but a new one was added not when a new one was added fresh (@todo: needs update function before deployed)
- moved settings to not be the default packaging tab
- packaging form now contains both language and project rebuild forms at once
- fixed project autocomplete description and code to properly handle that a title is actually input there
- modified schema to have a "status" column instead of "package" to be more clear on what does that mean
- other minor changes like use of a nonexistent $download variable where $files should have been used
We definitely need to do in the short term:
- have an outside CLI packager, so we can make the packaging independent of the Drupal cron run; drupal.org needs to have this running only on one of the webnodes (where the filesystem is local) and also, this will take huge amounts of time, so we need to possibly run it more often initially at least
- add upgrade path to l10n_server to fix the badly recorded uid_approved and time_approved records so the current latest time SQL will work properly; something like
UPDATE l10n_community_translation SET uid_approved = uid_entered, time_approved = time_entered WHERE is_suggestion = 0 and uid_approved = 0;
basically solving the problem that initial "instant" translations (those which were never suggestions originally) were recorded without approval data while further instant translations had this approval dataI'm highly interested also in:
- writing tests for the packager
- possibly renaming the 'checked' column, and/or try to better document what it means exactly
- fix the compact export mode to not export empty translations since that could help to reduce the file size again dramatically for incomplete languages
I'm interested also in:
- fixing the project / release selector to use the Ajax code suggested for the export screen (we should fix this at both places)
Because the attached patch is now committed (and all existing tests still pass), we can close this down and work on the above in follow up patches. Thanks again for your continued help!
Comment #7
Gábor HojtsyHere is the update path I'm adding in.
Comment #8
Gábor HojtsyResults of the first run (which I took the liberty to run via a browser, so it was slower then just running from www1 probably):
Now moving on to #593282: Set up syncing of l10n_packager files to ftp.drupal.org so that we can test with a client once the code is there. For later packaging, we still need the cli script explained above, since running from a browser is just too time consuming due to the file system setup.
Comment #10
Gábor HojtsyChanged tag.