Add drush integration for extraction

The command line script does not run in the context of the website, so it does not have a connection to the database. Therefore it is not currently possible to get translations in the command line version.

We can however migrate the command line script to be a drush (http://drupal.org/project/drush) command instead, which could then access the database. Basically no need to keep our own command line wrapper when we can just reuse drush :) Retitled issue for this.

Here is a quick attempt at adding this feature. It adds a potx.drush.inc to run with drush in place of the potx-cli.php that was the standalone executable. Since now we have the full potx runtime in the Drupal directory, we need to exclude all the files from potx, so the self-skipper also needed updating.

The current status reporting does direct output printing, not using drush_print(). Not sure how big of an issue is that. Reviews are welcome!

Ps. this only rebuilds the existing functionality as a drush script, and then we can extend this with support for exporting translations in .po files. It should be pretty trivial beyond this patch :)

Ps. we will also need to update the README to reflect this new way of command line support.

BTW the drush command help looks like this

$ drush help potx
Generates translation templates from the given Drupal source code in the current working directory.
Usage: drush potx [potx-mode] [files]

Where the possible potx modes are:
 single    Single file output mode, every file folded into the single output file (default).
 multiple  Multiple file output mode, .info files folded into module .pot files.
 core      Drupal core output mode, .info files folded into general.pot.

If no files are specified, potx will autodiscover files from the current working directory. You can specify
concrete files to look at to limit the scope of the operation.

You want to enrich potx_drush_command() with the available arguments, options, and examples. See core commands for an example. basically, move info from unstructured help hook to potx_drush_command().

Subscribe.

I'm also thinking we should tap into Drupal's knowledge of modules and take a module name instead of a file path to traverse. eg. instead of drush potx single sites/all/modules/og I'd do drush potx single devel. I'd love to start out with an extensible way to do things, so I'm a bit reluctant to commit this patch right away. We need some "command design" here, or UX for the command line in other words, so we will not end up with broken scheduled jobs with people if we change command syntax. How is this handled in other drush supportive projects? Am I overly cautious?

Drush integration and #6 would be great steps forward here. I bet we could figure out some kind of backwards compatibility for the command that would add a warning about transitioning to the new command format.

Would be a great feature! +1

#2: potx-drush.patch queued for re-testing.

subscribing

Opened a 7.x-2.x branch for the module to do more drastic feature reorganization / API changes / API cleanup. Removed the potx-cli.php file from that branch in preparation of including actual drush support. There is no excuse for supporting a one-off obscure command line PHP thing... This will hopefully eventually be backported to Drupal 6 too, but currently on 7.x-2.x now. Will work on including the drush support code later.

subscribe

subscribe

subscribe

@Gábor Hojtsy: what's the status on this?

Per conversation with @Gábor Hojtsy I've re-rolled #2 for 6.x-3.x. Have also added Drush command options 'files' and 'modules' so you can pass specific files or module names. Passing neither uses drush_cwd() to extract strings from files in the current directory.

Here's the output of drush help potx:

Generates translation templates from the given Drupal source code in the current working directory.

Possible potx modes are:
 single    Single file output mode, every file folded into the single output file (default).
 multiple  Multiple file output mode, .info files folded into module .pot files.
 core      Drupal core output mode, .info files folded into general.pot.

If no files are specified, potx will autodiscover files from the current working directory. You can specify concrete files
to look at to limit the scope of the operation.

Examples:
 potx single                               Extract translatable strings from applicable files in current directory and
                                           write to single output file
 potx multiple --modules=example           Extract translatable strings from applicable files of example module and write
                                           to module-specific output file.
 potx                                      Extract translatable strings from example.module and write to single output
 --files=sites/all/modules/example/exampl  file.
 e.module


Arguments:
 mode                                      potx output mode e.g. single multiple core


Options:
 --files                                   Comma delimited list of files to extract translatable strings from.
 --modules                                 Comma delimited list of modules to extract translatable strings from.

A little long but wasn't sure whether to put mode as a command option or keep it as an argument.

Quick update with drush_print() instead of potx_status().

Retry

Bah, hadn't merged latest 6.x-3.x commit, sorry. This is better.

This looks pretty cool in itself. I think we should take a folder option as well and an API version option. The combination of these would allow us to parse the Drupal 8 codebase from a Drupal 6 setup for example so long as they are on the same filesystem.

Fix for build mode bug and now supporting options folder and api:

Generates translation templates from the given Drupal source code in the current working directory.

Possible potx modes are:
 single    Single file output mode, every file folded into the single output file (default).
 multiple  Multiple file output mode, .info files folded into module .pot files.
 core      Drupal core output mode, .info files folded into general.pot.

If no files are specified, potx will autodiscover files from the current working directory. You can specify concrete files
to look at to limit the scope of the operation.

Examples:
 potx single                               Extract translatable strings from applicable files in current directory and    
                                           write to single output file                                                    
 potx multiple --modules=example           Extract translatable strings from applicable files of example module and write 
                                           to module-specific output file.                                                
 potx                                      Extract translatable strings from example.module and write to single output    
 --files=sites/all/modules/example/exampl  file.                                                                          
 e.module                                                                                                                 
 potx single --api=8                       Extract strings from folder projects/drupal/8 using API version 8.             
 --folder=projects/drupal/8                                                                                               


Arguments:
 mode                                      potx output mode e.g. single multiple core 


Options:
 --api                                     Drupal core version to use for extraction settings.                         
 --files                                   Comma delimited list of files to extract translatable strings from.         
 --folder                                  Folder to begin translation extraction in. When no other option is set this 
                                           defaults to current directory.                                              
 --modules                                 Comma delimited list of modules to extract translatable strings from.       

Amazing, committed! Fabulous!

Expanded patch for call _potx_process_file() to pass $api_option

Woops! Here's the updated patch, fixes case where api_option isn't sent to file processing.

Superb, thanks! Committed.

Should be on 7.x-2.x too of course :)

Ok here's a reroll for 7.x-2.x . Well actually it's more or less a complete rewrite since the API's in 7.x-2.x are quite different. Please check if everything is appropriate.

Forgot to remove some debug statement. Or would it maybe make sense to implement verbosing as a feature/drush option?

I was hoping to add the module as a custom command to my general drush installation, removing the requirement to install the module in every site I want to extract translation strings for and just have it available as a generic drush command. I placed the contents of the module directory in the commands directory of my drush installation (/usr/share/php/drush/commands/potx) and called 'drush potx' just to see what happens. Unfortunately, it broke:

include_once(/potx.inc): failed to open stream: No such file or directory potx.drush.inc:67                                           [warning]
include_once(): Failed opening '/potx.inc' for inclusion (include_path='.:/usr/share/php:/usr/share/pear') potx.drush.inc:67          [warning]
PHP Fatal error:  Call to undefined function potx_parser_source_files() in /usr/share/php/drush/commands/potx/potx.drush.inc on line 107
PHP Stack trace:
PHP   1. {main}() /usr/share/php/drush/drush.php:0
PHP   2. drush_main() /usr/share/php/drush/drush.php:16
PHP   3. _drush_bootstrap_and_dispatch() /usr/share/php/drush/drush.php:61
PHP   4. drush_dispatch() /usr/share/php/drush/drush.php:92
PHP   5. call_user_func_array() /usr/share/php/drush/includes/command.inc:175
PHP   6. potx_drush_extract() /usr/share/php/drush/includes/command.inc:175
Drush command terminated abnormally due to an unrecoverable error.                                                                    [error]
Error: Call to undefined function potx_parser_source_files() in /usr/share/php/drush/commands/potx/potx.drush.inc, line 107

Unfortunately, I don't have time to debug right now, but I thought I'd let you know. This is with the patch from #30.

I figured the fix might be fairly simple, and it was. I replaced the call to drupal_get_path with a call to dirname(__FILE__). Attached is the updated patch.

Not sure why the last patch failed testing, it must've been against a later version of the repository. I've changed it so it only differs from the one in #30 for the one line that changes the call to drupal_get_path() into a call to dirname(__FILE__).

It looks like the extractor will repeat the extracted strings multiple times. The behavour I observed for a .pages.inc (file A), .module (file B) and .info (file C) files:

Strings for file A
Strings for file A
Strings for file B
Strings for file A
Strings for file B
Strings for file C
Strings for file A
Strings for file B
Strings for file C
Strings for file A
Strings for file B
Strings for file C

This may be slightly off, but I think what may be going on is that for each file it scans, it will also include the strings for the previous files. The directory I scanned contains a few more files, they just don't include translatable strings. I'm not certain if this is an issue with the drush patch, or with the extractor itself (I looked for any existing issues, but didn't see any). Maybe somebody can run with this info and provide another patch.

Here's a patch based on #34 which outputs the pot files in the directory where drush is ran if no module options are provided and also outputs the path of each created file (previously the files where all created at the webroot and I couldn't find them!). Also when ran with the multiple option, the file name is now the trailing directory name with the date (instead of "extract").

I haven't encountered the problem in #36 and haven't tested this patch when specifying modules.

If there are bugs with the drush integration, please fix those in the 6.x-3.x version which is our reference version and then work from there to port it again to 7.x.

Bugs in the 7.x-2.x version may not exist in the 6.x-3.x version because the first patch for 7.x-2.x isn't actually a port, it's a rewrite. Not 100% sure though, thus 6.x-3.x should be tested. Unfortunately i don't have time at the moment.

Using the unmodified potx.drush.inc file from the 6.x-3.x branch inside the 7.x-1.x branch, I did not observe the behavior reported in #36.

Is there any reason not to commit this file to the 7.x-1.x branch ?

Once done, this issue could be closed and Drush related issues could be handled like any other issues.

For reference (or usage un Drush Make files), the attached patch add a copy of the potx.drush.inc from the 6.x-3.x branch and should apply on the 7.x-1.x branch.

Ported potx.drush.inc to the 7.x-2.x branch. This is a closer match to the 6.x version than previous patches, and doesn't contain the bugs described in the comments.

Ported to 7.x-3.x as it has been branched directly from 6.x-3.x