files totaly different from old style

1. What should we do there? Have just the module file name?
2. You mean CVS header lines, or the "generated from" lines you quoted yourself?
3. Randomly? Or is there some noticable order in them anyway?
4. This is how msgcat works, extractor.php did export one-line strings all the time. If you run msgcat on the potx resulting PO file, you will get the line breaks as you expect from msgcat.

1. hass, with extractor.php you get paths relative to where you have put extractor.php. It could be in a subfolder in case you are translating a submodule, or in the root folder, so you get different paths. What we can do in potx.module is to generate paths as if you would put extractor.php into the folder you selected as POT generation root. That does not guarantee compatibility with extractor.php, but maybe works more like how *you* used extractor.php before.

2. Yes, that is exactly because of the join() error you already reported, and which I made a duplicate now.

3. Maybe you can attach a pot file done with the extractor+msgcat way and with potx module, so we can do a diff?

4. We are not using xgettext.

Just comitted a fix for subissue 2 here. I tested with several modules on my test site, and it seems to work nicely.

On top of yours here a "fix" for number 1, normalize the filenames in output to the directory we request a pot-file for.

That to everybody's liking?

Yes, that seems to be logical. Except that you would write a capital letter at the start of the documentation of $strip_prefix

+ * @param $strip_prefix
+ * An integer denoting the number of chars to strip form filepath for output

Otherwise seems to be fine, feel free to commit.

Hass, do we know at what column msgcat wraps? As far as I have found out, it wraps at "width of the output device" which is, well quite ambiguous.

Hass, I tried to point out that if you put your file in "sites/all/modules" in the hopes of getting translation files for all modules (which you got before with extractor.php to the file system), you will have your pot files generated with different locations, compared to putting extractor.php into "sites/all/modules/pathauto" for example. Shortening the path to all PO files for the modules you use can end up in different results, and noone told translators to put extractor.php to any particular place, so they could be creative.

Now we can enforce a path style with potx.module, which is better again then extractor.php possibly having any type of path style. I just tried to point out that what we come up might still have diffs with your previous practice, because the previous practice was largely undefined.

hass, you have a dream, I have a Google SoC project to do that :) So it will happen this summer. Still we need to fix it here, because it will not get right in the build system otherwise.

Commited my patch with Gábor's suggested documentation change.

I don't think we can do anything about #3, but the order should stay the same once the pot-files in use are those generated by potx, right?

#4 probably also shouldn't be too hard if we set a fixed number for the cutoff point. If we really want that.

Looked through the ordering stuff. Diffed the two files you sent me. Noticed that the locations make a good difference (but that is fixed, so run a big sites/all/modules/views/ replace to remove these). By looking at the remaining files the following issues appear:

- file list missing from top -> already fixed
- potx version has bad file name at top in parenthesis -> already fixed

- wrapping done on "extractor version"
- "extractor version" has following order: general.pot, translation file for files from subfolders, translation files from the same folder
- potx version has following order: translation files from the same folder, translation files from subfolders, general.pot strings are intermixed with others

So the wrapping account for most of the diffs, while reordeing accounts for some others. I put "extractor version" to quotes because it is really a combination of extractor.php and msgcat we are replacing with potx module's web based extractor. Ordering is not really surprising, because instructions said that you should use this msgcat line: msgcat --use-first general.pot [^g]*.pot (regardless of how buggy that is for module names starting with "g"). So no wonder general.pot stuff comes to the first position. The other files are used in file time order I guess, but extractor.php seems like doing files in the same order as potx.module does, that part of the code did not change.

But forgot to mention that the strings appearing in multiple files does not seem to appear at all in the potx generated version.

Hass, run latest potx.module on views module, to compare with your files. The repeated strings appear again, due to the bug fix done on the "generated from files" list. The "generated from files" is more accurate then what you have before with extractor.php and msgcat, because that only kept your general.pot list, but we have a complete list now.

Ordering and wrapping is surely different still. Ordering differences are mainly due to the two factors:

- Repeatedly appearing strings intermixed in the pot (not at the start as was before) -> we can help this by ordering the $strings array by number of occurances (that would still not be exactly as before, but would still push the repeated strings to the beginning).

- Mass ordering differences erupt from how msgcat and potx scan through files. Extractor generated files into the current folder. msgcat goes through them in alphabetical order, so you get general.pot, views-module.pot, views_book-inc.pot, views_comment-inc.pot and so on. The inc files are in subfolders, but because the extractor naming algorithm does not distinguish between them, you get that order from msgcat. Now with potx module, you basically get the order of the result of the glob() call in _potx_explore_dir(). Seems like it orders the results in "brace order" (that is order of extensions), then the order of file names. So you get views.module and other .module files in the same folder, then .install, .info, and so on files. Then it does down to subfolders once all files are parsed in the same folder and goes on the same order.

So the difference is that msgcat orders by absolute alphabetical order based on extractor.php results, while potx module orders by "folder deepness", then file extension, then file basename.

That said, it is not impossible to do the same ordering in potx then with extractor.php + msgcat, we would just need to order the $files array by name of file, without considering the path to the file, but I don't think that ordering is really logical. Do you?

Hass, yes, the reason of modifying potx-cli.php to use a single file mode by default was made exactly to help translators (remove the msgcat requirement altogether) *and* at the same time resemble what is generated by potx.module.

It is not named views.pot because we don't know that it should be named views.pot. You can have so many files in the folder with submodules and so on. Maybe we can guess the desired filename by looking one folder up, but that only works if the working directory is the same where your files are (eg. in autodiscovery mode). Granted, that is the default, but people can do "php potx-cli.php --files=c:/mydrupal/modules/mymodule.module" Maybe this is not much realistic, but still a possibility. Naming the resulting file after the name of the folder potx-cli.php runs from seems to be a possible default, but then we need a command line option for that (and a better option parsing code, now that it gets important to support multiple options at a time).

"Additional i compared a command line generated pot file with the web generated pot file (both created with potx) and - they are *different*. They must be both the same or we will have senseless diff's..."

Again would be nice if you could collect the differences disturbing you. You have the files already, why would we redo all the hassle you went through already? :)

Sure, we can put potx-cli.php on a blacklist.

The second patch you sent shows that for some reason, the .info files were not picked up by the command line version. That should have something to do with special handling of .info files in certain modes. This could be in _potx_build_files(), but all I see there is some $filename variable used in CORE and MULTIPLE modes, but that $filename is not defined before it gets used. Maybe it was intended to be $file. Anyway, that should not affect this behaviour because we should be in SINGLE mode...

Hmm, different timestamps and the info files missing.
The first one is "works as designed".

Gabor, any idea about the 2nd?

Well, we made a lot of changes on hass' bug reports, and I think we fixed this along the way. A try with a fresh potx version would be great.

I have just committed a new version, which fixes the .info file issue (among lots of feature additions :). Please grab the latest CVS version and see if it works to your satisfaction.

Well, if it is only the potx.inc, then it is fixed with my patch. See the issues you opened. There is no point in keeping three issues open for the same problem.

Well, you remember right that we already discussed that msgcat works in the order of the generated file names, while potx works in the order of the source file names and paths, which are quite different. Potx does not know the names of the generated files throughout the process, only at the very end of the file generation cycle, which makes it quite hard to do an msgcat type of merge.

Look, I am trying to concentrate my powers to make the web based translation editor a possibility, which means that all .po files will be wiped from CVS and imported to a database, which will have a nice web interface to edit all translations for all versions of all modules, share translations between them easily and so on. Even with included PO(T) import/export functionality for the desktop geeks, and people on the road to work offline. I think my energies are better concentrated there. The recent updates to potx were also part of that effort. I API-ifed the extraction process, so we can save extracted strings to the database instead of generating template files.

BTW as the diff shows we are better with extracting strings from .info files, so that's good ;)