The language-specific front pages on l.d.o display a list of "further top projects" (currently the 20 most popular translations).
To help plan the translation work (i.e. to figure out which modules to work on), it would be nice to have access to a longer version of this list.
The list is generated from the following query in the l10n_packager.module:
db_query_range('SELECT title, uri, pid FROM {l10n_server_project} WHERE status = 1 ORDER BY weight ASC', 0, 20)
Is it possible to get, say, the top 100 results from this query, or should I try to create a patch against l10n_server instead?
| Comment | File | Size | Author |
|---|---|---|---|
| #24 | do_flag.png | 44.75 KB | artusamak |
| #21 | 1095806-21-project-sort-options.patch | 7.91 KB | wulff |
| #19 | 1095806-19-sort-options.png | 20.25 KB | wulff |
| #10 | l10n-mockup-2.png | 16.04 KB | wulff |
| #8 | l10n-mockup.png | 19.08 KB | wulff |
Comments
Comment #1
gábor hojtsyCustom sorting was recently added to language lists in #1089788: Add sorting to highlighted project (front page) and language list. For the project list, #806944: project list needs alpha-based navigation is somewhat related. But I think we should have the same ordering capabilities in the project list too (and probably default to popularity maybe instead of alphabetic?).
Comment #2
gábor hojtsyBTW the reason it is difficult is that the project listing page is a mashup of data from different queries, and is not really one query we could easily tack on a sort... It needs lots of manual implementation, see the language list sorting issue, which had the same problem...
Comment #3
wulff commentedIt would be a huge improvement if we could somehow add the ability to sort by popularity.
I'll try to come up with a proposal for how the project overview can be improved.
Comment #4
gábor hojtsyWell, I have a simple idea for that. Projects already have weights. The localize.drupal.org project weights are now set based on usage data, but to have the right order, they use -$overall_usage as the weight. I think we can re-weight the projects when we save usage data (since we don't need the relative weight value), and just number them from 1....8000 based on their order in weight. Then we can expose that number as "popularity index" or something in the project table. Finally, we need to implement the cumbersome custom data ordering thing as in the language tables... I think this is a fine/reasonable plan, just needs help/time for implementation :)
Comment #5
wulff commentedThat does sound like a reasonable plan. I think I can try to take a stab at that, if you don't mind.
Is it possible to get a sanitized DB dump from l.d.o to use for testing?
Comment #6
gábor hojtsyNo database dump available, but you can use the process described in http://hojtsy.hu/blog/2010-may-05/using-drupal-collaborative-software-tr... to set up a simple testing environment.
Comment #7
wulff commentedNo problem, I'll just use some existing stuff from the Danish project for testing.
Comment #8
wulff commentedAs a followup to #806944: project list needs alpha-based navigation I would like to propose a minor change to the layout of the project overview table on translate/projects:
With the advent of the l10n_update module I would argue that the project overview is now mostly a tool for translators and developers, since most site administrators never need to visit l.d.o to download translations.
I would like to add two columns to the overview table (in addition to adding a sort option): Project URI (short name) and Popularity. I have attached a quick mockup.
I'd like to see the table sorted on project URI or popularity by default, with the project name used as a secondary means of identification, since the short name is what we use to refer to projects in all other development-related activities (e.g. when using drush, creating make-files, building install profiles, etc.).
Comment #9
gábor hojtsyI think "Popularity" should be the first column and we should default to sorting by that IMHO. I've heard many translators are eager to see that information to help guide their translation teams. Also, if we can find a shorter name for it, it would use less vertical space :)
Comment #10
wulff commentedMakes sense. Here's an updated mockup.
Comment #11
gábor hojtsyThat looks much better :) Now about the internal name... In reality, the autocomplete on top of the page, the other project listings (eg. for download, etc) all display the project title as primary information and it is linked. Not sure we should deviate from that. I'm trying to make the listings consistent, not deviate more from each other :| I agree some kind of way to look at the machine name would be good. A machine name based "lookup" is already possible via the URL :)
Comment #12
wulff commentedO.k. then maybe we should make the order of the colums: Popularity, Project name, Project URI, Progress, Untranslated, Suggestions?
I'll update the patch in #806944: project list needs alpha-based navigation to filter on the title instead and try to figure out a way of handling titles starting with symbols or numbers (e.g. #translatable, the 960 grid system modules, etc.).
Comment #13
gábor hojtsyI've prepared this table for more data display via #1102568: Make project listing theming simpler.
Comment #14
wulff commentedCool. I'll take a stab at making the table sortable later today.
Comment #15
gábor hojtsyI think probably it is key to avoid a LIMIT in the project list SQL query itself, since we'll need to sort based on data coming from other sources, and THEN limit the list of projects based on that sort and pager. So that makes it even more complicated than the language list which always fits on one page (so we assume :).
Comment #16
gábor hojtsyAlso, once we remove the paging on the SQL level, we can finally throw out "No data available" projects from the list... The only reason those appear is because we do paging on the SQL level, and it would be awkward to then display variable items per page... Once we move paging upwards, we can finally get rid of those...
Comment #17
gábor hojtsyThe no data issue should in fact be solved (for projects where no release was non-dev) with #1102648: Disable projects removed on drupal.org. For projects, where releases were parsed but no strings were found, it might be valid to list them as such, and they should not be that numerous at all. So maybe forget about that comment for now :D
Comment #18
gábor hojtsyMarked #780828: view - project by completeness duplicate.
Comment #19
wulff commentedI have attached a screenshot of my current project overview.
It is now possible to sort the overview table on an arbitrary column.
The next step is to try and get rid of the pager_query() in a sensible way.
Comment #20
gábor hojtsyLooks great. How did you even solve sorting without getting rid of the pager? Or are you just sorting inside the page for now?
Comment #21
wulff commentedThe first version just did sorting inside the page.
The attached patch adds support for sorting and paging the "explore projects" page without using
pager_query(). This requires that all projects are loaded on each page view, but I'm kinda counting on the fact that the project listings are mostly of interest to active translators and not the busiest parts of the site.The patch adds a new function which handles paging of the listed projects as well as a function for sorting on arbitrary project columns (based on the existing
l10n_server_sort_by_column_language()function).l10n_server_get_projects()has been modified to make it possible to retrieve all projects with a specific initial.Comment #22
wulff commentedForgot to mark it as "needs review"
Comment #24
artusamakInteresting to see that this never went in. :D We even lost the usage access on projects since.
It looks like there was a dedicated table that was populated on D6 version: https://git.drupalcode.org/search?search=project_usage_week_release&nav_... but it's not here anymore. By any chance, does anyone remember why this information has been cut off?
We could leverage such information. I don't know if we have an API parsable way to extract usage stats from d.o. We could ask to have it exposed as we extract the releases maybe?
It could be interesting to be able to extract "promoted" releases that are exposed on project pages on d.o in order to focus translations' suggestions on those.
Eg for Flag project, it would be:
- 8.x-4.0-beta4
- 7.x-3.9
Comment #25
artusamakAhah, after obscure digging i found something! (Searches for d.o API information that are really related to d.o itself are not easy to find with all the modules that pops for related features :D).
The key page is https://www.drupal.org/drupalorg/docs/apis/rest-and-other-apis#s-get-the... that lists exposed endpoints.
There is also https://www.drupal.org/drupalorg/docs/apis/update-status-xml that could help extract releases information (in order to identify if they are still supported for instance).
We can extract project usages from it, eg: https://www.drupal.org/api-d7/node.json?field_project_machine_name=flag
Two drawbacks:
- it doesn't look like there is an aggregated query available, leading to multiples calls that will have to happen against the node endpoint (one per project every x weeks).
- the usage are aggregated per branch(?) and not per release so, we may not be able to lead to the latest releases if we want to guide translators. It is balanced by the fact that we'd probably like to promote projects as a whole more than specific releases.