Add historical issue credits to Drupal.org user profile

This is an important feature. A Drupalist's d.o profile is essentially his or her résumé. Additionally, many contributors, especially new contributors, find the absence of core contribution on their profiles disappointing or even upsetting.

I like the idea behind this, but I think this needs to be moved to the infrastructure issue queue? Because I think we need custom code to parse the commit logs and display it, or alter it into the project commit info. There's nothing about this the webmaster team can really do here aside from 'approve' it?

If there's one major author, how about simply using the author option in Git (and doing the above for additional authors)? For example, I was the only one who worked on #2055019: Syslog configure link missing on Extend/Modules page and I even provided Git metadata with git format-patch, but neither "git am" nor the "--author" option was used. Is there a particular reason why core committers aren't doing this? I'm a little bit confused because we do this sort of thing in contrib all the time.

It would help user cred on Ohloh, which builds each user's experience based on the number of commits authored. With the approach described here initially, core commits wouldn't count for anything over there.

There have been many discussions about proper Git attribution in core. For example:

#1717676: Core commit credit and log messages are inconsistent between patches and merges
https://groups.drupal.org/node/161659
...

Sadly, it's totally inconsistent. When people work on core via sandboxes, they do proper commits themselves. When those commits get merged into the real core, some people get actual commits in core. Everyone else who works via patches do not. The differences are the heart of what #1717676 is trying to resolve.

Meanwhile, parsing the commits for the old-school "[#x] by foo, bar" and crediting users foo and bar for that commit is sort of a pain. We'd probably want to put the code in drupalorg_versioncontrol so perhaps this issue should be moved to "Drupal.org customizations". But, it gets lots more visibility in this queue, so we can leave it here for now until we decide if/how to handle this.

One fundamental problem is that Git itself only has the notion of a single author for each commit. So I suspect that the Version Control API data structures we're dealing with (to generate the commit listings on profile pages) aren't going to be able to handle the case where a single commit is "authored" by multiple contributors.

If we don't basically lie to VCAPI about the commits, and handle this in a totally separate data structure and display it separately, that seems crappy, too, and will throw off the counts in the "native" commit listings...

Input from the Version Control API maintainers (e.g. sdboyer, marvil07, et al) would be most welcome.

Cheers,
-Derek

As a new contributor to core just like others, I would also like to see core mentions on my profile page. I agree here with @xjm that d.o. profile is drupalist's resume and without any contribution to contrib module and/or theme, profile page looks pretty empty.
I have added my input here - https://groups.drupal.org/node/313108 for significant improvement for Drupal.org after I saw the announcement here - https://association.drupal.org/node/18358.

I'd also like to point out Eric Duran's project:

https://github.com/ericduran/drupalcores
http://ericduran.github.io/drupalcores/
http://ericduran.github.io/drupalcores/data.json

make it a link

...we now have a dedicated place to put related issues ;)

This would fit nicely with other issues related to finding what motivates d.o users to contribute.

It could fit nicely into the open badges issues which are being discussed.

This is a big issue. I do think it would be useful though. Right now there is little, no recognition for contributing. It can be a good motivator.

Given the work to clean up Drupal profiles in #2281763: Make Drupal.org user profiles more robust, I think it's even more important to find a solution for this.

As Cathy asked above, who has the power or abilities to make something like this happen? Clearly it's possible given the existence of Eric Duran's Drupal Cores site. Would it be possible to repurpose some of that code to make this work?

This can be a lot more straightforward on Drupal.org. The commit messages are available locally in the versioncontrol_* tables, and can be parsed as they happen too. What we would want from Eric Duran's work is the actual parsing algorithm; for example, making sure "Dave Reid" is parsed correctly with the space.

That parsing is still intensive, it can't be done on profile view. We will probably want to store the data in a table like (commit, user), one row for each mention. Probably something along the lines of the code that finds issue node IDs in commit messages, http://cgit.drupalcode.org/versioncontrol_project/tree/versioncontrol_pr... and http://cgit.drupalcode.org/versioncontrol_project/tree/versioncontrol_pr...

This should all be doable on a dev site, https://www.drupal.org/node/1018084, if someone wants to work on this.

This should be pretty simple to re-implement on drupal.org

I haven't worked on drupal.org specific code in a while so I'll need to get another dev d.o up and running but besides that it shouldn't take much work.

The parsing is mostly 20 lines the one tricky thing we do is we keep a local mapping for when people change usernames. That could just be moved to being handle on a case by case basis based on the infra-queue.

That being said @drumm is right, we do not what to do this on view (that would be really bad).

We we'll need to keep the data in a separate table but that also should be pretty straight forward. We can do the initial import based on a drush script or an update (queues) hook but I'll pretty much go with a drush script so we can able to re-built it if anything. This will only be needed for the initial import after that we could just use the version_control_api hooks to parse the commits as they come in and give proper attributions.

In short i'm ok contributing this feature, or someone else can work on it, if they beat me to it.

Is this a good match for GSOC? With #2369159: Extend crediting UI to include organizations & customers this should be a few hours designing and building a View. My understanding is that it would be too small for an all-summer project.

Just wanted to mention that there is also json data available of the commit mentions for people http://drupalcores.com/data.json . With a small change it would be possible to provide this data also before a specific date if wanted so.

Just checking to see if there's any kind of ETA on exposing the commit credit data? I know the intent was to hold off until we have a bit more data captured. Do we feel like we're nearing that point?

Also, to keep things simpler, I would highly recommend in this issue we only get the plain credit count on the org/user profiles in some fashion. And explore aggregate visualizations like drupalcores.com etc. elsewhere, since that's a much broader topic and can have community implications. (e.g. leaderboard of individuals vs. leaderboard of companies, large companies vs. small companies, etc.)

This is ready to work on, when someone has time. I made sure it is in our staff backlog.

To further simplify this, I broke out backfilling old data into a separate issue, #2495441: Parse commit logs and store mentions as issue credit. Do we want to launch this issue only after old data is backfilled, or go ahead and mention that credit is generally only there for issues updated after [don't know the day offhand] 2015?

I have been maintaining scripts to do this for a long time, but it is based on a drupal core git repository. The last time I rewrote them I used php.

In this case that git extraction will not really be needed since we have the commits in the database, but I would say we want to do the same process, that is, creating an sqlite database with the data (let's not fill more d.o db), so we can analyze it later from several angles, instead of trying to parse every time. Notice that those scripts take in account a way to "fix wrong messages", e.g. typos, by providing a way to override the messages.

I would be glad to help/collaborate with someone to do this.

Let's track the parsing in #2495441: Parse commit logs and store mentions as issue credit. Sticking to MySQL, maybe temporary tables, is encouraged. The DB servers have plenty of power and do not need additional setup.

Do we want to launch this issue only after old data is backfilled, or go ahead and mention that credit is generally only there for issues updated after [don't know the day offhand] 2015?

Spoke to Dries about this question, and his proposal was slightly different, so moved to a separate issue: #2498497: Expose recent issue credits data on user & organization profiles

In short, because the credit UI is actually tracking new data (issues someone contributed to closing), there's no reason to block exposing it on profiles based on the parsing/storage of historical data. And tracking only the past ~3 months of data is arguably better anyway, as it incentivizes organizations to keep their employees' participation active and ongoing.

Adjusting the issue title for clarity, since I got confused in #28 thinking this was about issue credit data.

No, what we track in the DB is different than commit credit; we actually track issue credit. And only for closed issues, from what I understand. That issue credit UI is also used to generate default commit messages, but commit messages can include other people not present in the issue, and also can (and do) exist without any d.o issue at all. So these really are separate concerns, I think.

And right, I forgot that contrib sometimes uses the "Issue #3434 by foo, bar" commit message format instead of --author. Trying to figure out a way to make this clear from the issue title. Let's try this.

See #32 and #2495441: Parse commit logs and store mentions as issue credit. This issue is not about parsing.

Per IRC, re-titling to be a bit more clear what we mean here vs. #2498497: Expose recent issue credits data on user & organization profiles.

Until this issue is fixed, I think pages like https://www.drupal.org/u/tim.plunkett/issue-credits/3060 should be appended to say "in the last 3 months".

#2495441: Parse commit logs and store mentions as issue credit is the blocker for this issue.

https://www.drupal.org/u/tim.plunkett/issue-credits/3060 actually does have the whole history that we have in the database, starting when the Credit & Committing fieldset was expanded to store credit.

Yes, I think the idea is "Denote that this list is only since XXX date" rather than making it appear as though the #2 contributor to core has only ever fixed 32 core issues. ;)

All recorded credit is now accessible from user profile pages. Once #2495441: Parse commit logs and store mentions as issue credit is done, this issue will be done too.