Posted by stodge on June 16, 2009 at 12:03pm
| Project: | Version Control / Project* integration |
| Version: | 6.x-2.x-dev |
| Component: | Code |
| Category: | feature request |
| Priority: | normal |
| Assigned: | boombatower |
| Status: | needs work |
| Issue tags: | git phase 3 |
Issue Summary
As far as I'm aware, there's no direct link between a changeset and a bug/ticket. A critical feature IMO is providing such a link. I realise this could be raised against the versioncontrol_svn module as well but I thought I would start here.
This feature would facilitate implementing code reviews e.g.
Comments
#1
How do you imagine a "direct link", and what should a preliminary user interface for this feature provide? Version Control API itself offers the repository link for issue/case/bug trackers which you can enter in the repository edit form, if such a link template is present then Commit Log will link Drupal-style issue numbers ("#12345 by jpetso: blah blah") to their corresponding issue URLs.
There's no database table capturing such relations though (so backlinks from the issue are not currently possible), and the issue number format in the commit message is also not extensible. I will gladly accept patches that change these shortcomings.
Oh, and I think the main API module is appropriate as target for this feature request; after all, commit<->issue relations are one of the primary use cases.
#2
I just started reading through the version control code so I'm not familiar with it yet; I'm just thinking at a higher level to incorporate existing functionality we have at work with Team Foundation Server and Trac.
So as a high level example, say we have a content type ChangesetOperation in the database that contains details of a single commit operation (update/add etc). The ChangesetOperation would have a field called ticket_id, which is extracted from the commit message. E.g. the commit message is:
refs #100
The changeset operation's ticket_id would be set to 100. Then when you view ticket 100, you can see a list of all changesets (basic details to save space). Exactly as you do in ClearQuest/ClearCase and Team Foundation Server.
Hopefully that makes sense. I'm only working at a high abstract level as I'm not familiar enough with version control modules to relate it to their data. I'm trying to find a screenshot to help but I haven't found anything useful yet.
Thanks
#3
The nearest I could find in a hurry is this Redmine page:
http://www.redmine.org/issues/3279
Note that the "Associated revisions" are shown when viewing the ticket.
#4
Right, that's approximately what I expected. Valid request, although instead of a content type we might just have a table relating two values, the version control operation (a.k.a. vc_op_id) and the issue (a.k.a. nid). Then provide a block that appears in issue nodes and links to the operation URL returned by versioncontrol_get_url_commit_view(). Also provide a setting that the admin can customize for the regular expression in the commit message in order to correctly retrieve the issue id.
This functionality can and should be implemented as an extension module, and is probably pretty easy to implement, using hook_versioncontrol_operation() with the regular expression determining the issue id from $operation['message']. Still takes some work, of course, but all the infrastructure is there and the requirements are straightforward. I'm tagging it with the "Newbie" tag because this one seems like an ideal way to get familar with Version Control API.
#5
I don't claim to yet understand most of what you wrote, but I think it makes sense. :)
#6
new features on the 2.x branch
#7
I have marked #782588: Add support for issue nid hyperlinking based on commit as a duplicate of this issue
#8
tagging with views integration
#10
I don't think this issue has anything to do with views integration, that's probably the issue over at #782588: Add support for issue nid hyperlinking based on commit.
This issue is about providing a new feature for drupal.org and VC API, VC Project and Project issue module to integrate and provide a link, or followup message when a commit is parsed that contains a reference to the issue NID. For example making the commit at http://drupal.org/cvs?commit=467500 would automatically post a followup issue comment with a link to, or meta data about the commit.
Since this is a new feature, I'm going to move this to phase 3 for now. If we can get this done by the launch, that'd be awesome, but I don't want to hold up phase 2 on a new feature.
Note that this will probably be implemented with hook_versioncontrol_entity_commit_insert() and/or hook_versioncontrol_entity_commit_update().
#11
Note: I think we just want a block on issue nodes that shows all commits that reference the issue, not a new comment for every commit that references it. I could be wrong, but it seems like a comment is overkill.
In fact, I'm tempted to call this duplicate with #443000: When viewing an issue, display a list of commits that reference that issue # and move that over here. ;)
#12
I would dearly love to see this.
#13
Subscribe! :D
I really really really want this:
dww says this is the place to be. :)
#14
@webchick, Why (and how) would commits be attached to an individual comment?
Wouldn't it be more useful to have all the commits that reference an issue be listed in a single place?
#15
Well, I meant they'd be interspersed in the view in chronological order, along with the comments. I'm only so good with firebug. :) They might even be comments from "System message," or the committer, or or whatever.
I think a block showing all commits at the top of the page or something is also useful for those who want an overview of what happened in an issue. But for the purposes of the conversation about a change (commits are a BIG part of that conversation, esp. if/when we move to per-issue repos), having the commits in-context is also super helpful.
I would be happy with either of the above though. I just like to think big. :)
#16
After using github issues I thought that making comments would be a great idea and even marking an issue as fixed (and/or having a setting for the auto-fixed part). The most common case for issues is to have a single commit that fixes the issue.
If there are more then one commit having them be interspersed in chronological order (as webchick wrote) seems like the best idea as you can easily see the conversation related to commits. It's also extremely simple to implement. We can add things like blocks later if we want.
Only question then, should this be a drupalorg module addition or general "Version Control / Project* integration" addition? I'd be happy to write code for this.
#17
Should probably provide per-project setting for marking issue as fixed on commit and the formatting of the commit/detection of issue reference may have more proper/better ways to be done. I'll leave it up to others to tinker, but this should provide the basic structure.
Since this is being done generically in versioncontrol_project I figured we cannot assume we have uid like we should be able to on d.o thus the
if ($operation['uid']).Also as part of formatting we might want to add label. I just followed webchick's example for this patch.
#18
This looks just lovely.
#19
I love it, for sure
#20
Cool, progress... thanks!
I haven't tested, but here's my review of the patch itself:
A) Huge -1 to automatically marking an issue fixed just because it's referenced in a commit (per-project setting or not). Imagine that in conjunction with per-issue repos. ;) Certainly when working on patches with feature branches that's a bad idea. Even for core, usually the issue needs to move to some "to be ported" state, or whatever, not just marked fixed. The only reason this works at all on Github is because there's special syntax you use if you actually want to resolve an issue via a commit. That's really a totally separate feature that belongs in a separate issue. Please don't try to introduce that in here, or we're going to bungle both and delay things unnecessarily.
B) (minor nit) It'd be nice (and conform to our coding standards) if the code comments used proper grammar. The 1st comment mentions "a issue". The 2nd is full of errors. We want something like:
// If the Drupal user is known then format the author as a link and post the
// comment as the detected user. Otherwise format the author as the VCS
// username without a link and post the comment as the auto-followup user.
C) The code seems needlessly confusing with how it's handling
$changes['comment']and the t() placeholders. Can't we just save the placeholder values in a separate array so it's easier to grok what's happening? I thought it was a bug, then re-read and saw "oh, wait, no there's just tricky stuff happening" but I'm too tired to tell if it's actually a bug or not. There's no reason to make this complex. A separate array is basically 0 cost and is a win in saved developer time and more robust code.At first I wasn't sure I liked auto-commenting at all, and would have rather seen this more like an activity stream (e.g. along with notifications about edits to the issue summary, etc). But, in practice, I basically always comment on an issue once I commit something for it, so it seems like a win to just automate that. And it's definitely easier to implement this as a comment since we don't yet have a way to render all the comments interspersed with other activity notifications.
In summary: yay! ;)
Thanks,
-Derek
#21
A) Didn't know you could close issue from commit on github, but making a special syntax seems reasonable. Although, I would have to say that per-issue repos, feature branches, and core represent the minority. Defaulting to disabled, but allowing contrib projects to do so since most work on single commit to fix issue wouldn't be a bad idea. I'm fine with making a separate issue since there are many options to discuss. Created #1291886: Support marking an issue as fixed via a commit.
B) Yea noticed the bad comments after uploading. heh
C) Sure, not biggy. Actually went back and forth when writing. My primary concern was that there was a better way to do the comment message maybe even using same formatter used in commitlog?
#22
Fixed comments, removed auto-fixed, and moved t() parameters to $comment.
#23
Sorry for the outdated api file, I just fixed the chunk related to operations so you can take a look at it and the operation class.
Thanks for the effort here!
#24
Definitely appreciate the separate issue for A -- I'll reply there.
Now I'm just bikesheding, but wouldn't $placeholders be a more accurate name for this array than $comment?
Also, is
$operation['revision']already formatted as a link or is it just the text of the revision? Given that you're referring to it via !revision I'm assuming it's coming through as a link, but I just wanted to double check. (Oh, I noticed marvil07 just replied as I was typing this -- maybe that's what he's concerned about).I've got no other visual concerns with the patch.
Anyone got a test site that's attached to Git that can try this? I have no idea what's the status of our Git test environment(s), puppet builds for VMs and all that stuff. Probably worth pinging sdboyer in #drupal-vcs in IRC to see what's up and where/how to test this before deployment.
Thanks,
-Derek
#25
Just a side note that I'd still like to follow this up with a second issue that introduces a new table that stores associated commits with issue nids, and provides a block or format for adding to issues that lists ALL commit messages that reference the issue number in a single place.
This is looking like a great complement and a great start on a much needed feature though.
#26
Agreed with mikey_p. Comments are hard to scan, but a block doesn't give the context that you'd get from a comment followup, so having both seems really useful.
#27
I'm also +1 to a mapping table and a block. However, on d.o I'd like to avoid the block rendering in a whole sidebar region, since then we lose tons of screen real estate on the whole issue just for a tiny bit of info at the top of the page. I'd rather that "block" was floated to the right of the "Jump to" links or something. Anyway, that's out of scope for arguing about here, just wanted to mention it while it's on my mind. ;)
D) I just realized this patch isn't going to handle the case of a single commit that references multiple issues. I do that a fair bit -- e.g. if there's a separate issue to fix a bug introduced in an earlier feature, I do stuff like:
[#1234567] by dww: Fixed a bug in the foozbangle introduced in #1234000.To me, that commit is relevant to both #1234567 and #1234000. Not sure if it should generate an auto-comment in both -- it probably should. Certainly I'd want to see the commit in the block of commits for both issues.
#28
$operation['revision'] is not currently rendered as a link, but as I noted I was looking for tips on how to render all that stuff. Specifically with that a function to render it as a link, but otherwise I can construct it manually.
Looks like things outstanding are:
- Update hook and usage since api documentation was out of date.
- Render revision as a link.
- Deal with multiple issues in single commit.
#29
Left some @todos which marvil07 sounds like he is willing to fix.
EDIT: missed the $operation['message'] in preg_match_all()
#30
This is a stretch to exploit, but technically this is a XSS vulnerability:
+ $placeholders['!committer'] = $operation->committer;In the other code path it's okay b/c l() check_plain()'s for you. But you should check_plain() here since we're using !committer. Sorry I missed this earlier.
Otherwise, looks good to me. Still haven't tested anything, so no idea if it actually works. ;)
#31
What's new on the patch:
Also adding a diff against last patch state.
I also did not really tried this patch on a proper environment :-(
#32
In the line $changes['comment'] = t('Commit !revision on @labels by !committer: @message"', $placeholders);, there is an unmatched and unneeded double quote.
#33
+++ b/versioncontrol_project.module@@ -508,6 +508,59 @@ function versioncontrol_project_versioncontrol_entity_commit_insert($operation)
+ $label_names = $label->name;
Assume you wanted $label_names[] = $label->name;
+++ b/versioncontrol_project.module@@ -508,6 +508,59 @@ function versioncontrol_project_versioncontrol_entity_commit_insert($operation)
+ $changes['comment'] = t('Commit !revision on @labels authored by !author, committed by !committer: <pre>@message</pre>', $placeholders);
Have to see how this works out after groups.drupal.org/node/161659, but probably fine for now.
+++ b/versioncontrol_project.module@@ -508,6 +508,59 @@ function versioncontrol_project_versioncontrol_entity_commit_insert($operation)
+ $placeholders['!committer'] = $operation->committer;
I believe this is where dww wanted check_plain which makes sense.
Powered by Dreditor.
#34
Thanks for the corrections, we really need to test this on a proper environment ;-)
#35
Seems good and agree we need to test somewhere. Not sure how we proceed.
#36
Good call on using theme('username'). However, you're using it wrong. ;)
+ $committer->name = check_plain($operation->committer);+ $placeholders['!committer'] = theme('username', $committer);
In Drupal's never ending quest to confuse you and get you to not handle strings correctly, theme_username() takes care of the check_plain() for you. If you can access user profiles, you get run through l() (which in turn does the check_plain()) and if not, theme_username() does the check_plain() itself. Fun, huh? ;) So your patch is double-escaping the usernames.
While I'm wearing my security team hat, it occurs to me that we should be checking node_access() here, too. Just because I put "#12345" into a commit message and could push that to your site doesn't mean I should actually have permission to view or comment on issue #12345. So, after node_load() we should both check
$node->statusand call node_access().Thanks,
-Derek
#37
Really happy to see this being picked up...but unfortunately, the approach here is wrong. Implementing the insert hook is not the way to do it - that hook is fired when a commit is recorded in the database, not when it initially arrives into the repo. The latter necessitates the former, but the former does not imply the latter - when we resync a repo (which we do fairly regularly for various reasons), we blow away all commit records then rebuild them from scratch. So every single commit in the history gets reinserted, and this hook is retriggered for each one. Imagine the cavalcade of automated issue comments :)
For when the commit truly first arrives in the repo, we have #1092062: Introduce framework for generic repository history synchronization. There are a couple smallish bugs related to merging that back in, but at this point the only thing that's blocking it is the fact that I don't have a git-dev to really test it on properly. re: that, I got git-dev just about set up again three weeks ago, but did it on an old Gentoo vm, instead of the new centos stagingvm2. Which is now temporarily borked because of an internal OSL/yum repo availability question. That's supposed to be resolved today, at which point I can resume efforts to bring git-dev back online.
That being said, I expect most of the actual logic here can be picked up and dropped into another hook implementation (and eventually shifted into a reposync event plugin). The hook to be implemented will be
hook_versioncontrol_code_arrival(VersioncontrolRepository $repository, VersioncontrolEvent $event)(invocation here).Agreed. And another reason why the event is very important - the relevant user to check for perms is *not* the commit's author/committer, but the person responsible for the network operation (the push). Which is not really accessible from the commit itself in its insert hook, but is a built-in part of
VersioncontrolEvent: http://drupalcode.org/project/versioncontrol.git/blob/gsoc-activity:/inc... - though we should really add a getter method for it and maybe return a full user object.#38
And btw, a big reason we make this an optional plugin on a per-repo basis (rather than a straight hook) is so that we can have different rules for different repos - such as confining closing/responding to issues to pushes made to a main repository, but never a per-issue repo. That's a little further off, though, so it's fine to do it as a straight hook for now.
#39
And, reading more/further back - my ultimate preference would be to thread the commits in chronologically with the comment times as separate bits of information. That is what github does, though they go by committer timestamp, which changes when you rebase to the time you rebased, so commits almost always end up stacking in a single place in their interface rather than being interleaved. That'll be much more useful in a per-issue repo/pull request context, though, so I see no problem with auto-commenting as an intermediary solution, as getting together a good framework for interleaving the data could be a bit complex.
As for highlighting names in the commit messages...well, maybe. That data is a bit fuzzy, though, and I'd prefer to either a) read it from git-notes or b) use actual commit metadata to show who authored/committed the commit or c) just not highlight them at all. If we want to do that highlighting, we'd need a whole separate place to store these db records with the commit message analysis already performed, otherwise we'd be doing potentially expensive mysql searches for username matches on the fly. Minimum one string-based matching query against {users} per commit shown on a page.
#40
Summary:
- use theme('username') instead of calling theme_username() directly
- get rid of check_plain() from data passed to theme('username') [possibly just use user_load()?]
- move comment code to hook_versioncontrol_code_arrival(VersioncontrolRepository $repository, VersioncontrolEvent $event)
- perform a node access check before posting comment
I would vote for changing the comment user to the push event user (that was what I was looking for originally).
I want to note that posting a comment doesn't give you access to view a node right? So the viewing part seems off, but not being able to post is definitely a good idea.
@sdboyer: Not sure what you are referencing with "highlighting names." The only usernames that will be links are those that are already known based on data passed to the hook. Additionally, any look-ups are already performed and will only be done when comment is posted note repeatedly.
#41
@dww: thanks for the clarification :-), so yeah, we need to remove it for strings that are then used in theme('username') (hint: not the last one)
@sdboyer: agree on waiting for git-dev to test history sync branch properly and use pusher to review permissions instead of author/committer. agree on using a plugin to implement this logic so we can be more granular too for d.o and easily change it when git notes was official attribution.
IMO boombatower is right on his comment about highlighting messages, we are using what's on the operation itself already(committer_uid/author_uid), and that's is performed once(on the history sync hook world) just before saving the comment.
Other notes:
In conclusion, I guess we need to postpone this :-/
#42
@boombatower sorry, that was based on what now seems like a misreading of the 'jhodgdon'-linkification in #13. Initially it I mistook it for reading the commit message contents, discovering d.o usernames, and linkifying them. Not easy. If it's the actual author of the git commit itself, though, it's easy.
@marvil07 Why would we need to access the Git-specific data? We use the mapped uid in {versioncontrol_operations}.author and/or {versioncontrol_operations}.committer, and then just use that to load a user account. What's the problem?
We gotta wait for the staging site for final testing regardless, but I have no problem (AT ALL) with adding more tests.