Relationships between issues: support for parent issue and related issues

both http://drupal.org/node/57782 and http://drupal.org/node/64408 are duplicate with this same basic idea. for certain values of the status (the existing "duplicate", my proposed "blocked" -- which now that i've read it, is basically sara's proposed "parent/child") and in certain other cases (sara's proposed "related to", not sure if that's useful as a status setting in the same way duplicate and blocked are), it'd be great to be able to input the node id of another issue. ideally, changes to the status of the related issue could automatically alter the status of the other (e.g. when the parent is closed/fixed, the child can move from blocked to active, or when a duplicate is set, the other issue gets a new comment added automatically ("url/title marked as a duplicate of this issue"...). not sure how to best handle this. maybe we should move discussion over to http://groups.drupal.org/relationships-site-structuring and/or http://groups.drupal.org/issue-tracking-and-software-releases.

created http://groups.drupal.org/node/555, which i cross-posted to both of the groups i mentioned above. let's move discussion there for brainstorming and design ideas. we can come back to this issue once we have a plan and can start dealing with project-specific patches, etc. if you don't have a groups.drupal account, you can just use your existing drupal.org username with "@drupal.org" appended to it (e.g. "dww@drupal.org").

Nice idea. Giving it a real version. Could be done with node reference I guess.

@catch: yes. However, it's still postponed. I don't want to discuss it in here, I want to discuss it over at http://groups.drupal.org/node/555 first to brainstorm how we want to go about it, then come back here once we have a reasonable plan we can agree on and implement.

subscribing. looks like i'll be working on something similar for work.

There was a quick BOF at the end of Paris Drupalcon where that issue was also brought up as being a key issue in usability. Furthermore, I don't feel the groups.d.o discussion is particularly complicated or leading away from a consensus, so I think this should be considered active again.

I have also posted extra specs on the groups.d.o page.

so it's been almost a month since I posted those specs on the group and almost two years since dww postponed the issue. when do we know we agree?

i would very much like to see this in, i guess the only implementation-level question remaining here is whether this is implemented standalone or we reuse existing code.

Ok, so what do we need to do to start the ball rolling, development wise? Be great to start working on this. I too feel core is losing contributors (myself included) because the issue queue is too hard to unravel. I wouldn't set the priority as "minor" either, because I don't think it is minor!

Honestly, I think #569552: Provide a mechanism for issue meta discussions is more important in the short term for improving the workflow on d.o. That would be greatly helped + facilitated by wrangling a decision on #651484: Enable CCK and node_reference...

Sigh. If only we had a mechanism for issue meta discussions, this would be so much easier. ;-)

Heading over there to read up now.

@dww - you should hold up an umbrella when you walk around or something. You're a very useful tour guide! ;-)

subscribing...

http://drupal.org/node/651484#comment-3398242

... as suggested by Derek (dww), I am posting here the screenshot of how bugzilla.mozilla.org handles issues relationships:
 
 

I just had some ideas about relationships between issues during ironing (call me a Drupal nerd) and saw that there isn't any active discussion on g.d.o. so I'm taking the liberty to air them here.

Basically what I'd like to see is a way to define (m-n) relations between issues like klonos shows in #14. If there's a relationship between issues it's possible to order them in importance and show an issue tree. A project owner could define a mile stone issue (a new release for example) and define which issues are blocking this mile stone. This would give everyone the opportunity to give priority to issues.

An extension to this would be to be able to define an estimate time amount for each issue. If this would be defined a basic scheduling algorithm could be used to define the critical path or to give an estimate time for a issue chain to be resolved.

If only we would've had this tools for the upcoming D7 release.. But of course the "Project" and "Project issue tracking" projects are used in many smaller issue tracking projects as well, and the above addition could come in very handy.

This is going to be really easy in the D7 version. See #1545922: [META] Issue page redesign for more. We'll just add a pair of entity_reference fields:

- Multi-valued reference for "related issues"
- Single-valued reference for "parent issue"

We can just add these to the default project_issue node type we'll provide when you install project_install. Anyone can update them and the auto-generated comment should display the useful changes. If not, we can easily patch nodechanges accordingly.

True, we can do this easily in D7, and we should architect for it, but we won't be actually implementing this idea until after the fundamentals of the D7 Upgrade initiative land solidly on their feet.

Perhaps it would be useful to have:
- Duplicate of
- Has duplicates
- Mentions
- Is mentioned by

EDIT: Oops, I think duplicate is already talked about, thought this issue was about blocks/depends only

There's no sense trying to record the inverse relationships. Everything should be recorded in one place. The inverse can be computed and displayed automatically. E.g. instead of "Has duplicates", just have a field on an issue "Duplicate of" and then we'll have a block (or whatever) on an issue showing all the other issues where that field points to the current issue. If you maintain these relationships in two places, they're going to get out of sync and it makes it much more of a pain in the ass to work with.

Wasn't suggesting to reference both ways, makes no sense, just noting what might offer us some nice usability UI side.

I'm not sure what "might offer us some nice usability UI side" means. Are you suggesting that although we only store the parent issue in the child node, that while you're updating the parent issue X, you could fill in something like "Issue Y is my child" and that would go out and also update issue Y to set the "my parent is:" field to X? I'm not sure that's a win, especially since that seems like a lot of extra magic to implement it. But maybe I misunderstand your proposal.

Thanks,
-Derek

See #1628160: Port issue nid field formatter to D7 as a first step to make this easy.

Assigning to @bdragon.

Let's make sure to encapsulate this functionality in a fieldset or a field group so that the Bluecheese Team can more easily move it around and theme it according to UX workflow.

Right, see project_issue.field_group.inc for exporting our own field_group definitions.

Estimated 24hr, feature desired and discussed in person.

Unassigning from @bdragon due to lack of a workable tech spec.

Do we actually need this during the Drupal 7 upgrade? I think it should be rolled out later as a feature.

Yes, it could definitely be rolled out later. SUCH a critical feature though; it'd be great to roll it out very soon after. :)

Yep, and it will be a lot less stressful to roll out smaller deployments.

Tags

Tags?? I'm only asking because:

#1056932: Sometimes comments do not show they changed tags.
#1940572: Sometimes tags added to issues do not show up in the issue summary.

...comment moved to #1972612: Relationships between issues: 'duplicate of'/'postponed on' (integrated with the status field)

...related #1965592: Rename the 'postponed (maintainer needs more info)' status to 'postponed (more info required)' and provide a 'from' field.

I'd like to tackle this sometime this week.

@amateescu: Great! For now, please stick to the plan in #16 so this doesn't get too complicated and scope-creepy. Should be a patch against includes/project_issue_node_type.inc to add and configure the new fields, a patch to project_issue.field_group.inc to put the fields into a sane fieldset on the issue edit UI, and a new default view that provides a block on an issue page showing all the issues that think the current issue is their parent. Maybe we want a similar block for all the issues that think the current issue is related (although mostly we just want to display the related issues, and not rely on the "who points here?" like we will for parent/child). #1628160: Port issue nid field formatter to D7 gives us a nice display formatter for entityreference fields we can use so that you get the cool behavior that the [#xx] text filter provides.

The rest of this (e.g. a "duplicate of" field that only appears if the status is "duplicate") should be punted for a follow-up patch so that we don't complicate or delay this.

Thanks!!!
-Derek

Agreed! We should tackle one thing at a time. Here's a follow-up #1972612: Relationships between issues: 'duplicate of'/'postponed on' (integrated with the status field)

Thanks. Better title still. ;)

Re #35: How did you guess that I was going for the full thing? :D I can understand the desire to start it simple, so thanks to some free time during lunch today, here it is :)

In addition to the fields, field group and the view, I also created a custom row plugin so we can use that nifty theme function in the view output.

In case this will need a reroll, I'll have to update the sloppy copy/paste from the @file header of project_issue_related_issues.view.php.

Oh, and I also found this while I was there: #1972738: Specify the dependency on Entity API.

Awesome! Some initial problems on a quick skim:

A) We don't want to hard-code the project_issue bundle name into the new entityreference fields. Instead, we want #1836934: Entityreference selection handler for project behaviors.

B) The block for showing related issues seems okay, but the more important one is the block showing child issues. When you're looking at a meta/parent issue, we definitely want an auto-generated list of all the subtasks/child issues.

C) It's not clear to me why the views plugin is a *row* plugin (and why we need it at all). Isn't that specifically for tables? Do we need this to be a table? Or, is the idea that you can already use the field formatter if you want to see it as a separate field (e.g. inside a <ul>), and you need this plugin if you want the list in a table? But is that even true? What's wrong with a table of entity reference values, rendered with the right display formatter?

I haven't actually tried testing this yet, but I'll wait to do that until there's a new patch.

Thanks!
-Derek

p.s.:
+$handler->display->display_options['block_description'] = 'Project: child issues';

D) That should be "Project issue" not just "Project".

p.p.s.:

E) I think it makes sense for these two fields to live in a new fieldset on the issue edit UI, not just a new div within the current "Issue settings" fieldset. I think of the current one as "stuff about this issue" while these are "how does this issue relate to the rest of your queue?". And with #1952792: Make the default project + project_issue install easy to use for non-software project management in mind, I think it's good to keep this separate so it's easy to hide (e.g. start collapsed) for sites that want to keep things simple.

Thanks!
-Derek

A) fixed. now that you mentioned this, I noticed that I also harcoded the bundle name in the view, so I had to write a project_issue_nid validator instead. also, I re-exported the field and instance structure with field_inspector, so the interdiff won't be too helpful there.

B) actually, only the 'child issues' block was included in the initial patch, but I see why the view name was confusing so I renamed it and also added a block for 'related issues'.

C) discussed in IRC a bit about this, there were two reasons for introducing that row plugin

1) a small performance improvement by bypassing the regular entity rendering process: views query -> theme function -> entity_load view results (don't pass go, don't collect $200 :D)
2) we couldn't use the field formatter in this view because we're not displaying the parent or related fields directly as they are not part of entities that result from the view query (they are on the "other side", the *referencing* entity, and we display the *referenced* entity)

D) fixed.

E) fixed as well.

Addressing #40C and #44C, using a row style here is a good idea. No they are not just for tables, and yes the skip the entity_view process (which it doesn't seem is needed).

I didn't do an indepth review, but that approach is sound.

Excellent, thanks (both for the new patch and the review)! Finally had a chance to rebuild my local site to see all this in action. We're so close it hurts. :)

F) I think people will freak out unless they can paste an issue nid directly into either of these new fields. Ideally, it'd support both (nid or title). I guess that means we need a new EntityReference selection plugin? Seems like something ER could just provide in general, this isn't specific to an issue tracker or anything. In fact, I could have sworn I already wrote this functionality, but maybe that was into D6 node reference CCK and somehow it got lost in the entity reference re-write. :/

G) Once you enable the new blocks on issue pages, it's slightly confusing that we have the "Related issues" field being diplayed with one set of values (all the issues this one points to), and then a separate block with the same title and a different list (all the issues that point here). I'm not sure the best solution, but I think one or both lists needs to have a different title so it's clear why they're different.

H) The logic in project_issue_plugin_argument_validate_project_issue_nid is faulty. Release nodes might also have a field_project. Really, I think we just want this:

  function validate_argument($argument) {
    if (is_numeric($argument) && ($issue = node_load($argument))) {
      return project_issue_node_is_issue($issue);
    }
  }

I know it's nice not to necessarily do a node_load() in an arg validator, but a) we need to load at least the the node type to be able to validate the nid with the project behavior settings, and b) generally we're going to be on a page where the node in question is already loaded so we're just hitting the cache and doing a separate query would actually be worse for performance.

I) There's a typo in the fieldgroup config ' fieldgroup' (the leading space) which was breaking the config.

Tagging for a UX review for F and G, since they're very user-facing aspects of this.

Otherwise, this is looking basically perfect.

Thanks!!
-Derek

p.s. Decided to just push your patch exactly as-is to 7.x-2.x:

http://drupalcode.org/project/project_issue.git/commit/200ccc0

And then pushed fixes for H and I:

http://drupalcode.org/project/project_issue.git/commit/f4aab3b

So, please git pull and the next patch can just be a trivial one for G -- I hope F will be fixed upstream.

p.p.s. A few ideas for follow-up issues if anyone's interested:

- Reconsider if we should allow multiple parents.

- Consider a way to view more layers deep in an issue tree. Right now you can see the current node and direct descendants - what about sub-sub-tasks? That's nasty since we don't validate it as a tree and prevent cycles and such, so probably what we're doing already is the only thing we can safely support, but I know some automated way to visualize at least 3 layers of the tree would be really helpful for larger initiatives.

First of all.. YAY!

Now, about the possible follow-ups:

- Reconsider if we should allow multiple parents.

I would be against that, we still have tagging for 'customized' relationships.. :)

- Consider a way to view more layers deep in an issue tree.

This could be pretty easy! We would just need to use the Tree behavior for Entity Reference (which is also slated for a possible inclusion in D8 core: #1915056: Use entity reference for taxonomy parents)

I believe multiple parents is a bad idea too.

As for G, how about we merge these two. "Related" does not imply any direction. For directional relationships w have parent/child (or depends/blocks).

Finally, the tree view (what I proposed back in #14) should be a separate issue and shouldn't block d.o upgrade or this issue here. So: #1975806: Provide a way to represent the relationships between issues in a tree list format.

- Yeah, I also think multiple parents is probably a bad idea, but I think it's maybe worth considering in a followup. Or not. Now that it's just a field, it's not really our problem. Sites that actually want that can do it themselves with a few clicks.

G) I don't like merging the two since when you edit the issue to update that list, you'd only see some of the values, continuing the confusion. I think we just need to accept the fact that these relationships *are* directional, and there are two ends of the arrow. One list (the field) is the list of arrow tails that start at this issue and point elsewhere. The other list (the block) is all arrows where the head points here. So long as we have reasonable names for these two, I think they should be separate lists.

- Yes, tree needs to be a separate issue and won't block this -- that's why I said: "p.p.s. A few ideas for follow-up issues if anyone's interested:". So, thanks for being interested and spinning that off into a separate issue. ;)

---

The things holding up this issue from being deployable are F and G. Let's focus on those two for now.

Thanks!
-Derek

Here's the approach discussed in IRC for F. I created a new view that can be used to reference project issues nodes and used it for both Parent and Related fields.

Do we have to provide provide an upgrade path for updating the settings of those two fields?

Love it! Works great in local testing. Pushed:

http://drupalcode.org/project/project_issue.git/commit/faacc30

Only had a minor nitpick with the UI. The slick thing you did to prepend '#nid: ' was weird since it only worked on the fields you added to the list, not the ones that were already there. More importantly, it duplicated the nid that appears at the end in parens, anyway. On d.o with 6-7 digit nids, duplicating them would suck. I pushed a trivial fix for that:

http://drupalcode.org/project/project_issue.git/commit/efcc423

No, we don't need to worry about an upgrade path for people who happened to use -dev in the last few days, and the d.o test sites regularly rebuild themselves from scratch so they'll pick this up soon enough.

Even though everything is pushed to Git here, I'm going to leave this as RTBC until this is actually in d.o's bzr, ideally also with a drupalorg update that automatically enables and places the new blocks on issue nodes. If you had any interest in that (it'd be a new drupalorg_project_update_N() function), by all means, knock yourself out. ;)

Also, it wouldn't hurt to get that final UX review once this is live on git7site. I already talked to tvn about it and the plan is to have this fieldset collapsed by default. The idea is to have the following order of fields when editing an issue: title, issue settings, tags (collapsed), relations (collapsed), body, files, revision reason. I'm cleaning that up at #1970046: Properly configure issue fields on rebuild but some of the commits are to project_issue or other places.

Meanwhile, thanks so much for getting this done! You're a hero to the community. :)

Cheers,
-Derek

Sorry, whoops. I was getting so excited about how close this is that I forgot about G (see comment #46). ;) We really need to address that before we call this "fixed".

Note, to see this live on the git7site sooner, I wrote a project_issue_update_N() to create these fields and configure them properly. The update code (like the code we use when creating the node type in the first place) checks to make sure the field doesn't already exist before creating it. Maybe in the case of the update hook if the instance already exists we should update it (e.g. to force the new ER config, weights, etc). Anyway, here's the commit:

http://drupalcode.org/project/project_issue.git/commit/990f1d6

Working with tvn right now on trying to resolve G. Stay tuned.

Bojhan also offered his help today in IRC, but he needs to see either a screenshot or a test site because the problem is not really clear from our text conversations :/

Yeah, it took me a while to try to explain the problem to tvn in IRC, too. :( Sorry.

Test site is here:

http://git7site.devdrupal.org

after the drupal/drupal htaccess you can use your d.o username and password to log in.

Then, I clicked together some issues to demo the problem. Here's a meta issue with children:

http://git7site.devdrupal.org/node/1545952

One of the subtasks is here:

http://git7site.devdrupal.org/node/1833684

It is pointing to both of these as related:
http://git7site.devdrupal.org/node/1964202
http://git7site.devdrupal.org/node/1965078

However, these issues now point back to #1833684
http://git7site.devdrupal.org/node/1964202
http://git7site.devdrupal.org/node/1962930

Currently, the field (with #1964202 and #1965078) is displayed inline with the files and body, where as the "Related issues" block in the sidebar shows all issues that point to #1833684 (#1964202 and #1962930).

SO...

G.1) If we merge the two lists of related issues into a single block, it would contain all 3 issues: (#1964202, #1965078 and #1962930). But when you went to update the list, you'd only see two options (#1964202 and #1965078). So that kind of sucks.

G.2) If we keep the two lists separate, we need clear titles for each one so we know what we're talking about, and why you can only edit one of the lists by updating the current issue you're looking at. I still don't have a short + clear label to propose.

G.3) Magic where the act of marking an issue related also goes out and updates that issue to add the inverse relation is a rat's nest of complications and scope creep.

At this point, I think G.2 is our best option, if we can come up with good labels.

We can also show only 1 block with the issues current issue specified as related (basically the field only).

So in this example case only: #1964202 and #1965078.
Not showing #1962930.

If choosing between G1, G2, G3 I agree that G2 is the best. Though we'll have somewhat duplicated lists.

Maybe block titles could be:
'Related to issues:'
'Issues related to current:'

BTW, I think people have already clicked away my example on the test site. Luckily, I grabbed a screenie at the time, just forgot to post it. ;)

I don't understand the value of splitting those out? To me I care about "related issues" and only want to look in one place for all of them. Whether they're linking to or away from the current issue seems like that should just be a separate heading in the body alongside "Parent issue" and "Related issue"?

In IRC, bojhan suggested "Referred by" and I countered with "Referenced by".

So, an issue would potentially have the following 4 lists (potentially all as separate blocks):

Parent issue:
#xx: whatever
---
Child issues:
#xy: something
#xz: something else
#yx: something else
---
Related issues:
#yy: foobar
#yz: fubar
--
Referenced by:
#zx: something fooey
#zy: barfu-foo

re: #58: Again, the problem is that if you mix them all together into a single list, when you go to edit the issue, it'd weird if you can only update a subset of the "related issues".

You cannot change the fact that other issues reference issue A by editing issue A. You must edit the issues that reference A to change that.

Not sure how else to say it. ;)

From my point of view, the "Child issues" is unnecessary precision. That would be fine if its folded into Referenced by, I think in the end it will be confusing if there are 4 blocks; of which 2 are the ones you added to the issue, and 2 are coming from other places.

I propose actually folding the four categories into just two categories, Related issues and Referenced By. Then you always know whether they are explicitly added to the issue you are looking, or whether they are coming from somewhere else. Making this distinction very explicit, will avoid any confusion around this,

I tried to illustrate it in http://screencast.com/t/pmtfssR6 , I also added the qualifications in line - I do imagine this will be out of scope. For parent, I might suggest to still go for this UI parent, since its simply less intrusive than a full new block.

I think there is a difference between child-parent relationship (as main task and sub-tasks) and various other possible relations between issues. If we basically merge them on display, we might as well just have one field instead of two. (And 'Blocked by' qualifications are indeed out of scope, so this makes layout in #61 more confusing).

oops

Based on IRC chat with tvn, I'm going to take this and do some initial UI cleanup:

- Rename the block of issues that point here to "Referenced by"
- Add two new blocks to display the parent issue and the related issues the issue points to
- Hide those two fields on the issue node view itself
- Automate bluecheese block configuration/placement.

We'll see how people like it during public testing of the D7 issue UI, and we can always revisit this if it's a problem.

Also, tagging this for the d.o upgrade again...

Pushed some fixes for this:
http://drupalcode.org/project/project_issue.git/commit/41ee7f1
http://drupalcode.org/project/project_issue.git/commit/48dbf87
http://drupalcode.org/project/project_issue.git/commit/7cf70bd

Also pushed a drupalorg_project_update_N() to configure all of this automatically on site rebuilds:
http://drupalcode.org/project/drupalorg.git/commit/9e5e733

Then realized that the two new blocks weren't hiding themselves if an issue has no parent or related issues, and fixed that, too:

http://drupalcode.org/project/project_issue.git/commit/2df0856

;)

Merged into bzr and deployed on git7:
http://git7site.devdrupal.org/node/1545952
http://git7site.devdrupal.org/node/1833684
...

Calling this fixed. Unless something's majorly broken with any of this, let's handle other changes in new issues.

BTW, see also #1980852: Correctly display multi-valued fields

Thanks!
-Derek

git7 links don't work.

Every time git7 rebuilds, the relations people click together are gone. We'd have to write a drupalorg_project_update_N() that automatically pre-populated some real relationships that we want once this goes live. That mostly seems like a waste of effort, although it might be useful for during QA to always have some issues with these fields populated. I can't justify working on it, but if someone else wants to run with it, please do! Perhaps open up a new issue in the drupalorg queue.

Thanks,
-Derek