Refactor project issue module to use Views

yeah, i've thought about using views in both project and project_issue quite a bit, and mentioned it in passing in numerous places. the main reason i haven't actively proposed it like this is that from what i understand, dries is as-of-yet uncomfortable/unwilling to run views on drupal.org. apparently, it's a lot of code, and dries hasn't personally reviewed it all, and so he doesn't want to run it. given that project/project_issue are central to d.o, i've been hesitant to take the code in a direction that requires views. maybe once we have multiple branches of contrib for each drupal core API, we could do all the views integration on a "4.7.x-3.N" series (where 1.N is stable, 2.N is regular new features, slated for d.o, and 3.N is extra-special devel stuff that won't necessarily run on d.o)...

and yeah, i'm totally slammed with other projects, so i'm not going to be able to take the lead on this issue, at least in the near term.

but, it's a great idea. ;)

at the drupalcon session i led about the future of the project module (http://drupal.org/node/86696) we discussed the possibility of making a couple of small optional modules called "project_views.module" and "project_issue_views.module" which exposed whatever stuff has to be exposed for you to be able to define views on projects, releases, issues, etc, etc. that way, folks that want to build nice views-based filtering and displaying of issues could do so, but we'd leave all the existing filtering code in the base modules (or even move them into a separate .inc file or something so they only get pulled in if you use the default, non-views stuff)...

just a thought. only to be considered once the new release system is done and everything is ported to the 5.x API, of course. ;)

Has any more thought gone into this since? We're looking at some features that would be greatly helped by this. Has any further work been done towards adding views support?

views support will hopefully happen for the D6 branch of the module, if all goes well...

Initial patch. Only exports all the data from the {project_issues} table. Doesn't define any default views, filters, arguments, etc, and still contains some // todo comments. However, it's definitely a start, and I wanted to post it before anyone else duplicated the effort.

Here's an update of the above patch. It has the basic handlers and filters in place. It also has a default view.

More work needs to be done, but this is another step. Marking as needs review to get eyes on it.

PS- We may want to consider getting this in with the limited functionality so that it's easier to track future patches.

Here's an updated version that adds handler and options for project node title field.

Here's an updated patch that:
1. Fixes the handler for project node title.
2. Adds handler for rid/version so release versions are displayed properly.
3. Adds a new field for project issue changed time so we can display this the way we do now (the default views handler for changed displays it as 3 hours 33 min ago--we don't need the 'ago').
4. Added a theming function to add the proper css classes for each row based on the status of the issue in that row.
5. Updated default view to include new fields mentioned above.

The functionality with this patch is getting pretty close to the current functionality of project_issue. Other than @todo comments in the patch itself, here are some things that need to be done:

1. Add the Create, Statistics, Subscribe, and Advanced search links at the top of the page.
2. Decide how we're going to handle menu paths, etc. Do we want to keep the menu path handling in project_issue.module and then embed a view in the page while passing arguments for searching and filtering directly to the view? Or do we want to allow views to handle everything directly? In some ways the former is more complicated and reduces the ability of project_issue administrators to customize the project_issues views on their own sites. However, I think the later option will result in linkrot as the URLs that views uses would be different than the current URLs we're using.
3. Removal of the current code from project_issue that will be made unnecessary by using views. We are going to rip out this code, right (and therefore make views required for project_issue)?

@aclight: great work, thanks! i can't reply in full right now, but a quick reply on point 3: yes, we want to remove the existing code, but i'd like to do that as a 2nd patch. let's get all the views support in, optional at first, and make an official release once the dust settles a little. then, we can work on making views requried and ripping out everything in another patch (we can keep using this issue, that's fine), and make another official release after *that* dust settles. by then we'll be ready to branch and start porting to 6.x in HEAD.

Another update with the following additions:
1. Added 'uri' field for project short name. We need this to allow /project/issues/ URLs. I probably haven't added this field in the ideal way though, since as is you have to add this field to the view itself to be able to access it in argument handlers--that's not what we want, since this should be a hidden field. I'll have to look into how to get this data but not display it.
2. Added project_issue_views_arguments() (implementation of hook_views_arguments()).
3. Added views_handler_argument_project_issue_project() handler to handle /project/issues/ and /project/issues/ URLs. This function is almost complete--currently the title isn't being set quite right, but otherwise it functions normally.
4. Updated view that adds the project_issue_project argument described above.

Here's a slightly different version from the last. It implements hook_views_pre_view() which rips out the project short name field since we don't need to actually display that but since when it's not included as a field in the view the arguments handler doesn't work properly. It might be possible to use the query_handler somewhere to add {project_projects}.uri into the query somewhere, but after spending about a day working on this unsuccessfully I give up.

I'm pretty much tapped out on this patch, so someone who knows more about the views API than I do will probably need to take over from here.

FYI: Now running on d.o. ;) See http://drupal.org/node/197286

Ugh, this was broken in project_issue_views_tables(). The "Project issue: Updated Time" definition was clobbering the 'node' entry. :( We need to use a different array name to avoid conflicts. While I was at it, I fixed the code for how we load the various views hooks, as per a discussion with Earl a few weeks ago about the most fool-proof method for conditionally loading views support that plays nicely with all the cache stuff.

woah....i didn't realize last night that you were actually running this *patch* on d.o. Did I mention that it wasn't exactly well tested? At least it only broke views once :)

why the whole project_issue_views_*() -> _project_issue_views_*() mapping? why not just put the project_issue_views_*() functions in the .inc file? it seems like it's just adding extra function calls.

Views sometimes invokes these hooks to populate its own tables before hook_menu() is invoked. :( Merlin and I have discussed this and decided this was the best approach. See http://drupal.org/node/183417. It'd be great is someone put that into the views dev handbook.

After this patch was installed on d.o for use with helping to track the GHOP tasks, we noticed a few things that were less than ideal about the patch with regards to it's use for tracking GHOP issues.

One of those was that the issue status filter was a select box, and thus it was not possible to select all issues with one of several statuses. I've attached a patch which changes the status filter from a select box to a list box, so that multiple statuses can be selected. I also changed the operator to allow this.

The only part of the patch that's changed is this block from _project_issue_views_tables() (this is the new version):

      'sid' => array(
        'name' => t('Project Issue: Status'),
        'operator' => 'views_handler_operator_andor',
        'list' => 'views_handler_filter_project_issue_status',
        'list-type' => 'list',
        'value-type' => 'array',
        'help' => t('Filter by issue status'),
      ),

I accidentally capitalized one letter of one function name in the previous patch. This fixes that.

After applying patch 5, everything seems to work, except I now get a white screen when saving any view. Memory isn't the problem and bottom of all files are clear of php end tag. Thoughts?

I didn't see a filter for issues assigned to current user, so I added one and rerolled the patch.

Okay, so I came back to it and was tinkering more tonight. I found that when using exposed status filters, I wanted a single option for selecting the 'default' statuses as indicated in the project issue status configuration. To do this I had to write a small filter handler that adds the default sids to the list then passes things back to the default filter handler. I'm not sure if this is the best approach, but it seemed to be the cleanest and quickest to me.

Patch is attached. Someone stop me if this isn't how I should be going about things... still a little new in the community.

Modified one line to use require_once instead of require to avoid fatal error in cases where cache is cleared and site is redirected and line of code would be hit again. Rerolled patch against 5.x-2.0

Just a note: If you use any iteration of this patch with any module that implements node access restrictions (og, tac, etc), you may encounter an error on some views using any filters or fields related to the Project Issue Project Node set.

This is related to Drupal core bug #206224: db_distinct_field mangles some queries, for which I've submitted a patch.

I appreciate everybody's work on this and I'm looking forward this makes it into a stable release.

Reroll to keep up with head and remove CRs.

Subscribing to this thread.

just FYI, I'm working on this this weekend. If someone else has a burning desire to work on this as well, please post here and maybe we can coordinate who does what. I'll post a patch tonight and probably tomorrow with my progress.

This is still a work in progress, but I've made good progress so far today. Here's a list of the more major things that I've fixed:
1. Created a "Project issues table" style plugin instead of using the hard coded theme function for theming the issues in the table by status.
2. Rewrote several field and filter hook implementations to be cleaner and less deprecated.
3. As far as I can tell, it's not going to be possible for us to use an argument in the URL for the project, like we've been doing. The reason is that if you select a different project to show issues for by using the project filter, the argument used in the URL doesn't change, and you get no issues. As an example, say you're at /project/issues/project and then you change the project filter to choose the Project issue tracking project. When you hit the submit button, the URL you'll be redirected to is /project/issues/project?filter0=75232 (the nid for the project_issue module). That won't give you any issues since issues are associated with only one project. So, I've dropped using the project uri/nid as a views argument and am just using the filter argument. Right now this only works if the filter argument is the nid, not the uri. I'm not sure if the later is possible, but I'll investigate.
4. By default, if you view a list of projects from all issues, you'll see the project name column but won't see the column of release version (this is how the issue tracker works now). If you narrow down to a specific project, the project name column disappears and the version column is now present. There is also an option for the version column to have it available all of the time, regardless of whether you're looking at projects from multiple issues.

Things that still need work:
1. I've only worked on the project/issues and project/issues/?filter0=pid type views. I haven't worked on the advanced search or issues by user yet. I don't think these will be that difficult since I've laid most of the groundwork for those already.
2. When you're viewing issues across multiple projects, the links in the project column go to the project node instead of a list of issues from only that project, which is how it works now. To some extent I think this makes more sense, but it's definitely a change from the way things work now and will surprise many people. Unfortunately, I'm not sure how to make these links go to the list of issues for that project. The only way I've thought of that will work is to basically hard code these links to the specific view that's creating them. But if someone were to change the order of the filters, or remove filters, then the links won't be right.

I've attached a screenshot of the view of issues from a specific project. If you happen to see something that looks more different than it should, let me know. I'll try to get the other views worked up by sometime tomorrow.

This patch adds some new fields and corrects a few other fields, including those from the users table. I've also fixed the filters so that they allow multiple selection.

In terms of functionality, this patch adds another default view, project_issue_projects_search, which (somewhat poorly) implements the advanced search functionality. The form at the top of the page for the advanced search is really ugly, but it seems to at least work.

Some additional things that need work:
1. The advanced search links do not point to the right place at the moment. They need to add filter arguments but aren't doing that.
2. I haven't added a field for participants in an issue (ie. those who have created and/or commented on an issue), so there's no search by participant in the advanced search.

I should also note that for now the paths for these views is project/issues2 and project/issues2/search. The view doesn't seem to override the menu item created by project issue, so I'm not sure how we're going to keep the old functionality while offering the new functionality of views (during the transition period). I'm sure we can figure this out later however.

My feeling is that it should be possible to do all of these views using only one actual view, but I am not quite sure how to do this (and haven't tried much).

If anyone is interested in testing this, I recommend using the drupal.org testing installation profile. I also recommend that you apply the most recent patch at #231640: Auto-generate some release nodes, which will automatically generate project release nodes, to this profile. Do so before you install Drupal.

Attached patch:
1. Adds the filter for participants and exposes this filter in the advanced search view.
2. Fixes the assigned filter so that any user can be selected and used for the filter, not just the current user.

Just in case anyone else is watching this issue, the plan is to commit what I've done so far for Views 1 (soon) and then postpone the rest of the views integration until we start working on the port of project issue to Drupal 6. See http://groups.drupal.org/node/9500 for more info.

Here's the final patch. As far as I can tell it exposes the relevant project_issue fields and creates filter handlers. There are two default views, but no attempt is made to completely replace current views-like functionality in project_issues.

Here's a new version with the queries in views_handler_filter_project_issue_participant() and views_handler_filter_project_issue_assigned() modified a bit.

In testing these all work fine with the exception that if you use the advanced search view, when you use the "Is None of" operator for the participant field, and you select several participants, you'll sometimes get issues back that have those people as participants.

So, for example, if I select that the participant operator is "Is none of" and select participants 1, 2, 3, 10, and 11, the query that's created is this:

SELECT DISTINCT(node.nid), project_issue_project_node.title AS project_issue_project_node_title, project_issue_project_node.changed AS project_issue_project_node_changed, project_issue_project_node.nid AS project_issue_project_node_nid, node.title AS node_title, node.changed AS node_changed, project_issues.sid AS project_issues_sid, project_issues.priority AS project_issues_priority, project_issues.category AS project_issues_category, project_issues.rid AS project_issues_rid, pi_node.changed AS pi_node_changed, pi_users.name AS pi_users_name, pi_users.uid AS pi_users_uid, project_issue_project_projects.uri AS project_issue_project_projects_uri FROM node node LEFT JOIN comments comments ON node.nid = comments.nid INNER JOIN project_issues project_issues ON node.nid = project_issues.nid INNER JOIN node project_issue_project_node ON project_issues.pid = project_issue_project_node.nid LEFT JOIN node pi_node ON node.nid = pi_node.nid INNER JOIN users pi_users ON project_issues.assigned = pi_users.uid INNER JOIN project_projects project_issue_project_projects ON project_issues.pid = project_issue_project_projects.nid WHERE (node.status = '1') AND (node.uid NOT IN (1,2,3,10,11)) AND (comments.uid NOT IN (1,2,3,10,11)) GROUP BY node.nid ORDER BY pi_node_changed DESC LIMIT 0, 50

One of the issues that is returned by the view has one comment by user 10.

But other than this problem, things work well. I'll keep this at CNR because I can't think of anything else to get this to work right and the rest still needs reviewing. At worst we could just omit that filter, I suppose.

I use the participant field sometimes, but can't think of a reason to use 'is none of' - could we not just lock the operator for that?

@catch: I guess that's better than just removing the filter entirely. A theoretical use case would be if you wanted to find issues on a project that a particular user or two had not participated in. You might want to do this if there are some users that tend to pollute issues and get them off track to the point where they become useless. I can think of a few cases on d.o where this might be handy, but at the same time with 200k users having a multi select field for username is not really practical. But on smaller sites it might be more useful.

That said, if nobody else has suggestions for *why* the query doesn't work right when "Is none of" is used as the operator, then I'll just reroll with that filter locked for now.

changing title

committed to 5.x-2.x and HEAD with some minor formatting and syntax corrections.

final work can be viewed at: http://drupal.org/cvs?commit=116454

Since I've marked about 10 issues as postponed based on this issue, I'm setting this back to postponed. We'll do full views integration and require views in the D6 port of the project issue module (see #157693: Upgrade project issue to Drupal 6.x).

Now that #98278: project* namespace bugs in $node is in, I'm actively working on this. My plan is to expose the data to views, use a stub project_issue.module that just implements hook_views_api(), and see how much of project_issue I can re-implement in views2. Then, I can rip out all the corresponding code in project_issue*. Then, I can use deadwood on the (presumably much smaller) codebase over at #157693: Upgrade project issue to Drupal 6.x.

Here's a start. If you install 76725_43_project_issue.module.txt as your project_issue.module in the latest HEAD, and apply the 76725_43_pi_views2.patch to populate your views subdir, you have a sort of functioning issue page for project/issue (all projects) and project/issue/[project_short_name]. TODO:

A) Row coloring based on status. #359122: Add an "Issue table" view style plugin to provide row coloring and the links at the top of the page

B) The little set of links above the exposed filters for "Create a new issue", "Search", "Subscribe", etc. #359122: Add an "Issue table" view style plugin to provide row coloring and the links at the top of the page

C) Using a project nid instead of a short name as the argument for project/issue/%.

D) Linking from the project name on the project/issue view to the issue queue for that project.

E) Showing an empty cell instead of "Anonymous" for unassigned issues.

F) Removing "ago" from the "Last updated" column. #379584: Remove 'ago' from the "Last updated" column on issue views

G) I'm not positive what to do about the project filter. On the project_issues_all_projects view, I'm just using a text field instead of the gigantic drop-down box (#201435: Huge list of projects hard to use, consumes excessive d.o. bandwidth, can hang browser). However, it's currently doing autocomplete on both project and release node titles, and I'm not sure why. I just removed the filter entirely on the per-project view, since I think that's going to be confusing and weird. Perhaps it's better to have a separate little text area that appears as if it's another exposed filter that really redirects you to project/issues/[short_name] based on the project title you type in...

H) There's no support for the "default issue status" set -- the config setting for which status values should be used for the default query. Currently, everything is just defaulting to "Any".

I) The page title for the per-project view is just using the URL arg (project short name) not the real project title. E.g. "Issues for project_issue" instead of "Issues for Project issue tracking"

J) There's no exposed filter for the issue version on the per-project view, even though that'd be really nice to add.

K) I'm not sure that the order of the items in the RSS feeds is right, even though I added an override for the sort I'm normally getting from the table style configuration. In fact, it doesn't appear that the feeds are showing the right items at all. #379608: Force a certain order in the issue queue RSS feeds

L) The "Issues" menu item that's supposed to be under navigation from the project_issues_all_projects view isn't appearing.

Then of course there's there's nothing yet for:

M) My issues #359129: Convert the "My issues" page to a view

N) My projects #359130: Convert the "My projects" page to a view

O) Advanced search #359131: Convert the issue queue "Advanced search" to a view

(These 3 might just want to move to separate issues, honestly).

But other than that, this works great! ;)

Assuming we resolve all of that (and I have complete faith we will) I'm pretty sure we can rip out at least all of this code.

(A) and (B) should be fixed over at #359122: Add an "Issue table" view style plugin to provide row coloring and the links at the top of the page by anyone who wants to help. That should be fairly self-contained...

(C) and (I) are both fixed by the patch at #359119: Add a views argument validator plugin for project nids which takes nids or project short names and the new patch here.

Earl agrees that (G) is a mess, and supports removing the "Project" filter from project/issues/[project] entirely. I think it'd be best to add a little "View issues" navigation block to make it easy for folks to jump to another issue queue if they really want. In fact, maybe we should just merge this in with the existing "Project navigation" block and have two buttons "View project" and "View issues" (or something). That block should also have a config knob to control if you want a giant drop-down or an autocomplete text area. All that said, it's still not clear why the exposed filter at project/issues is auto-completing both project and release node titles. :(

(M): #359129: Convert the "My issues" page to a view

(N): #359130: Convert the "My projects" page to a view

(O): #359131: Convert the issue queue "Advanced search" to a view

Re: (G) and autocomplete -- I'm just being sleep deprived -- there's no autocomplete from views at all, it was just my browser making suggestions. *sigh* However, Earl agrees that what I thought views was doing would be a killer feature. ;) It'd just be a lot of work, so I don't know if I'll be able to tackle that on my own with everything else I'm doing right now. Instead of the holy grail general solution, we could do some form_alter() to use the project autocomplete path for this particular exposed filter...

Whoops, I forgot the -N on the cvs diff for #45. ;) Try this...

Committed #44 and #47 to HEAD. Setting back to active for everything not crossed out in #43.

This fixes #43 (D) -- adds a field handler to give you a link to a project's issue queue.

Committed #49 to take care of (D), and updated the appropriate default views to use it. I fixed (G) via #364383: Alter "Project navigation" block form to add a "View issues" button. Back to active for the rest of #43...

#43 (L) works for me. What is the issue there?

I fixed (E) via #368234: Add a textfield for the value to print for anonymous users in views_handler_field_user_name (now in views 6.x-2.3).

(L) appears to have been something broken about my test site -- it's all working fine on d6.d.o.

There are no longer any critical blockers in here for the d.o upgrade, so I'm down-grading the priority.

(K) is still a bit worrisome, although I have no idea how wide-spread RSS subscriptions to the issue queues are...

Split (F) out into #379584: Remove 'ago' from the "Last updated" column on issue views

Split (K) out into #379608: Force a certain order in the issue queue RSS feeds

This is hereby done. (Yay!). ;)