As announced at https://groups.drupal.org/node/333658, we want a new metric: the average wait time for an issue to receive a response. This is a good way to measure health of a project, and with D7 Drupal.org it will actually be possible to capture this. [May have been just as easy in D6, but it seemed true at the time.]
The time to response is: min(comment creation where uid ≠ node uid) - (node creation)
When graphed over time, this needs to be shown in a way that does not penalize following up on old issues. The time could be measured by node creation. This means times in the past could go up when old issues are responded to.
drush -v sampler-update-storage-plugin project_issue responses_by_project drupal_database_table_per_metricdrush -v sampler-sample project_issue responses_by_project --object_batch_size=10000 --startstamp=1010275200Todo
| Comment | File | Size | Author |
|---|---|---|---|
| #9 | out.tsv_.txt | 4.42 KB | drumm |
| #7 | chart_1-2.png | 24.37 KB | drumm |
Comments
Comment #1
drummComment #2
mgiffordLooking forward to seeing this! Thanks @drumm!
Just adding some related GDO links:
https://groups.drupal.org/node/314018
https://groups.drupal.org/node/312828
Comment #3
mgiffordNot sure how or why it got marked postponed. But since it's mentioned here I assume it's not any more https://association.drupal.org/content/drupalorg-team-week-notes-22
Comment #4
drummThis was postponed since metrics in general were not working. Since they have moved forward, this can move forward. The next step is to write a metric to collect this data; see the others in the metrics directory of this module.
Comment #5
xurizaemonOver the lifetime of a project, the level of community engagement changes - will an average indicate *current* engagement, eg when a project was once active but has fallen into disuse, or recently invigorated with a new maintainer? A poor average on full project lifetime might be a hard stat to change once established.
Comment #6
drummThis is plotted over time. The metrics from #1998044: Port issue metric sparklines in statistics block on project pages to D7 are graphs over the last 2 years, with each point being a week. This should be the same.
Since these are in week-long bins, we probably also want response rate for the week. Responding to an old issue shouldn't bring up your average for the current week. Metrics are designed to have static numbers for the past — responding to an old issue shouldn't change previous times.
Comment #7
drummTo sanity check this metric, I ran this query on staging for issues posted in the last 2 years:
SELECT days, count(1) c FROM (SELECT ceil((min(c.created) - n.created) / 60 / 60 / 24) days FROM node n LEFT JOIN comment c ON c.nid = n.nid AND c.uid <> n.uid AND c.uid <> 180064 AND c.status = 1 WHERE n.status = 1 AND n.type = 'project_issue' AND n.created > (unix_timestamp() - 60 * 60 * 24 * 365 * 2) GROUP BY n.nid ORDER BY NULL) d GROUP BY days DESC;44% got a response within a week, 30% within a day. 30% have not had a response at all. The raw results are attached.
This seems okay, 44% average 1-week response rate gives projects room on either side to be either above or below average.
Comment #8
mgiffordNice.. Thanks Neil! Nice to have that guideline.
Comment #9
drummExcluding issues posted by the projects' maintainers, which are often notes not needing a response, have 46% 1-week response rate, 31% 1-day, and 27% no-response.
SELECT days, count(1) c FROM (SELECT ceil((min(c.created) - n.created) / 60 / 60 / 24) days FROM node n INNER JOIN field_data_field_project fp ON fp.entity_id = n.nid LEFT JOIN comment c ON c.nid = n.nid AND c.uid <> n.uid AND c.uid <> 180064 AND c.status = 1 WHERE n.status = 1 AND n.type = 'project_issue' AND n.uid NOT IN (SELECT pm.uid FROM project_maintainer pm WHERE pm.nid = fp.field_project_target_id) AND n.created > (unix_timestamp() - 60 * 60 * 24 * 365 * 2) GROUP BY n.nid ORDER BY NULL) d GROUP BY days DESC;Comment #10
drummComment #11
drummComment #12
drummComment #14
drummComment #16
drummThis is ready to be deployed, but the metrics are still being calculated.
Comment #17
drummNow deployed.
Comment #18
mgifford@drumm thanks for doing this. I'm trying to go from these squiggles that you've added to having some insights to the health of various modules. If I don't get it, and I've been following this issue, probably most other people don't either.
I'm thinking a link to a human readable explanation of New issues, Response rate, 1st response, Open bugs & Participants would be great. Each of those linking off to the exact SQL (#9 in this case) that's being used to generate it.
Anyways, looking at Core + 15 top modules:
Core 163 68% 13hrs 6004 263
Views 13 0 0 1319 39
CTools 6 0 0 398 19
Token 1 0 0 58 3
Pathauto 1 0 0 108 10
Libraries 1 0 0 8 5
Admin Menu 0 0 0 100 2
Date 1 40% 9hrs 591 11
Webform 10 80% 4hr 207 30
IMCE 1 0 0 3 2
Entity 2 0 0 241 10
WYSIWYG 3 50% 54hrs 69 12
Google Analytics 5 100% 2hrs 4 7
Backup and Migrate 1 0 0 56 5
CKEditor 2 14% 1hr 257 15
Link 1 0 0 110 8
Which of these has the best stats?
Comment #19
rjacobs commentedThanks for this useful and handy metric! However, in echoing what mgifford said, I think there is a slight UX issue with the presentation, specifically the "summary" number shown beside the graph. It seems that in cases where there is no new issue activity for the past period (week), the "Response Rate" and "First Response" values are shown as "0%" and "0 hours" respectively (this seems to be quite common). From a logical perspective this makes sense, but from an end-user perspective this is confusing. Given that these two values imply "rate" or "average", a value of 0 at any given time sends a strange message, and without additional explanation it's a bit of a head scratcher. A rate of "0" for the other metrics ("Open bugs", "Participants", etc) makes sense, as these can easily be interpreted as absolute snapshot values, but these new metrics don't necessarily fit that pattern.
All that said, I know this is a new feature, so if I'm looking at cached or incomplete data (and jumping to conclusions on how this works), I apologize.
Perhaps the number next to the graph should really be the 2-year running average, or if no data is available for a given period something like "n/a" should be shown? Perhaps this should be considered a separate "labeling" issue, that warrants another queue entry?
Comment #20
drummThe code for the dynamic queries is at http://drupalcode.org/project/project_issue.git/tree/refs/heads/7.x-2.x:.... The metrics added by this issue are in the
ProjectIssueResponsesByProjectclass.A project can have few new issues because it is stable, and/or included in D8; or because it isn't used much. The lower numbers don't necessarily mean they are bad. Project quality is complex, I don't think it can be totally simplified down to best or not as great by software.
The way to read these new numbers is: 68% of the new core issues from the week-before-last were responded to by the end of last week. Those that got a response took an average of 13 hours. A response is the first comment by someone other than the original reporter or System Message. Issues posted by the project's maintainers are not considered.
Comment #21
drummI posted a followup at #2259341: Better represent projects with few new issues in responses metrics, which I think will be an improvement for projects that have weeks with no new issues.
The numbers on the right are for the last week, labels for the red dots. A 2-year average would be too long for anyone to make a visible impact, without a lot of patience.
Comment #22
mgiffordSo much variety.. Not an easy thing to represent in one squiggly line.
Comment #23
rjacobs commentedThanks drumm for opening up the separate issue, I'll move any of my own through over there. I think this feature is great, my only concern was that it's usefulness may not be immediately clear.