currently, if you have views_sparkline configured to show, say, two years worth of data, but only one year's worth of data gets returned by the view, the resulting sparkline stretches that data over the full time period of the dataset, which is an incorrect display.

there are perfectly valid reasons why less than a full set of data would be returned. in the case of project issue metrics, a project might only be a year old.

the easiest solution is to have the module examine the 'maximum items per page' setting of the view, and if less than that number of results is returned, then zero pad the data set before passing it off to the chart code.

a simple setting of pad left/pad right/don't pad would handle the most common cases of partial data. however, it would not handle, for example, a partial dataset that fell within the middle of the display range (eg. six months of no data, a year of data, six months of no data), this doesn't seem like as common of a case, and would be much more difficult to implement properly. i personally don't think that the module needs to handle any more than the basic cases.

Comments

mikey_p’s picture

mikey_p commented

Conceptually the problem here is that we are using SQL (via views) to fill in the datapoints when SQL doesn't really work that way. The two scenarios here that could cause issues are the following:

  1. A new project is created after the back fill of data. All old projects have been backfilled with data by the backfil process, but a new module might only return 20 points instead of 104 (for 2 years of weekly data). Views sparkline module itself can tell that there is a discrepancy between the view "items per page" setting and the number of rows returned, but flot itself will take the 20 points and stretch them to fill the entire plot, which is definitely not what we want.
  2. A project or bug in the individual metrics of some module could prevent the metrics from being recorded for some reason. Or the job to run the samples isn't triggered. In this case there could be fewer points because more recent data is missing (instead of least recent data in scenario 1.
  3. The other scenario is a gap in the data, and this is not a problem that views_sparkline or even SQL can easily solve.

Either of the first two scenarios could be handled by a setting in views_sparkline to "pad left" or "pad right" with NULL values to make sure that the correct time period is displayed on the plot. The confusing bit is that there could be a scenario where both of these conditions is true, and then the data wouldn't truly be accurate at all, and views_sparkline has no way of resolving that condition.

Possible solutions:

  1. Add the pad left or right option. This doesn't really ensure data integrity on output and has no way of determining if the data is correct. I'm not a huge fan of this option but it is the quickest way to get things going. This would also require redoing the views to not use a filter (may speed things up) and having it return results in descending order on timestamp instead of ascending. A views_sparkline option for the order of points would be needed.
  2. Don't display anything until the correct number of rows is returned. This doesn't really solve problem scenario 2 or 3.
  3. Rip this out and make some custom code in sampler for doing this that knows what the sample timestmaps should be and can line up the values fetched from the DB with a prefilled array of all the points that it knows should be on the plot for a given period.
  4. Write some kind of crazy views handler in sampler that tries to fetch all the correct points and generate null data for any missing rows based on it's default sample size period.
  5. Add some kind of dependency option to views_sparkline that uses sampler to compute where the sample timestamps should be and then maps the returned result into that data.
dww’s picture

dww
we/he/they
commented

Option 6: Don't care. Label the graph with the actual time period returned. The main thing for readability IMHO is that all the metrics on the same page have the same time scale. I'm assuming that'll still be true under the various scenarios here. I don't care as much that Project A has 2 years of data while Project B only has 14 weeks. So long as all the sparklines when viewing B are 14 weeks, you can still meaningfully understand trends, etc.

I'm not necessarily opposed to the other 5 solutions, but I wanted to throw #6 into the mix here for consideration. ;)

Cheers,
-Derek

mikey_p’s picture

mikey_p commented

The thing with labeling is that the only way to accurately label would be to label each sparkline with a start and end time (and again, assume there are no gaps) and that seems very redundant considering that there would be a label for each sparkline on the page. Further, making these labels pretty would probably be as much if not more work than some of the other solutions.

Personally from what I can tell adding labels would really go against the spirit of sparklines being high-density inline data.

dww’s picture

dww
we/he/they
commented

Agreed. I'm not talking about labels on the sparklines themselves. I meant in project_issue where we're injecting this into the issue cockpit block. I (perhaps incorrectly) assumed there'd be an easy way to figure out how many weeks of data our sparkline views are returning, and instead of always saying "Statistics (2 Years)" or whatever, that we could just create that label based on how much data we actually have. That'd also solve my concern that if the answer is 0 weeks of data we can just hide that whole section of the block...

mikey_p’s picture

mikey_p commented
Status: Active » Fixed

Pretty much went with option #1 from my comment. Just trying to get this out so we can deploy.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.