Support the Visualization API

Hi Alex,

We are indeed quite interested in improving our API. One approach would be to start supporting visualisation API.

The main reasons why we would want to upgrade our API are outlined on the charts graphs homepage:

1. Allow for multilevel datasets.
2. Allow for links in labels & values.
3. Allow for library specific configuration forms.
4. Create exportable global graph configurations.

1 + 2 need work in visualisation API too.

But 3 + 4 have at least the basics working in visualisation API, because its using ctools.

In general, I'd say, I like the fact that the charts_graphs classes are more specific than the visualisation API classes, which makes switching between libraries very easy for views_charts and facetapi_charts. (and maybe soon openlayers_charts)

Let's start a conversation about this and work out a plan.

Reinier

I'm coming from from @batje's invitation in #2039615: Discussing generic APIs. Sure, I'd be up for discussing that - not sure this issue is really the best place to do it, but maybe as good a place as any.

In case it's helpful, I'll briefly explain the approach I took in the 7.x-2.x branch of Highcharts, based on a few very general requirements.

When creating Drupal charts, I want:

1. All of the options available in the JS charting library at my disposal

   The intention was to create an API that allows unmediated use of the full Highcharts JS library, as opposed to supporting only a subset of the options. Most other Drupal charting modules that I'm aware of take the latter approach (including Visualization API, Data Visualization API, Charts and Graphs, and quicksketch's recent work updating Charts). To achieve the former requirement, my feeling was that a wrapper API module should not rewrite specific options, but instead support them all by only making sure we properly pass the options (necessary for the chart we want) in the format the JS library is expecting. In other words, if I have a hard-coded form to choose the xAxis etc, something is wrong. This is why the module file is basically an implementation of hook_library(), a main render function, and a few helper functions (required for overriding Highcharts methods, since they're converted to strings when passed through JavaScript settings). Getting all of the options into Views is a different story, and still needs a bit of UI love to complete (definitely open to suggestions here! The current plan in outlined in #1674928: Allow full use of Highcharts library Options in Views).

2. Total flexibility in what kind of data gets presented in various chart types

   This was my reasoning behind adding the (optional) options classes, which can be used as templates to quickly build common chart types that are also passed to the clean API above. You can see these within the Views integration submodule style plugin. The reason I think we need classes that can be overridden is that we don't know exactly which data people want to present in exactly what ways as charts. I explained some examples in point 4 in this issue comment - I would love to discuss adding "some sensible settings with a way to let other modules add new ones for additional use cases", if you have any interest in this.

I wonder if my requirements are compatible with the goal of having a generalized, pluggable graphing API? For example, different JS charting libraries offer actually quite different options, and the UIs in these other modules appear to support those places where the various libraries overlap (whereas I wanted to support all of the options without the API having to care or even really know about them). It's possible this approach above could be abstracted slightly to work within a wrapper that also includes Google charts etc, but I'm not sure what else could really be shared between them beyond that wrapper? Personally I love the idea, and am open to collaborating on something like this, as long as the above concerns are shared.

scottrigby thanks for your response. Your concerns about specialisation are quite common with people who maintain a module that implements a specific graphing library (in fact, I think, this is one of the most common issues in ICT in general: specialize or generalize. Drupal or custom code, right?)

So far, with charts_graphs what is happening with the most powerful modules (flot and d3) is that there are 5! modules involved:

Charts Graphs defines the API definition, the base classes and functions to generate a graph
Charts Graphs Views and FacetAPI Graphs generate data an call the Charts Graphs API to generate graphs
Charts Graphs Flot and Charts Graphs D3 (and google & bluff) implement the Charts Graphs API and translate the Charts Graphs API to a specific graphing library.
And lastly, there are specialised flot & d3 modules that implement their graphing libraries in a Drupal way. They each have a specific way of doing this that is specific to the way the creator/maintainer sees fit. The flot module knows a lot about Flot, the d3 module tries to be as thin as possible. This is very much what highchart also tries to do.

The flot/d3/highchart modules should totally focus on giving drupal developers the most out of their respective libraries, I completely agree on that. Charts Graphs (and visualisation API and others) can and will only implement a subset of these functions.

I really like your idea of the Options Classes. They remind me bit of the FacetAPI Widgets. FacetAPI widgets *do* implement option forms. Every widget type has its own Options Form and all Option Forms are keyed & sent to the browser. When a user changes from one widget (or in our case, graph type) the option form changes along.
I would think Options Classes & their Forms would be implemented on the charts_graphs_d3 or charts_graphs_highcharts level. Like in FacetAPI, there could also be some generic options (like width and height). But generally, every library + chart type can implement their own options.

The problem (as you mention in the comment you link to in item 4) is the data. We would need to figure out a generic way to pass on data from views/facetapi/whateverthinggeneratesdata) through charts_graphs_highchart, throught ChartsGraphsOptionsClassPiechart->mangledata to the actual piechart.

If we can come up with a more or less generic way to pass on data from the data providing modules to the charts_graphs_d3 layer, that would be very helpful.

Wether we then implement this in the visualisation API or in charts_graphs is not very important imho.

Let me take a first shot at that. This would only be the data(series), not the axis definitions etc.

The series property sets the data to be plotted on the graph. This definition is taken from the current charts_graphs:

"It's an associative array of arrays. The keys are the data series names (which will appear on the legend of the graph). Each inside array holds the values for that data serie."
I'd like to extend that so it looks like this:

$data = array (
  "elephant sightings" => array(
    "2008" => array (
       "#value" => 200,
       "#tooltip" => t("elephant sightings in 2008. Click link to view more elephants"),
       "#url" => "http://graphs.com/elephants/2008",
       "#classes" => "elephants, 2008",
       "#data" => array (
         "January => array (
           "#value" => 6,
           "#hover" => t("elephant sightings in January 2008. Click link to view more elephants"),
           "#popup" => '<img src="myelephant.jpg"/>',
           "#url" => "http://graphs.com/elephants/2008/January",
           "#classes" => "elephants, 2008, January, winter",

etc.

This allows for multi-dimentional graphs, tooltips, and clickable graphs (very nice for filtering). Remember, this will never arrive like this in your specialised highchart/flot/d3 modules, there is a layer in between that will interpret and filter this data, so you only get what you implement.

Hope this is yet another slow, small step towards a combined effort to make graphing and charting simple for a lot of Drupal users out there.

To whom it may concern ... I am a (fairly new) co-maintainer of the Forena module, of which earlier this week a major new D7 release became available. Not many people seem to be aware of this module, but I believe it does an amazing job in charting also, using the PHP SVG Graph library. Checkout the community documentation to get an idea of the supported chart types.

I recently started a Comparison with other charting solutions. This comparison is not yet complete, at least the Visualization API seems to be one of the missing solutions there (I plan to add that soon to it).

However, as a result of working on that comparison, I decided to create the forena feature request #2364427: Renderer for Charts and Graphs APIs, which includes some details about why it would be beneficial to both modules involved. Subsequent to trying to get something going, I then stumbled on this interesting issue (which seem to not have moved anymore for over a year though ...). What really caught my attention was this part of #3:

If we can come up with a more or less generic way to pass on data from the data providing modules to ...

.

Not sure if it would be of any help, but for forena "charting" is just one of the various things you can do with it (it's just one of quite a few ways to render/format the data). Instead one of forena's major key features seem to be that you can use it to go get your data from database formats like Oracle, MS SQL, a Drupal database, any PDO compliant database, Postgress, XML, SQLite, ...

If anybody is interested in further exploring if forena can help, I'd be happy to consider participating.