Spectra Analytics

Why Spectra?

What we are looking to develop is a self-hosted analytics server which can be used as a platform for a wide variety of applications:

• Combine with monitoring programs like Zeomine Server Monitor, to build server monitoring dashboards.
• Decouple your reports from your application, i.e. no need to bury resource-intesive reports in an admin/* page!

Data specification

Statements

The core data object in Spectra is a subject-verb-object statement, with options to add a text category or "adjective/adverb" to each. In addition, the object includes a configurable context, and may be referenced by data objects. Data are therefore recorded as "statements" of the following structure:

ACTOR performed ACTION on OBJECT, in the given CONTEXT. Further information may be found in referencing DATA.

Schema-less data storage

Each entity in the statement has a JSON field for storing additional data, such that statements may support schemaless entry of a variety of data. For example, a server monitor may report access log data and server performance data as key-value pairs, with reports generated based on the key corresponding to the desired data.

Flat Statements

In addition to the core statements, as of 8.x-1.0-alpha13 we have a "flat" statement, which will consist only of an entry date, text category (and secondary category), and a JSON field. This is meant mainly for cases where you are looking to index/partition by category, and are looking to insert a large number of arbitrary data objects.

Data retention

As of version 8.x-10-alpha12 , Spectra comes with the Spectra Entity Expiration module, which permits users to create data retention policies which are managed by cron. This module depends on the Policy-based Entity Expiration module.

Database structure

The core table for this module is the statements table, and each statement may link to an actor, action, object, or context. Data links in the other direction, the spectra_data table links to the statement table, as one statement may have multiple pieces of data attached.

API

Currently we have the core API, at /spectra/post , and at this time it accepts JSON POST data with a basic handler that assumes you are posting objects with the same names as the database tables/fields.

We have also set up an annotation that is being used by src/Spectra/Plugin/DefaultSpectraPlugin.php You may extend this class and create a new annotation, and if you pass the "id" of the annotation using the key "plugin" in your JSON, and the API will pick this up and generate statements using your custom API plugin.

Example use cases

We are working on some case studies/example cases to add as a documentation guide. The first planned use case will involve a mailing list submodule located at https://www.drupal.org/sandbox/laboratorymike/3020691

Related modules and Distributions

• Spectra Connect API: API module for managing interactions between Drupal modules and a Spectra server.
• Spectra Server Distribution: Drupal distro which builds a Spectra server on installation.
• Policy-Based Entity Expiration: This can be used to create a data retention policy. Spectra also comes with a "Spectra Expire" module to handle deletion of un-used entities.
• Zeomine Server Monitor: Python library with the option to export data to a Spectra server.

Major updates coming for alpha

• As of 8.x-1.0-alpha15, we have chosen to add a dependency on the Key-based Authentication module. The decision was made after creating derivative modules that depend on this feature from Spectra, such as the Spectra Connect API.
• SpectraEntities will be simplified, in that actors, objects, and contexts will collectively be SpectraNoun entities, and actions will be SpectraVerb Entities. SpectraData and SpectraStatements will be unchanged by this, and we strongly encourage updating your module one 8.x-1.0-alpha16 is released.
• Once this update is complete, we will start documenting the API and plugins.
• The SpectraAction, SpectraObject, SpectraContext, and SpectraAction entities will be deprecated as of 8.x-1.0-alpha16, and will be deleted on the release of 8.x-1.0-beta1.