Just thought it'd be awesome to have a sparkline graph (Google Charts?) that displays performance; both indexing and search. =) Also might be helpful to know if/when searches are taking "too" long.

CommentFileSizeAuthor
#7 apachesolr_stats-7.zip6.79 KBjanusman
#7 apachesolr-408990-7.patch17.08 KBjanusman
#5 apachesolr_stats.zip6.39 KBjanusman
#4 408990-4 sample report.jpg84.81 KBjanusman
#4 apachesolr-408990-4.patch15.03 KBjanusman

Comments

janusman’s picture

janusman commented
Title: Admin candy: sparkline graph showing Solr performance » Keep track of Solr performance

Toning this down: it would at *least* be helpful to know long query, MLT, indexing (and other?) operations are taking. Any pointers to existing projects like this one?

Scott Reynolds’s picture

Scott Reynolds commented

apphend to the query queryDebug=true and you get a whole lot of useful stuffs

http://wiki.apache.org/solr/CommonQueryParameters#head-f45a9396425956a4d...

Be cool to log some of that away, and only when enabled

edit: i don't believe it works with mlt though :(

robertdouglass’s picture

robertdouglass commented
Version: 6.x-1.x-dev » 6.x-2.x-dev

A debug mode would be interesting.

janusman’s picture

janusman commented
Status: Active » Needs review
StatusFileSize
new apachesolr-408990-4.patch15.03 KB
new 408990-4 sample report.jpg84.81 KB

First proposed patch; this is a contrib module that leverages Apache Solr's hooks to add debugging info. to each query and log query results on query completion.

Instructions:
* Patch applies to 6.x-1.x and 2.x, as does not modify existing files, just adds a new module in contrib/
* Enable the module "Apache Solr Statistics" under admin/build/modules under
* Go to admin/settings/apachesolr/stats and enable logging.
* Do some searches, and go to admin/reports/apachesolr/stats to see the report (see attached sample).

It's still missing quite a lot:
* Want to register clickthrus to report which facets were most valuable to obtain those... this way I'd know a little about what my users are using (and ignoring).
* Want to register clickthrus for the spellchecker suggestions.
* Want to report indexing performance.

janusman’s picture

janusman commented
StatusFileSize
new apachesolr_stats.zip6.39 KB

In case someone wants just the contrib/apachesolr_stats folder, here is a ZIP file you can place in your modules/apachesolr/contrib folder (although it'll work fine in sites/all/modules)...

janusman’s picture

janusman commented
Status: Needs review » Needs work

Just a note, the graphs' granularity is "per minute", which will surely break when the log begins to fill up. Will issue a new version soon.

janusman’s picture

janusman commented
Status: Needs work » Needs review
StatusFileSize
new apachesolr-408990-7.patch17.08 KB
new apachesolr_stats-7.zip6.79 KB

New patch and .zip file.

Changes: This now uses the shorter data format from Google Charts, so that more points can fit in a single image. It also now aggregates results per hour, not by minute.

robertdouglass’s picture

robertdouglass commented 

  if (! variable_get('apachesolr_stats_enabled', 0)) {
    return t('Logging is disabled in the !link. Enable it to log Apache Solr queries.', array('!link' => l('module configuration page', 'admin/settings/apachesolr/stats')));
  }

could that be part of the access checking for the report path in hook_menu so you never get to the report if logging isn't enabled?

robertdouglass’s picture

robertdouglass commented

with regards to granularity: Something like a set of links to change granularity on the reports page would be great: month | day | hour | minute

And then somehow change this:
$result = db_query_range("SELECT * FROM {apachesolr_stats} ORDER BY timestamp DESC", 0, 1000);

Except... result sets will get really really big. Hmmm.

robertdouglass’s picture

robertdouglass commented

I know we all trust google... but can we do something to protect this from abusive input?

  $image_url = $chart_prefix . $encoded_values; #&chdl=". t('Queries per minute');
  // Add labels
  $image_url .= "&chxl=0:|". intval($chd_min) ."|". intval($chd_max);
  if ($average !== FALSE) {
    $image_url .=sprintf("|1:|%s=%.2f", t('average'), $average);
    $image_url .= "&chxp=1," . intval($average/$chd_max*100);
    $chxt = "y,r";
  } else {
    $chxt = "y";
  }
  $image_url .= "&chxt=$chxt";
  return "<img src='$image_url'>";
}

Peter - do you see any risk here?

robertdouglass’s picture

robertdouglass commented

Awesome functionality! I think the trend is to make separate projects instead of adding to /contrib. I think you should just make a new project and commit it =)

janusman’s picture

janusman commented
Status: Needs review » Closed (fixed)

Thanks for your review and help... opened up a new project:

http://drupal.org/project/apachesolr_stats

Closing this out. =)

kenorb’s picture

kenorb commented

Something similar: #725928: Show the query being generated