Keep track of Solr performance
janusman - March 21, 2009 - 00:41
| Project: | Apache Solr Search Integration |
| Version: | 6.x-2.x-dev |
| Component: | Miscellaneous |
| Category: | task |
| Priority: | minor |
| Assigned: | Unassigned |
| Status: | closed |
Description
Just thought it'd be awesome to have a sparkline graph (Google Charts?) that displays performance; both indexing and search. =) Also might be helpful to know if/when searches are taking "too" long.
#1
Toning this down: it would at *least* be helpful to know long query, MLT, indexing (and other?) operations are taking. Any pointers to existing projects like this one?
#2
apphend to the query queryDebug=true and you get a whole lot of useful stuffs
http://wiki.apache.org/solr/CommonQueryParameters#head-f45a9396425956a4d...
Be cool to log some of that away, and only when enabled
edit: i don't believe it works with mlt though :(
#3
A debug mode would be interesting.
#4
First proposed patch; this is a contrib module that leverages Apache Solr's hooks to add debugging info. to each query and log query results on query completion.
Instructions:
* Patch applies to 6.x-1.x and 2.x, as does not modify existing files, just adds a new module in contrib/
* Enable the module "Apache Solr Statistics" under admin/build/modules under
* Go to admin/settings/apachesolr/stats and enable logging.
* Do some searches, and go to admin/reports/apachesolr/stats to see the report (see attached sample).
It's still missing quite a lot:
* Want to register clickthrus to report which facets were most valuable to obtain those... this way I'd know a little about what my users are using (and ignoring).
* Want to register clickthrus for the spellchecker suggestions.
* Want to report indexing performance.
#5
In case someone wants just the contrib/apachesolr_stats folder, here is a ZIP file you can place in your modules/apachesolr/contrib folder (although it'll work fine in sites/all/modules)...
#6
Just a note, the graphs' granularity is "per minute", which will surely break when the log begins to fill up. Will issue a new version soon.
#7
New patch and .zip file.
Changes: This now uses the shorter data format from Google Charts, so that more points can fit in a single image. It also now aggregates results per hour, not by minute.
#8
<?phpif (! variable_get('apachesolr_stats_enabled', 0)) {
return t('Logging is disabled in the !link. Enable it to log Apache Solr queries.', array('!link' => l('module configuration page', 'admin/settings/apachesolr/stats')));
}
?>
could that be part of the access checking for the report path in hook_menu so you never get to the report if logging isn't enabled?
#9
with regards to granularity: Something like a set of links to change granularity on the reports page would be great: month | day | hour | minute
And then somehow change this:
$result = db_query_range("SELECT * FROM {apachesolr_stats} ORDER BY timestamp DESC", 0, 1000);
Except... result sets will get really really big. Hmmm.
#10
I know we all trust google... but can we do something to protect this from abusive input?
<?php$image_url = $chart_prefix . $encoded_values; #&chdl=". t('Queries per minute');
// Add labels
$image_url .= "&chxl=0:|". intval($chd_min) ."|". intval($chd_max);
if ($average !== FALSE) {
$image_url .=sprintf("|1:|%s=%.2f", t('average'), $average);
$image_url .= "&chxp=1," . intval($average/$chd_max*100);
$chxt = "y,r";
} else {
$chxt = "y";
}
$image_url .= "&chxt=$chxt";
return "<img src='$image_url'>";
}
?>
Peter - do you see any risk here?
#11
Awesome functionality! I think the trend is to make separate projects instead of adding to /contrib. I think you should just make a new project and commit it =)
#12
Thanks for your review and help... opened up a new project:
http://drupal.org/project/apachesolr_stats
Closing this out. =)