git clone http://git.drupal.org/sandbox/alanmackenzie/2099953.git views_usage_audit

The aim of this project is to aid refactoring of a Drupal code base by detecting unused view displays and providing information about the menu paths that view displays are executed on.

It achieves this by logging the view name, display name and the menu path when a view is executed.

Once you have this data you can run two types of reports either from drush or through the UI.

1) A list of all menu paths that a view display is executed on.
2) A list of all displays that have never been executed to date.

It is designed to be highly performant so it can be enabled on production environments, hence the use of drupal_static(), caching api, locking api, shutdown functions and logic to prevent unnessessary INSERTs.

After installing the module you must exercise your Drupal instance so a number of views are executed, otherwise the reports will be useless.

You can view the reports at admin/reports/views-usage-audit-reports or run the following drush commands:

drush views-usage-audit-report-unused
drush views-usage-audit-report-usage

Comments

klausi’s picture

Status: Active » Needs review

I guess this needs review? See https://drupal.org/node/532400

PA robot’s picture

We are currently quite busy with all the project applications and we prefer projects with a review bonus. Please help reviewing and put yourself on the high priority list, then we will take a look at your project right away :-)

Also, you should get your friends, colleagues or other community members involved to review this application. Let them go through the review checklist and post a comment that sets this issue to "needs work" (they found some problems with the project) or "reviewed & tested by the community" (they found no major flaws).

I'm a robot and this is an automated message from Project Applications Scraper.

Anonymous’s picture

Status: Needs review » Needs work

You are getting in pareview

FILE: /var/www/drupal-7-pareview/pareview_temp/views_usage_audit.module
--------------------------------------------------------------------------------
FOUND 0 ERROR(S) AND 1 WARNING(S) AFFECTING 1 LINE(S)
--------------------------------------------------------------------------------
7 | WARNING | There must be no blank line following an inline comment

http://pareview.sh/pareview/httpgitdrupalorgsandboxalanmackenzie2099953git

klausi’s picture

Status: Needs work » Needs review

Minor coding standard errors are not application blockers, please do a real manual review.

alanmackenzie’s picture

@klausi, Thank you for catching the failure in reading comprehension on my part.

Updates since original post:

  • Project now has a README.txt.
  • Squashed a minor @TODO statement causing PAReview to complain.
  • Improved inline documentation of the shutdown function responsible for inserting view usage data.
oresh’s picture

hi alanmackenzie,

nice module!
In your .info file you don't have to use "" around text.
And I wonder if this should go to Views package rather than Development, thought the purpose is different.
And this module should have dependency on Views, right?

In your views_usage_audit_reports() function you're using count() two times to get the same result in a foreach loop and again every iteration. Would you consider changing it to a variable? like this may be:

    $i = 1;
    foreach ($data as $view => $displays) {
      $rows[][0] = $view;
      foreach ($displays as $display => $paths) {
        $i++;
        $rows[$i][1] = $display;
        $path_column = array();
        foreach ($paths as $path => $value) {
          $path_column[] = $path;
        }
        $rows[$i][2] = implode('<br/>', $path_column);
      }
    }

The comments are also great, everything is well commented.

minoroffense’s picture

In your .info file you don't have a dependency on the Views module which you probably should (considering you're using the administer views permission in your hook_menu).

Speaking of hook_menu, perhaps changing the path from admin/reports/views-usage-audit-reports to just admin/reports/views-usage-audit as having the word "reports" in there seems redundant. Not really a bug, just an observation.

In your hook_schema, perhaps add the foreign key markers for the Views table keys you're using. https://api.drupal.org/api/drupal/includes!database!schema.inc/group/sch...

Great module though, looking forward to using it.

minoroffense’s picture

Issue summary: View changes

Forgot to add a link.

bogdan1988’s picture

Hi alanmackenzie, thank you for a very good module and nice code. Here is some suggestions:
1) For views block all info is shown, but we can hide admin paths like admin/structure/views/nojs/preview/%/%. If you think that we need them then please explain why;
2) Like in this comment https://drupal.org/comment/7990199#comment-7990199 you are still using count() twice;
3) Please read this wonderful article about drupal caching http://www.lullabot.com/blog/article/beginners-guide-caching-data-drupal-7. After reading you will agree with me that _views_usage_audit_fetch_audit_data() function could be written in easier way

function _views_usage_audit_fetch_audit_data() {
  $data = & drupal_static(__FUNCTION__);

  if (!isset($my_data)) {
    if ($cache = cache_get('views_usage_audit_data')) {
      $data = $cache->data;
    }
    else {
      $results = db_select('views_usage_audit', 'va')
        ->fields('va')
        ->execute();

      foreach ($results as $result) {
        $data[$result->view][$result->display][$result->path] = TRUE;
      }

      cache_set('views_usage_audit_data', $data);
    }
  }

  return $data;
}

Thank you, very nice module!

bogdan1988’s picture

And please add git clone to description. git clone http://git.drupal.org/sandbox/alanmackenzie/2099953.git views_usage_audit

alanmackenzie’s picture

Issue summary: View changes

Adding git clone command.

miroslavbanov’s picture

Status: Needs review » Needs work

As per comment #7, you have not declared dependency on views module. If you access the reports page (admin/reports/views-usage-audit-reports) with root account and no views module, you get
Fatal error: Call to undefined function views_get_all_views() in ../views_usage_audit/views_usage_audit.module on line 176
This is the only critical problem I see.

But there are a few minor ones:
- In _views_usage_audit_fetch_audit_data() you may consider adjusting your caching logic as per #8. Some of the checks you are making are unnecessary. #8 is very similar to drupal core code - see core function menu_load_all(). So it must be good.
- in README.txt "displays" is misspelled "dispays".
- in views_usage_audit_reports() you are calling count twice per iteration, and you can make it more readable and call count 0 times if you use something like:

$row = array()
$row[0] = $view;
..
$rows[] = $row;

I know that this is not the scope of this project, but have you thought about making this a full-blown statistics module with information for when a view was last used, and how many times in total?

alanmackenzie’s picture

Status: Needs work » Needs review

@All Thank you for your reviews so far, I've learned a few tricks that I'll be using in future :)

Below is a list of commits that solves all problems mentioned above.

  • f438895 The reports page will now include themed link to the views edit page for each view is the views_ui module is enabled.
  • e2a3088 Removing redundant check for empty data in views_usage_audit_reports().
  • b60c1e7 Added views_usage_audit_ignored_paths persistant variable the default value includes the ajax and nojs preview paths from from the views_ui module.
  • 0a8bbf5 Adding foreign key relationships for the views_usage_audit table.
  • 8aacf39 Refactored _views_usage_audit_fetch_audit_data() to be simpler and inline with similar core functions.
  • ea76e2b Added a @TODO dealing with a potential small code quality improvement.
  • 3a9759a Added views as a dependency of this module.
  • a30c18a Refactored views_usage_audit_reports() to not use count(), making it simpler to maintain.
  • 4f4897d Removing duplicated word 'report' for the path on which the usage reports are generated.
  • 6bfe7e2 Fixed spelling mistake in the README.
  • 2f36174 Removing quotes from .info file.

I know that this is not the scope of this project, but have you thought about making this a full-blown statistics module with information for when a view was last used, and how many times in total?

The thought has crossed my mind however this is a much more difficult problem to scale as it would not be possible to completely avoid INSERTs after the initial warm up period. Perhaps it could be possible by logging to watchdog then running a drush command that does some analysis on the messages generated.

I have had the module enabled on bbcgoodfood.com for a few weeks with zero performance impact, so we can be sure it preforms at the highest levels in its current form.

miroslavbanov’s picture

Status: Needs review » Needs work
Issue tags: +Needs database review

There is a small issue in views_usage_audit_reports(), where you should be building one row per display, but in fact, you are overwriting the same row for every display of a view.

  $rows = array();

  foreach ($data as $view => $displays) {
    $row[0] = theme('views_usage_audit_report_link', array('view' => $view));

    foreach ($displays as $display => $paths) {
      $row[1] = $display;

      $path_column = array();
      foreach ($paths as $path => $value) {
        $path_column[] = $path;
      }
      $row[2] = implode('<br/>', $path_column);
    }

    $rows[] = $row;
  }

Other than that, I see no other issues (except possibly performance). The drush commands, and the general usage both work.

One thing I can't really review with confidence is the performance implications of this module. I think the scenario for performance problem is when a view is deployed on a very busy site for the first time on the front page, and many hits are made all at once, write-locking the database. I am not sure how to test this at all. I think someone that understands databases better should review.

alanmackenzie’s picture

Status: Needs work » Needs review

@MiroslavBanov, Thanks for catching that bug with the reports. I've fixed that now.

In terms of database performance I do not think this will be an issue for a number of reasons:

Contention - We're using Drupal's database locking api to avoid and row contention on INSERT in the database.
Volume - In my experience the watchdog module is likely to generate a larger number of INSERTs on a page request than this module.

As I said in my last post I've already deployed this to www.bbcgoodfood.com, according to www.topdrops.org it's the 99th largest Drupal site in the world so I'm fairly happy to say that is has been proven production ready.

miroslavbanov’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: -Needs database review

Thank you for the explanation, and for contributing this great module. I am marking this RTBC.
As for the performance/database concerns, the locking API is used to prevent db_merge from locking the database, and the potential writes/locking are rare edge-case thing, and less severe then what dblog can cause.

kscheirer’s picture

Status: Reviewed & tested by the community » Fixed

Do you need cache_clear_all('views_usage_audit_data', 'cache'); after every View is executed in the shutdown function? Seems like that would slow down the site slightly. Perhaps a better idea to clear cache when the report is generated? Or allowing the admin to configure cache lifetime.

Thanks for your contribution, alanmackenzie!

I updated your account to let you promote this to a full project and also create new projects as either a sandbox or a "full" project.

Here are some recommended readings to help with excellent maintainership:

You can find lots more contributors chatting on IRC in #drupal-contribute. So, come hang out and get involved!

Thanks, also, for your patience with the review process. Anyone is welcome to participate in the review process. Please consider reviewing other projects that are pending review. I encourage you to learn more about that process and join the group of reviewers.

Thanks to the dedicated reviewer(s) as well.

----
Top Shelf Modules - Crafted, Curated, Contributed.

alanmackenzie’s picture

Cheers for the promotion.

@kscheirer

There's code to detect if display execution has already been logged and in that case not to bother with an INSERT. Delaying the clearing of the cache would likely end in a substantially worse hit ratio and a less performant site.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.