Tried to turn on caching for one of the most expensive query "tracker". I had data caching for 30 minutes and html for 0 sec.
What I noticed after this the pager disappears. The cache seems to be working.

At a purist level this is what it was supposed to since it cached only x results of data from query that is what you see.

From a users perspective that odd since I would expect atleast the pager should show and after going to next page it will cache that data since technically the query has different started at a different offset.

If we can get tracker view caching data, that would be a boon to many webmasters to reduce one of the expensive query.

CommentFileSizeAuthor
#44 views-6.x-3.x-dev-550962.patch552 bytesLes Lim
#9 views_cached_pager.patch3.16 KBigor.ro
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

dawehner’s picture

dawehner
Primary language German
CreditAttribution: dawehner commented

Could you please export your tracker and post it here?

I tested it again, and it worked well.

Perhaps thats a problem with the bug in time based caching, did you tryed the dev version of views, or just the stable release?

ajayg’s picture

ajayg CreditAttribution: ajayg commented

I used 2.6 stable with patch mentioned here #503774: Cache Time based does not reset

Here is my tracker view. It came default from OG

$view = new view;
$view->name = 'og_tracker';
$view->description = 'Shows active posts across the whole site.';
$view->tag = 'og';
$view->view_php = '';
$view->base_table = 'node';
$view->is_cacheable = FALSE;
$view->api_version = 2;
$view->disabled = FALSE; /* Edit this to true to make a default view disabled initially */
$handler = $view->new_display('default', 'Defaults', 'default');
$handler->override_option('fields', array(
  'tid' => array(
    'label' => 'साहित्य प्रकार',
    'alter' => array(
      'alter_text' => 0,
      'text' => '',
      'make_link' => 0,
      'path' => '',
      'alt' => '',
      'prefix' => '',
      'suffix' => '',
      'help' => '',
      'trim' => 0,
      'max_length' => '',
      'word_boundary' => 1,
      'ellipsis' => 1,
      'strip_tags' => 0,
      'html' => 0,
    ),
    'type' => 'separator',
    'separator' => ', ',
    'empty' => '',
    'link_to_taxonomy' => 1,
    'limit' => 1,
    'vids' => array(
      '9' => 9,
      '10' => 0,
      '7' => 0,
      '5' => 0,
      '8' => 0,
      '1' => 0,
      '4' => 0,
      '6' => 0,
      '11' => 0,
      '2' => 0,
      '3' => 0,
      '12' => 0,
      '14' => 0,
    ),
    'exclude' => 0,
    'id' => 'tid',
    'table' => 'term_node',
    'field' => 'tid',
    'override' => array(
      'button' => 'Override',
    ),
    'relationship' => 'none',
  ),
  'title' => array(
    'label' => 'शीर्षक',
    'alter' => array(
      'alter_text' => 0,
      'text' => '',
      'make_link' => 0,
      'path' => '',
      'alt' => '',
      'prefix' => '',
      'suffix' => '',
      'help' => '',
      'trim' => 0,
      'max_length' => '',
      'word_boundary' => 1,
      'ellipsis' => 1,
      'strip_tags' => 0,
      'html' => 0,
    ),
    'link_to_node' => 1,
    'exclude' => 0,
    'id' => 'title',
    'table' => 'node',
    'field' => 'title',
    'relationship' => 'none',
    'override' => array(
      'button' => 'Override',
    ),
  ),
  'name' => array(
    'id' => 'name',
    'table' => 'users',
    'field' => 'name',
    'label' => 'Author',
    'link_to_user' => 1,
    'relationship' => 'none',
  ),
  'comment_count' => array(
    'label' => 'प्रतिसाद',
    'alter' => array(
      'alter_text' => 0,
      'text' => '',
      'make_link' => 0,
      'path' => '',
      'alt' => '',
      'prefix' => '',
      'suffix' => '',
      'help' => '',
      'trim' => 0,
      'max_length' => '',
      'word_boundary' => 1,
      'ellipsis' => 1,
      'strip_tags' => 0,
      'html' => 0,
    ),
    'set_precision' => FALSE,
    'precision' => 0,
    'decimal' => '.',
    'separator' => ',',
    'prefix' => '',
    'suffix' => '',
    'exclude' => 0,
    'id' => 'comment_count',
    'table' => 'node_comment_statistics',
    'field' => 'comment_count',
    'relationship' => 'none',
    'override' => array(
      'button' => 'Override',
    ),
  ),
  'group_nid' => array(
    'id' => 'group_nid',
    'table' => 'og_ancestry',
    'field' => 'group_nid',
    'label' => 'Groups',
    'type' => 'separator',
    'separator' => ', ',
    'empty' => '',
    'relationship' => 'none',
  ),
  'last_comment_timestamp' => array(
    'label' => 'शेवटचा प्रतिसाद',
    'alter' => array(
      'alter_text' => 0,
      'text' => '',
      'make_link' => 0,
      'path' => '',
      'alt' => '',
      'prefix' => '',
      'suffix' => '',
      'help' => '',
      'trim' => 0,
      'max_length' => '',
      'word_boundary' => 1,
      'ellipsis' => 1,
      'strip_tags' => 0,
      'html' => 0,
    ),
    'date_format' => 'small',
    'custom_date_format' => '',
    'exclude' => 0,
    'id' => 'last_comment_timestamp',
    'table' => 'node_comment_statistics',
    'field' => 'last_comment_timestamp',
    'relationship' => 'none',
    'override' => array(
      'button' => 'Override',
    ),
  ),
  'timestamp' => array(
    'label' => '',
    'alter' => array(
      'alter_text' => 0,
      'text' => '',
      'make_link' => 0,
      'path' => '',
      'alt' => '',
      'prefix' => '',
      'suffix' => '',
      'help' => '',
      'trim' => 0,
      'max_length' => '',
      'word_boundary' => 1,
      'ellipsis' => 1,
      'strip_tags' => 0,
      'html' => 0,
    ),
    'link_to_node' => 0,
    'comments' => 0,
    'exclude' => 0,
    'id' => 'timestamp',
    'table' => 'history_user',
    'field' => 'timestamp',
    'relationship' => 'none',
    'override' => array(
      'button' => 'Override',
    ),
  ),
  'new_comments' => array(
    'label' => '',
    'alter' => array(
      'alter_text' => 0,
      'text' => '',
      'make_link' => 0,
      'path' => '',
      'alt' => '',
      'prefix' => '',
      'suffix' => '',
      'help' => '',
      'trim' => 0,
      'max_length' => '',
      'word_boundary' => 1,
      'ellipsis' => 1,
      'strip_tags' => 0,
      'html' => 0,
    ),
    'set_precision' => FALSE,
    'precision' => 0,
    'decimal' => '.',
    'separator' => ',',
    'prefix' => '',
    'suffix' => ' नवीन',
    'link_to_comment' => 1,
    'no_empty' => 1,
    'exclude' => 0,
    'id' => 'new_comments',
    'table' => 'node',
    'field' => 'new_comments',
    'relationship' => 'none',
    'override' => array(
      'button' => 'Override',
    ),
  ),
));
$handler->override_option('sorts', array(
  'last_comment_timestamp' => array(
    'id' => 'last_comment_timestamp',
    'table' => 'node_comment_statistics',
    'field' => 'last_comment_timestamp',
    'order' => 'DESC',
    'granularity' => 'second',
    'relationship' => 'none',
  ),
));
$handler->override_option('filters', array(
  'status' => array(
    'id' => 'status',
    'table' => 'node',
    'field' => 'status',
    'operator' => '=',
    'value' => 1,
    'group' => 0,
    'exposed' => FALSE,
    'expose' => array(
      'operator' => FALSE,
      'label' => '',
    ),
    'relationship' => 'none',
    'expose_button' => array(
      'button' => 'Expose',
    ),
  ),
));
$handler->override_option('access', array(
  'type' => 'none',
));
$handler->override_option('cache', array(
  'type' => 'none',
));
$handler->override_option('title', 'संपूर्ण मायबोलीवरचं नवीन लेखन');
$handler->override_option('items_per_page', 25);
$handler->override_option('use_pager', '1');
$handler->override_option('style_plugin', 'table');
$handler->override_option('style_options', array(
  'grouping' => '',
  'override' => 1,
  'sticky' => 0,
  'order' => 'desc',
  'columns' => array(
    'type' => 'type',
    'title' => 'title',
    'name' => 'name',
    'comment_count' => 'comment_count',
    'group_nid' => 'group_nid',
    'last_comment_timestamp' => 'last_comment_timestamp',
    'timestamp' => 'title',
    'new_comments' => 'comment_count',
  ),
  'info' => array(
    'type' => array(
      'sortable' => 0,
      'separator' => '',
    ),
    'title' => array(
      'sortable' => 0,
      'separator' => ' ',
    ),
    'name' => array(
      'sortable' => 0,
      'separator' => '',
    ),
    'comment_count' => array(
      'sortable' => 0,
      'separator' => '<br />',
    ),
    'group_nid' => array(
      'separator' => '',
    ),
    'last_comment_timestamp' => array(
      'sortable' => 0,
      'separator' => '&nbsp;',
    ),
    'timestamp' => array(
      'separator' => '',
    ),
    'new_comments' => array(
      'separator' => '',
    ),
  ),
  'default' => 'last_comment_timestamp',
));
$handler = $view->new_display('page', 'Page', 'page');
$handler->override_option('path', 'group/tracker');
$handler->override_option('menu', array(
  'type' => 'tab',
  'title' => 'मायबोलीवर नवीन',
  'description' => '',
  'weight' => '5',
  'name' => 'navigation',
));
$handler->override_option('tab_options', array(
  'type' => 'none',
  'title' => '',
  'description' => '',
  'weight' => 0,
));
$handler = $view->new_display('feed', 'Feed', 'feed');
$handler->override_option('style_plugin', 'rss');
$handler->override_option('style_options', array(
  'mission_description' => FALSE,
  'description' => '',
));
$handler->override_option('row_plugin', 'node_rss');
$handler->override_option('row_options', array(
  'item_length' => 'default',
));
$handler->override_option('path', 'group/tracker/feed');
$handler->override_option('menu', array(
  'type' => 'none',
  'title' => '',
  'description' => '',
  'weight' => 0,
  'name' => 'navigation',
));
$handler->override_option('tab_options', array(
  'type' => 'none',
  'title' => '',
  'description' => '',
  'weight' => 0,
));
$handler->override_option('displays', array(
  'page' => 'page',
  'default' => 0,
));
$handler->override_option('sitename_title', FALSE);
catch’s picture

catch
he/him
Primary language English
CreditAttribution: catch commented
Title: caching: Pager disappears while caching tracker: Is this the way it is intended? » Caching query results but not rendered output breaks pager

I'm also getting this - I'm caching the query results of the tracker, but not the output, so that #new links etc. still work per-user. I can't reproduce this if I cache both output and query, or neither.

Since pager_query() also populates all the globals for the actual pager etc., this makes sense, but means it'd need some special handling to fix it.

Phoenix2020’s picture

Phoenix2020 CreditAttribution: Phoenix2020 commented

I am also having this problem with my views:
- Caching "Query results" + "Rendered output": pager is there and working
- Caching none: pager is there and working
- Caching only "Query results": pager disappeares after the query results have been cached (after reloading the page).

eaton’s picture

eaton CreditAttribution: eaton commented

Please note, too, that a number of subtle but annoying cache invalidation bugs have been fixed in the 2.7 release. Is there any way you can test this again using Views 2.7, rather than 2.6?

Phoenix2020’s picture

Phoenix2020 CreditAttribution: Phoenix2020 commented

I just made a test with Views 2.7 and the problem is still there. Exactly as described above:
1) Caching only "Query results"
2) Calling the view the first time (no cached queries): pager is there and working
3) Reloading the view / Calling it while queries are cached: pager disappears

ajayg’s picture

ajayg CreditAttribution: ajayg commented

Reconfirming the bug exists in View 2.7. If the query is cached, the pager disappears.

Phoenix2020’s picture

Phoenix2020 CreditAttribution: Phoenix2020 commented
Version: 6.x-2.x-dev » 6.x-2.7

Any news on this issue? A working solution to cache the query results would be really great.

igor.ro’s picture

igor.ro CreditAttribution: igor.ro commented
FileSize
views_cached_pager.patch3.16 KB

Problem was in synchronization views pager data with global pager variables.

Please test with several pagers on one page.

Patch attached.

igor.ro’s picture

igor.ro CreditAttribution: igor.ro commented
Version: 6.x-2.7 » 6.x-2.8
Status: Active » Needs review
Phoenix2020’s picture

Phoenix2020 CreditAttribution: Phoenix2020 commented

I just tested the patch from #9 and it works. The pager is NOT disappearing if the page is cached.

MJH’s picture

MJH CreditAttribution: MJH commented
Status: Needs review » Fixed

The Patch from igor.ro works perfectly in my case. The function to only cache the "Query results" is finally working with the pagers. Thanks

dawehner’s picture

dawehner
Primary language German
CreditAttribution: dawehner commented
Status: Fixed » Reviewed & tested by the community

Thats the wrong status :)

izmeez’s picture

izmeez CreditAttribution: izmeez commented

subscribing

ajayg’s picture

ajayg CreditAttribution: ajayg commented

I confirm the patch above fixes the issue I reported. Tested with views 2.8

merlinofchaos’s picture

merlinofchaos CreditAttribution: merlinofchaos commented

Because the patch moves a chunk of code and teh change appears to be in this chunk, I can't easily tell what was changed within this chunk.

Also, Views 3 handles pagers differently, so in order to commit this i'll need a patch that works against both 2.x and 3.x

ajayg’s picture

ajayg CreditAttribution: ajayg commented

@merlinofchaos
the patch is to move the chunk of code but there is no change in that chunk of code itself. That chunk of code is now a function call which is now called from two places (instead of previous only one place). The chunk itself has not changed.

merlinofchaos’s picture

merlinofchaos CreditAttribution: merlinofchaos commented
Status: Reviewed & tested by the community » Active

Still need patch against 3 before I can commit to 2.

chawl’s picture

chawl CreditAttribution: chawl commented

subs

ajayg’s picture

ajayg CreditAttribution: ajayg commented

Anybody familiar with views 3.x, can you please look at the patch in #9 and create one for views 3.x so both can be commited? This would be a quite good performance improvement.

seanr’s picture

seanr
he/him
CreditAttribution: seanr commented

This definitely still happens, and it's even more specific than the bug reported above - both caches have to be set to the exact same time limit in order for the pager to appear consistently.

izmeez’s picture

izmeez CreditAttribution: izmeez commented

subscribing

secretuser’s picture

secretuser CreditAttribution: secretuser commented
Version: 6.x-2.8 » 6.x-2.11

The problem is still here with version 6x-2.11

crea’s picture

crea CreditAttribution: crea commented

subscribing

dawehner’s picture

dawehner
Primary language German
CreditAttribution: dawehner commented

Several questions to merlinofchaos here

  • Currently the cache plugin uses $this->view->total_rows, which is not set anymore. The cache plugin could a) load the data from the pager plugin, or b the pager plugin could set $this->view->total_rows
  • In views3 there is pager::update_page_info which does the same as synchronize_pager() in the patch. Perhaps we could change the patch a bit.
  • In the first item of this list, we could add a new function view::get_total_rows or view::get_total_items
yched’s picture

yched CreditAttribution: yched commented

and the last question remained unspoken :-)

dawehner’s picture

dawehner
Primary language German
CreditAttribution: dawehner commented

Oh this was just a non-closed tag. Ignore it.

ronnbot’s picture

ronnbot CreditAttribution: ronnbot commented

Also have been experiencing the same issue. Haven't tried the patch, but we've been using a work around by implementing the hook 'hook_preprocess_views_view' to add the pager back to the view when needed. Ironically it does so by caching the pager when it exists. 

function hook_preprocess_views_view(&$vars) {
  $vars['pager'] = _view_pager_cache($vars['view'], $vars['pager']);
}

function _view_pager_cache($view, $pager) {
  $ttl = $view->display_handler->options['cache']['results_lifespan'] > $view->display_handler->options['cache']['output_lifespan'] ?
           $view->display_handler->options['cache']['results_lifespan'] :
           false;

  if ($ttl > 0) { // query caching is longer than render cache; therefore, we need to cache the pager, as it will surely be gone
    $args = isset($view->args) && is_array($view->args) ? implode("|", $view->args) : "";
    $pager_cid = "view_pager.{$view->name}.{$view->current_display}.$args.{$view->pager['current_page']}";

    if ($pager  != null) { // cache it while we can
      cache_set($pager_cid, $pager, 'cache', time()+$ttl);
    }
    else if ($data = cache_get($pager_cid)) { // pager doesn't exist, grab it from the cache if available
      $pager = $data->data;
    }
  }
  return $pager;
}
dawehner’s picture

dawehner
Primary language German
CreditAttribution: dawehner commented

Question to everyone:

Do you use caching of the views?

ajayg’s picture

ajayg CreditAttribution: ajayg commented

Yes I use. I use different cache settings for different views. It reduces load considerbably for some of my views that don't need to be refreshed immediately.

dawehner’s picture

dawehner
Primary language German
CreditAttribution: dawehner commented

I think there was a issue about problems with pagers and caches

ajayg’s picture

ajayg CreditAttribution: ajayg commented

The issues is there if you want to just cache query but not HTML. If you cache both then there is no issue. A great usecase where this is required is "tracker" type pages. You want to cache query but to preserve new comments links you don't want to cache html.

Vacilando’s picture

Vacilando CreditAttribution: Vacilando commented

Subscribing.

catch’s picture

catch
he/him
Primary language English
CreditAttribution: catch commented
Status: Active » Needs work

@dereine, this is the issue. However the patch needs work rather than active since it's just a porting issue.

dawehner’s picture

dawehner
Primary language German
CreditAttribution: dawehner commented

oh i just have to many issues. I will work on a solution and search the other issue

catch’s picture

catch
he/him
Primary language English
CreditAttribution: catch commented

Patch in http://drupal.org/node/550962#comment-2353282 for 6.x-2.x

kika’s picture

kika CreditAttribution: kika commented
Status: Needs work » Needs review

Confirming the problem on both views 2.11 and latest 3.x and that the patch on #9 works for me.

voxpelli’s picture

voxpelli CreditAttribution: voxpelli commented
Version: 6.x-2.11 » 6.x-3.x-dev
Status: Needs review » Needs work

According to #18 this patch needs a version for Views 6.x-3.x prior to being committed. Therefor the status for this is "needs work" and nothing else. The fact that it works for Views 6.x-2.x doesn't change that status.

I also want to point out that this is related to #636128: Don't have the cache rely on $_GET which in turn is dependent on #635990: Make the cache respect db_rewrite_sql() and substitutions.

wojtha’s picture

wojtha CreditAttribution: wojtha commented

Subscribing.

Aurochs’s picture

Aurochs CreditAttribution: Aurochs commented

That patch did not help me with Views 6.x-2.11, installed that by hand.

Aurochs’s picture

Aurochs CreditAttribution: Aurochs commented

Upgraded to 3.x and that solved the problem but now i have to fix tons of pager issues and some other cause i need to change seetings almost for all views.

srjosh’s picture

srjosh
he/him
CreditAttribution: srjosh commented

Subscribing; I've also seen in some cases where setting both causes the pager to break, but it's inconsistent.

catch’s picture

catch
he/him
Primary language English
CreditAttribution: catch commented

The pager works when you get a cache miss, it's on cache hits where it doesn't get initialized.

Les Lim’s picture

Les Lim
he/they
Primary language English
CreditAttribution: Les Lim commented
Status: Needs work » Needs review
FileSize
views-6.x-3.x-dev-550962.patch552 bytes

The fix for 6.x-3.x-dev should hopefully be much simpler. Patch attached, which works in limited testing.

mstrelan’s picture

mstrelan CreditAttribution: mstrelan commented

subscribe

podarok’s picture

podarok
🇺🇦 he/him/his
Primary language Ukrainian
Location Ukraine
CreditAttribution: podarok commented
Status: Needs review » Reviewed & tested by the community

#44 works good for 6.x-3.0-alpha3 too (with line shift)
so RTBC

also tested with #1089034: True views cache for a multiple blocks with multiple pagers

merlinofchaos’s picture

merlinofchaos CreditAttribution: merlinofchaos commented
Status: Reviewed & tested by the community » Fixed

Committed to all branches.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

Vacilando’s picture

Vacilando CreditAttribution: Vacilando commented

Does 6.x-2.x-dev contain the solution as well?

Les Lim’s picture

Les Lim
he/they
Primary language English
CreditAttribution: Les Lim commented
Version: 6.x-3.x-dev » 6.x-2.x-dev
Status: Closed (fixed) » Needs review

Nope, still needs action for 6.x-2.x. There's a patch that needs to be re-reviewed in #9 above.

Les Lim’s picture

Les Lim
he/they
Primary language English
CreditAttribution: Les Lim commented
Status: Needs review » Closed (fixed)

Whoops, never mind. Here's the commit message:

https://drupal.org/commitlog/commit/1174/6ce5bb11927ac8824667f7497008446...