The code below causes the statistics day timestamp to drift by the time it takes to execute the db operation to clear all the daycount fields. A small site will never notice but with 10's of thousands of nodes it becomes noticeable.

  if ((time() - $statistics_timestamp) >= 86400) {
    // Reset day counts.
    db_query('UPDATE {node_counter} SET daycount = 0');
    variable_set('statistics_day_timestamp', time());
  }

It should say something like

  if ((time() - $statistics_timestamp) >= 86400) {
    // Reset day counts.
    db_query('UPDATE {node_counter} SET daycount = 0');
    variable_set('statistics_day_timestamp', $statistics_timestamp+86400);
  }
CommentFileSizeAuthor
#31 521858-31.patch1.14 KBdimKarp
#21 521858-21.patch1.27 KBwim leers

Comments

bryrock’s picture

This is actually noticeable on small sites as well. It was after I noticed my daycount stats were reseting in the late afternoon that I started looking into it and found this (http://drupal.org/node/521858).

This is also somewhat beyond "drift." Even if the db operation takes only a second or two, with cron running hourly, the first test (if ((time() - $statistics_timestamp) >= 86400)) generally won't pass until the 25th hour because in the 24th hour it will be just a few seconds shy of 86400, so statistics_day_timestamp gains an hour a day.

jpp’s picture

Here is the code I'm using now - it's needs to be made timezone aware (the 3600*8 is a hack for pacific time) and I've commented out the log delete because I keep all my logs.

/**
 * Implementation of hook_cron().
 */
function statistics_cron() {
  $statistics_timestamp = variable_get('statistics_day_timestamp', '');
  $t = time() ;
  $t = $t - ($t % 86400);
  $t = $t + (3600*8);

  if ((time() - $statistics_timestamp) >= 86400) {
    // Reset day counts.
    db_query('UPDATE {node_counter} SET daycount = 0 WHERE daycount != 0');

    variable_set('statistics_day_timestamp', $t  );
  }

  // Clean up expired access logs.
  //db_query('DELETE FROM {accesslog} WHERE timestamp < %d', time() - variable_get('statistics_flush_accesslog_timer', 259200));
}
bryrock’s picture

Thanks for this. I'll give it a whirl. I've been going into the db and resetting this manually about every two weeks.

Two questions:

1 - Then Central time zone would be "$t = $t + (3600*6);" ?
2 - What is "$t = $t - ($t % 86400);" doing? (I'm not clear on the "%" operator in this context)

One thing I've been curious about and have not been able to find documented, when and how does statistics_day_timestamp get initialized the very first time? One might assume that daycounts are meant to start over at midnight, local time, but would it not depend on what time of day an instance of Drupal was fired up for the first time?

jpp’s picture

The % operator is modulo so $t = $t - ($t % 86400) will give you number of seconds at the last whole multiple of 86400 (IE midnight UTC) then add the fidge for the local timezone to get local midnight.

At the end of it all $t == last midnight in seconds since the unix epoch, then you just need to see if 86400 seconds have elapsed and if so we've hit midnight so do our thing. The other optimization is to set day count only on the records where it's not already zero in sites like mine with 30K+ modes that makes a big difference.

andypost’s picture

Version: 6.13 » 7.x-dev
Status: Active » Needs work

This issue should be fixed in 7.x first!

Marked as duplicate #733636: Statistics cron does not reset the day-count properly

andypost’s picture

Day statistics used only at block http://api.drupal.org/api/function/statistics_block_view/7

And because cron are rinning every 3 hours for D7 by default so we should only change setting to variable to take into account a site's time zone.

marcingy’s picture

Version: 7.x-dev » 8.x-dev
Issue tags: +Needs backport to D7

Moving to d8

koppie’s picture

Issue tags: -Needs backport to D7

I have tested this code with D6 and seems to work well. Any love for the old-timers?

webchick’s picture

Fixing tags.

andypost’s picture

There's a 2 different problems:
1 - statistics reset caused by statistics_cron()
2 - statistics display that does not take into account a site's timezone

andypost’s picture

Priority: Normal » Major
Issue tags: +Needs tests

This is functionality bogus at storage and display levels

catch’s picture

Priority: Major » Normal

Right that makes it a bug, but I don't think it's major in the slightest. Statistics counts are not anywhere in the critical path, and this only affects accuracy not anything else.

iamEAP’s picture

Is this actually still an issue now that those instances of time() have been replaced with REQUEST_TIME? This is true even for 7.x.

andypost’s picture

Yes, because cron running is unpredictable

andypost’s picture

Version: 8.0.x-dev » 8.1.x-dev
Issue summary: View changes

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.0-beta1 was released on March 2, 2016, which means new developments and disruptive changes should now be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

wim leers’s picture

Status: Needs work » Needs review
StatusFileSize
new1.27 KB

I think something as simple as this is all we need?

andypost’s picture

@Wim this is only half of problem, because cron run randomly by default, so some drift will exists

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

ressa’s picture

Title: Drift in statistics_day_timestamp yeilds inaccurate daycounts. » Drift in statistics_day_timestamp yields inaccurate daycounts.

Fixing title.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

dimKarp’s picture

StatusFileSize
new1.14 KB

I'm not sure if it is the right issue to mention that if someone views the node after the day change but before the cron run then the views of the previous day will be transferred to the current day. I used this as a temp solution. I am not very familiar with PHP. What do you guys think? or should I create a new issue for that?

Status: Needs review » Needs work

The last submitted patch, 31: 521858-31.patch, failed testing. View results

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

quietone’s picture

Status: Needs work » Postponed

Statistics is approved for removal. See #3266457: [Policy] Deprecate Statistics module in D10 and move to contrib in D11

This is now Postponed. The status is set according to two policies. The Remove a core extension and move it to a contributed project and the Extensions approved for removal policies.

It will be moved to a contributed Statistics project once the project is created and the Drupal 11 branch is open.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

quietone’s picture

Project: Drupal core » Statistics
Version: 11.x-dev » 1.0.0-beta1
Component: statistics.module » Code
Status: Postponed » Needs work