I noticed quit a few people have very large table sizes for ad statistics (200-700mb+).

Currently ad statistics creates a new row every hour, every day, for every ad (if it's viewed). This means that there will be 24 * 365 = 8760 rows per year per ad. That's a lot of rows to keep unless we really need to go back 2 years earlier and to a specific hour in time (which 90+% of people don't). This is more granular than google analytics. Not to mention, when viewing the statistics page it has to calculate all these rows.

The display block only shows 'this hour, last hour, today, yesterday, last seven days, this month, last month, this year, last year, and all time'. Based on that there's no reason there shouldn't be an option to aggregate the statistics.

Meaning, why can't we aggregate statistics and have, for example, a table for: 1) current hour and last hour (remove the rest), 2) the last seven individual days (so we can calculate today, yesterday, and last 7 days), 3) month (aggregated by adding the last 7 days together for 3 weeks), 4) year (aggregated by adding the last 12 months), etc.

This would reduce the 8760 rows down to about 14 (depending how you do it). If we don't really need granularity down to the last hour for an infinite amount of time, there's no reason to keep all that data, right?

Any thoughts on this and how maybe we could implement this and change the database structure for aggregating data into less granular chunks? Even if we just saved past 30 days for each ad along with the current days hourly count it would significantly reduce the amount of uneeded data that's saved.

Comments

cerup’s picture

Title: Aggregate statistics so reduce table size » Aggregate statistics to reduce table size
joshuautley’s picture

Subscribe.

michael_lessard_micles.biz’s picture

+1

This is important.

I am currently moving my website, and the ad_statistics table is causing me a lot of trouble and time...

Even when dividing the mysql dump, only for that table, in 10 parts (!), it is still too heavy for phpMyAdmin to import normally. I might have to split it in 15 to 20 different .sqp dump files!!!

This has to be fixed.

jenlampton’s picture

Great idea. It would also save me a lot of headaches if we could det this table under control.

andrewtf’s picture

Late to the game, but +1 on this.

lrwebks’s picture

Status: Active » Closed (outdated)

Drupal 6 is EOL and no longer supported. Closing this as outdated for that reason. Thanks for your contribution!