Make refresh rate of feeds configurable
Raines37 - May 6, 2009 - 21:14
| Project: | FeedAPI |
| Version: | 6.x-1.x-dev |
| Component: | Code feedapi (core module) |
| Category: | feature request |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | closed |
Description
This patch updates feedapi.module to use variable_get() instead of the constant FEEDAPI_CRON_MIN_REFRESH_TIME. The constant is still used as the default value. I'm using settings.php to override 'feedapi_cron_min_refresh_time' to a lower value than the default 30 minutes. Eventually it would be nice to have this as a setting on admin/settings/feedapi.
| Attachment | Size |
|---|---|
| cron_min_refresh_time.patch | 838 bytes |
#1
#2
I think there is somewhere else an issue for that...
However, it has been requested more than once that the refresh rate of feeds should be configurable. Right now FeedAPI tries to update as often as possible, up to every 30 minutes.
If we do that, we should make this a configuration option on a per content type / per feed level like any other FeedAPI option.
- Introduce a refresh_rate variable in feedapi_feedapi_settings_form().
- Convert checked to next_refresh_time, make it hold not the time last refreshed, but the time the feed should be refreshed next.
- When creating or refreshing a feed, set next_refresh_time to time() + refresh_rate.
- When querying feeds for refreshing do SELECT ... FROM ... WHERE next_refresh_time < time()
#3
Giving this a shot.
#4
Implements configurable refresh rates following the approach in #2, also consolidates configurable refresh time with skip flag: skipping a feed is a refresh_time of 0, refreshing a feed as often as possible is a refresh_time of 1, refreshing a feed once in 30 minutes is a refresh_time of 1800 and so forth.
Note that FeedAPI cannot guarantee a refresh time - the actual refresh time depends at least on the number of feeds in the system and on your hardware.
If you review this patch, start with the changes to feedapi_feedapi_settings_form(), _feedapi_insert(), _feedapi_update() and feedapi_update_6106() - these contain the core of the new functionality.
I tested hook_update, setting refresh_time back and forth and cron updates. There are some adjustments to the views integration that I didn't test at all. I did not test installing feedapi with the patch applied either.
_Back up your databases before testing_
#5
Alex, your patch seems to work. However that level of configurability is not something I'd like to have when managing a huge number of feeds. I mean if you want to fine tune the system you may need to update the refresh time in a thousand feeds, one by one.
I'd rather have just a global setting or a per-content type one, but not per feed.
Also, can we make something about FEEDAPI_CRON_FEEDS ? Just making it configurable with a single variable...
#6
#5
Very true. And not only for feed refresh settings, but also for feed _item_ content type settings, refresh on creation, Delete items older than, etc, etc...
This is an old problem in FeedAPI. I'd love to solve this on a global level: #201056: Only save per-node settings if different from per-content-type settings.
#7
#5: in regards to FEEDAPI_CRON_FEEDS, check out the patch on #370318: Keep exceeding cron time with FeedAPI
#8
I fixed one bug in the patch: when the feed was created, the Never refresh option was effectless.
Otherwise, I tested it and reviewed the code also, looks good!
#9
Reviewed, found some issues, fixed them:
1)
Update was not updating refresh_time correctly:
// before
$settings['refresh_time'] = FEEDAPI_CRON_DEFAULT_REFRESH_TIME;
// now
$settings['refresh_time'] = $settings['skip'] ? FEEDAPI_CRON_NEVER_REFRESH : FEEDAPI_CRON_DEFAULT_REFRESH_TIME;
2)
Feed updates where not behaving correctly if next_refresh_time was FEEDAPI_CRON_NEVER_REFRESH:
//before
elseif ($old_config->feed->settings['refresh_time'] != $node->feedapi['refresh_time']) {
// now
elseif ($old_config->feed->settings['refresh_time'] != $node->feedapi['refresh_time'] || $next_refresh_time == FEEDAPI_CRON_NEVER_REFRESH) {
#10
I did more testing on this patch.
Running a refresh manually reset next_refresh_time also when refresh_time was NEVER_REFRESH.
Fixed with:
<?php
- db_query("UPDATE {feedapi} SET next_refresh_time = %d, half_done = %d, hash = '%s' WHERE nid = %d", time() + $settings['refresh_time'], $half_done, $feed->hash, $feed->nid);
+ // Set next_refresh_time to FEEDAPI_CRON_NEVER_REFRESH if refresh_time is FEEDAPI_CRON_NEVER_REFRESH.
+ $next_refresh_time = $settings['refresh_time'] == FEEDAPI_CRON_NEVER_REFRESH ? $settings['refresh_time'] : (time() + $settings['refresh_time']);
+ db_query("UPDATE {feedapi} SET next_refresh_time = %d, half_done = %d, hash = '%s' WHERE nid = %d", $next_refresh_time, $half_done, $feed->hash, $feed->nid);
?>
#11
I tested it a lot again. The functionality is okay, fortunately I catched some issues again:
checked column name appeared in the views integration and in the test suite. So i did some find'n'replace in the feedapi package :) and replaced it when it was needed.
#12
Committed.
#13
Automatically closed -- issue fixed for 2 weeks with no activity.