Better exception handling, use OAuth call if the account is authenticated

Posted by pheraph on December 1, 2011 at 4:15pm

8 followers

Issue Summary

I followed the steps in the ReadMe and in the end I was able to add a twitter-account to user 1, but the cron fails to import its tweets. Everytime cron is executed I get the following error message and it's unclear where 150 requests should come from (cron is executed twice per hour):

exception 'TwitterException' with message 'Rate limit exceeded. Clients may not make more than 150 requests per hour.' in /path/sites/all/modules/twitter/twitter.lib.php:250 Stack trace: #0 /path/sites/all/modules/twitter/twitter.lib.php(193): Twitter->request('http://twitter....', Array, 'GET') #1 /path/sites/all/modules/twitter/twitter.lib.php(78): Twitter->call('statuses/user_t...', Array, 'GET', '0') #2 /path/sites/all/modules/twitter/twitter.lib.php(126): Twitter->get_statuses('statuses/user_t...', Array, '0') #3 /path/sites/all/modules/twitter/twitter.inc(128): Twitter->user_timeline('313283513', Array, '0') #4 /path/sites/all/modules/twitter/twitter.module(146): twitter_fetch_user_timeline('313283513') #5 [internal function]: twitter_cron() #6 /path/includes/module.inc(794): call_user_func_array('twitter_cron', Array) #7 /path/includes/common.inc(5050): module_invoke('twitter', 'cron') #8 /path/modules/system/system.admin.inc(2301): drupal_cron_run() #9 [internal function]: system_run_cron() #10 /path/includes/menu.inc(503): call_user_func_array('system_run_cron', Array) #11 /path/index.php(21): menu_execute_active_handler() #12 {main}

Does anyone have an idea from where to begin? I am using OAuth 7.x-3.0-alpha2.

Edit: Just tested the "Post to Twitter"-function and it was posted successfully. So basically the connection is working. Somehow.

Regards,
Raphael

Comments

#1

Posted by steinmb on December 1, 2011 at 6:43pm

Priority:

major

» normal

You are triggering https://dev.twitter.com/docs/rate-limiting at the Twitter API, so you should be fine after 1h. On the other hand, we should not throw exception like that when this happen. twitter.lib.php prob. need some TLC.

#2

Posted by bugster on December 2, 2011 at 9:03am

from twitter:

Unauthenticated calls are permitted 150 requests per hour. Unauthenticated calls are measured against the public facing IP of the server or device making the request.

OAuth calls are permitted 350 requests per hour and are measured against the oauth_token used in the request

Because the twitter module uses oauth, should it not be 350?

#3

Posted by bugster on December 2, 2011 at 8:03am

After rereading:

The submodules twitter_post, twitter_signin and twitter_actions do require the OAuth module

So shouldn't this module also use oauth for receiving feeds? because outside oauth the rate is measured on ip level. For servers hosting multiple instances the 150 limit is easily reached...

#4

Posted by pheraph on December 2, 2011 at 9:18am

steinmb and bugster, thanks for your help so far. The Twitter modules are active since the day before yesterday and not a single cron job was able to fetch the latest tweets. It appears to be related to the 150 requests per hour per IP. Since this Drupal installation is on a _big_ hoster chances are high that there are too much Drupal/Twitter-installations trying to fetch Tweets. Using OAuth for receiving feeds might be helpful, but maybe this module won't work on big hosters because of Twitter's API limitations.

#5

Posted by bugster on December 2, 2011 at 9:56am

If you use OAuth the requests are measured by the oauth token

OAuth calls are permitted 350 requests per hour and are measured against the oauth_token used in the request

So I think updating the Twitter Module for using OAuth in receiving tweets is not a bad idea...

BTW: I experience the same problem, webserver hosting multiple sites and is now having issues updating the feeds.

#6

Posted by pheraph on December 2, 2011 at 10:57am

Category:

bug report

» feature request

So, this would rather be a feature request than a bug report.

#7

Posted by steinmb on December 2, 2011 at 1:25pm

Title:	Tweets aren't imported; Cron fails with "Rate limit exceeded"-exception	» Better exception handling, use OAuth call if the account is authenticated
Version:	7.x-3.0-beta3	» 7.x-3.x-dev

#8

Posted by 2bsharp on January 31, 2012 at 8:16pm

I had the same issue and was wondering if there was any work being done on this?

Due to Twitter's limits any site on a shared host will rarely (if ever) be able to update their tweets with this module. Since I assume a decent portion of sites are on shared hosts, this makes this module unusable for most people for retrieving tweets. I only cite this to suggest raising its priority.

#9

Posted by r.aubin on February 1, 2012 at 9:23pm

Having a similar problem in Rackspace Cloud Sites environment. There's simply no way this one site is exceeding the per hour rate and we only wish to read from the Twitter feed, not post. Installed OAuth module and configured it, but am I right that the Twitter module isn't using OAuth to check for new tweets?

#10

Posted by r.aubin on February 23, 2012 at 5:46pm

I'm using the latest stable 6.x, and have found that changing the user's account "protected" flag to 1, in the {twitter_account} table will force OAuth authentication, but unless it is actually a protected account, it will eventually change back to unprotected and stop using OAuth to pull latest tweets.

exception 'TwitterException' with message 'Rate limit exceeded. Clients may not make more than 150 requests per hour.' in /mnt/target02/350461/501661/www.thesite.org/web/content/sites/all/modules/twitter/twitter.lib.php:283 

Stack trace: 
#0 /mnt/target02/350461/501661/www.thesite.org/web/content/sites/all/modules/twitter/twitter.lib.php(225): Twitter->request('http://api.twit...', Array, 'GET') 
#1 /mnt/target02/350461/501661/www.thesite.org/web/content/sites/all/modules/twitter/twitter.lib.php(110): Twitter->call('statuses/user_t...', Array, 'GET', '0') 
#2 /mnt/target02/350461/501661/www.thesite.org/web/content/sites/all/modules/twitter/twitter.lib.php(158): Twitter->get_statuses('statuses/user_t...', Array, '0') 
#3 /mnt/target02/350461/501661/www.thesite.org/web/content/sites/all/modules/twitter/twitter.inc(102): Twitter->user_timeline('84667223', Array, '0') 
#4 /mnt/target02/350461/501661/www.thesite.org/web/content/sites/all/modules/twitter/twitter.module(140): twitter_fetch_user_timeline('84667223') 
#5 [internal function]: twitter_cron() 
#6 /mnt/target02/350461/501661/www.thesite.org/web/content/includes/module.inc(497): call_user_func_array('twitter_cron', Array) 
#7 /mnt/target02/350461/501661/www.thesite.org/web/content/includes/common.inc(2783): module_invoke_all('cron') 
#8 /mnt/target02/350461/501661/www.thesite.org/web/content/cron.php(10): drupal_cron_run() 
#9 {main}

I'm not sure why we're using the account->protected flag to decide whether or not to use OAuth in $twitter->user_timeline.

Twitter.inc (102)

$statuses = $twitter->user_timeline($account->id, $params, $account->protected);

#11

Posted by rob_dean on March 7, 2012 at 9:04pm

Category:

feature request

» bug report

My Twitter Feed has stopped updating a week ago.
I'm getting the Rate Limit Error in Drupal, but when I check the Twitter API immediately after running CRON, it says I still have 119 hits remaining. Any ideas where the disconnect might be occurring? I'm just trying to receive the feed.

Drupal Error

exception 'TwitterException' with message 'Rate limit exceeded. Clients may not make more than 150 requests per hour.' in /mnt/Target01/338050/610803/dev.site.org/web/content/sites/all/modules/twitter/twitter.lib.php:284 Stack trace: #0 /mnt/Target01/338050/610803/dev.site.org/web/content/sites/all/modules/twitter/twitter.lib.php(227): Twitter->request('http://api.twit...', Array, 'GET') #1 /mnt/Target01/338050/610803/dev.site.org/web/content/sites/all/modules/twitter/twitter.lib.php(112): Twitter->call('statuses/user_t...', Array, 'GET', '0') #2 /mnt/Target01/338050/610803/dev.site.org/web/content/sites/all/modules/twitter/twitter.lib.php(160): Twitter->get_statuses('statuses/user_t...', Array, '0') #3 /mnt/Target01/338050/610803/dev.site.org/web/content/sites/all/modules/twitter/twitter.inc(125): Twitter->user_timeline('0', Array, '0') #4 /mnt/Target01/338050/610803/dev.site.org/web/content/sites/all/modules/twitter/twitter.module(143): twitter_fetch_user_timeline('0') #5 [internal function]: twitter_cron() #6 /mnt/Target01/338050/610803/dev.site.org/web/content/includes/module.inc(795): call_user_func_array('twitter_cron', Array) #7 /mnt/Target01/338050/610803/dev.site.org/web/content/includes/common.inc(5121): module_invoke('twitter', 'cron') #8 /mnt/Target01/338050/610803/dev.site.org/web/content/modules/system/system.admin.inc(2310): drupal_cron_run() #9 [internal function]: system_run_cron() #10 /mnt/Target01/338050/610803/dev.site.org/web/content/includes/menu.inc(517): call_user_func_array('system_run_cron', Array) #11 /mnt/Target01/338050/610803/dev.site.org/web/content/index.php(27): menu_execute_active_handler() #12 {main}

Twitter API message

<?php
<hash>
<reset-time-in-seconds type="integer">1331156442</reset-time-in-seconds>
<reset-time type="datetime">2012-03-07T21:40:42+00:00</reset-time>
<remaining-hits type="integer">119</remaining-hits>
<hourly-limit type="integer">150</hourly-limit>
</hash>
?>

#12

Posted by rob_dean on March 9, 2012 at 6:20pm

*UPDATE
On my local machine, I upgraded Drupal from 7.12 to 7.13-dev, and the issue seems to have gone away. The tweets are now updating.

**UPDATE
After replacing all Files and Database on the Live Server, from the known working local copy, the Twitter has again stopped updating.

Could this have anything to do with hosting on RackSpace Cloud?

#13

Posted by steinmb on March 14, 2012 at 8:22am

Is it still an TwitterException with message 'Rate limit exceeded. Clients may not make more than 150 requests per hour.'?

#14

Posted by nicobo on May 6, 2012 at 9:09am

r.aubin, you can change the line to :

$statuses = $twitter->user_timeline($account->id, $params, _twitter_use_oauth());

-> this will use oauth even if protected is 0

#15

Posted by nicobo on May 6, 2012 at 9:11am

Why should it be a bad idea ?
On shared hosts, it looks logical to authentify against the oauth token...

Are you afraid to reach 350/day with all your operations on a given account ?

Project:	Twitter
Version:	7.x-3.x-dev
Component:	Twitter API
Category:	bug report
Priority:	normal
Assigned:	Unassigned
Status:	active

Better exception handling, use OAuth call if the account is authenticated

Jump to:

Issue Summary

Comments

Drupal Error

Twitter API message