I followed the steps in the ReadMe and in the end I was able to add a twitter-account to user 1, but the cron fails to import its tweets. Everytime cron is executed I get the following error message and it's unclear where 150 requests should come from (cron is executed twice per hour):

exception 'TwitterException' with message 'Rate limit exceeded. Clients may not make more than 150 requests per hour.' in /path/sites/all/modules/twitter/twitter.lib.php:250 Stack trace: #0 /path/sites/all/modules/twitter/twitter.lib.php(193): Twitter->request('http://twitter....', Array, 'GET') #1 /path/sites/all/modules/twitter/twitter.lib.php(78): Twitter->call('statuses/user_t...', Array, 'GET', '0') #2 /path/sites/all/modules/twitter/twitter.lib.php(126): Twitter->get_statuses('statuses/user_t...', Array, '0') #3 /path/sites/all/modules/twitter/twitter.inc(128): Twitter->user_timeline('313283513', Array, '0') #4 /path/sites/all/modules/twitter/twitter.module(146): twitter_fetch_user_timeline('313283513') #5 [internal function]: twitter_cron() #6 /path/includes/module.inc(794): call_user_func_array('twitter_cron', Array) #7 /path/includes/common.inc(5050): module_invoke('twitter', 'cron') #8 /path/modules/system/system.admin.inc(2301): drupal_cron_run() #9 [internal function]: system_run_cron() #10 /path/includes/menu.inc(503): call_user_func_array('system_run_cron', Array) #11 /path/index.php(21): menu_execute_active_handler() #12 {main}

Does anyone have an idea from where to begin? I am using OAuth 7.x-3.0-alpha2.

Edit: Just tested the "Post to Twitter"-function and it was posted successfully. So basically the connection is working. Somehow.

Regards,
Raphael

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

steinmb’s picture

Priority: Major » Normal

You are triggering https://dev.twitter.com/docs/rate-limiting at the Twitter API, so you should be fine after 1h. On the other hand, we should not throw exception like that when this happen. twitter.lib.php prob. need some TLC.

bugster’s picture

from twitter:

  • Unauthenticated calls are permitted 150 requests per hour. Unauthenticated calls are measured against the public facing IP of the server or device making the request.
  • OAuth calls are permitted 350 requests per hour and are measured against the oauth_token used in the request

Because the twitter module uses oauth, should it not be 350?

bugster’s picture

After rereading:

The submodules twitter_post, twitter_signin and twitter_actions do require the OAuth module

So shouldn't this module also use oauth for receiving feeds? because outside oauth the rate is measured on ip level. For servers hosting multiple instances the 150 limit is easily reached...

pheraph’s picture

steinmb and bugster, thanks for your help so far. The Twitter modules are active since the day before yesterday and not a single cron job was able to fetch the latest tweets. It appears to be related to the 150 requests per hour per IP. Since this Drupal installation is on a _big_ hoster chances are high that there are too much Drupal/Twitter-installations trying to fetch Tweets. Using OAuth for receiving feeds might be helpful, but maybe this module won't work on big hosters because of Twitter's API limitations.

bugster’s picture

If you use OAuth the requests are measured by the oauth token

OAuth calls are permitted 350 requests per hour and are measured against the oauth_token used in the request

So I think updating the Twitter Module for using OAuth in receiving tweets is not a bad idea...

BTW: I experience the same problem, webserver hosting multiple sites and is now having issues updating the feeds.

pheraph’s picture

Category: bug » feature

So, this would rather be a feature request than a bug report.

steinmb’s picture

Title: Tweets aren't imported; Cron fails with "Rate limit exceeded"-exception » Better exception handling, use OAuth call if the account is authenticated
Version: 7.x-3.0-beta3 » 7.x-3.x-dev
2bsharp’s picture

I had the same issue and was wondering if there was any work being done on this?

Due to Twitter's limits any site on a shared host will rarely (if ever) be able to update their tweets with this module. Since I assume a decent portion of sites are on shared hosts, this makes this module unusable for most people for retrieving tweets. I only cite this to suggest raising its priority.

r.aubin’s picture

Having a similar problem in Rackspace Cloud Sites environment. There's simply no way this one site is exceeding the per hour rate and we only wish to read from the Twitter feed, not post. Installed OAuth module and configured it, but am I right that the Twitter module isn't using OAuth to check for new tweets?

r.aubin’s picture

I'm using the latest stable 6.x, and have found that changing the user's account "protected" flag to 1, in the {twitter_account} table will force OAuth authentication, but unless it is actually a protected account, it will eventually change back to unprotected and stop using OAuth to pull latest tweets.

exception 'TwitterException' with message 'Rate limit exceeded. Clients may not make more than 150 requests per hour.' in /mnt/target02/350461/501661/www.thesite.org/web/content/sites/all/modules/twitter/twitter.lib.php:283 

Stack trace: 
#0 /mnt/target02/350461/501661/www.thesite.org/web/content/sites/all/modules/twitter/twitter.lib.php(225): Twitter->request('http://api.twit...', Array, 'GET') 
#1 /mnt/target02/350461/501661/www.thesite.org/web/content/sites/all/modules/twitter/twitter.lib.php(110): Twitter->call('statuses/user_t...', Array, 'GET', '0') 
#2 /mnt/target02/350461/501661/www.thesite.org/web/content/sites/all/modules/twitter/twitter.lib.php(158): Twitter->get_statuses('statuses/user_t...', Array, '0') 
#3 /mnt/target02/350461/501661/www.thesite.org/web/content/sites/all/modules/twitter/twitter.inc(102): Twitter->user_timeline('84667223', Array, '0') 
#4 /mnt/target02/350461/501661/www.thesite.org/web/content/sites/all/modules/twitter/twitter.module(140): twitter_fetch_user_timeline('84667223') 
#5 [internal function]: twitter_cron() 
#6 /mnt/target02/350461/501661/www.thesite.org/web/content/includes/module.inc(497): call_user_func_array('twitter_cron', Array) 
#7 /mnt/target02/350461/501661/www.thesite.org/web/content/includes/common.inc(2783): module_invoke_all('cron') 
#8 /mnt/target02/350461/501661/www.thesite.org/web/content/cron.php(10): drupal_cron_run() 
#9 {main}

I'm not sure why we're using the account->protected flag to decide whether or not to use OAuth in $twitter->user_timeline.

Twitter.inc (102)

$statuses = $twitter->user_timeline($account->id, $params, $account->protected);
rovo’s picture

Category: feature » bug

My Twitter Feed has stopped updating a week ago.
I'm getting the Rate Limit Error in Drupal, but when I check the Twitter API immediately after running CRON, it says I still have 119 hits remaining. Any ideas where the disconnect might be occurring? I'm just trying to receive the feed.

Drupal Error

exception 'TwitterException' with message 'Rate limit exceeded. Clients may not make more than 150 requests per hour.' in /mnt/Target01/338050/610803/dev.site.org/web/content/sites/all/modules/twitter/twitter.lib.php:284 Stack trace: #0 /mnt/Target01/338050/610803/dev.site.org/web/content/sites/all/modules/twitter/twitter.lib.php(227): Twitter->request('http://api.twit...', Array, 'GET') #1 /mnt/Target01/338050/610803/dev.site.org/web/content/sites/all/modules/twitter/twitter.lib.php(112): Twitter->call('statuses/user_t...', Array, 'GET', '0') #2 /mnt/Target01/338050/610803/dev.site.org/web/content/sites/all/modules/twitter/twitter.lib.php(160): Twitter->get_statuses('statuses/user_t...', Array, '0') #3 /mnt/Target01/338050/610803/dev.site.org/web/content/sites/all/modules/twitter/twitter.inc(125): Twitter->user_timeline('0', Array, '0') #4 /mnt/Target01/338050/610803/dev.site.org/web/content/sites/all/modules/twitter/twitter.module(143): twitter_fetch_user_timeline('0') #5 [internal function]: twitter_cron() #6 /mnt/Target01/338050/610803/dev.site.org/web/content/includes/module.inc(795): call_user_func_array('twitter_cron', Array) #7 /mnt/Target01/338050/610803/dev.site.org/web/content/includes/common.inc(5121): module_invoke('twitter', 'cron') #8 /mnt/Target01/338050/610803/dev.site.org/web/content/modules/system/system.admin.inc(2310): drupal_cron_run() #9 [internal function]: system_run_cron() #10 /mnt/Target01/338050/610803/dev.site.org/web/content/includes/menu.inc(517): call_user_func_array('system_run_cron', Array) #11 /mnt/Target01/338050/610803/dev.site.org/web/content/index.php(27): menu_execute_active_handler() #12 {main}

Twitter API message

<hash>
<reset-time-in-seconds type="integer">1331156442</reset-time-in-seconds>
<reset-time type="datetime">2012-03-07T21:40:42+00:00</reset-time>
<remaining-hits type="integer">119</remaining-hits>
<hourly-limit type="integer">150</hourly-limit>
</hash>
rovo’s picture

*UPDATE
On my local machine, I upgraded Drupal from 7.12 to 7.13-dev, and the issue seems to have gone away. The tweets are now updating.

**UPDATE
After replacing all Files and Database on the Live Server, from the known working local copy, the Twitter has again stopped updating.

Could this have anything to do with hosting on RackSpace Cloud?

steinmb’s picture

Is it still an TwitterException with message 'Rate limit exceeded. Clients may not make more than 150 requests per hour.'?

nicobo’s picture

r.aubin, you can change the line to :

$statuses = $twitter->user_timeline($account->id, $params, _twitter_use_oauth());

-> this will use oauth even if protected is 0

nicobo’s picture

Why should it be a bad idea ?
On shared hosts, it looks logical to authentify against the oauth token...

Are you afraid to reach 350/day with all your operations on a given account ?

richH’s picture

Hi,

I've installed Twitter 7.x-3.1 and OAuth 7.x-3.0 today. I also set up a new twitter account. I have created a Twitter Application and entered the keys into the Twitter/OAuth configuration fields and also the Callback URL.

I've only got a single user (admin) and I have created a twitter account for that user. In the list of twitter accounts for admin, I can see the twitter account and everything looks ok.

But when I run Cron, I also get this rate exceeded error. My site is on shared hosting. Cron only runs once an hour and there are no other Apps querying the twitter account.

The error report comes from the user "Anonymous". Is this the problem, that it isn't the user Admin which is running Cron (which would mean the request is being made by the authenticated user and is apparently limitless) but the Anon user which means the IP is used to log requests?

The error code is:


exception 'TwitterException' with message 'Rate limit exceeded. Clients may not make more than 150 requests per hour.' in /home/example/www/demo/athlete/sites/all/modules/twitter/twitter.lib.php:214 Stack trace: #0 /home/example/www/demo/athlete/sites/all/modules/twitter/twitter.lib.php(155): Twitter->request('https://api.twi...', Array, 'GET') #1 /home/example/www/demo/athlete/sites/all/modules/twitter/twitter.lib.php(59): Twitter->call('statuses/user_t...', Array, 'GET', '0') #2 /home/example/www/demo/athlete/sites/all/modules/twitter/twitter.lib.php(88): Twitter->get_statuses('statuses/user_t...', Array, '0') #3 /home/example/www/demo/athlete/sites/all/modules/twitter/twitter.inc(127): Twitter->user_timeline('745145539', Array, '0') #4 /home/example/www/demo/athlete/sites/all/modules/twitter/twitter.module(147): twitter_fetch_user_timeline('745145539') #5 [internal function]: twitter_cron() #6 /home/example/www/demo/athlete/includes/module.inc(826): call_user_func_array('twitter_cron', Array) #7 /home/example/www/demo/athlete/includes/common.inc(5150): module_invoke('twitter', 'cron') #8 /home/example/www/demo/athlete/modules/system/system.admin.inc(1624): drupal_cron_run() #9 /home/example/www/demo/athlete/includes/form.inc(1460): system_run_cron_submit(Array, Array) #10 /home/example/www/demo/athlete/includes/form.inc(859): form_execute_handlers('submit', Array, Array) #11 /home/example/www/demo/athlete/includes/form.inc(376): drupal_process_form('system_cron_set...', Array, Array) #12 /home/example/www/demo/athlete/includes/form.inc(131): drupal_build_form('system_cron_set...', Array) #13 [internal function]: drupal_get_form('system_cron_set...') #14 /home/example/www/demo/athlete/includes/menu.inc(516): call_user_func_array('drupal_get_form', Array) #15 /home/example/www/demo/athlete/index.php(21): menu_execute_active_handler() #16 {main}

Thanks
Rich

juampynr’s picture

Status: Active » Closed (works as designed)

I can do nothing regarding exceeding rate limits. However, I have just committed a fix to react on Exceptions from failed requests so they are logged and the error is returned in an alert.

As stated at #2, using OAuth to authenticate your Twitter accounts in your Drupal site will give you a higher rate ignoring the IP address.

juampynr’s picture

rovo’s picture

I confirmed through Twitter's API that I had not exceeded the rate limit, but I was receiving that error from the Twitter Module in the Drupal error logs. Do you think using OAuth to authenticate the Twitter Account will help this, even though the rate limit is not actually being exceeded?

juampynr’s picture

Yes I think so. Twitter module does not keep track of your requests, but simply receives those stats form api.twitter.com after a request.

richH’s picture

Hi,

as I wrote, I am using OAuth to register the twitter account and also adding it to Admin so that I can have the block. Maybe I'm doing something really wrong, but I followed all the instructions and have OAuth running and still get this error even though this is the only App accessing my twitter account.

So I think it is causing too many requests because the requests are coming from the IP and not from the OAuth user.

How exactly can I set this up so that I can see twitter posts for a twitter account on my Drupal website? I'm not interested in having twitter accounts allocated to users, so it's OK for me to just have a "site wide" twitter access.

Thanks
Rich

juampynr’s picture

Title: Better exception handling, use OAuth call if the account is authenticated » Use OAuth to pull tweets if the account is authenticated
Status: Closed (works as designed) » Active

Reopening. I will review the hook_cron implementation to ensure that it uses OAuth when it is enabled and the account was authenticated.

justindodge’s picture

FileSize
584 bytes

I was running into api limits on shared hosting due to the shared IP address and I needed to have an authorized request just for retrieving tweets.

Here's my patch to make that work.

hughey’s picture

Thank you!

I am having trouble getting through to get my tweets too. I only need it to pull one feed (for user 1) and cannot do so because of the error given by Twitter.

richH’s picture

Hi,

OK - got it!! The best way to put twitter onto the website is to use the pre-prepared Widgets from Twitter. Simply log onto your twitter account, goto settings and create the widget. The tweets are then pulled from the client and not the server which bypasses the 150 limit.

Paste the code into a block (using Full HTML mode) and away you go.

Probably really obvious but I gotta share the knowledge ;-)

cheers
Rich

xurizaemon’s picture

Status: Active » Needs work

This should go in. Untested, but it looks good to me - just needs true/false constants capitalised.

justindodge’s picture

Here's a fix for the mentioned coding standards.

ryan.gibson’s picture

This is adding whitespace errors still.

xurizaemon’s picture

Status: Needs work » Needs review
FileSize
565 bytes

If you're hitting API limits, you can also help by letting us know whether the functionality in Justin's patch is working as expected.

xurizaemon’s picture

Status: Needs review » Reviewed & tested by the community

Tested and works as expected on 7.x-3.x, preventing the auth errors and restoring import when API limit has been hit.

Thanks Justin.

justindodge’s picture

Thank you for cleaning up the patch!

xurizaemon’s picture

Status: Reviewed & tested by the community » Fixed

Fix tested and applied to 7.x-3.x in e348ac6 and 6.x-3.x in 45bee07.

No problem, Justin.

FYI - It's really easy to set up most text editors (see "configuring $EDITOR for Drupal" etc in that page) to do things like trim EOL whitespace and auto-expand tabs, so you can bid goodbye to the whole "ew look a tab" malarkey because it just happens when you hit save. All I did to clean it up was apply your patch, visit and save the changed files, and git diff. No thinking required :)

Dreditor is nice too because it makes whitespace issues visible when reviewing patches.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

Anonymous’s picture

Issue summary: View changes

Tested the "Post to Twitter"-function