RSS feeds of Releases (/node/X/release) are bringing the site down. They are currently blocked.

E.g.: https://drupal.org/node/3060/release
https://drupal.org/node/3060/release/feed

~ $ curl -I https://drupal.org/node/3060/release/feed
HTTP/1.1 503 Service Unavailable
Server: nginx/1.4.3
Date: Wed, 06 Nov 2013 20:23:47 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 211
Connection: keep-alive
Retry-After: 0
X-Varnish: 703585667
Age: 0
Via: 1.1 varnish
X-Cache-Svr: www6.drupal.org
X-Cache: MISS

Comments

eliza411’s picture

I'm seeing this message now: Our servers are currently a bit busy, please try refreshing the page after a few seconds.
Thanks!

I'll see what I can find for a parent issue.

eporama’s picture

So I got access to the devwww server but none of the existing development environments seem to have the same problem. This is a 503 error, so if it doesn't happen on devwww we might need to get someone to look at logs on the actual production web servers.

Is it worth the effort for me to request a separate development environment ala https://drupal.org/node/1018084 point #2? I would be glad to troubleshoot this, but would think that it should be present in the latest updated devwww environment, but it isn't.

tvn’s picture

Issue tags: +Drupal.org 7.1

Infrastructure team might have blocked release feeds because of performance issues. I am checking with them now.
In the mean time feel free to choose any other issues with Drupal.org 7.1 tag to work on :) https://drupal.org/project/issues/search?text=&projects=&assigned=&submi...

nnewton’s picture

Correct. I currently have release feeds blocked globally as they were taking down the site. This need to be addressed.

-N

DamienMcKenna’s picture

Any idea why they were taking down the site?

DamienMcKenna’s picture

tvn’s picture

Status: Active » Postponed

nnewton, can you give more information? Why they were taking the site down? How can we fix it?

tvn’s picture

Project: [Archive] Drupal.org D7 upgrade QA » Drupal.org infrastructure
Component: Code » Other
Assigned: Unassigned » nnewton
ricardoamaro’s picture

I saw that most of the servers are blocking it still but for some reason www5 is not:

ricardo@ricardo-VPC:~$ curl -I https://drupal.org/node/3060/release/feed
HTTP/1.1 503 Service Unavailable
Server: nginx/1.4.7
Date: Fri, 28 Mar 2014 13:50:29 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 211
Connection: keep-alive
Retry-After: 0
X-Varnish: 771684814
Age: 0
Via: 1.1 varnish
X-Cache-Svr: www1.drupal.org
X-Cache: MISS

ricardo@ricardo-VPC:~$ curl -I https://drupal.org/node/3060/release/feed
HTTP/1.1 503 Service Unavailable
Server: nginx/1.4.7
Date: Fri, 28 Mar 2014 13:50:33 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 211
Connection: keep-alive
Retry-After: 0
X-Varnish: 1248611135
Age: 0
Via: 1.1 varnish
X-Cache-Svr: www7.drupal.org
X-Cache: MISS

ricardo@ricardo-VPC:~$ curl -I https://drupal.org/node/3060/release/feed
HTTP/1.1 200 OK
Server: nginx/1.4.7
Date: Fri, 28 Mar 2014 13:50:52 GMT
Content-Type: application/rss+xml; charset=utf-8
Content-Length: 338801
Connection: keep-alive
X-Powered-By: PHP/5.3.28
X-Drupal-Cache: MISS
Etag: "1396012027-0"
Content-Language: en
Cache-Control: public, max-age=0
Last-Modified: Fri, 28 Mar 2014 13:07:07 +0000
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Vary: Cookie,Accept-Encoding
X-Varnish: 1217607366 1217469374
Age: 2624
Via: 1.1 varnish
X-Cache-Svr: www5.drupal.org
X-Cache: HIT
X-Cache-Hits: 12
Front-End-Https: on

so I would advise for us to move the configurations from CFEngine to puppet, so every Infra Sysadmin can change them and update them across servers

This is the actual code that's blocking the feeds:

default.vcl:

     if (req.url ~ ".*/node/[0-9]+/release/feed") {                                                                                                            
       error 503;                                                                                                                                             
    }     
tvn’s picture

Title: Release Feed causing 503 » Various RSS feeds on Drupal.org are blocked
Issue summary: View changes
tvn’s picture

Title: Various RSS feeds on Drupal.org are blocked » RSS feeds bringing Drupal.org down
Issue summary: View changes

Updating issue summary.

tvn’s picture

While we're figuring out how to fix this issue - is there any possibility to provide a better message on this blocked feeds? Currently it says:

Our servers are currently a bit busy, please try refreshing the page after a few seconds.
Thanks!

Which is not really a good idea, considering refreshing won't help.

drumm’s picture

Status: Postponed » Postponed (maintainer needs more info)

I'm not sure what the root cause is here. These pages are another Display of https://drupal.org/node/3060/release, which is doing okay, as far as I know. We may be able to turn these back on, any pay attention to the DB and logs.

ricardoamaro’s picture

@drumm you mean someone can remove this code from varnish?

   if (req.url ~ ".*/node/[0-9]+/release/feed") {
       error 503;
    }

or is that still a problem?

drumm’s picture

Yes, I think that should be removed. (Or we need to know what is problematic about those pages.)

ricardoamaro’s picture

@drumm where can i change those vlcs?

ricardoamaro’s picture

$ git commit -am"drupal.org/node/2129427 - RSS feeds"
[master 5c3f7f2] drupal.org/node/2129427 - RSS feeds
 1 file changed, 4 deletions(-)
$ git push
Counting objects: 15, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (7/7), done.
Writing objects: 100% (8/8), 656 bytes | 0 bytes/s, done.
Total 8 (delta 5), reused 0 (delta 0)
To git@bitbucket.org:drupalorg-infrastructure/puppet.git
   aa15496..5c3f7f2  master -> master
ricardoamaro’s picture

this is back to normal now

while true; do curl -sI https://drupal.org/node/3060/release/feed  | egrep "HTTP|Cache-Svr"; sleep 1; done
HTTP/1.1 200 OK
X-Cache-Svr: www5.drupal.org
HTTP/1.1 200 OK
X-Cache-Svr: www6.drupal.org
HTTP/1.1 200 OK
X-Cache-Svr: www6.drupal.org
HTTP/1.1 200 OK
X-Cache-Svr: www5.drupal.org
HTTP/1.1 200 OK
X-Cache-Svr: www7.drupal.org
HTTP/1.1 200 OK
X-Cache-Svr: www2.drupal.org
HTTP/1.1 200 OK
X-Cache-Svr: www7.drupal.org
HTTP/1.1 200 OK
X-Cache-Svr: www2.drupal.org
HTTP/1.1 200 OK
X-Cache-Svr: www7.drupal.org
basic’s picture

Status: Postponed (maintainer needs more info) » Fixed

Thanks Ricardo

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.