I was slashdotted and dugg (and didn't go down)

Posted by fintler@wii-lin... on October 10, 2006 at 11:35am

I'd just like to say that caching support kicks ass. My site (http://www.wii-linux.com/) was slashdotted and dugg at the same time yesterday, peaking at around 18,000 pageviews. The server is a dedicated single processor xeon running apache2/mysql on ubuntu breezy badger server with no special tweaks for speed. I had the cache set to 15mins, and throttled some of the more processor intensive stuff and there was only a slight slowdown in usability overall. If anyone is questioning the efficiency of Drupal's caching support, I'd just like to say it works great. :)

Comments

Server

Posted by joshua_cohen on October 10, 2006 at 2:28pm

If you don't mind me asking, what kind of line are you running the server on to handle all that traffic? From your description I'm guessing you're running your server from home. I'm thinking about doing the same rather than using a hosting company.

Regards,

Josh

It's on a dedicated server

Posted by fintler@wii-lin... on October 10, 2006 at 11:01pm

It's on a dedicated server hosted at netsonic on multiple backbones. I'm pretty sure my cable company would shoot me if I tried this at home :)

Colocate?

Posted by joshua_cohen on October 11, 2006 at 1:32pm

I just took a look at Netsonic's dedicated hosting options and I don't see a dedicated server offering running Ubuntu as you mentioned. Are you colocating a server with them or was this just an option not shown on their website? Their services look fairly good, but I would like to utilize Apache 2 as well as PHP 5 and it appears like their standard offerings only include Apache 1.3.33 and MySQL 4. Granted with root access upgrading shouldn't be an issue.

Thanks again for the info, it's all very helpful!

Josh

Hmm, it seems like they

Posted by fintler@wii-lin... on October 17, 2006 at 1:10pm

Hmm, it seems like they removed the ubuntu option. We got a dedicated running ubuntu breezy badger about 350 days ago for $100 a month. I guess they changed the offering. If you were to contact them I'm sure they'd work with you if you really wanted ubuntu installed. Their staff has no problems making exceptions for just about anything.

The server originally came with apache 1, php4 and some management utility called ispconfig. We removed everything and installed apache2/php5/fcgi/ruby using apt and a few manual tweaks. It was basically a fresh install of ubuntu with a few popular packages and ispconfig installed.

More specifics

Posted by kbahey on October 10, 2006 at 3:12pm

Good job there. Thanks for sharing ...

Can you give us more specifics?

Things like, Drupal version (I am assuming 4.7, but ...), list of modules enabled, any custom code in themes, PHP version and type (FCGI or mod_php), memory size of server.

How many Apache processes do you allow (from apache2.conf).

Also, did you run any monitoring tools while the pounding was active? Like vmstat or top? Would be interesting to see the utilization.
--
Drupal development and customization: 2bits.com
Personal: Baheyeldin.com

--
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc.
Personal blog: Baheyeldin.com.

Ok, the modules installled

Posted by fintler@wii-lin... on October 10, 2006 at 11:16pm

Ok, the modules installled and not throttled were:

captcha, codefilter, gsitemap, locale, menu, page, path, pathauto, print, profile, quote, statistics, story, taxonomy, and tracker

Throttled modules were:

aggregator, db_maintenance, comment, drupal, extendeddescription, forum, help, image, image_attach, image_gallery, ping, poll, smileys, syndication, textimage, upload, and upload_image

This is drupal 4.7, no custom code, standard foliage theme. (This was an unexpected slashdotting). Cache time was set to 15 mins.

PHP 5.0.5-2ubuntu1 (cli) (built: Oct 8 2005 23:17:38)
Copyright (c) 1997-2004 The PHP Group
Zend Engine v2.0.5, Copyright (c) 1998-2004 Zend Technologies

Server version: Apache/2.0.54
Server built: Oct 4 2005 07:50:10
Server's Module Magic Number: 20020903:9
Architecture: 32-bit
Server compiled with....
-D APACHE_MPM_DIR="server/mpm/prefork"
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_SYSVSEM_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D HTTPD_ROOT=""
-D SUEXEC_BIN="/usr/lib/apache2/suexec2"
-D DEFAULT_PIDLOG="/var/run/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_LOCKFILE="/var/run/accept.lock"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="/etc/apache2/mime.types"
-D SERVER_CONFIG_FILE="/etc/apache2/apache2.conf"

Linux salinger.firepair.com 2.6.12-8-386 #1 Tue Aug 30 22:41:30 BST 2005 i686 GNU/Linux
19:09:49 up 301 days, 3:22, 1 user, load average: 0.23, 0.42, 0.44

From apache conf:

StartServers 10
MinSpareServers 10
MaxSpareServers 20
MaxClients 40
MaxRequestsPerChild 100

Apache modules:

Compiled in modules:
core.c
mod_access.c
mod_auth.c
mod_log_config.c
mod_logio.c
mod_env.c
mod_setenvif.c
prefork.c
http_core.c
mod_mime.c
mod_status.c
mod_autoindex.c
mod_negotiation.c
mod_dir.c
mod_alias.c
mod_so.c

And I was wrong, the other dedicated is a xeon, this one is a...

vendor_id : AuthenticAMD
cpu family : 15
model : 44
model name : AMD Sempron(tm) Processor 3000+
cpu MHz : 1800.536
cache size : 128 KB

and from proc/meminfo:
MemTotal: 240736 kB

We were running bmon and top for a bit but didn't record anything. Load averages were up in the 30 range when the story was on the bottom of slashdot. Other than running slow, I don't have any load averages for when it was first hit.

What are your hardware specs

Posted by 3dsoft on October 11, 2006 at 8:58am

What are your hardware specs (ram, processor, harddrive...)?

Edit: thanks... seems like you were faster.

horse ranch
-----------------
www.joelle.de

Thanks so much for sharing.

Posted by DanielTheViking on October 10, 2006 at 9:45pm

Very interesting, thanks for sharing.
Would become even greater hero if you would provide some meat on kbahey's questions :-)

Btw, we should provide a specific page with collection of "best links" to such user case posts on a separate page somewhere in the handbook, is there one already which I haven't seen?

Such a page should be cross-linked with / available from these pages:

System requirements: http://drupal.org/requirements
Server performance tuning: http://drupal.org/node/2601
Perfornance case studies: http://drupal.org/node/44740

host

Posted by mwu on October 10, 2006 at 10:52pm

Hi, if you are using a hosting service, would you mind sharing that please?

arg...double post

Posted by fintler@wii-lin... on October 10, 2006 at 11:18pm

arg....double post

wait a second

Posted by mwu on October 10, 2006 at 11:48pm

according to the timestamps, you wrote which host you are using AFTER I posted my question

I thought Joshua's question was asking you about what kind of server you are using. He thought you were hosting it at home. So I didn't think of it as a double post.

I appreciate you answering all the questions above so patiently. I had not meant to double post.

Sorry for the confusion. I

Posted by fintler@wii-lin... on October 11, 2006 at 12:12am

Sorry for the confusion. I meant that I double posted. I accidently submitted the specs twice O:-)

Anyway....the hosting provider is netsonic (http://www.netsonic.net/)

thank you

Posted by mwu on October 11, 2006 at 3:18pm

thank you very much for clarifying

Digg is hitting me now. I

Posted by carlmcdade on October 16, 2006 at 4:47pm

Digg is hitting me now. I will post some info later on the web host and code used. I am running a hybrid of the 4.6 caching fix. So far over 18,000 page views with bursts of 100-200 in seconds. This is interseting because I always thought my site would go down under such pressure.

Hiveminds Magazine
http://www.hiveminds.co.uk
for web publishers and community builders

Drupal Web Developer - Stockholm Sweden
Drupal Sweden

Please post the traffic

Posted by FiReaNG3L on October 18, 2006 at 5:12am

Please post the traffic behavior of Diggers if you can. I stopped 'advertising' on Digg even if I was kinda successful (one story a month on average on the front page). The reason? You get hit by thousands of people who :

a-) Don't check more than 1 page
b-) Don't come back
c-) Don't post links to your web page
d-) Don't click ads
e-) Don't comment on the article

So yeah, tons of traffics, but its useless any way you look at it. It's just a waste of bandwith at this point, but still, it's a great feeling when your website if visited by masses :(

---
e! Science News

You know your stuff

Posted by carlmcdade on October 18, 2006 at 10:52am

a-) Don't check more than 1 page /// Google analytics says you are right
b-) Don't come back /// I recieved only 5 more members as a result so again you are right
c-) Don't post links to your web page /// I was somewhat satisfied here. I recieved about ten links that I could find. Not a lot but it was pleasing to see.
d-) Don't click ads /// Right again. I recieved not one more click than I would on a normal day or week.
e-) Don't comment on the article /// Here I was amazed. I made a slight change in my template to try and pull comments. I recieved about 60+ and even got Rick Klau (Feedburner.com) to post once.

In total the tusnami had 30,000 with the second day being 10% of the first. Your point about the good feeling is right also. It was nice but I don't think I want it to happen more than once a month because of potential for disaster without gain.

Hiveminds Magazine
http://www.hiveminds.co.uk
for web publishers and community builders

Drupal Web Developer - Stockholm Sweden
Drupal Sweden

I had a digg 'attack' a year or so ago...

Posted by mcurry on February 6, 2007 at 4:08am

... on my old, crufty, non-drupal corporate site - someone had written about a free download on my site, and whoosh! I thought my site was under a DDOS attack, because I was getting admin notifications about db errors & other messages, and the site was sooo slow that I was tempted to take it offline. Then someone sent me an email with a link to the digg article.

It's a mixed blessing, for sure - lots of attention, all at once.

Anyway, traffic was up 100x, google ad revenue jumped to about 50x normal. (For a day, anyway - tapered off pretty quickly afterwards.) I can't comment on length of visit or depth (I'd have to go back and check out my old logs, but I think the average was three or four pages visited per user that day.)

I did get a bunch of new inbound links, from other sources (mainly bloggers and utility hounds like lifehacker, etc.). Google for 'whyreboot' and see what I mean. Prior to being dugg, google knew about 10 links for WhyReboot; after: >12,000. :D (WhyReboot had been around for years with little notice, it was one of those silly little utilities I whipped up on a whim...)

This is one of the reasons I decided to begin working with a real CMS, and settled on Drupal - my hand-wrought, lame, IIS/ASP-based 9-year-old site just can't handle this kind of stress, and it's not worth rewriting. I've moved WhyReboot and most of my other content to my new Drupal site at http://exodusdev.com, and things are going very smoothly.

Thanks for the great info!

Saw That

Posted by techsoldaten on October 18, 2006 at 1:46pm

I saw that site yesterday and said to myself, 'That guy has to be using caching'.

Good work!

Interesting

Posted by ruturajv on October 19, 2006 at 5:12am

Thats really good news!
By the way did you have URL Rewriting enabled (as in did you name your URLs, cause they are loaded into memory..) ?

Find me @ http://www.ruturaj.net/