UPDATE4:Thank-you to the 41 people and 22 organizations that signed for membership since this announcement was posted. We now have over 157 paid members and have raised 6162.01 EURO in memberships out of 7500EUR to pay for this hardware.
As part of our ongoing efforts to support the Drupal.org infrastructure, the Drupal Association has further bulked up our hardware resources with 10 GB of RAM and a new master database server. These improvements will help us deal with the exponential growth curve the Drupal community has been experiencing, and help us prepare for the anticipated growth due to Drupal 6.
The Drupal Association spent approximately $11,000 USD on hardware. To help pay for these upgrades and support the Drupal.org infrastructure, please purchase a membership (individual, organization). Your membership will help the association plan and spend money to support the Drupal.org infrastructure and Drupal events such as Drupalcons for the next 12 months.
You can learn about the evolving architecture of our infrastructure in these diagrams. You can learn about our efforts to scale the Drupal.org architecture in this presentation. You can also view a picture of the new database server.
These upgrades included 2GB RAM for each of three web servers and 4 GB for the database slave server. The new master database server is a a dual quad core Xeon processor with 16GB of RAM with four 15,000 RPM SCSI disks. We believe that this database hardware combined with new web servers should allow the Drupal.org infrastructure to double its capacity to 30 million pages a month and 50,000 unique authenticated users per month.
| Attachment | Size |
|---|---|
| drupal-org-db-master.jpg | 1.69 MB |
Comments
Great! Do you use
Great!
Do you use lightweight http-server (like nginx, tux or lighttpd) as a web-frontend?
Practice shows that such two-tier architecture (visitor --> [light httpd server --> apache] --> db) is effective.
Such servers can send static files very effictively (much better than apache taking much less memory). Also they can proxy requests to apache so the latter releases faster and don't take much memory whaiting while user is getting a page. This decreases number of apache processes and memory consumption.
Yours,
Eugene
squid
If I'm not mistaken there are squids running in front of the apaches, basically doing the same thing.
Good to hear that Drupal.org is constantly improving, but...
Good to hear that Drupal.org is constantly improving, but why the 1.69 MB picture of the drupal-org-db-master using up resources?
Bandwidth generally not the problem
PHP processing and db queries are the big bottlenecks. So enjoy the jpg =)
- Robert Douglass
-----
Lullabot | my Drupal book
Drupal.org website serves 500GB/mo not including downloads
Hello, I evaluated making a smaller picture. But since the site uses so much bandwidth I decided that the few people who really like hardware would appreciate the picture in full resolution.
Enjoy!
Cheers,
Kieran
CTO CivicSpace
Try hosted and pre-configured Drupal 5 profile
http://civicspacelabs.org/create
Kieran Lal
Thanks for pointing that out
Robert: thanks for pointing out that the bottleneck lies with processing power rather than bandwidth.
Kieran: I like the picture even though I haven't got the faintest clue what I am looking at. I am just happy that Drupal.org is running smoothly and its infrastructure is in good hands.
Interesting insight into the
Interesting insight into the hardware setup. Money well spent IMO.
------------------------------
Alex Cochrane
Spoon Media
VERY big improvements, I
VERY big improvements, I have been waiting a while for this day to come. Keep up the good work guys, sounds like a pretty sick cluster!
As some one that has
As some one that has contributed to drupal association before and am planning to contribute more in the future, i wholeheartedly say that this is a money very well spent.
Sometimes something interesting appears on http://litwol.com
Paypal
Shame you use paypal, otherwise I'd contribute :-(
If you need to setup a proper ecommerce account I can write a module for which ever provider you wish.
Cheers,
Malc
If it ain't broke, fix it!
You can send a check
Hi, the address for the association and the bank account are available on the website. You can contribute that way as well.
Can I mail you a check for a donation or a membership payment?
You can send a check to:
Drupal vzw
Langveld 2 Bus 12
2600 Berchem
Belgium
Kieran
CTO CivicSpace
Try hosted and pre-configured Drupal 5 profile
http://civicspacelabs.org/create
Kieran Lal
in the post
That's great, I do feel bad not contributing, considering how much drupal helps me every day!
Cheers,
Malc
If it ain't broke, fix it!
Cheque
When I'm not poor because I spend all my time fiddling with Drupal you can expect a cheque in the mail. ;) Agreed with Malc on Paypal being evil.
Great to hear about the improvements to the infrastructure. Things were getting a bit flaky there for a while!
----
Web Design, GNU/Linux and Drupal.
woo-hoo!
I'll show my age here by gasping at "10 GB of RAM." Wow.
I remember a conference years ago when Kai Krause was talking about how his buddy Graham Parsons (I think thats who it was, but it was some rock star) had 1 GB of RAM and everyone in the audience was awed. These days it isn't quite so shocking, but still....
Congrats on the upgrade!
Facebook uses a terrabyte of RAM
Facebook serves over 2 Billion pages a day and uses a Terrabyte of RAM to do it. They also use the same LAMP stack that Drupal uses.
Cheers,
Kieran
CTO CivicSpace
Try hosted and pre-configured Drupal 5 profile
http://civicspacelabs.org/create
Kieran Lal
Make that 2 terabytes...
Yowza.
http://blog.facebook.com/blog.php?post=2245132130
PHP5?
Since Drupal's future move to PHP5 has already been announced, I'm curious to see that the configurations mentioned all use PHP 4.4. Is that upgrade planned soon, and if so, what is implicated? More RAM? More webserver heads?
I'm not even sure if that is an infrastructure question, but if it isn't I'm not sure where it would fall. I saw it mentioned in the diagram, so I'm asking here ;)
PHP 4 is faster than PHP 5
Dries measured PHP4 and PHP5 and found that PHP4 was faster so that's why we use PHP4 on the main web heads. We do use PHP5 in some cases. We will move to PHP5, for Drupal 7 will require it, and have planned for adequate hardware improvements that may be necessary.
Cheers,
Kieran
CTO CivicSpace
Try hosted and pre-configured Drupal 5 profile
http://civicspacelabs.org/create
Kieran Lal
The architecture of Drupal
The architecture of Drupal is good enough that I don't think it is critical to move to PHP5 to get a performance penalty.
Good work
I noticed the "My Recent Posts" page loads faster than it used it. It might be a good idea to link that directly from the user menu rather than the more generic and generally less useful "Recent Posts" page.
--
John Forsythe
Need reliable Drupal hosting?
Better and better
This hardware upgrade prove that Drupal get better and better. Page load faster so we can help forum faster too. Many thanks to Drupal hardware team!
Encouraging membership fees/donations
A simple membership benefit that could encourage new members and reward commercial members would an official "DRUPAL sponsor" logo which can be placed on a members own website and linked back to their Drupal page confirming their sponsorship at stating how important membership is to continued Drupal development and maintenance (blah, blah, blah).
Drupal using businesses of various sizes may be already want to put money back into Drupal in return for the commercial benefit they have had from it but may not be allowed to, or not be able to justify, giving out free money when they don't have to. To be allowed to do the right thing, they need to be able to say to the accountant and tax-man that this is a marketing/publicty expense (unless Drupal.org is a registered charity at a national level) and therefore need to show a commercial gain from giving the money.
Either there could be two versions of the logo for "Personal" and "Organisation" membership respectively or it might be only given to a new "Sponsor" membership-level at say €150. Or possibly some permutation there of where all levels have their own type of logo but sponsorship level includes rights to various sizes & formats and rights to use in printed literature etc.
Regards
John Bryan
www.ALT2.com
Application Integration Specialists
Tel: UK 08700 3456-31
Please review the results of the survey
Hello, please review the results of the membership pricing and benefits survey.
Benefits will be added as volunteers step forward to implement them and work through the process of deploying features on Drupal.org.
Cheers,
Kieran
CTO CivicSpace
Try hosted and pre-configured Drupal 5 profile
http://civicspacelabs.org/create
Kieran Lal
Would it be possible to get
Would it be possible to get a visio-ish look at the infrastructure from load balancer (if you are using multi-headed access) through to database with specifics for things like memecache and squid?
I'd love to see how you are stacking your server.
Nevermind... I found it. :-P
It seems as really big
It seems as really big performance improvements.
Nice.
---
Drupal Theme Garden
Its good to see that this
Its good to see that this post generated so many new memberships to the association.
------------------------------
Alex Cochrane
Spoon Media