drupal.org is so slow i want to shoot myself in the head - why is that ?
flexy123 - June 20, 2007 - 15:26
i want to get modules and more....but the drupal.org site is so slow.
Are having your mysql server on another host ?
I noticed that on a testrun with drupal on a host of mine, the mysql server must be the same machine or everything is slow like molasses. Just wondering. Its not really good for promotion if the main site is so slow... ;(
Wow it really is as slow as, well, pudding
Okay, i am behind some slow server in a Mexico City office, but still, wow this is slow!
It took me 5 minutes to even open this forum to complain about it :-)
I hope they make it a bit faster soon, because I have a lot of reading to do if I want to start using Drupal for creating my websites.
Seventies WebDesign
Blog Review Blog
Hm, works for me...
Let's see how long this post takes, for me I'm seeing two to three second page load times when browsing/reading, no major problem here.
Ok, back here editing after post creation, creating this reply originally took about twenty-one seconds to store after preview.
--
Joe Kyle
--jjkd--
If I'm correct...
Drupal is hosted about 15 miles from me on the OSU campus. And it's still dog slow. They have (I believe) one or two web servers and one or two SQL servers. There's no local SQL calls being made, its all over the local OSU network to grab SQL data.
Drupal is not designed for over-the-network SQL calls, only local on-machine calls which makes things nice and speedy if your SQL server is on the same box as your HTTPD. That's why Drupal is slow on Dreamhost where the SQL servers, file servers and HTTP daemons are all run on different machines.
This isn't gospel truth so YMMV.
you are wrong.
Drupal works just fine with remote databases. Thousands of sites world wide have used this configuration successfully for many years now.
Your distance from drupal.org isn't relevant in network terms as your connection might actually take a completely different network path across the world depending on your service providers routing.
The current database Drupal.org is hosted on is getting pounded by our growing community. The number of logged on users has consistently grown for several years. Logged on users have a higher impact of the database server. A move to a different back end setup is planned but still in testing.
-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide -|- Black Mountain
So its not just me!
I'll have to admit, drupal.org has been very slow for me too, and for weeks...
Being as I'm working from the UK I haven't said anything, there are any number of reasons for slow america-england traffic (Not least the internet cops, caching and tracking away!). But one week (3 weeks ago) in particular was very bad, it took minutes to load every api page until the point at which the api sub-domain just stopped responding at all!
Perversly, today drupal.org has been like lightning!!! It seems the day you guys have got your problems is the same day my access speeds have cleared up! Perhaps they've moved the servers over here in the night (when you wern't looking??)???
Either way, can we expect performance to improve/stabilise any time soon? Or is this likely to be (as alluded to above) a server(s) and bandwidth constraint??
ZuluWarrior
First of all, that Drupal is
First of all, that Drupal is slower when the SQL server is not running on the same machine as the webserver is just nonsense. Ever thought about the fact that drupal.org has simply far too much traffic to be handled by one single server?
For more information, read http://drupal.org/node/144228.
And if you actually want to help, try testing the performance-related improvements for Drupal 6 listed in the patch spotlight.
People are talking about
People are talking about drupal.org site itself ! I confirm, very slow these last days...
I was simply putting two and
I was simply putting two and two together (Drupal.org has always been just a bit slow for me, though the steadily increasing traffic over the last 18-24 months has made things progressively worse). Naturally I realize that the Drupal admins know what they're doing, I was not implying anything.
mysql DB on another server
i had a test install on a host (NO TRAFFIC AT ALL) and the mysql db was on another host...it was SO slow with me as ONLY one on the system i *almost* dropped the whole idea of using drupal for my new site !
I then went to anothe hosting firm where i know mysql is on the same server...and the difference is like night and day !
I am not making this up, i am telling you what i experienced.
and?
You are extrapolating a conclusion based on insufficient evidence and poor test setup.
Is the first setup connecting to an over subscribed database? Is your authentication setup correctly on the database? Is the name resolution working properly? Is the network routing correct or are you traveling an incorrect path? Is the database running Raid1 or raid 5? Are the log files on a separate disk spindle? Are they RAID1/5? It makes a difference.
Is the new setup on a more powerful box? Are you the only one on it? Is apache tuned correctly for LAMP?
Any number of variables would have to be considered. I can assure you that MTV UK site does NOT run off a box with a single web server and database on the same server.
Just because you are using Drupal does not mean you can skip any of the tuning steps from the hardware configuration, the OS, Web Server config and Database tuning. To many people seem to assume that you can.
-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide -|- Black Mountain
mysql DB
>>
Just because you are using Drupal does not mean you can skip any of the tuning steps from the hardware configuration, the OS, Web Server config and Database tuning. To many people seem to assume that you can.
>>
Steven,
well i dont doubt what you say. You are certainly more involved than me who has that system for two days now :)
I dont know anything about that one hosting company's mysql server setup, or WHAT the reason was for the slowness. But see it from a users point of view. (And NOT someone into DB internals and server setup !) Yes, its and ASSUMPTION, but there is a likelihood this problem appears more often w/ other people using drupal and ANY hosting - EXCEPT we assume it is an exception only happening on THIS particular server and with THIS setup. I do not know that ! I mean, there IS a likelyhood that hosting company has some things odd....for sure.
TOO slow, means too slow...
.. no matter if mysql is in the same host or not.
Just because you are using Drupal does not mean you can skip any of the tuning steps from the hardware configuration, the OS, Web Server config and Database tuning. To many people seem to assume that you can.
Default Apache-PHP-Mysql installation is as good as any other, and if drupal needs some kind of special tweaking it should appear in the documentation. Otherwise people will get bored trying to manage a >1000 users site.
Have you tried http://drupal.org/project/Modules/name ? I can't believe this >90secs of loading time is comming from php rendering stuff, neither from themeing functions, laMp odd configuration or things like that. I can guess most of the loading time comes from the (lots of nodes) mysql query process to load all the projects, but I will never know.. however, I'm sure the guys at drupal.org should know.
Anyway, it means to me that drupal may not be so "ready to plumb hundred of users" as said, at least it's not working so well here..
By the way.. your post arguments are so.. well.. you forgot almost all these:
- Is the netwok balancer busy with traffic from other hosts?
- Is the Host's cpu overloaded with the emule..
- Is the mysql running as root?
- Is /tmp directory empty?
... so many other...
comm'n.. if Drupal is so environment dependant we should start considering the SO-HO architecture instead of multi-module init-load per_module-invoke and so many other drupal (by default) enabled code being executed per request. Anyway, 2 or 3 second of loading time is acceptable in a site like this.. more is dangerous..
sigh
Have I tried that link? Gosh I don't know. Maybe once or twice.
Feel free to download project module and take a look at the SQL query necessary to generate that page. Then count the number of modules on that page after it loads. Perhaps we should prevent people contributing their modules?
Random extrapolation and gigantic leaps to conclusion from my answer to one post that contained a conclusion that was at odds with that persons test setup. I answered specifically his extrapolation from results of a setup with to many variables to get to that conclusion. I didn't feel that a one front end web server to one back end database merited getting into the specifics of load balancers and other esoteric database details. Merely introducing the concept that there are a lot of other considerations involved before coming to the conclusion that poster did.
Any complex application, such as a database driven content management solution, should be sized to it's purpose. Do you think that MTV UK just tossed a site out there for hundreds of thousands of users without performance testing? That Sony didn't consider their traffic and support requirements when they put up a bunch of Drupal based sites?
That the default setup is sufficiently tuned that people can quite often get away with 'generic hosting provider' just fine for small sites and 'quality hosting provider' for mid sized sites pretty much out of the box. It means that when you get into the scale that sites like Drupal.org, NowPublic, KernalTrap see with users and traffic, you have to move beyond the assumptions of a web server with static html pages considerations.
Drupal.org is six years old. It's database is fairly large. It's not thousands of users, it more then 150,000 users with more then 150,000 nodes of content and more then 244,000 comments. It is running on one stand alone database server and two front end web servers. It is gaining in popularity and seeing increased traffic. It is getting more from the CNET announcement that is new traffic.
I have said in two other comments now that the database server is overloaded. This is known. A solution is being worked on. No one is happy that it is taking a while to get it worked out. It just is.
You
-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide -|- Black Mountain
clear like water..
I agree with you 100%, and I share most of your prosa.. . Thanks for your reply, now there's no way to confusion. LAMP is good, and configured is better, not for drupal, but any application.
I hope you to be lucky with the database issue, and I guess, as a community, a call for help would be assumed by most of the users, including me.
By the way: forget about trying that link.. Yes, we should close the project submissions :P
Drupal.org was never fast on me
It's always slow... that was the reason why I have hesitated to try it at first.
I think they should move their host or something.
It's kind of embarrassing.
I love drupal. I have about 5 Drupal websites.
All I'm saying is that Drupal.org is very slow.
Drupal.org is Slow here in Utah too for the past couple of days!
I can't wait until they get it resolved. It takes forever to post a comment and to pull up pages!
Aloha!
Jeff
For me, it's also very slow
I agree with some of the above posters : drupal.org is very very slow for two or three weeks... I have no problem at all with other websites, so I can imagine the problem comes from the website's running environment...
Would it be possible to have some feedback from a technical team, mainly regarding the causes (hardware ? number of visitors ? database logs not on a separate writefast disk ? number of nodes too high ?)... May be put a survey on the home page to asses the users feeling on drupal.org responsiveness ?... and split the documentation apart (almost unusable these days) ?
I'm really sorry to say that, but it makes me feel better ...
One of my morning concerns : "And how fast's gonna run drupal.org today ?"
Thanks to all the teams and communities for their job.
Regards
Until maybe the end of May,
Until maybe the end of May, Drupal.org was a pain to work with. 30-40 seconds of response time, that sort of stuff. Then, a miracle happened (described in other threads). A few very smart patches were written, and all of a sudden, those 30-40 seconds were reduced to maybe 3 or 5 seconds. I felt so happy! Finally, drupal.org was a genuine tool, a showcase for Drupal itself, the living proof that a big site with a lot of traffic could be managed very well by Drupal. It was also a proof that software remedies actually work, and that the age-old cry for "more hardware" is not necessarily the only solution. In short: drupal.org felt the way it is supposed to feel.
And then, for reasons unbeknownst to me, disaster struck again. For about a week now, those horrendously long waiting are back. And I don't have the faintest idea why...
What makes me really sad, is that I have witnessed the fact that Drupal can be fast and that drupal.org can be a nice site to work with, but that for whatever reason we're back to square one. Why?????? And will there be any way out, restoring those yummy 3-5 seconds response times?
Ludo
...
As I said above, the database server is being pounded.
Drupal.org is a 6 year old Drupal site. The database is fairly large. Software remedies such as SQL query tweaks and improvements are great and help tremendously but that still doesn't solve the communities growth over the last 18 months impact on the database server. There are some folks working on it. But that takes time and testing by a small pool of people. The DB server is CPU bound and probably Disk IO bound as well.
-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide -|- Black Mountain
I understand what you're
I understand what you're saying, Steven, and I think it must be hell to run a forever growing site like Drupal.org. Still, as I wrote, for 2-3 weeks drupal.org was running smoothly. I don't think the site grew sooo much in that time as to bring it back to its knees (and to 30-40 seconds delay) again. So, something must have been changed, some setting, whatever. I hope this can once again be unchanged, to bring the workable speeds back...
Ludo
I agree...Check out this link from two years ago!
What you've both said makes sense. AND if it worked fine for 2-3 weeks with the "patches" in place AND then started going slow again, I'm recommending they have their hosting company switch to a different dedicated server with a lot more ram. DISCLAIMER: That's my non-technical two bits.
Check out this link!: http://kairosnews.org/node/4362
***I've shown the content below***
Help drupal.org buy a dedicated server
Submitted by cel4145 on July 10, 2005 - 10:27.
The following message has been posted to drupal.org, which is still down except for the main page:
Dear friends and supporters of Drupal,
Quite a few people have pointed out that drupal.org has been slow lately. We know it's been slow, and have been working on optimizing Drupal.org; adding new features to help keep (evil) crawlers out, fine tuning MySQL and Apache, etc. The fact remains that as the result of Drupal's growing popularity, the server is saturated pretty much all day. This explains drupal.org's poor performance.
To make a long story short, our current server doesn't cut it anymore. Our unprecedented growth in traffic requires more and better hardware. To buy a new server we need your help to raise $3000 USD. Read more about the details below, or just click the Paypal donation button on the right.
Where we are now
Currently, drupal.org runs on a shared server paid for and maintained by Kjartan. The server is a single Pentium Xeon 3Ghz with 1 GB of RAM. There are about 20 sites running on the server, including some of our sites like http://drupal.org/, http://drupaldocs.org/ and http://cvs.drupal.org/. In addition to the websites, the server hosts our mailing lists, mailing list archives and CVS repositories. Last month, drupal.org alone served more than 3 million pages for 100 Gb of traffic (this does not include any of the other sites or services; non Drupal websites, Drupal mailing list traffic, etc).
What we have planned
In the few past weeks we have been talking to the Open Source Lab (OSL) at Oregon State University and they generously offered to provide free rack space, free bandwidth, free power, free backup facilities and onsite support. Scott Kveton, Associate Director of the Open Source Lab, explains:
"The OSL currently hosts several open source projects such as Mozilla, Gentoo, Debian, Freenode and the Apache Software Foundation. The hosting we do is to help facilitate projects as they grow and leverage an economy of scale by hosting them all in the same facilities. The services hosted at the OSL currently touch well over 20 million unique visitors a day and growing at a phenomenal rate.
As part of the hosting we do here, we offer other services such as DNS, database, backups, mail relay etc to the community to free up their hardware to do the "main thing" for their project. We have offered up rack space, bandwidth, power and our "smart hands" service to the Drupal project because we want to help a great project that is having a significant community meeting one of our goals; enabling communities."
In order to take advantage of this generous offer, we need to supply our own server.
What we need to get there
We would like to buy a Dell PowerEdge 1850 1U (or equivalent hardware) with two Pentium 2.8Ghz Xeon CPUs, at least 2 GB RAM and two 70+ GB SCSI disks with a RAID controller. The total cost of such hardware is approximately $3000 USD ... and this is where we need your help . It is time for us to move to a new home.
Once we have collected enough money to buy a new server, we'll get it to OSL's data center, and we'll move the Drupal sites and services from the current server to the new server. At the same time, we hope to grow our team of server administrators, as well as extend the services we offer to the community. Things we plan to provide include a subversion mirror, an infrastructure for nightly tests, and so on.
How we are doing this
As many of you know, Drupal does not currently have a non-profit or foundation status. We are working on this and discussing with other large Open Source projects how they have handled it themselves. This will help in determining what will be best for us. No matter what we decide, filling out forms and filing paperwork will take time and money. Time we don't have.
Currently all funds are held by Dries so the equipment purchased will also be owned by Dries until we have an organization in place that can own equipment. The equipment and funds will then be transfered over. While we have some funds in the bank that have been used to pay for some marketing materials, we are hoping to use the majority of those funds we currently have to set up the foundation. In any case, it is not enough to purchase a server, which we need now. Until then, donations are not tax-deductable.
What you can do to help
Donate now! If you have ever considered donating, now is the time. Your contributions to a new dedicated server will help us lay a new foundation for current improvements and future expansion. It will gain us increased infrastructure support and monitoring. The opportunity to leverage OSL's services is just to good to pass up.
Your donation will be appreciated by the community and you will have everyones thanks for helping to keep us going.
tags:
* drupal
* cel4145's blog
* Login or register to post comments
Comment viewing options
Select your preferred way to display the comments and click "Save settings" to activate your changes.
cel4145's picture
16 hours, almost $6500
Submitted by cel4145 on July 10, 2005 - 19:24.
The Drupal community came out with strong support, donating almost $6500 (over double the target) in 16 hours. The note at drupal.org has been updated to say that the additional funds will be put to use for a better server (or two servers) and/or getting the not-for-profit foundation established.
Also, Slashdot covered the call for donations.
* Login or register to post comments
cel4145's picture
Donation total: $10,000+
Submitted by cel4145 on July 13, 2005 - 09:13.
Drupal tripled it's donation goal, receiving over $10,000. It will soon be moving to new hosting space on new hardware at OSL.
Aloha!
Jeff
drupal.org
I want to echo the fact that drupal.org is very slow for me to . I am in USA and using cable broadband. I used to think its my wireless, but its slow from home or office. I just want to report this as an issue so that someone can check if there is a potential issue.
The slow loading is not consistent, meaning at times pages do load fast, but most times its slow.
______________________________________
Drupal Tips, how-to, Themes @ http://mydrupal.com
Discover Best IT Content @ http://itdiscover.com
umm... uh...
see here? http://drupal.org/node/153288#comment-244443
The database server is being pounded. Slammed, over worked.... This has happens periodically during the communities growth. Generally folks work on a solution but those solutions often take time and time is something that is required from multiple specific people at the same time.
Where you are has very little to do with it. When you are does have some. During US Daylight hours the load goes up significantly as logged on users rise. Logged on users clicks are direct, live queries on the database server. So;
more popular == more logged on users == database performance impact.
As the overall user load goes down (people log off) then the database has less load and performance goes up.
-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide -|- Black Mountain
I could be totally wrong
I could be totally wrong about this, but wouldn't it help reduce the load if we changed a couple of the default views on the site? For example, the Downloads > Modules > Browse by name currently defaults to
<all>versions, and returns every single module that has ever been created (that is, if you stick around the 2 minutes it takes to return this page).I don't know what kind of tricked out caching is being used on drupal.org, and where, but this seems like a pretty expensive request and this has got to be one of the more popular pages on the site. Wouldn't it help to remove
<all>as the default, if not altogether, and return these as paged results?-zach
--
harkey design
I'll be happier if drupal manages to improve its speed....
yes Drupal org has become too slow.....
I joined 5 weeks 6 days before....my userid was 143297
this is the very new user to drupal at this moment....http://drupal.org/user/156200
so the new users for the past 5weeks and 6 days is 12,903....
I'm proud to be a drupaller and happy for drupal's fame...
I'll be happier if drupal manages to improve its speed....
I definitely agree with the
I definitely agree with the slowness. I am in Ohio on a very fast fiber connection and it just crawls. I am in the web hosting business myself and I must admit that Drupal.org would be a tough site to host. In order to speed the site up, you will need a nice cluster of servers with load balancing equipment. In all honesty, since the site has so many international visitors, it would be highly beneficial to have redundant clusters at several data centers. While I can write up this recipe to fix the slowness in a single paragraph, the amount of money required for such a design would be extremely hefty, especially for an open source project.
My suggestion would be to try to get a connection with someone at Google and see if Google would help out with hosting. Google is a huge supporter of open source software, so I would think that it would feasible.
Something surely needs to be done though. Drupal.org is my primary source of information. It is like a manual to me, along with almost every Drupal user. If the site is consistently slow, it can be extremely frustrating. The reputation of the software suffers as well. It is almost comical to me that Drupal advertises their unique caching system to improve speed when their own site is a turtle.
I don't know much about site
I don't know much about site hosting. But after months and months of irritation over drupal.org's slowness (say: 30-40 seconds in between page clicks), it felt as a Huge relief to experience a couple of weeks of acceptable response (say: 3-4 seconds). There is a thread in which the nifty tweaking of the software leading to this dramatic speed increase is documented. This happened about one month ago, maybe from May 15 till the beginning of June or so (I'm not sure about the dates).
Anyway, my point is that drupal.org Has shown that it can have speedy response times, without costly hardware changes.
There may have been an increase in traffic in the beginning or middle of June, as is explained elsewhere, but I fail to see how this increase could lead to the disastrous result we see now, i.e. we're back to the deplorable 30-40 seconds response time.
What I'm hoping for, is that the wizards who made the first speed increase happen, would again find the time to tweak the software, without having to resort to exotic solutions like redundant clusters at several data centers or without having to depend on Google charity ("Proud to announce the birth of Goopal..." :-). Mind you: if that is the only way out, fine with me. Anything goes, as long as Drupal.org can once again become a usable tool.
Ludo
slow fast slow fast
I dunno, but sofar today's been a breezy experience with sub 5 second responses from Drupal.org. Interesting development taking into account the horrendous response times seen (again) lately. Troubleshooting performance problems usually is more art than science though.
One tidbit of information I picked up at a recent meeting is that OSUOSL has received a couple of heavy duty servers donated by SUN to Drupal that need to be installed and configured to take on database duties. No idea when that's planned though.