How to improve overall Drupal site load time?
I have some concerns about the overall load time of my Drupal site (http://www.hollywoodchicago.com). I know a lot of factors could be contributing to a slight slowdown. It could either be related to Drupal or my somewhat undesirable shared hosting from GoDaddy.
It seems like the site takes longer to "think" upon initial load of a page that it should and there's room for improvement here. We're talking in the seconds, but still it seems like it can definitely be loading faster than it does. I have already had "normal" page caching (with no minimum cache lifetime) enabled along with the aggregating and compressing of CSS files (in /admin/settings/performance).
What are some things I can look at or do to improve load time?
This test shows you the load time of HollywoodChicago.com along with some past tests for the site down below on that page. In terms of load times, it looks like these are some of the slowest-loading items:
.../sites/default/themes/hollchi1_hgbm/backgrounds/bg-hg-wrapper.gif
.../sites/default/themes/hollchi1_hgbm/backgrounds/bg-hg-left.jpg
.../sites/default/themes/hollchi1_hgbm/backgrounds/bg-hg-right.jpg
.../sites/default/themes/hollchi1_hgbm/backgrounds/bg-hg-center.jpg
.../sites/default/themes/hollchi1_hgbm/backgrounds/bg-body.jpg
.../sites/default/themes/hollchi1_hgbm/backgrounds/bg-hg-header.jpg
=-=
site seems to load quickly for me, albeit I'm on cable.
How many querries are being called on the pages you deem slow?
Just made some improvements...
Just made some improvements. See this new test. I've reduced the sizes of some of the images listed in my first post without hurting quality too much so the total file size load for the site is less.
As for your question, see this link. Do you see anything out of the ordinary there that can be improved? Do you think this is the way to go to improve load time in general or should I be looking at something else entirely?
By the way, load time of the main page itself appears to be faster than the load time of inner pages (i.e. here and here).
Home page: total page load
Home page: total page load about 18 or 19 secs (measured using Firebug). Time to generate the basic page approx 1 sec (reasonable, but somtimes a few secs on other pages); however it's about 10 secs before enough assets have downloaded to enable the browser to render it.
Have you enabled the page cache (normal mode) at admin -> settings -> performance?
I wouldn't expect godaddy to be super-fast TBH but to get a site as "heavy" as yours to load quickly you may need to spend more on your hosting.
gpk
----
www.alexoria.co.uk
Drupal vs. hosting
Yes, I do and have had cache in normal mode enabled.
So for further load time improvement, do you think this is a matter of upgrading beyond GoDaddy shared hosting into a virtual dedicated or dedicated environment or is there anything else I can do on the Drupal end before going that route?
It just seems like there's a short "waiting" period to load internal pages and I don't know why. I can't tell if that's a hosting thing or a Drupal thing.
I just tested a bare bones
I just tested a bare bones site on a VPS with a fraction of the graphics that you have on your site and it came in at 2.6 seconds. It seems like your site is loading fine.
----------------------------------------------------------------------
http://classicvinyl.biz
http://music.bwv810.com
http://davidhertzberg.com
http://association.drupal.org/user/1207
I am a writer, researcher and solo drupal freelancer
Graphics
Yeah, my site is a graphics-heavy site considering its an arts and entertainment site. So is your feeling that the load time is within range for what it is?
Yes, absolutely. Only light
Yes, absolutely. Only light travels faster. In any event, I think that studies have shown that users will generally wait a maximum of what,, about 8 seconds, for a page to load before they depart. This is, of course, a function in large measure of the desire for instant gratification, which is to some degree borne out of excessive human engagement with things like video games and internet coding. The next generation of internet users will expect a faster load time, perhaps 4 seconds or so, so you are safe for the time being. ;-)
Having said that, it could be that your shared hosting is working in your favor, at the moment. This could change without you being aware of it.
A VPS is really the way to go if you are expecting a lot of traffic and can afford the extra cost. As far as ecstasy in the server world goes, there is really nothing like being able to set your own php memory to 300mb.
----------------------------------------------------------------------
http://classicvinyl.biz
http://music.bwv810.com
http://davidhertzberg.com
http://association.drupal.org/user/1207
I am a writer, researcher and solo drupal freelancer
You should really be looking
You should really be looking at the 2nd possibility that you mentioned and that is GoDaddy.
I just helped someone move from their host (they had load times of 11sec).
I helped them move to eapps.com vps. Load times dropped to 4seconds. Alot of this has to do with amount of resources available to php. Just like software on your computer the more ram available the faster it is able to process items for display. The same holds true for web apps. Godaddy is an absolutely terrible host, and the forums are littered with complaints of speed issues, memory issues, etc. This is what you get for $5.00 hosting.
If you don't know much about setting up servers and such, I suggest going with a managed vps, they are just as easy as a shared host, if they have a control panel. I was recommended to eapps from a couple dev's as a starting point, since then I've helped 3 people move to it, none will turn back I assure you that. 1 from GoDaddy, 1 from site5, and one from another shared host. I myself moved from Hostgator.
Once you've moved you open up a huge variety of options you can utilize to speed up your site. Like increased ram (288MB vs 64MB on godaddy). Complete control over apache, mysql, etc settings, so you can set cache's and optimize however you like.
You can also use php accelerators like memcache and eaccelerator which will help increase load times tremendously.
My hosting plan is $20, but they have ones as low as $10.00. Remember the golden rule you get what you pay for.
Think, you got free software running on the cheapest server space on the planet (next to free). how well do you expect it to run?
I Second This
I second this suggestion. I also noticed slow load times on my Drupal sites on low budget shared hosts. I switched to a hosting provider which has separate servers for the database and the site, and it is just as fast as a VPS and less expensive. (Cartika).
Very helpful
This is a very, very helpful post. My site will be increasing its traffic by leaps and bounds over the next several months. I've known all along that it was only a matter of time until I'd outgrow GoDaddy shared. It's getting very close to that point. Thank you for the hosting suggestion as well.
Database optimization?
HollywoodChicago.com (HC) was just having load problems again. I called GoDaddy again. This time, though, they told me something new. They said everything looked good on their end, but they believe the live HC site is being slowed down by the way it's pulling data from the database. They're saying it's being done in a "tedious" way and it needs "optimization".
What's interesting about this is for the first time in a long time I tested the load times on HC's development server. It's a replica of the live site but with much less data. The dev site loads blazing fast. I'm not used to HC loading that fast. The dev site has much less data in its separate database.
Even switching to another hosting provider may not fix the load times on the live site. This may call for hiring someone to do "database optimization". Any thoughts on this? Does this sound like something valid? Does anyone have any information on how I can "optimize my database" or might I need to hire someone to do this?
"Optimize table" function
I had hundreds of megabytes worth of "overhead" on both my live site and my dev site. I just used the "optimize table" function in phpMyAdmin on all tables with overhead on both my live site and my dev site.
My dev site is still running blazing fast and my live site is still slow. :-(
If you've already optimized
If you've already optimized your site, and your load times are slow, then it's the shared hosting. Most shared hosts are slow! It's a fact of life.
You could switch to a dedicated plan, or you could switch to a more expensive host with better hardware. I did this recently, and I have noticed a significant increase in speed. I am using Fused Network - excellent service, great hardware. I fully recommend them. But you pay for what you get - so in this case you pay a lot.
But slow live site vs. fast dev site?
But how do we explain and much faster dev site as compared to a much slower live site while on the same host?
I just optimized all database tables with overhead (and there was several hundred megabytes worth for both the live and dev database; I don't know whether or not that's a lot) and it didn't seem to make any difference.
I now wonder whether it's indeed the shared hosting (which I still can't explain the live vs. dev situation) or if it's something on the live site that's not on the dev site (i.e. certain modules, scripts, etc.; there are many changes made to the live site and modules installed there that haven't been made to the dev site).
You already said yourself
You already said yourself that there is much less data on the dev site. That's like asking why your fresh install is faster then your install with 50 modules. The bigger the database the longer it takes for each query to retrieve it's results. More data = more rows, longer searches. Anyone that is scared to move to VPS due to costs, need NOT ignore the fact they start as low as $10.00 already surpassing the quality of shared hosts. People have a COMMON MISCONCEPTION that vps are really expensive, so they avoid them without research. Simple fact is if your site is not a low traffic static site, don't use shared hosting. Your site's quality and reliability are at the mercy of your host. It's amazing how we can have 20+ of these performance threads each with atleast 5 people saying moving hosts has improved things DRASTICALLY (that's over 100 people), yet people still avoid them out of fear?
Anyways, when it comes to database optimization there's a few different things you can do. Most of those things are locked out on shared hosts.
A professional company will laugh if you ask them to tune your site on a shared host, as they have no power.
There's several things you can do on the database level are index large tables that don't change frequently
Increase or enable MySql cache
Increase max simultaneous connections
Only the 1st one is possible on a shared host.
Also other things are php caching (ALL major sites use some sort... memcache, apc, eaccelerator, etc). Can't be used on shared hosts.
Dedicate more memory (max of 64mb on most shared hosts, and that's SHARED, meaning you only get it if it's available).
VPS = Dedicated ram, normally with extra burst (shared ram). Example mines 288 Guaranteed.
Other things to keep in mind that many shared hosts (including godaddy), throttle bandwidth.
There are about 100 performance related articles with benchmarks, tips, do's and don't s. At 2bits.com I suggest you read them if you plan on running any sort of successful site. But nothing is free, and most things aren't cheap.
I've said it before: Running free software (but enterprise level) on the cheapest host in existence, what do you expect?
If you move to vps, you can add ram and resources for pennies a month as you need it. All I can say is that if you're on a shared host and your site gets dugg, expect it to go down and you to owe LARGE to your shared host.
I suggest using devel + the
I suggest using devel + the net tab on firebug to find out what's taking the longest. Surfing your site watching load times with firebug, average was easily 20 seconds + although most of the page could be seen before that, sometimes feedburner NEVER full connected or something, leaving the loading bar incomplete for several minutes.
There is an article on 2bits about 3rd party services slowing down your site as you wait for them to complete a request. You may want to look into that. But 1.5sec to load a 110K image makes me believe it's more then just your database hits.
(BTW when I called my original shared host (hostgator, who has MUCH better reviews then godaddy), they said the same thing to me. The database calls are outrageous (related to cache tables like menu_cache), and you should talk to the devs of your software. So I switched to a low cost vps, and well everythings good now.
Thanks, everyone
Thanks, everyone. It sounds like we can talk about this until we're blue in the face, but what it comes down to is that I'll get more performance if I leave shared GoDaddy hosting.
Just had a look at your site
Just had a look at your site and noticed you have the random image block activated. As I understand it that means the pages NEVER get cached. Obviously very bad for load times.
I might be missing something or this might be my first ever useful post!
A good theory...
Certainly a good theory. See, it could be a million things (hosting, modules, scripts, etc.). The only thing I'm hearing consensus on is that upgrading to a VPS and getting away from GoDaddy shared hosting will most certainly do good things for performance.
Per your theory, though, I did just disable that random image block. The site's certainly not running any slower after that, but I'm not sure it's really running any faster now either. Do you notice any difference?
CAPTCHA can impact performance, caching
In doing some more research, it appears that the CAPTCHA module can impact performance and caching. The thing is I have the CAPTCHA module installed on my live site (which is slower) and my dev site (which is much faster).
The difference is I had the comment forms with the CAPTCHA appearing on every page rather than separate pages, but changing that to separate pages on my live site still doesn't seem to help things.
The thing I like about the CAPTCHA theory being the culprit is that the pages on my live site without a CAPTCHA appear to load faster (i.e. the main page) than pages with the CAPTCHA (i.e. an internal news story page). This text about CAPTCHA conflicting with caching and performance is noted here:
I also just made this code change to the CAPTCHA module based on this, but that also doesn't seem to be helping.
This is like trying to find a needle in a haystack. It sounds like either it's my GoDaddy shared hosting situation and it'd be improved if I went to a VPS (and the reason my dev site is faster than my live site is because there's much more data in use on my live site) or it's a module/script/etc. that has been installed on the live site that's not on the dev site.
Would it be worth it to disable each and every module that's on the live site that's not on the dev site to see if I can find something that improves load times?
Firebug
When I load the main page (www.hollywoodchicago.com) in Firebug, I get these results:
68 requests, 234 KB (161 KB from cache), 16.1s (with 4.15s to load www.hollywoodchicago.com)
When I load an internal page (like this), I get these results from Firebug:
96 requests, 526 KB (428 KB from cache), 32.46s (with 11.18s to load that page alone).
That's the problem. Why is it taking 4 seconds to load the main page of the site and then 11 seconds to load an internal page? Why is it "thinking" before all that other stuff on the page is even loaded? I do notice this in the response header:
Not sure why it's saying "no-cache" there, but then Firebug is reporting that a lot of the content gets cached on reload of a page. What I'm ultimately trying to figure out here is what's happening in the first 10 seconds or so while you're just waiting for an internal page to load and then how to fix that.
When I do Firebug on an internal page on my dev site, the results are totally different:
33 requests, 88 KB (34 KB from cache), 10.72s (with the page itself only taking 3.14s to load)
So what's taking 10s on my live site is only taking about 3s. That's the difference, but I can't figure out why. The response headers on the dev site internal page are mostly the same (except for the "max" number on "Keep-Alive"):
Last thing I noticed for the night...
In addition to my dev site loading much faster than my live site (mostly in internal pages), what's also particularly peculiar is that not only does the main page on my live site load much faster than internal pages but admin pages on my live site also load much faster, too.
Basically the pages loading slowly are pages with content/nodes (i.e. a news story, discussion forum node or anything with view/edit/revisions/track on the top of the page).
So http://www.hollywoodchicago.com/about loads slowly but http://www.hollywoodchicago.com/contact loads quickly. What the heck is slowly down the content/node pages?!
Like I mentioned above are
Like I mentioned above are you bringing in any information from 3rd parties? What often happens is content on your site will sometimes not load till the connections to 3rd party are closed, that 3rd party can slow down your site. Like I mentioned, 2bits has an article on it.
First... Remove all blocks pulling 3rd party information, make sure all your caching and aggregating is turned on.
Sounds like the database is slow, and perhaps your php. Y
Third-party information...
I'm not bringing in any third-party information into blocks. I do have third-party code installed in the form of scripts (i.e. Google Analytics, FeedBurner stats, phpMyVisites stats, Quantcast, etc.). Caching has always been on normal and so has aggregating.
Something is still causing the live site to "think" for the first 10 seconds or so before everything else starts loading (just on internal content/node pages and not on admin/etc. pages without node content). Whatever that is, that's really the source of the slowdown and the problem. That's what's *not* slowing down my dev site (for whatever reason).
Running devel.module query
Running devel.module query stats/analysis per pageview would possibly indicate what's going on. But you might want to do that on a copy of the live site (certainly take a backup before installing it).
"Thinking" is probably queries or PHP processing, or possibly iowait (the server waiting for a chance to read something from the HD, e.g. a database table).
gpk
----
www.alexoria.co.uk
Well connecting to those to
Well connecting to those to provide stats, is still using 3rd party services... Here's an example... your site hit 24sec loading time and still not complete... waiting on feedburner... The loading icon in firefox spun on that page for over 2minutes although the whole page seemed to of loaded just fine and I could see/navigate, but every page load it would hang on feedburner. Just because you aren't bringing info into boxes doesn't mean you are not waiting to send or recieve from a 3rd party. If you send something you need to wait for a response. It's best to try things before shooting them down. Concidering your site froze on feedburner on almost every page, for over 2 minutes, is a good indication there's ATLEAST a problem there, even if not connected. You should really READ on drupal optimization, javascript, etc, because it seems others are debugging your site for free at the moment... Like we said, devel + the net tab of firebug will let you know exactly what is taking a really long time. And I would have to say that most are server related. A 100k image should not take over 1 second to load (and every single one on the page did). I download at almost 1600K which causes fingers to point at your server, again.
You could always try a vps,
You could always try a vps, they almost all have 30day money back guarantee's. If you are trying to run a serious site, it's best to usea serious host and equipment. If it doesn't improve things drastically then you'll know. Also remember that php accelerators, cache the php, which will reduce that "pre-load thinking" time DRASTICALLY. Also mysql can cache (if you have the ability) frequent queries (which will improve those queries 10fold). Other things that you may want to concider is indexing the larger columns in your database for faster retrieval, it works like the index of a book (allows mysql to find information MUCH MUCH MUCH faster, however the downside is the index needs to be updated when new items are inserted... that means inserting will be slower, selecting will be faster... I think selecting speed is CONCIDERABLY more important).
Information on everything above can easily be found on google. They are all VERY VERY simple to set up and most of the time require very little knowledge of anything. I indexed the title and nid columns because my autocomplete was taking 5-10 seconds to return results making it unusable. Now it autocompletes in under a second. Problem is that there's no way to check how smart the queries are built for EACH module that uses them, so you have to be aware that queries may not be optimized within the code (often contrib modules). And you have to account for that by other means. If you need further assistance with this I would be happy to set up an account on my vps for you to TRY.
Just installed the devel module
As advised, I just installed the Drupal devel module. It's giving me a ton of information, but I don't know what to do with it.
It seems like there are a lot of entries for buddylist_get_buddy_groups taking up lots of load time along with links_load_links_for_node, but what should I be looking for here?
There's something that's causing an internal/node page to "think" for 10 to 15 seconds before even loading. That's not the case on the main page of the live site, the admin pages on the live site and all pages on the dev site.
It brought down your site
Anon users get this message -
Fatal error: Call to undefined function: user_access() in /home/content/p/r/o/projecthc/html/sites/all/modules/devel/devel.module on line 243
What the?!
What the?! I have no idea why. Anon users could still access internal pages with the devel module active, but I am seeing that same error on the main page when in anon mode. That makes no sense to me. I just disabled the devel module. The main page is back up for anon users.
Still no solution to the overarching problem.
There is an open issue for
There is an open issue for the error in devel.module: #309086: Devel + Caching = undefined function user_access in devel_init.
gpk
----
www.alexoria.co.uk
All FIXED! GoDaddy shared hosting was largely the problem
All fixed! We just launched HollywoodChicago.com today on GoDaddy virtual dedicated hosting (instead of GoDaddy shared hosting). The new hosting comes to around $45 per month. I doubled the RAM to 512 MB and 2 GB bursted, but kept the bandwidth (500 GB a month) and disk space (10 GB) the same. I'm running on CentOS 5. I also added 10 GB of FTP backup for $3 a month.
The site is now BLAZING and appears stable. This switch got rid of the mysterious 10- to 15-second load time problem on internal pages. The Drupal developer I hired also installed the Drupal boost module (to further improve caching) and the Drupal db_maintenance module (to optimize my databases).
Yay! Thanks everyone for all your help!
Publisher, HollywoodChicago.com
Glad everything worked out,
Glad everything worked out, sounds like my advice to a tee, funny how many people doubt it. Often when you can't figure out the issue it's server configuration, and if you're not the one configuring it, then you may just run into issues. Again, very glad everything worked out.
.
If you ever get tired of godaddy, hotdrupal.com has a great set up, ready to fly.
page load defects
I have fought with bad page load times ever since the first day with drupal. It is a problem that needs solved for drupal to make better progress into the mainstream.
I finally had to change hosts to get one site running to an almost acceptable level, but it still is a HUGE issue or DEFECT even on a very good vps. There is a very fast flat file system in every server available for things like this.
Supposedly the cache_router fixes the drupal defect, but never have gotten it to work. I hope someday this defect will finally get fixed and the discussion can come to an end.
Then you either have it
Then you either have it configured wrong, or have a slow host, cause I have no troubles with it at all on the hosts I run it on.
Try playing around with the cache settings in admin->site configuration->performance.
boost
I finally found the boost module....all pages load extremely fast now on a shared host even with a clogged up database.
The subscriptions are not working or I would have been back here.. long ago
=-=
there has never been subscriptions to drupal forum threads.
_
Of course there's always room for improvement, but sony, mtv, google, virgin, symantec, nokia, disney, etc, etc, etc are pretty darn maintstream. See http://buytaert.net/tag/drupal-sites for a pretty impressive list.
If you're using the update status module, try turning it off.
===
"Give a man a fish and you feed him for a day. Teach a man to fish and you feed him for a lifetime." - Lao Tzu
"God helps those who help themselves." - Ben Franklin
"Search is your best friend." - Worldfallz
How much php memory have you
How much php memory have you committed to your site? I find that this makes a significant difference in page load times.
----------------------------------------------------------------------
http://classicvinyl.biz
http://music.classicvinyl.biz
http://association.drupal.org/user/1207