It's alive! http://ubersoft.net

It took about three months, but it's finally up and running. My brain is fried, but it's finally up and running. Every once and a while I twitch uncontrollably... but it's finally up and running.

And worth it, I think. It was worth the trouble it took to get done.

A little background:

I publish a webcomic called Help Desk. I've been doing it since 1996, and for most of that time it was all static pages. When I joined Keenspot (a webcomics collective) in 2000 I added some templating tools they developed that allowed the static pages to be built automatically every night, but it was still not a particularly dynamic system, and I always thought that was a shame. I'd always wanted the site to be sort of like Slashdot -- post a comic and let people comment on it if they wanted to.

I first tried to implement such a system by using WordPress -- I did this in early 2006, and it worked OK but there were limitations. I have nothing bad to say about WordPress, it's great at what it does and its extremely easy to use and theme, but to get it to do some of the things I wanted it to do I was really pushing it to the limit. So in late 2006 I started looking at Drupal 5, and in late December I actually started building the new site. It's live now, and while only time will tell if it will actually do what I want, I have to say I'm really impressed with what Drupal let me do.

In the process of building the site, and specifically building the site archives for my webcomics (I have 3), I have really become a fan of Drupal's taxonomy system. I use it a lot -- not only do my comics have a "default" category that identifies which of the three comics they are, but there are also categories that list all the characters that appear in each comic and the name of story arcs that occur within each comic. This allows me to tag each comic as I create it, and when someone views each node they see links for the story arc and for each character that appears in a given episode. By clicking on the story arc link they can view the entire series of comics that appear in that story arc, and by clicking on a character name they can view only the comics where that character appears.

The other key ingredient of my setup is Eaton's Custom Pagers module -- the standard Drupal pager works contrary to the standard paging method for webcomics (left for previous, right for next), and Custom Pagers fixes that quite handily. Such a "small" thing, but my site would fall apart without it.

Finally I use views for just about everything -- the front page, the comic archives, character lists, almost everything except browsing one comic at a time is a view. Views actually allows me to frontload my comics (i.e., publish them ahead of schedule and have them display at a specific predetermined date) because you can set the filters to exclude any content that was published on a date that hasn't occured yet. And the views bookmark module gave my readers a feature they've wanted for a while: the ability to bookmark individual comics as they're going through my archives.

It's a home-grown theme, and it's not what I'd call a "pretty" or "elegant" design, but it works for what I want it to do. Some day I'd like to make it more xhtml compliant (Impossible at this point, since the Keenspot branding elements were created back when html 3 ruled the day, so they are chock full of invalid tags).

Anyway, I'm rambling a bit too much, but I really am impressed with how Drupal was able to handle all the tasks I wanted it to do. It was a struggle, but worth doing.

Comments

rszrama’s picture

Congratulations, and welcome aboard! I daresay we're almost like second cousins or something... working to get Drupal known as the "uber"-CMS. :P

----------------------
Current Drupal project: http://www.ubercart.org

mrknowitall’s picture

Got to love the comics.

aren cambre’s picture

Can you tell us where your hosting? It seems relative fast for a drupal site. Also, what modules are you using, and how did you use them?

ubersoft’s picture

58 modules in all -- I don't use all of the CCK and views modules, but they're active because you never know...

Front Page
CCK Field Permissions
CCK Content Copy
CCK Fieldgroup
CCK Image
CCK Link
CCK Node Reference
CCK Number
CCK Option Widgets
CCK Text
CCK User Reference
CCK View Field
Aggregator
Comment
Contact
Drupal
Help
Menu
Ping
Poll
Taxonomy
Tracker
Upload
Block Cache
Image Attach
Image Enhanced Scaling
Imagecache
Privatemsg
User registration notification
Atom
Comment closer
Content Templates
Custom Breadcrumbs
Custom Pagers
Date API
Read More Tweak
Footnotes
Google Analytics
Paging
Pathauto
Poormanscron
Service Links
Spam
Token
Update status
System Info
Calendar
Views
Views Calendar
Views daterange
View Bonus Pack
Views Bookmark
Views Calc
views_fastsearch
views_filterblock
Views RSS
Views Theme Wizard
Views UI

litwol’s picture

Come now, tell us the truth. why is your site so fast? :-D.

even on my localhost drupal acts very slow. but then again i didnt go and make my own mysql index tweaks.

tell us what you did...

Sometimes something interesting appears on http://litwol.com

ubersoft’s picture

was so fixated on the modules I forgot the other part of the question.

I didn't do anything specific to make the site fast. I'm running Mysql 5 and php 5, if that makes any difference. Mysql has its caching system turned on, and I have site caching turned on and use blockcache as much as possible. I'd like to try to get a php caching system put in place as well, but I haven't yet.

a2hosting.com has pretty good servers.

Steven Vermoere’s picture

Unable to connect to database server
If you still have to install Drupal, proceed to the installation page.

If you have already finished installed Drupal, this either means that the username and password information in your settings.php file is incorrect or that we can't connect to the MySQL database server. This could mean your hosting provider's database server is down.

The MySQL error was: Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2).

Currently, the username is ubersoft_wrightc and the database server is localhost.

Are you sure you have the correct username and password?
Are you sure that you have typed the correct hostname?
Are you sure that the database server is running?
For more help, see the Installation and upgrading handbook. If you are unsure what these terms mean you should probably contact your hosting provider.

ubersoft’s picture

Site loads fine for me.

I have seen this happen from time to time, however... not sure why at the moment.

Though now that you mention it, I wonder: is there a way to change/customize that page at all? It seems so... exposed when it displays.

ubersoft’s picture

Looks like my host is having some issues.

Update: my host provider just told me they had to suspend my account. They think someone is DOS'ing it.

...

That's a first for me.

Paul Adler’s picture

I just clicked on your site and redirects to this page: http://a2s16.a2hosting.com/suspended.page/ saying that this account has been suspended. Maybe too many Drupal people looking at your work :) I am looking for a new webhost, so I would be interested to know the cause.

Paul.

ubersoft’s picture

... they tell me they think someone is orchestrating a Denial of Service attack on my domain. They had to suspend my account to keep the server up.

That's never happened to me before. I can't imagine why anyone would bother.

markbl’s picture

The 'DOS" is probably just the volume of visitors coming from your posting on the drupal site.

ubersoft’s picture

... because I'm getting conflicting information from them now. One minute it's "DOS," another minute it's "too much traffic!"

I'm feeling a bit like Icarus today.

ubersoft’s picture

Drupal brought my site to its knees.

On its second day live.

This does not bode well.

That said, it gives me a chance to show off how Drupal allows you to customize your "site offline" page. ;)

Walt Esquivel’s picture

I can only begin to feel your pain. Sorry your site went down on its second day. Well, it wasn't my fault because I was on it yesterday, its FIRST day, and not today! ;)

On a more serious note, have you enabled the "throttle" core module and, if so, have you furthermore tried checking off the many throttle boxes on your admin module page? When I launch my production site, I plan on having many of the modules I've enabled to also be throttled.

First, go here and ensure your "throttle" module is enabled:
yoursite.com/admin/build/modules

Also, go here:
yoursite.com/admin/settings/throttle

Throttle - If your site gets linked to by a popular website, or otherwise comes under a "Denial of Service" (DoS) attack, your webserver might become overwhelmed. This module provides a congestion control throttling mechanism for automatically detecting a surge in incoming traffic. This mechanism is utilized by other Drupal modules to automatically optimize their performance by temporarily disabling CPU-intensive functionality.

In addition, have you played around with the cache options?
Go here:
yoursite.com/admin/settings/performance

I'm no expert when it comes to enhancing your site's performance, but throttle and cache do come to mind. You should perhaps join the High Performance group on groups.drupal.org, state your problem, and ask if there are any cache-related contributed modules that might help (in addition to asking for help in general).

Good luck and please let me know by posting back here if my suggestions help any...uh, once your site comes back online.

Walt Esquivel, MBA; MA; President, Wellness Corps; Captain, USMC (Veteran)
$50 Hosting Discount Helps Projects Needing Financing

ubersoft’s picture

It was my understanding that throttle was incompatible with the caching options. I do have caching enabled, though it isn't set to aggressive because apparently aggressive is incompatible with some of the modules I have enabled, and I have cached the cascading stylesheets as well. I also use blockcache, though not as much as I would like due to some limitations in it.

I will definitely be lurking in the High Performance Group...

Walt Esquivel’s picture

I visited my cache settings page here...
mysite.com/admin/settings/performance

...and it states:
"The following enabled modules are incompatible with aggressive mode caching and will not function properly: statistics, throttle."

So it sounds like as long as you have the "Caching mode:" set at "Normal (recommended, no side effects)", which you state you do, you should be OK. Have you throttled as many modules as possible? Of course, it would be almost impossible for you to now test your site under the same load as when it first appeared on Drupal's home page, but throttling, IMO, can be a very useful tool to help prevent your site from going down. (As a side note, it doesn't seem like your host was very forthcoming or helpful with the specific issue you were experiencing when it initially claimed DOS.)

If you set it to "Aggressive (experts only, possible side effects)", that's where you're correct in terms of potential throttle/cache conflicts.

Walt Esquivel, MBA; MA; President, Wellness Corps; Captain, USMC (Veteran)
$50 Hosting Discount Helps Projects Needing Financing

ubersoft’s picture

I didn't really understand how it worked until your post prompted me to look into it more. It's actually a really simple idea, and a pretty good one.

eaton’s picture

Caching?

you might also make sure that you're using caching, AND turning on caching in the custom_pagers module. It can be expensive if people are thrashing it, and the caching option makes sure it doesn't churn your DB unecessarily...

--
Lullabot! | Eaton's blog | VotingAPI discussion

--
Eaton — Partner at Autogram

ubersoft’s picture

... I had no idea Custom Pagers had a caching option... is that a relatively new feature?

eaton’s picture

That was added in (I believe) the most recent or the previous version and is pretty easy to miss if you were working with pagers that you'd already set up.

--
Lullabot! | Eaton's blog | VotingAPI discussion

--
Eaton — Partner at Autogram

ubersoft’s picture

Well I turned it on a few hours before they shut down my site (again) -- it's being moved to a less crowded server so I can debug without killing the rest of their customers...

... I am, apparently, a problem child.

ligures’s picture

Agreed. Happened to once when I had a spike in traffic hosting admin thought that it was dDos, it things like that that maku you use dedicated hosting

Paul Adler’s picture

Should tell your host that this is terrible advertising for them and putting you back onto a fast server would be a good idea to showcase their capabilities.

pamphile’s picture

try using http://www.modwest.com they can handle the load
I've seen them handle database intensive website counters with joy.

Marcel
http://01ftp.com

drupal-is-OK’s picture

The price / benefits ration seems rather high for modwest. I guess they don't oversell their servers as much as others do. If I ever have a big site that I need to be bulletproof, I guess I'll give it a shot.

_________
uofl forum

benc’s picture

Nice site! The theme looks very familiar (wink!). Congratulations and thanks for the info on caching and throttling.

"Drupal and the Power of Categories (Taxonomy)"
http://digitalsolutions.ph/couchkamotereviews/power_drupal_categories

ubersoft’s picture

The basic files in the theme were taken from Garland, though most of what makes Garland unique was stripped away (I didn't want to, but the way Garland worked was incompatible with Keenspot's branding bar)... the two-sidebars-on-the-right look was inspired by Amadou, but instead of using their layout I based my css on the "Holy Grail" theme (using negative margins for column positioning).

So when I originally described this as a "custom" theme, I think it's actually more accurate to say "this theme is an unholy frankenstein amalgamation of ideas from a bunch of other themes out there, all apologies to the original creators." :)

ubersoft’s picture

Ubersoft.net will be reverting to its old, static page design late tomorrow evening. This is not, I need to make clear, a reflection on Drupal itself: Drupal allowed me to build exactly the site I wanted to have, and I will be restoring the site to its database-driven glory a few months down the road.

However, one of the things being linked to from Drupal's front page taught me was that database-driven sites require quite a bit more resource management and fine tuning than a static page -- "no kidding," I hear you say, and I knew that was the case going into it, I just didn't have a clear picture of what that really meant. There are plenty of things I can do to get the site running smoothly, but I need to be able to do them... some of the solutions require time, others require money, and at the moment I don't have enough of either to do the job right.

I expect that when Ubersoft.net returns to Drupal it will be much better able to withstand the kind of scrutiny it received last week.

aren cambre’s picture

Can you share lessons learned in this process? I'll bet you have valuable insight on things the rest of us need to know.

ubersoft’s picture

Which forum would be most appropriate for that? I do have some interesting information based on the experience...

litwol’s picture

here would be just fine since all of us are already following this thread closely.

Sometimes something interesting appears on http://litwol.com

Walt Esquivel’s picture

Yes, here would be very appropriate.

Walt Esquivel, MBA; MA; President, Wellness Corps; Captain, USMC (Veteran)
$50 Hosting Discount Helps Projects Needing Financing

sepeck’s picture

http://drupal.org/forum/49
Just make a comment here linking to the new topic.

We're trying to get people to help out with this type of documentation and experience because it's important to understand the relationship of the OS and drive array configuration, Database and webserver configuration in regards to Drupal and type of site traffic and content when deciding where the bottlenecks and performance issues actually hit you.

Caching for anonymous users can help tremendously and built in caching is actually pretty good, but you hit a point where you need to engage a dedicated caching solution etc etc and so on.

People migrating from html or simpler backends sites can really get caught out by this. On the good news, there are more people aware of these types of issues and some are startig to share their experiences.

-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide -|- Black Mountain

-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide

EHCruzan’s picture

Ubersoft, can you give an estimated number of users around the time, or on the day, that your problems occurred? Thanks.

ubersoft’s picture

I'm going to try to write something up today. I've been taking notes and comparing traffic -- the only thing I don't have any real data on (and that would be more useful than anything else) is how much mysql activity there was during peak times.

ubersoft’s picture

It's taking a lot longer than I expected to finish writing this. The main problem is that I don't know what information is relevant and what isn't, so I'm including everything I can think of. That's making this post really ridiculously long and ponderous.

druvision’s picture

My brain is fried, but it's finally up and running.

Beware of getting your brain fried too much. It's your life, dude!

See my drupal site on EMF Pollution and Remediation . Fried brain is one of the symptoms.

Amnon
-
Drupal Focused Search Engine | Drupal Israel
Personal: Hitech Dolphin: Regain Simple Joy :)

ubersoft’s picture

Took me long enough... here's an analysis of the problems I encountered after going live that forced me to return to the static pages:

The birth, death, (and hopefully rebirth) of Ubersoft.net on Drupal 5.1 -- a site performance autopsy

Hope it is helpful to some.

Artliquide’s picture

I've been reading your comic for a couple of years now. Love it :D I'm considering Drupal for my own webcomic, which I'm now building. This post will be invaluable. Thanks.