By pembeci on

Hi all,

I am planning to develop two sites with Drupal. One of them will be a community blog with low traffic in the short term but I hope the other one will get popular within a year and it will need storing and serving hundreds of images. I checked the hosting options in this forum and although they look fine for budget purposes and features, they generally have low disk space for my purposes and I am not sure I'll get the type of control I'll need.

So I am considering self hosting. Although I am good at programming, I am a newbie at the hosting arena. I was wondering what shall I need for self hosting. I am willing to buy a new computer and spend the time for installing a linux distrubition and other things required for Drupal (php, mysql, imagemagick etc.if they are not already at the linux distro). I guess I'll need a DSL connection to my home. Then what else shall I need? If I need a web hosting company to forward page requests to my server sitting at home, what is it called (I said I am a newbie :)?

This looks like a better option financially then having a dedicated server. Those things look pretty expensive. I also found something called Linux Virtual Private Servers (at linode.com). They also seem to offer the control I want and looks cheaper than dedicated servers. In sum, I need a summary of the steps need to be taken for self hosting and pros and cons (i.e compared to having a linux virtual private server). Any help/suggestions will be appreciated. I am getting excited to exploit Drupal's taxonomy and module system but unfortunately first I have to decide on this. Oh, and also what does this hosting plan allows N mysql databases mean? How many mysql databases Drupal uses? I understand a limit about the size about the tables but a limit on number seems absurd to me.

Thanks a lot.

Comments

TDobes’s picture

TDobes commented

You will probably find your DSL line's upload capabilities to be insufficient to support anything other than a very small site... it certainly won't be able to handle a lot of images without making your visitors impatient. You will need a DNS host somewhere to self-host as well. There are a number of freebies that do this with no advertising. I have used ZoneEdit for years on my sites and have never had a problem. You do not need to pay anyone (other than your DSL provider) to self-host. You do not need to worry about disk space limits other than those imposed by your own hardware. The major downside is the relatively low upload (a.k.a. outgoing) bandwidth of your DSL line. Internet via. cable has similar shortcomings, but is more variable in my experience. (speeds are more inconsistent)

Are you sure you can rule out a shared hosting environment? Drupal plays well with others and can fit rather comfortably onto one of those inexpensive shared hosts. Of course, those will have the restriction on disk space which you're trying to avoid. Also, if you're looking into these, make sure safe_mode is off. Safe mode will cause you nothing but trouble.

As for the virtual private servers, they use User-mode Linux and provide you with your own virtual environment. This has the upside of much more configurability and never having to worry about unannounced updates or downtimes, but still the downside of low disk space allotment unless you're willing to pay a premium. You might also look into Quantact for this sort of thing... I opened an account with them last week as a sort of trial thing and they seem to have a nice service. (and are $5/month cheaper than Linode) However, I haven't moved any production sites there, yet.

I've also researched building a system then co-locating it in a high-bandwidth datacenter, but that's almost as prohibitively expensive as renting a dedicated server. I'm working on setting up a sort of hybrid system which will host the site and commonly-accessed media on my VPS and host the less-accessed media on my home server via. DSL. It's turning out to be somewhat complicated. It might be an option if you're clever, but I wouldn't recommend it. It's frustrating to be required to pinch every kilobyte when a 200GB hard drive can be purchased so inexpensively.

Drupal requires one mysql database. If necessary, you can make Drupal share that one mysql database with other applications by using table prefixes.

A good self-hosting setup would require:
* computer with *nix installed (Linux or FreeBSD recommended)
* web server (Apache recommended)
* PHP - latest version 4 release recommended for now. Drupal 4.6 (when released) will support PHP 5.
* connect PHP to apache using either cgi or mod_php... I'd go with mod_php
* database (mysql or postgresql... some contribs support only mysql, so I'd recommend it)
* whatever else you want to run your site... maybe imagemagick, maybe an FTP server, maybe sshd for remote administration

killes@www.drop.org’s picture

killes@www.drop.org commented

If you have no experience in running your own server I really cannot recommend that you start out by running one yourself be it from home, dedicated server, or as a VPS. There is a lot of stuff to be considered when doing self hosting. I've onl yrecently started to run my own (dedicated) server and I had not dared to do that if I had not a more experienced administrator with me.
--
If you have troubles with a particular contrib project, please consider filing a support request. Thanks. And, by the way, Drupal 4.6 will support PHP 5.

--
Drupal services
My Drupal services

sepeck’s picture

sepeck commented

I host my own server and have for years. My current server hardware is a former desktop system with IDE RAID 1, Pentium3 500 with 512MB Ram. Also hooked up is an older system setup as a dedicated firewall (www.smoothwall.org or www.leaf-project.org) and the web server is running in a DMZ. The TOS of my DSL provider allow me to run my server off the line. All these systems as well as the DSL modem and the switch they are connected to plug into a UPS. Web servers for small sites do not in general need a lot power.

In general, I do not recommend people self host. It is nice for a small family site where downtime is not a big deal. It is really nice to run as your test server where you can test and demo things.

You will have to consider backups, uptime, system OS patching, security (firewall), support, hardware failure, power, network reliability and bandwidth, email, dns, .............

I started off self hosting as a live lab and test environment and have continued ever since. Originally I needed somewhere to learn how to setup web and mail servers so I could make the jump to that next job position and this was how I did it (some friends also did the same). I don't use it for that much anymore, but I have two friends whose email and web sites that I support on the system. If it goes down, they help out. If I am out of town, they keep an eye on it. You need to be willing to consider this kind of stuff if you want to self host.

It is surprising how important to you even sites that start off as a bit of fun can become. :)

-sp
---------
Test site...always start with a test site.
Drupal Best Practices Guide

-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide

Uwe Hermann’s picture

Uwe Hermann commented

Oh, and also what does this hosting plan allows N mysql databases mean? How many mysql databases Drupal uses? I understand a limit about the size about the tables but a limit on number seems absurd to me.

Drupal only needs one database. And yes, a limit on the number of databases is IHMO quite absurd, but occurs very often, unfortunately.

Uwe.
--
hermann-uwe.de | crazy-hacks.org | unmaintained-free-software.org

mmdc’s picture

mmdc commented

If you've installed several LAMP applications, you'll notice that they usually give you the option of using a unique table prefix. This will allow you to use one database and not have overlaps on tables named "users" and the like. You can have a table called drupal_fred_users and wp_foo_users and they shouldn't, in theory, conflict.

The limit on databases really doesn't make sense, but if you're stuck with it, you can work around it.

kbahey’s picture

kbahey commented

As usual, the correct answer is: it depends.

If your web site is low volume and targeted only to friends and family, i.e. you do not need or want it to be in search engines, then hosting it at home on DSL makes sense. I already do this for test sites, photo albums, ...etc.

DSL at home can be fun to do at first, but then it can be tedious later. You have to keep the machine powered on all the time, and if you are running mail on it too, it would be a target for spam, script kiddies, ...etc. Moreover, DSL and Cable are often slower in upload than inb download, meaning that anyone who browses your site will think it is very slow.

If your site is targeted to a general public audience and you want it to be in search engines, but is still fairly low volume (e.g. 3-5 GB per month, 1,000s of page views per month, then shared hosting makes sense. You do not need to upgrade or patch the operating system and software.

So far, my sites are in this category, and I do not see the need to move to the next step (yet!). Using Drupal's multi site support, I host the 3 sites on a single account, and in a single database (using prefixes), so the cost is minimal.

If your needs are more than that, then consider VPS hosting, or even a real dedicated server.

I also recommend Zoneedit, and have been using it for years. They have great service and it is free for 5 or less domains. They allow me to have subdomains on the home test server for various things.
--
Consulting: 2bits.com
Personal: Baheyeldin.com

--
Drupal performance tuning and optimization, hosting, development, and consulting: 2bits.com, Inc. and Twitter at: @2bits
Personal blog: Ba

pembeci’s picture

pembeci commented

Using Drupal's multi site support, I host the 3 sites on a single account, and in a single database (using prefixes), so the cost is minimal.

Are you doing this by this patch: http://drupal.org/node/5942

But you still need to have different IP's one for each site, right? (assuming sites have different base URLs)

kbahey’s picture

kbahey commented

I am using no patches at all. All the sites have the same IP. They are all hosted on the same shared hosting account.

The trick is to use a different conf.php (settings.php in 4.6) for each domain. Each config file will have a different database prefix. All of the domains will use the same database, same database user and same database password.

This feature was one of the reasons I chose Drupal: multisite support out of the box.

If you will not allow users to select their own themes, then you do not need this patch. 4.6 has this functionality already.

My production sites run on 4.5, but 4.6 has the same functionality (using the sites/domain-name.com/settings.php

To explain more, here are steps needed:

1. Your hosting provider must allow you to have domain aliases, i.e. domain1.com and domain2.org display the same site (if you are using regular index.html), and have the same IP in DNS.

2. In Drupal, you have a different config file for each site. So you will have sites/domain1.com/settings.php and sites/domain2.org/settings.php.

3. Each of them has the same db_url, but different $db_prefix and different $base_url, as so. 

$db_url = "mysql://dbusername:password@localhost/databasename";
$db_prefix = "d1_";
$base_url = "http://domain1.com";

$db_url = "mysql://dbusername:password@localhost/databasename";
$db_prefix = "d2_";
$base_url = "http://domain2.org";

If you have more than one database in your account, you can use one for each domain, and no prefixes.

That is it. You now have totally different sites running from the same account and from the same IP, and same Drupal code.

Oh, and by the way: Pembe means pink. Right?
--
Consulting: 2bits.com
Personal: Baheyeldin.com

--
Drupal performance tuning and optimization, hosting, development, and consulting: 2bits.com, Inc. and Twitter at: @2bits
Personal blog: Ba

beate_r’s picture

beate_r commented

Hi, for those interested in self hosting multisite drupal installation, a few additions for the special case of selfhosting with dynamic IP address and dynamic dns (well, of course, if Your provider allows this)

If You use ddns services like dyndns.org, it is often possible to enable wildcard dns (i.e., arbitrary subdomains) on your router/firewall/whatever, say
http://www.example.com and
http://test.www.example.com

As long as there is no mapping of these two names to the ip of your local web server, drupal will fall back to its default configuration. So You might consider to specify these names as aliases to the hostname of your webserver, e.g., in /etc/hosts

As already noted elsewhere, it is highly recommendable to use relative URLs pointing to the installation directory of drupal in $base_url, e.g.:
$base_url = "/drupal46" if Your installation folder is $DOCUMENT_ROOT/drupal46

Michael

Eric Scouten’s picture

Eric Scouten commented

... read the terms of service from the DSL/cable service provider. They may prohibit you from running a server. Regardless of whether this is true, your bandwidth is almost certainly insufficient to support any more than the most minimal site. I certainly wouldn't attempt to host anything serving "hundreds of images" over any home-based server.
--
http://www.ericscouten.com

mmdc’s picture

mmdc commented

I've self-hosted for years. It has some definite advantages and disadvantages, but all-in-all, if you want to learn how this stuff works, go for it.
Here's a brief rundown on what works for me:

I use an IPCop router/firewall to control access and do NAT stuff. Make this out of the oldest pentium you can lay your hands on. It will outperform anything a typical home user can afford as far as hardware firewalls. It also has a DYNDNS update script that works well. Then you get some "foobar.dyndns.org" free address.

The router connects to the hub or switch that connects to the web server or servers. For that, grab an old 500mhz or so box and get a nice big hard disk.

Install Fedora Core 3 or whatever you like and webmin, unless you are a configuration textfile wizard.

Play with the router until you have opened port 80 and forwarded it to your web server.
...
Profit!
Bandwidth could be a problem, but better your ISP shuts you down than getting a massive bill from your hosting provider if you get slashdotted. Actually, having a slow connection will teach you to not be sloppy about posting lots of massive pictures.

I have both a home server, a friend's home server and a textdrive account that I use. They all have different advantages. Textdrive has amazing technical support and vast amounts of knowledge. But at home, I have root and can do whatever crazy thing I think up after too many beers. Just be sure to back up the stuff you care about. Did I mention that you should back up? Really. You'll sleep better knowing you won't rm -Rf your family photos.

Dru’s picture

Dru commented

Hi,
I'm using linode.com for two public sites, and a few test projects. I used to work in a unix environment in uni and partially at work for a few months. And I have to admit there was still a learning curve of a couple of months.
If you're interested in a simple installation on the linode forums you can find step by step installation instructions for a few different packages.
I started with a very simple small debian woody installation, and started customizing it. I'm now running a debian sarge (testing packages) and a few packages I compiled on my own. I mainly did that in order to have support of a few packages that highly increased my site speeds, including GD2 and turck_mmcache (which in my case had a x10 increase on the drupal powered sites).
I really recommend linode, it has a small community but their are very helpful, and most of the things can be found on the different communities dedicated to the different packages.
Hope that was helpful, if you need any help installing a system similar to mine feel free to email me.
Cheers,
Dru

Backpacker Hostel I'm Building in Cusco, Peru

stephenhendry’s picture

stephenhendry commented

I have been self hosting for a few years now. Going to rebuild my web server in the next few days. There is a not a bad guide http://www.oneofthosedays.org.uk/wikka/DebianInstallWiki of how to set one up.

I have around 6000 personal images online which I serve to my friends. I never have more than 1 or 2 people on the site at the time. The speed is not too bad. I would love to provide a faster service to my friends but I cant afford the montly cost for the size of disk space I need.

pembeci’s picture

pembeci commented

Thank you all for the great responses. Sorry, I couldn't find time earlier to post a comment.

After reading all these, I realized that I was a little bit more romantic about self-hosting and underestimated the challanges. So my game plan for now is to start developing my sites at a Virtual Private Server and get my hands dirty. If I can't handle this, after a few months I can switch to a normal host and say goodbye to root access. I want to use developing these sites as an opportunity to learn more about being a linux admin and web developer dealing with all the low level stuff and hopefully earn my bread this way in the future but I guess self-hosting is not the best starting point for this.

For future readers of this thread, I found these on the web:

Self-hosting on a user-mode Linux virtual machine
Self Hosting for Home and SOHO
Self Hosting (not much info but may be good for hosting newbies like me)
http://user-mode-linux.sourceforge.net/uses.html (User Mode Linux page lists some UML providers)

I found Drupal completely by chance (while reading a Drupal based site) and before this I though every blogger was using WordPress or a blog hoster like Blogharbor. When I discovered through Drupal pages that there were a lot of other CMS packages around, I checked those but somehow feel connected to Drupal. Now after getting 10 comments and two private messages to my first and not too related question I am feeling even more connected. Thanks again.

kbahey’s picture

kbahey commented

Depending on how your hosting provider does it, VPS could still mean that you upgrade your own packages and you are responsible for patching, system administration, ....etc.

This is both a curse and a blessing.

Also, a VPS may not have a control panel for fequent web site admin tasks.
--
Consulting: 2bits.com
Personal: Baheyeldin.com

--
Drupal performance tuning and optimization, hosting, development, and consulting: 2bits.com, Inc. and Twitter at: @2bits
Personal blog: Ba

sepeck’s picture

sepeck commented

Learning through hands on practice is a great way to gain experiance in some real world issue's. Don't neglect 'business case' theoretical stuff as well while you are learning. As your site develops you will have a nice starting place to hang your resume.

Some additional suggestions:

  • Local test system to test things before you blow up your remote site :)
  • Backups of your data
  • If posible, hook up with a LUG (local linux group) -they can have some great resources
  • Never forget to pause and write your target plan out. If you know where you are headed you can set the steps up to get there in a nice achievable manner :)
  • Remember to have fun and take a break occassionally :)

Good luck

-sp
---------
Test site...always start with a test site.
Drupal Best Practices Guide

-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide

Uwe Hermann’s picture

Uwe Hermann commented

Some more suggestions:

  • Read some basic security articles before you start! Default installations are usually insecure, you should know how to fix them.
  • Try to maintain as much as possible in subversion (or CVS, if that's not possible). I suggest the most important config-files and all your sites (the HTML + PHP files, images etc. and the database dumps). You could even automate this via cron, e.g. dump the database daily, and insert that dump into svn). Using svn, you have a complete history of everything you ever did and changed and of all patches you applied. If something breaks, you can always go back to the last version which worked.

HTH, Uwe.
--
hermann-uwe.de | crazy-hacks.org | unmaintained-free-software.org

kbahey’s picture

kbahey commented

More points, if you are going to self host:

  • Make sure you install a NAT router between the server and the internet.
  • Open only port 80 to the outside world, and perhaps ssh (port 22, if you need to access it from elsewhere)
  • Turn off all the daemons that you do not use. If you have no need for Postrgres, DNS, NFS, Samba, ...etc. then TURN THEM OFF.

Often, after the novely of hosting your own site wears off, it becomes a burden to look after the machine, backups, upgrades, and other sys admin stuff. But it is fun to try it.
--
Consulting: 2bits.com
Personal: Baheyeldin.com

--
Drupal performance tuning and optimization, hosting, development, and consulting: 2bits.com, Inc. and Twitter at: @2bits
Personal blog: Ba

Mahatma’s picture

Mahatma commented

Yes, it seems like you are considering all the options, however I doubt you will be satisfied with less than the best that is available, at least after a while.

I've found the only REAL drawback to using a hosting service is the security of any super-good code you may write, but you could get around that by using a compiler. I don't remember many details of those, but there's bcompiler(open source) and roadsend(commercial) ...

Check out the hosting plans I ended up offering to people to help subsidize my website expenses at http://www.cyberocity.com

Generally speaking you can go up to about 10gb drive space and 100gb bandwidth before you need to start looking at a dedicated server.

cheers

DKL64’s picture

DKL64 commented

I was all primed to sign up for Linode or Quantact, but then I read about
unixshell.com. They provide Xen virtual hosting from $7.99/month.
They appear to just be starting the service -- not quite ready for prime time.

They use Xen virtualization software, which is reportedly much better performing than UML - especially for disk intensive operations (web serving, db access).

Linode has recently done a beta test with Xen technology. See xenode.net and http://www.linode.com/forums/archive/o_f/f_27/xen_testing.html

Does anyone have any experience with either Xenode or unixshell?

David

hedge funds   credit cards
mortgages   insurance

mgifford’s picture

mgifford
he/him
Primary language English
commented

I was looking around for ways to track apache logs in a subsite instance. Finally asked the folks on #apache and got this response, so figured it was well worth sharing:

setenvifNoCase referer ^http://(www\.)?example\.com example
CustomLog /var/log/httpd/example_access_log combined env=example

This would allow you to set a different apache log file for every domain pointing to your site. Very handy when folks want to have something other than the drupal statistics modules.

-- Update --
justdave pointed out that you can have multiple virtualhosts all pointing to the same DocumentRoot.. That would do the trick too and would catch people who first come to the site rather than those who link within it.

Mike
--
Mike Gifford, OpenConcept Consulting
Free Software for Social Change -> http://www.openconcept.ca