Running Non Drupal Sites Under Octopus

Another vote: http://groups.drupal.org/node/218284#comment-717509

:)

This is clearly postponed, so far.

@EdNet
Updated nginx_static_include.conf & nginx_wp_include.conf available here:
https://drupal.org/node/1416798#comment-7752611

non drupal sites pre upgrade

I add my vote also to supporting non-Drupal sites under BOA, both for some special reasons (like transitioning phase to Drupal from {other, static}) and for being more like general server and consistent handling of everything under BOA (not requiring under-the-hood changes that may be broken with upgrade).

So I pledge for:

1. Adding Static/PHP site under platforms that will not be a platform actually, just handling nginx and other system things
2. Adding required addition for serving those pages like nginx_static_include.conf
3. Handling them just as platform ones with aliases, services (backup maybe?)
4. Maybe even enable automatic DB creation (checkbox that will reveal access user/pass/db upon creation) and backup/deletion
5. Keeping those files upon upgrade so that upgrade procedure is safe for those sites too like it is for platform based ones.
6. Having place for them in standard user/aegir directory structure as /data/disk/o1/non-platform

(oh, my!)

This would go far, *far* beyond the BOA project scope. You can very easily manage any static/PHP/WP/whatever sites manually, using Chive and other configs as examples. Don't miss the fact that Aegir is designed to manage *Drupal* sites, even if there are plans to support "alien" apps, like WordPress or Textpattern. And don't miss the fact that BOA is a stack to easily install and upgrade high performance, but Aegir centric hosting system. We are not going to turn it into any kind of generic, Virtualmin-like stack.

Yes, I know, understand and respect... however, some things sound nice.

It is true we all need something else on our systems, from static site management (for instance, prototypes, launch page) to some tools that make life easier (ActiveCollab, Redmine).

And it is very true that BOA should be minimal, compact and narrowly focused and even I myself would strongly oppose to adding everything in and a kitchen sink.

I did manage (somehow) easily (after few days google-ing) to add static site (struggling to do so on Ruby for Redmine) but I am bit like "afraid" if some upgrade will break things.

And I would like to have my "alien" platfoms listed and somewhat managed in "sites" list.

Actually, I was even wondering on some add on module that would handle static.

You need to understand that it is not possible to create any kind of one-size-fits-all configuration for any third party app you may want to install, hence you will probably need to create custom Nginx include file anyway, as used by Chive, so looking at Chive vhost and its Nginx config include file should provide good enough starting point. You are asking for docs/tutorial or some extra magic while you have ready to use example (Chive) in front of you.

@Omega: true, there is no real one-size-fits-all recipe but I would like to make a case like this:

In order to stay Aegir style, I would propose to make a special platform that is called Non-Aegir and that would support any PHP-HTML based site.

Such a "platform" would support and do everything like regular Aegir custom built platform except one tiny thing: it would not make validate procedure and would also not support other Drupal specific tasks like Clone, Flush all caches, Reset password, Migrate, Rebuild registry, Run cron, Run cron and Verify (as it is not Drupal) but would support Backup, Delete backups, Clone, Delete, Disable and Restore as those can be implemented blindly, just plain archive and extract.

It even seems to me like a not too complicated task, just throwing some exceptions (conditions) on present code would do the trick (maybe, just an observer opinion for consideration, would not like to offend you!)

That way, users would gain that option to run simple things and yet also be able to have those sites on list, creating, deleting them, having users, sftp, ssh access and otherwise securely integrated into Aegir as it will share all benefits of regular Drupal site.

There are only two profiles actually needed, both the same as you already provided in #1416798: nginx does not open index.html on custom vhost the only difference being on whether nginx should use any index in any subfolder (for plain PHP/HTML site) or is some CMS that routes everything to /index.php.

That would make me, and I sincerely hope all the other users of Aegir/BOA.

Except it is not possible in Aegir.

Plus, you should probably follow up on some related existing requests in the Aegir queue since it is not really BOA job or issue etc.

I am newbie to BOA. I have installed in successfully on a VPS with plenty of RAM and CPU power. However, although it looks very handy for Drupal management, it is definitely not "hosting" friendly.

For example these are my needs that I can't find a solution for:
1. How can I set up mail server and mailboxes for the domains created?
2. Even installing with barracuda _EXTRA_LIST Webmin, nothing in Webmin actually works. Getting the Nginx module for webmin can't find the Nginx location by default (and I have no clue how to change it either for Webmin).
3. I added the Bind in the _EXTRA_LIST, but how can I access it and how does it works? I am OK using this with Webmin or Virtualmin, but BOA seems to be a totally different matter.

So for me, BOA can be the Ferrari of the Drupal environment, but definitely not a user friendly "Hosting" platform.

How does omega8cc manages to do real hosting with it?

I think I will revert it back to Virtualmin in which I can easily set up Varnish and Drush as well. MySecureShell is not BOA exclusive as I have set it up on Virtualmin and it is working great limiting users sftp or scp to their own directory for security.

Don't get me wrong, I have been really impressed with the BOA/Aegir platform. I am definitely missing something and the problem is that I can't find any pertinent documentation to help with these issues. Maybe someone can direct me to the right place?

Regarding setting up BOA on a fresh VPS installed, here is my 2 cent tip. I used a Debian 7 VPS:
When freshly installed, there is no way to have BOA installing correctly as there is always a invalid DNS issue!
To fix this, easier is to get a freeDNS account where you can add the hosts entry including the *.mydomain.com. I used the FreeDNS from Namecheap.com. This isn't mentioned anywhere in the google pages I found.

Hi marcnz,

I wrote quite a long answer but it was lost due to form time out so I'll try to remember and write again.

This answer is not only specifically for marcnz but for all others that may stumble somehow here. If there is a better place for this information and if it can be expanded to be better, feel free to suggest.

Hosting friendly

There are some misconceptions on what hosting friendly is. It is something very different depending on your own skill set and tool set and the way you are used to handle things.

And just for the sake of correctness, I'd refrain from giving hard critics but rather question myself if I am getting it right.

For me, personally, hosting friendly means that provider is letting me do the work in command line (a.k.a shell) and that I am not limited to do so with full power needed. On any root system (where you have root access) I feel completely hosting friendly having power to do anything.

At the same time, BOA is handling many things and checking (self healing) with cron every five minutes if everything is as it should be, in predictable and expected state so it can safely maintain itself.

BOA system is very complex and it's quest for consistency may bug someone but it is there for a reason.

To borrow from eastern philosophy, you are supposed to go with the flow, not against so everyone should keep that in mind when coping with BOA. With is the right way (tao) and you can only do that by knowing, which brings us to...

Knowing Aegir and BOA

And then, BOA as both Barracuda and Octopus as projects and Aegir on its own are free non commercial products and are aimed at professionals who understand what it is all about. There is a lot of documentation but be aware: power comes with complexity and really, there is no easy way to present what BOA is and how it may benefit you.

The first place to read is here: http://community.aegirproject.org/handbook

This is where you will understand how Aegir operates, what are presumptions and basic paradigm.

The second place to read is here: https://omega8.cc/library/aegir-basics-index

This series of articles will help you understand both Aegir and BOA and maybe will get you faster into the thing than Aegir Handbook. However, be prepared to read same articles over and over again — you will miss something as it is very densely information packed and one needs to go there from time to time. Year and over I still have to read and when I do, I wonder how did I miss some things?!

The third place to read is here: https://github.com/omega8cc/boa

This is hardest as it is most packed of all, aimed at professionals at very high level with brief information that means a lot to an experienced system administrator.

BOA maintainers (specifically, for Barracuda and Octopus) are Omega8.cc and they base their business on those scripts and they maintain it on github where anyone can grab the script and build their own server which is (so to say) core of their business. So you must understand why they are not answering and why answers are sparse - they have to make a living first and then to contribute back.

So I'll try to contribute back by trying to explain to you as well as to the others some misconceptions that I learned with my one year experience with BOA.

The essence

BOA is the script that builds the server for you from the scratch.

That server is highly optimized to drive you Drupal sites "the Aegir Way" meaning it uses platforms, migrations and multisite paradigms you may read about on http://www.aegirproject.org/

And while Aegir works for you with sites, Barracuda and Octopus scripts are working for you on building and maintaining server and Aegir.

In their essence, Barracuda and Octopus scripts does not influence Aegir operation. They do tweak some minor things for better performance and also do some very special things about nginx configuration together with other server services so you get highly tuned and performant server that runs Aegir

Summary: Most of your questions fall into Aegir operations not Barracuda or Octopus scripts

Mail server

Just like you, I wanted complete solution in one box that would include mail server but I wondered about other things like Ruby that would enable me to run Redmine and maybe some other niceties like nodejs.

And all of that is not good idea although I did argue in some other threads. So I read a lot and consulted with some sysadmin veterans and finally came to the same conclusion answered by Omega8cc guys and others: do not do that.

Seriously.

It is a bad idea for a few reasons:

• First there is security issue, you just do not want them to be on same IP and probably not the same machine.
  
  That is because any sort of attack on your web server or mail server would bring down other one as well and that happens a lot.
• Then, if your mail server get some security hole, it would blacklist your web server as well.
• Mailboxes can grow easily as people are used to send large attachments and you could easily eat up your space and web server would suffer
• Whatever web interface you bring to the bunch that would eat up resources as well (Zimbra or any other)
• Running mail server means some different skill set and will eat up your own resources
• Again, as it cannot be stressed enough, BOA is heavily optimized for running Drupal sites and every complexity that gets into its way is not desirable

As NetFast said, you should better bring up another VPS that would handle mail and sizing should be according to your needs. That means it is completely separated with another IP and own space where it runs.

I was wondering to use Barracuda script for building server for that too but because of some specifics of BOA server configuration it may not be wise idea as it would bring different feature set and tune ups than are needed for mail server.

BOA as Ferrari

BOA is fast and finely tuned and optimized, but I suppose creators would rather stack as Volvo as stability and security are more important for your business and speed is second to that.

That design is imposing many limitations you can or cannot live with. Those that can, go with BOA, those that can't may choose their own way.

Either way, the fact is that BOA is excellent with Drupal sites and it is more of a question if one can accept Aegir way of thinking or not. If Aegir is a way for you, then BOA gives you a good VPS/Dedicated build on steroids for free.

I strongly suggest testing before going with it as nothing can compare to real life testing. A month or two on $10 SSD should be enough to test drive if it is scenario for you.

Questions answered

1. Mail server is not a good idea and should be run on separate VPS or dedicated with separate IP
2. Webmin works but it is not independent as it is on plain server. On BOA it is bound by setup, limitations imposed by design including cron scripts that monitor, take care and correct changes outside BOA. However some Webmin tasks do work (users) and you may use it but I do no recommend that. I rarely use Webmin to check some things, never to change them. Again, BOA is heavily optimized for running Drupal sites and Webmin does not know about its specifics (unless someone writes a module for BOA which will most likely not happen).
3. BIND is a tricky beast and I gave up on playing with it. I am using provider DNS but not all hosting providers are having one. There are some books on BIND but I found that such service would be too much to think about. Also BIND does add to resource strain so I avoided it.
4. There are various sorts of limited shells and BOA one is so specific that it is hard to change it unless you are pretty highly skilled. Again, it is already optimized and configured for BOA scenario and one should not mess with it as it is set up good enough for you casual clients that would need it here and there. And they should not need more.
5. Omega8cc are extraordinary gentleman and I can only thank them by trying to support what they give for free and I use it to build my own small business. Probably this is the case with many others (not sure if they have statistics how many BOAs are there) so we all are building our business on BOA. A very real case of hosting. If that is not the case they wouldn't be able to operate 7 data centers all over the world.
6. You may revert to Virtualmin, you didn't pay anything so you are free to choose whatever suits you. But if you read this Quora article and this Omega Compare series of articles you will learn rationale on going with nginx and Redis and not with Varnish and memcached. One may agree or not but that is rationale. And being aulde dogs (in hosting since 1997), you may take their word for it. Or not and challenge with your own rationale.
7. Invalid DNS has nothing to do with BOA, it is DNS issue or BIND: you can't resolve unless you have resolver. As I said, I gave up on that to preserve myself from BIND headache and to handover that work (DNS resolution requests) to my hosting provider. So I point registration name servers to my provider which then routes all requests for my domains to my instance. That means, for every domain I own/register I point nameservers to my provider, then add DNS record in their control panel (ok, I use API actually) and then create site in Aegir. And it just works, really.

Great answer, MacMladen!

Re the DNS issue, I've noticed on DigitalOcean that you must remove the 127.0.1.1 entry in /etc/hosts or you will get an invalid DNS error.
Edit: filed an issue here.

Thanks!

(you have a typo: did you mean 127.0.1.1?)

I am on Digital Ocean and /etc/hosts can contain it without problem. Here is my untouched file

127.0.0.1 localhost
# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

No, my /etc/hosts contained both a 127.0.0.1 and a 127.0.1.1, and I had to remove the 127.0.1.1 entry.

Maybe it's IPv6 specific?

This is not related to this thread, but it doesn't really mean anything in particular (it's just another IP in the 127.0.0.0/8 block which is bound to loopback interface).

The reason it's used as the IP for your hostname is explained in Section 5.1.1 of the Debian Reference Manual https://www.debian.org/doc/manuals/debian-reference/ch05.en.html#_the_ho...

Each line starts with a IP address and it is followed by the associated hostname.

The IP address 127.0.1.1 in the second line of this example may not be found on some other Unix-like systems. The Debian Installer creates this entry for a system without a permanent IP address as a workaround for some software (e.g., GNOME) as documented in the bug #719621.

The matches the hostname defined in the "/etc/hostname".

For a system with a permanent IP address, that permanent IP address should be used here instead of 127.0.1.1.

For a system with a permanent IP address and a fully qualified domain name (FQDN) provided by the Domain Name System (DNS), that canonical . should be used instead of just .

Some software (e.g., GNOME) expects the system hostname to be resolvable to an IP address with a canonical fully qualified domain name. This is really improper because system hostnames and domain names are two very different things; but there you have it. In order to support that software, it is necessary to ensure that the system hostname can be resolved. Most often this is done by putting a line in /etc/hosts containing some IP address and the system hostname. If your system has a permanent IP address then use that; otherwise use the address 127.0.1.1.

I am closing this issue, since the module doesn't have releases for supported Drupal versions.