This commit breaks existing installs using previously available conf.d directory, since there is no upgrade path provided.

CommentFileSizeAuthor
#2 provision-revert-conf.diff.txt2.98 KBomega8cc

Comments

omega8cc’s picture

omega8cc commented

I believe it should leave conf.d as-is and simply add post.d to not break things for existing installs.

omega8cc’s picture

omega8cc commented
StatusFileSize
new provision-revert-conf.diff.txt2.98 KB

I'm attaching the patch to fix this.

anarcat’s picture

anarcat commented
Status: Active » Closed (won't fix)

The conf.d was introduced during the 0.4 development cycle, and we do not support upgrades from 0.4-whatever, really... those were all alpha releases. Furthermore, it's just a matter of moving conf.d to post.d, really here and besides, so I feel comfortable breaking that API that was introduced during the alpha releases.

Note that this changed many times during 0.4 dev: it used to be config/apache.d in march 17, then was changed to config/apache/conf.d, and now we're landing it in its final destination, post.d (and adding pre.d). It will *not* change once 0.4 is released.

I am opposed to having pre.d and conf.d: it's confusing, and unnecessary. If we want to keep the migration path, we need to have a server verification task that checks for conf.d and moves it to pre.d, but then I think we should have done the same for the old apache.d too, and we haven't, so I don't see why we do anything fancy here.

Just mv ~/config/apache/conf.d to ~/config/apache/post.d. :)

omega8cc’s picture

omega8cc commented
Status: Closed (won't fix) » Needs review

I'm sorry, but it is not acceptable. All Barracuda and Octopus installs are using it, and it shouldn't be changed. I don't see any reason why you suggest I (and many people) should "fix" theirs installs because you prefer different directory name.

anarcat’s picture

anarcat commented
Status: Needs review » Needs work

Okay, well maybe we can find a compromise here.

Basically, you do not want your barracuda and octopus installs to be broken, and I don't want to see conf.d anymore.

First, we do not support barracuda and octopus here, you'll have to support those yourself.

Second, we never committed to having intra 0.4 upgrades working, I witnessed that myself last week and bit the bullet to upgrade my funky 0.3+crazy patches to 0.4. That was my responsability.

Third, having pre.d and conf.d is silly: it's unclear and confusing. We want to have pre.d and conf.d. Your patch, as it stands, is "unacceptable". Sorry.

If you provide a patch that will fix server verification so that it moves the conf.d, I will not have a problem with that, but I don't think we should back down here.

You had to do this before, why is it a problem to do it again to have a clean final version for 0.4? Keep in mind you are running alpha code, and we're trying to stabilise a beta here, it may require a little (one command! you can even make a symlink for all i care!) manual configuration to keep up!

(Besides: don't you have your own update scripts in barracuda/octopus?)

omega8cc’s picture

omega8cc commented

My patch is not to keep pre.d and conf.d, it is to keep already used by *many* people conf.d and adding post.d, which will be never used in Nginx, by the way.

I believe you missed one point here. Aegir project now supports Nginx web server (not Barracuda and not Octopus), and you can't blindly introduce changes to the Nginx configuration because you need something fancy to be added/modified for Apache.

It is an arbitral change and it breaks Nginx support. It is not a problem for me, I can do whatever I want, but people are using Nginx with Aegir also without my wrappers/installers. If you don't care, I will have no choice and will use forked version, however I don't think it is a good idea and the proper way to resolve so little problems.

What is the reason you don't care if you will break the core Nginx config for people and you don't want accept already proposed compromise? Just leave this conf.d as is and add that post.d if you need it for some reason. Nginx users don't need it at all.

adrian’s picture

adrian commented

i'm wondering what the reasoning of the pre.d and post.d split is specifically ?

i agree we shouldnt just break them. The external db for the hostmaster site stuff was broken because it was never supported in the first place.

anarcat’s picture

anarcat commented

The rationale for pre and post.d is clearly explained in the template: pre.d/ allows you to override the configuration generated by aegir, as it is included before. post.d/ is the way conf.d used to work (although that was unclear and not quite explicit): it is "default" configuration that gets included *after* aegir-generated files, so aegir *overrides* those. It is useful for migrating legacy systems into aegir.

So let's see two examples here:

1. Admin thinks aegir screwed up in its vhost configuration of example.com and wants to override it. He slaps a modified example.com.conf in pre.d/ to override that config. Apache picks it up first and ignores the vhost in aegir's config. Win.
2. Admin has an already existing example.com on the server and wants to migrate that into aegir. He slaps that config in post.d/example.com.conf and then does magic things to import it into Aegir. Once aegir verifies the site, the final config is create in vhost.d and overrides the one in post.d, and the admin is happy: if it works, he removes the legacy config in post.d. If it doesn't, he can just delete the site (or faster: the config) and try again without too much disruption.

Heck, #2 is actually how I ended up upgrading my production server, as we migrate from /var/aegir/config/apache/vhost.d... (Where's the upgrade path for *that*?!) So I feel this is useful.

I don't know how this works in nginx, maybe you could instruct us about this.

All I know is that this is useful for apache, and could be useful for others, and makes it clearer.

There's also an easy way to support you guys that started running on an alpha release to migrate to that clearer design. Is there some issue I'm missing here?

Again: i'm all for supporting this upgrade path. Let's do that patch instead of just reverting what could be a great idea in the long term.

omega8cc’s picture

omega8cc commented

From your description I see it is not about splitting the conf.d directory, but rather about adding pre-vhost and post-vhost new directories to make legacy sites imports and emergency changes easier. It is a good idea and I like it.

Let me explain however, how it works in Nginx. The (previous) order is as below:

# virtual hosts
include /var/aegir/config/server_master/nginx/conf.d/*;
include /var/aegir/config/server_master/nginx/platform.d/*;
include /var/aegir/config/server_master/nginx/vhost.d/*;

The conf.d directory is the correct (and logical) place to put there global settings, if they are not included already above, in the main server config section. It is very handy if you require some global options not used in the default configuration, like Nginx specific fix for global gzip, upload progress etc - since some options don't work with older Nginx versions and we can't include them by default. We are using conf.d also to keep there the global HTTPS proxy wildcard vhost, so it can be later overridden in vhost section per SSL enabled site.

The platform.d is optional here, but works perfectly as a place to include Octopus instances configuration files (which later include its own vhosts).

The vhost.d directory is the directory for created by Aegir, site specific config files.

Now, if we want to introduce pre.d and post.d, we should probably better explain that both are vhost and not global config related. I would even suggest to consider more self-explanatory naming convention, since post.d has in fact nothing to do with "post" action (it refers only to the order of includes), it is in fact "pre" since its intended use is importing from legacy config etc.

What you think about something like: vhost_force.d instead of pre.d and vhost_legacy.d instead of post.d?

It could look like below:

# virtual hosts
include /var/aegir/config/server_master/nginx/conf.d/*;
include /var/aegir/config/server_master/nginx/platform.d/*;
include /var/aegir/config/server_master/nginx/vhost_force.d/*;
include /var/aegir/config/server_master/nginx/vhost.d/*;
include /var/aegir/config/server_master/nginx/vhost_legacy.d/*;

At any rate, we still need conf.d for reasons explained above, but I fully agree the idea behind your change is very good. Just let's make it more clear for the people.

omega8cc’s picture

omega8cc commented

Or even vhost_override.d instead of pre.d.

anarcat’s picture

anarcat commented
Status: Needs work » Closed (won't fix)

So after a good discussion with omega8cc through IRC, we agreed to let things as they stand now: we will not migrate people from conf.d, they will have to do it manually. I have added a note to UPGRADE.txt to that effect. Note that we don't migrate people's old vhost.d either, and they need to go through that painful process manually. I would rather work on addressing that issue instead of this one.

One thing we noticed during the discussion was that conf.d had different meanings in nginx and apache: in apache, it was included after the other configs, while in nginx it was included before. If we want to implement an upgrade path here, we'll at least need to take this into account. I started writing a verify_server_cmd() in provisionService_apache to do just that but then stopped when omega8cc agreed to handle the migration manually.

We agreed that post.d and pre.d are valuable and important distinctions. conf.d too generic, and vhost-legacy.d and similar are too specific: the admin can actually put any config in those directories, not only vhosts, so that naming would only blur things.

having a pre.d and post.d covers all the use cases, it clarifies the reason behind the .d directories and the include order.

Thank you for the compromise and the good discussion!

anarcat’s picture

anarcat commented
Issue tags: +FAQ

potential faq material

  • Commit ed2a4c3 on debian, dev-koumbit, dev-log_directory, dev-migrate_aliases, dev-multiserver-install, dev-simplerinstaller, prod-koumbit, dev-ssl-ip-allocation-refactor, dev-1205458-move_sites_out_of_platforms, 7.x-3.x, dev-subdir-multiserver, 6.x-2.x-backports, dev-helmo-3.x by anarcat: 
    #936720 - add upgrade note to tell people how to migrate away from conf....

  • anarcat committed ed2a4c3 on 
    #936720 - add upgrade note to tell people how to migrate away from conf....

  • anarcat committed ed2a4c3 on 7.x-3.x-1966886-context-to-entity 
    #936720 - add upgrade note to tell people how to migrate away from conf....

  • anarcat committed ed2a4c3 on 6.x-2.x-1995506-profile-option 
    #936720 - add upgrade note to tell people how to migrate away from conf....