Right now, we use verify to verify Drupal platforms and sites. As part of the multi-server support of 0.4, we need to check on some settings, server wide, and return some of those to the frontend. This will recognize the local webserver, mysql server, dns, and others (this should therefore be hookable, which is builtin to drush tasks), and create configuration settings for them if they are present.

Some of the things this can do:

1. find & store the location of drush
2. find & store the location of apachectl
3. discover existing vhosts on the server that live outside of aegir
4. discover the location of the apache configuration to help frontend configuration
5. store the mysql credentials
6. look into httpd -S

CommentFileSizeAuthor
#15 provision-webonly.patch377 bytesdrumm
#10 586000.patch674 bytesdrumm
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

anarcat’s picture

one of the things this needs to do is to manage the ~/config/apache.conf file, that is currently done through platform verification, in drush_provision_apache_provision_verify() in web_server/verify.provision.inc. It's simple to call _provision_apache_create_server_config() in the task.

once this is done, the initial apache.conf generation in install.sh should be replaced by the server verification task.

anarcat’s picture

Status: Active » Needs review

okay, so I started the work here, by creating a 'provision-server' task in provision.

the work is happening on the dev_server_verify branch on git.aegirproject.org.

i migrated most of the platform verification variables to the server verification task. furthermore, I have made sure those settings are *removed* from the platform drushrc to avoid confusion.

look for server.provision.inc files in the new branch. things that should be dealt with now:

* find & store the location of drush (through install.sh detection)
* find & store the location of apachectl (guessed normally)
* store the mysql credentials (prompted if not provided on the CLI)

It also does this:

* creation of the config, includes, backup, vhost.d, platform.d, etc directories
* creation of the apache.conf file
* creation of the global.inc file
* the following settings are now stored in ~/.drushrc.php: master_url, web_host, web_ip, web_group, script_user, restart_cmd, backup_path, config_path

This is still todo:

* discover existing vhosts on the server that live outside of aegir
* discover the location of the apache configuration to help frontend configuration
* look into httpd -S

Anonymous’s picture

For the record after IRC conversations

While testing the dev_server_verify branch, a couple of issues have been fixed and these are outstanding:

1) We need to pass web_ports to provision-server, otherwise when the apache.conf gets generated, the missing variable results in no NameVirtualHosts being written to the file

2) No drushrc.php is being generated in ~/

Anonymous’s picture

Status: Needs review » Needs work
adrian’s picture

I've gotten uneasy with generating drushrc.php files, especially with the new alias stuff that's come in.
At least on the server level, I don't want to store a single Drushrc.php file.

I'm looking to extend the site alias functionality in drush at the moment, so we can generate aliases for each of the servers / sites, and have them conditionally loaded.

adrian’s picture

As an example..

currently you can do

$options['site-aliases']['platform/1'] = array (
  'root' => '/Users/adrian/Projects/hosting/platforms/drupalnew',
);

in your ~/.drushrc.php

and you can then run any command against that platform using -

drush platform/1 command-here

I want to make it so that instead of maintaining one file, provision maintains an alias file for each server, platform and site in it's network, so we can make use of this from hosting.

This also gives us the place we need to store the server level information we need, not only about the current server .. but about the remote servers that we need.

For example, when thinking of the file service .. each web front end will need access to some place to store the backup files. This will allow us to store the necessary details on the remote side to make the connection to wherever FROM that side, without having to pass all the information along every request.

i think we are going to need to get used to the idea of implementing globally unique short_names for each of the components of the system too. Referring to NID's is going to get old.

adrian’s picture

short names become useful for the following -

drush sitealias provision-clone platformalias [newurl]

where sitealias could be on one server, and platformalias on another.

we can also write commands for hosting , such as

drush aegiralias hosting-add site url --platform=shortname --db_server=shortname 
drush aegiralias hosting-add task migrate sitealias --platform=shortname

and so forth.

anarcat’s picture

So yeah, site aliases are cool, but I'm not sure how it relates to the creation of drushrc files. Each server will need at least one of those files to store server-wide settings.

Are you saying we should have one drushrc.php file per server on each server?

If so, I don't necessarily object, but at any case, we need to start somewhere, and I think the branch should start by reproducing current functionality without any significant regression or crap in the upgrade path. I think all that is missing for that now is fixing the ports stuff and writing a single drushrc.php file. I chose the "user" context since it wasn't used yet, but the real context should probably be "system".

Let's keep the server alias discussion separate otherwise this will never get through, I think both features are orthogonal and can be developed separately or at least in stages.

adrian’s picture

Yeah, i started splitting it out into separate issues, and it's not a blocker on the server verify task.

#723288: Introduce the concept of 'shortnames' for all major entities.

Part of the problem we have is needing to keep multiple server's info concurrently already. ie: master db connection.

drumm’s picture

Status: Needs work » Needs review
FileSize
674 bytes

This has been merged into the dev-services branch. The only issue I've found so far is attached.

adrian’s picture

Status: Needs review » Needs work

merged the changes into dev-services, but the 'server' task was lost in the process.

I extended the PROVISION_CONTEXT_$X mechanism we use at the emoment to include a server context, and dropped the bootstrap level of the provision-verify command to only bootstrap drush.

I added configuration file classes for writing to $home/.drush/drushrc.php, and i additionally modified the verify task to write all the server specific information to the new configuration file, and the platform verification to remove it.

So this leaves us in a situation where the values need to move from one config file to another, and at the moment we do not have the benefit of the front end to guide us.

You should be able to do :

 cd /var/aegir
 drush --config=/var/aegir/platform/drushrc.php provision-verify --debug

and then follow that up with :

  cd /var/aegir/platform
  drush provision-verify --debug

The first will use the existing platform configuration file to bootstrap the values that need to go into the server drushrc,
and the next will remove the duplicated information from the platform level drushrc. (all that's left now is "publish_path" and "packages").

The first is redundant, and the second could be more elegantly handled (fwiw).

adrian’s picture

The idea behind the aliases is , is that we make provision/drush disregard the current environment entirely, and we can set the context by specifying it.

so :

drush @server_localhost provision-verify
drush @platform_atriumb5 provision-verify
drush @mysite.com provision-verify

so there's no confusion about what the current context is.

adrian’s picture

we're going to need to modify hosting for this too btw.

drumm’s picture

drumm’s picture

FileSize
377 bytes

This patch gets web-only sorta working. It is still trying to do the db actions, how should hosting communicate the type(s) of server?

drumm’s picture

This is moving forward on dev-services. Web servers get aliases written out, but on the remote server. It should change to
1. Write the alias
2. Use the alias in a sub-command to connect to and verify the server

adrian’s picture

Status: Needs work » Fixed

Done and in HEAD.

Status: Fixed » Closed (fixed)
Issue tags: -aegir-0.4, -multiserver

Automatically closed -- issue fixed for 2 weeks with no activity.

  • Commit cf2ddde on debian, dev-dns, dev-envobject, dev-koumbit, dev-log_directory, dev-migrate_aliases, dev-multiserver-install, dev-newhooks, dev-nginx, dev-ports, dev-purgebackup, dev-restore, dev-services, dev-simplerinstaller, dev-ssl, dev_server_verify, prod-koumbit, dev-ssl-ip-allocation-refactor, dev-1205458-move_sites_out_of_platforms, 7.x-3.x, dev-subdir-multiserver, 6.x-2.x-backports, dev-helmo-3.x by anarcat:
    #586000 - create a server verification task by moving all non-specific...
  • Commit ba5c144 on debian, dev-dns, dev-envobject, dev-koumbit, dev-log_directory, dev-migrate_aliases, dev-multiserver-install, dev-newhooks, dev-nginx, dev-ports, dev-purgebackup, dev-restore, dev-services, dev-simplerinstaller, dev-ssl, prod-koumbit, dev-ssl-ip-allocation-refactor, dev-1205458-move_sites_out_of_platforms, 7.x-3.x, dev-subdir-multiserver, 6.x-2.x-backports, dev-helmo-3.x by drumm:
    #586000