0.4 - automate platform creation

I have done *some* work here by simply adding some help text that tells the user how to create a new platform himself. No automation, but self-documentation at least.

drush_backend is in and working, and all drush commands can call each other.

What's needed now is the bits added to hosting to create release nodes from the drupal.org update repo, and passing the
platform version parameter to the provision verify task.

This is all pretty simple, so we're almost there.

drush_backend is in and working, and all drush commands can call each other.

What's needed now is the bits added to hosting to create release nodes from the drupal.org update repo, and passing the
platform version parameter to the provision verify task.

This is all pretty simple, so we're almost there.

We have decided that we should leave this for the 0.3 release.

While we would like to do it now, we simply do not have the right meta-information available to us. It does not help the user if we simply download drupal core itself, but not the right versions of any of the packages that his site needs to function.

So for this to be actually useful, we need to do some work on Drupal package management in general. I've written down some notes about what I'm working on to solve this here : http://groups.drupal.org/node/21295 , but this will not be ready for a timely 0.2 release, so to stop the 0.2 cycle from taking forever, we need to re-focus our efforts.

Marking as aegir-0.3

One of the things we should not forget here is: #529242: deleting a platform should remove the platform apache config. Also, should we completely delete the multi-site when we delete the platform? Maybe we don't want to allow platform deletion if there are still (non-deleted) sites on it?

Definitely, it shouldn't delete enabled sites when deleting platform, or should just refuse to delete platform with any enabled sites. Now it simply deletes sites together with its platform - without any warning - not a problem when you know what are you doing.

~Grace

As to "automate platform creation" - I believe it can be safely automated with well done Drupal distros like Open Atrium, Acquia or Prosepoint. But in the real world we want more customized distros and then it stops to be simple enough to be automated, IMO.

Two examples: try to use OpenPublish or MediaMosa as platforms - it will not work for different reasons, and can't be automated.

~Grace

I have attached patches to hosting and provision that create some basic platform management in Aegir.

Essentially we add a 'status' column to the hosting_platform table and this follows the same logic as the 'status' column in the hosting_site table.

Along with this, we add a Delete task for Platforms. A platform can successfully complete a Delete task if no sites are found in its sites/ subdirectory. If it finds any sites, it fails and rolls back. Sites must be removed or migrated off the platform for the Platform to successfully delete, and this is explained in the task log output.

On successful Delete task completion, it also deletes the platform vhost config per #529242: deleting a platform should remove the platform apache config .

The schema change to add the 'status' column sets the status for all existing platforms in hosting_platform to '1' which is Enabled. In the Platform view (hosting_platform_view) there is a field showing the status, similar to sites.

If a platform is Deleted, the status changes to HOSTING_PLATFORM_DELETED which is -2. Again, same as sites.

If a platform is Deleted, it doesn't show up in the Platforms summary (the block) nor does it show up in the site node form when adding a new site (because I modify _hosting_get_platforms() )

If a Platform is Deleted, the Delete task is no longer an option on the platform view (same as sites, except sites then show Enabled, and this is not applicable in the case of platforms).

Feedback is very welcome. Don't forget to run update.php to account for the schema change.

Attachment	Size
373062_hosting_platform_management.patch	7.35 KB
373062_provision_platform_management.patch	5.51 KB

*sigh* forgot to add the new field to the actual schema and not just the hook_update(). new hosting patch attached.

Attachment	Size
373062_hosting_platform_management_1.patch	7.65 KB

Ok tested this out and it appears to work as I would expect:
* Delete failed when it had sites on it as expected.
* Deleted when there were not sites as expected.

One problem I noticed is it wont allow me to add another platform at the old path... so I think when deleting the platform path should be also deleted since it doesn't exist anymore, or alternatively if you attempt to create a new platform at an old path on a delete platform it could just re-enable that platform node (ie: run verify again)

Also as discussed in IRC I think a profile disabled operation (or as discussed a better term for this task might be "lock" to avoid confusion between what site disabled means).

By locking a site it would prevent any new sites being added to the platform (e.g. add/clone/migrate), the use case for this is when you have a new release of a platform, but there are issues with migrating all the sites of the old platform. For security reason you probably want to prevent new sites being added to the platform but you also want to have full access/visibility of the platform to continue performing normal operations on the sites that are running on it.

One problem I noticed is it wont allow me to add another platform at the old path... so I think when deleting the platform path should be also deleted since it doesn't exist anymore, or alternatively if you attempt to create a new platform at an old path on a delete platform it could just re-enable that platform node (ie: run verify again)

Yes it's because the platform node still exists, same as when you delete a site, the site node still exists.

Interesting idea about 're-enabling' the node so to speak.

I do like the idea of a 'Lock' task on a platform - I'll implement this and reroll the patches.

Thanks for the feedback and the testing!

I've implemented Lock support, patch attached.

Battling trying to deal with adding Platforms and causing just a re-verify of an existing node if the existing node was the same platform but in a Deleted state, turning out not to be so simple.

Attachment	Size
373062_hosting_platform_management_2.patch	12.11 KB
373062_provision_platform_management_1.patch	9.02 KB

Sorry for the delay on this, must've got sidetracked.

This new reroll allows you to add a new Platform even if the old platform node exists, so long as that old platform node is a Platform that was deleted.

The provision patch hasn't changed, but I include it anyway.

Attachment	Size
373062_provision_platform_management_2.patch	9.02 KB
373062_hosting_platform_management_3.patch	13.1 KB

I tested the lock, unlock, delete task and they all worked.

I also attempted to add a platform with the same name as an old deleted one and it just re-verified the old node which is great.

Couple of minor issues:
* should the lock, unlock and delete tasks appear as tabs on the platform node?
* when you have no platforms enabled (ie: all locked), then when you attempt to add a new site it fail with an ugly php warning

I don't think that warning should hold this up though as its unlikely you will lock all your platforms.

Thanks for the testing!

* should the lock, unlock and delete tasks appear as tabs on the platform node?

In my testing, they did? Are you saying that they are not, for you? Or that you think they shouldn't?

Note that the recent mega-commits by anarcat related to the task refactoring as meant that what used to be tasks as MENU_LOCAL_TASK (hence tabs) have now become MENU_CALLBACK and hence are now meant to be included in the task listing and not tabs. I didn't account for this in my development since it began before the tasks refactoring and I'm not sure if I was meant to.

So are they appearing as tabs, and if so, are you saying they shouldn't, or are they not appearing at all?

I'll see what I can do about that corner case of having no unlocked platforms. Cheers!

I can confirm the platform management tasks are appearing as per anarcat's tasks refactoring and no longer tabs, see screenshot

Attachment	Size
platform_management.png	5.9 KB

Updated patch against Hosting.

I never saw the ugly foreach exception , however I did see the fact that no platforms were loaded in the form and hence I had an exposed 'hostmaster' and 'default' profiles. I re-read that, it wasn't a foreach exception like you said, it was a missing argument warning, and I did see that.

Updated patch should check to see if *any* enabled platforms were returned, and if not, print a drupal_set_message to the effect of 'You have no enabled platforms! Please add or unlock a platform before attempting to create a site.'

The form will still render, and submit is still possible.. obviously hook_validate won't let the site submit, but do you think the above is enough or should I more aggressively prevent the form from being submitted?

Attachment	Size
373062_hosting_platform_management_4.patch	14.52 KB

Ok, so i want automatic platform creation to be tied to drush make.

which means adding a new text field to platform nodes, that can either be a link or a local file to a makefile for the platform.

on platform creation we should have an install or make task, which then goes and rolls it out, before being verified.

It occured to me yesterday these patches of mine are spun out of #8 and not related to automatic platform creation at all. I'll create a separate issue and continue work on generalised Platform management in that maybe.

Sorry for the noise.

I am seeing what you see in #19, there are no tabs - I actually hadn't noticed that the tab were removed with the recent task changes on the sites, so thats all cool.

I think that message you added is sufficient, I think its fairly unlikely you will have all platforms locked anyway.

So what you have looks good to me.

Adrian has mentioned he'd like the 'Lock' idea moved into a more client permissions/access control area.

Since I'd like more discussion on this, and since I hijacked this ticket which was about 'automate platform creation', I've created a separate ticket explaining the state of things, and attached the two most recent patches.

For general platform management, please address in this ticket:

#602128: Platform management

Nothing for review here for automatic platform creation, my platform management patches are in #602128: Platform management