Add the ability to push deployment plans without user input (for instance from cron)

Right now this is not possible, because in order to maintain security, users must enter the remote server's username and password on every deployment. In order for this to happening another authentication mechanism would have to be used, ideally oAuth but another SSO implementation would work as well. There is a lot of interest in getting this done and I would expect it by the end of the year.

The module today is reasonably complete. One large gap is that it does not handle transferring attached files with nodes, due to a patch to Services that has not yet been committed and which I have not had the time to finish. It also does not handle some areas of Drupal core (most notably forums) and some popular modules. I would like to re-architect it to make it a bit more pluggable in various areas as well.

I'm leaving this issue open as a place to track the feature of unattended deployments.

I'm very interested in this.
What I'd like is a a system with hook_nodeapi so the nodes in a deployment plan automatically deploy themselves when updated.
This would require plans to have a setting to assign one or more servers to them, and a setting to say they are automated.
That side of things I can help with.

The oath stuff I know nothing about -- I'm guessing I'd need to know not only about oath but also about how services module uses it? or would work on services module be required too?

From a functional standpoint this is easy. The problem is how to securely authenticate against the remote server without entering a password or saving the password anywhere in plaintext. If this problem can be solved then the rest is easy, but I don't know a way to solve that problem at the moment without some kind system like oAuth integrated into Services. I'm open to suggestions though.

Subscribing :)

Another problem that will appear when trying to do this during cron is the batching of deployment operations. The Batch API needs support for javascript or at least meta-refresh by the client that executes the cron job. And that isn't really possible in most cases.

One solution (and a really good one) would be to add support for the backported version the Drupal 7 Queue API here: Drupal Queue for deployment operations.

Okey, I'll try to tackle this a bit today and see if I can come up with a patch.

My plan is to make use of the new #642584: Pluggable authentication for Deploy and implement an oAuth solution that heyrocker mentioned above, for security reasons. Then, on a per content type basis, you will be able to specify which servers deployment will happen to on insert and update. But we still have a problem - the dependency on Batch API and the design decisions behind oAuth. This will make it hard for deployment during cron. I'll leave that for later.

How does that sound? Shall I take another approach?

That sounds great. Have you seen Hugo Wetterburg's oauth modules?

http://drupal.org/project/oauth_common
http://drupal.org/project/services_oauth

I highly recommend them over the one maintained by brmassa.

I have a long plane flight on Sunday during which I plan to start digging into the authentication patch.

Yes, I've tested Hugo's oAuth module and we are heading for that solution!

I've attached the first take on this issue below. This isn't really "without user input" but instead you are just redirected to the node_operations/deploy_now callback on node save where you can select which server to deploy to. With the patch #642584: Pluggable authentication for Deploy that could be a server with any type of authentication. Not just oAuth. The default settings for deployment on save are configurable on a per content type basis, but could be overridden for each node.

But I have one problem, though. I can't find a way to handle new nodes. Are the nid even possible to grab on form submission of a new node?

Some things that still must be implemented in this patch:

• Implement hook_uninstall() to remove introduced variables.
• Document the functionality

After #642584: Pluggable authentication for Deploy has gone in (which I hope it will, when it's ready) should we ship deploy module with an oAuth implementation, or should that go in a separate module?

Nice job dixon-underscore ;) I'll definitely be keeping tabs on this development. I saw that you have a github account, you're now being followed.

Cheers,
Hugo

In the discussions over at #642584: Pluggable authentication for Deploy we are talking about storing some authentication credentials (maybe only for some authentication methods) so a totally automated workflow for deployment can happen. But that would only solve one problem for this issue. The other problem we need to solve is batching. Automatic deployment can never happen if we are stuck with the Batch API as that batching method requires javascript or at least meta-refresh for the client (browser) that executes the task.

The solution to this would be to implement a pluggable batching method or swap out the Batch API completely. Either way, the solution could be the backported Queue API from Drupal 7. I haven't tested that yet, but I might think this could work.

Maybe a pluggable system is the way to go so one could set up this on a per server basis to match your needs.

Right now there are API calls such that we can easily do non-batched deployments (I originally wrote it this way and then added Batch API.) It looks like this:

  deploy_init_deployment($pid, $sid);
  deploy_set_session($user, $pass);
  
  deploy_plan_check($pid);
  deploy_plan($pid);

  deploy_plan_cleanup();

So if we wanted to go that path we totally could. The one problem with this is that you end up running into timeouts on any deployments of any size. I like the queue API idea, but I don't know if I want another dependency. I'll look into it though, we could maybe make unattended deployment a submodule so that it's optional.

Subscribing to this amazingly cool issue.

seems like the backported Queue API and the oauth modules is the way to go...

I've tested my patch from #9 again. And it still applies and work as I described above.

But I still need a way to grab the nid when saving the node for the first time. Otherwise, deployment can only happen right after updating a node. Not right after saving new nodes. I'm afraid that we can't use hook_nodeapi()::insert for this. Because a redirection to the deployment form during that operation will break other modules reactions on the same hook.

Anyone that got an idea on how to solve this?

The docs are unclear (#543836: Clarify $ops paramter options for hook_nodeapi()) but 'insert' is called once the $node has been saved -- so you have a nid by that point.

@joachim As I mentioned in my comment I'm not sure we can use hook_nodeapi()::insert here. A redirect would mess up the visitor pattern happening there. So all modules wont be able to react on hook_nodeapi()::insert.

So I think we must find another way around this.

hook_nodeapi being monolithic (while we're on D6...), we could set a static flag in the 'insert' op, and then act when we're back at the view op for the edited node. Though that won't work for nodes edited from content admin, or Views. So we're possibly looking at a slightly hacky variable_set() that is then checked on the next page load, or next cron.

Yeah, that is one possible way to solve it. Although it could lead to problems with race conditions if we are doing it wrong. The problem with having one "global" variable when multiple users are editing the site is that they can overwrite each others deployments.

Another way is to prepare the plan in hook_nodeapi::insert and then register drupal_goto('whatever/batch/callback') with the PHP function register_shutdown_function() which PHP runs at the very end of the request. The problem is that I don't know if that would work, and it's not very "Drupal-ish".

A variable_set() of some kind is probably the right way to go?

Hmm unless we go the whole hog and just have a table that's:

uid | plan | nids (serialized)

or whatever we need to deploy the plan. I'd not thought of race conditions; that could get nasty.

FYI people interested in this may want to check out

#795408: Drush deploy action

The Drush support has been committed, however it still doesn't have the ability to take everything needed from the command line, which I think is what is needed to put this issue to rest.

Marking this as duplicate in favor of #893828: Expand Drush actions to allow unattended deployments which is about to be committed with support for completely unattended deployment.