Background

We are currently working on an integration between Drupal and a java desktop application called Newspilot. Newspilot is the leading application on the Swedish market when it comes to editorial workflows, archiving etc. for the newspaper industry.

When a journalist is writing something in Newspilot, they have the possibility to publish that article directly from the desktop to their Drupal website (in our use case a staging site) using Services. But we also wanted our staging site to be able to push changes back to the Newspilot server and the desktop where they work. So this is where the Deploy module becomes handy!

The problem

At the moment Deploy can only authenticate on a remote server with a valid session id, a username and a password. That makes total sense when deploying to another Drupal site, but not if we deploy to another system. Also, lets say we want more security when deploying between servers and want to use API keys or OAuth functionality. The Services module supports different authentication types on the server side. But the client (Deploy module) doesn't have a pluggable system for switching between authentication types.

The solution

Make the authentication functionality in the Deploy module pluggable!

What I've done is to remove any assumptions on how Deploy wants to authenticate on the remote server, and instead have that functionality defined in different implementations. These definitions are made in a new hook called hook_deploy_authentication_info() which works similar to Services' hook_authentication_info.

So, when you add a new server for the Deploy module, you will be forced to select which authentication type this server uses. This is saved in the table deploy_servers in a new column called auth_type.

Later, when you select server for your deployment the correct authentication form is loaded through AHAH (like the Services authentication form). How this form is structured depends on that specific authentication type. It can be textfields for username and password or just one textfield for API key.

With this patch, the Deploy module ships with three different authentication types:

  • API key (when using services_keyauth.module with Use keys)
  • Valid session id, username and password (when using services_keyauth.module with Use sessid)
  • API key, valid session id, username and password. (Not implemented in this patch yet. But will be used when using services_keyauth.module with both Use keys and Use sessid)

No usernames, passwords or API keys are currently stored anywhere with exception to the API key that is stored in a variable through all batch operations, but is removed in the cleanup callback later on.

About this patch

This is the first rough take on this problem. I've only patched the base system and node_deploy.module to test the functionality. This is still left to be done:

  • Small modifications to each entity module (taxonomy_deploy.module, user_deploy.module etc.)
  • Improve documentation of this new functionality
  • Update hook that alters the old column deploy_servers.api_key that wasn't used anywhere in favor of deploy_servers.auth_type
  • Separate the functionality in more files or maybe in new modules?

I will address the remaining issues in following patches.

Credits

This patch was sponsored by Infomaker and NodeOne.

Comments

gdd’s picture

Title: Ability to deploy to other systems than a Drupal site » Pluggable authentication for Deploy

This is totally great, thank you for it. I will try and take a look at it soon, although we are heading into a holiday here in the states and I will be out of touch until next week. I hadn't even realized I left that API key field in there, that is really old.

I'm really bummed I couldn't come to Sweden. Next year for sure, need another Kitten Killers gig :)

dixon_’s picture

Okey, great to see you like the idea! We will maintain and reroll the patch further here in the queue!

Yeah, too bad you couldn't come to Sweden! The Kitten Killers will definitely not disappear ;) Who knows, maybe Kitten Killers will be on stage in San Francisco next year!?

gdd’s picture

StatusFileSize
new21.46 KB

So, first off, this functionality is really excellent, thanks much for this patch. I did some testing on my local deployment setup and it worked right out of the box with session id functionality and no changes, which is cool because it won't break existing installations. I did have to add an update function for the table changes, and I also added the ability to specify a domain in the key auth settings. SERVER_NAME will change between environments and a lot of people use a hardcoded string for that, calling it 'domain' is really a misnomer since it can be any string. New patch attached.

The nice thing I realized about this is that using api keys we can actually have unattended deployments NOW. It may not be the ideal situation for them but it would work. We do need a way to store the api key information permanently, but we could add a setting for that in the deployment settings form.

dixon_’s picture

Nice!

Yeah, unattended deployment would be possible with some tweaks. But I see the security risk with storing API key and domain key equal to storing username and password as those are the combination of keys that gives you access to the service. Although, username and password are more serious as those are sent with the request each time, and could give you access to a whole site...

But, for making this possible I think settings like this should be stored on a per-server basis. Maybe add a data column to the deploy_servers table where we could store whatever suits the authentication type best (like API keys etc.). To do so, some AHAH sweetness should be added to the "add server" form, so the necessary data fields adds to the form when selecting the authentication type.

dixon_’s picture

Just noticed a small spelling error here:

+/**
+ * Implementation of hook_update().
+ *
+ * Add new auth_type field for pluggable authentication 
+ * and drop unused api_jey field.
+ */
+function deploy_update_6003() {

I'll give it a deeper look soon.

gdd’s picture

StatusFileSize
new21.46 KB

No you're totally right about storing the key and domain in plain text, that's a bad idea.

New patch with typo stupidity fixed.

dixon_’s picture

StatusFileSize
new30.29 KB

After some some consideration I think that storing the settings for the deploy_key implementation in plain text is ok. After all, that is how it usually works for other modules/applications. The API key isn't passed with any request and aren't exposed in an insecure fashion. Look at the Mollom module for example.

Two new features are introduced in this patch which is based on #3 with the spelling fix:

1) The form where servers are created now has the possibility to save settings specific to the authentication type (API keys etc.). All these settings are saved serialized in a new column in deploy_servers.settings. Note that saving settings is optional! It's up to the user to decide whether or not to store possibly sensible information in the database.

2) Both deployment callbacks (admin/build/deploy/push and node_operations/deploy_now) can now take an additional argument - a server id. If a server id is passed, and the server has stored settings, deployment will happen immediately with these settings. I'll leave the actual implementation of this to #537488: Add the ability to push deployment plans without user input (for instance from cron) after this is committed.

Along side of these changes I also renamed some functions relevant to this issue, to have better consistency.

Additional features we could consider:

3) Add a flag to each authentication type to tell if the settings are storable or not. We could flag the deploy_sessid implementation as not storable. Maybe this should be done on a per-setting-level, so we could store username, but not password?

The patch still needs some love.

dixon_’s picture

When (if) we get this committed I'd really love to see a cleanup/renaming issue to get better consistency over function names, and menu callbacks etc, before a stable release. The code is starting to get quite complex.

gdd’s picture

Status: Needs review » Needs work

Just FYI I recently committed the api_key fix as a separate patch so this is going to need a reroll

dixon_’s picture

StatusFileSize
new29.67 KB

Here is a reroll that applies to DRUPAL-6--1 with some adjustments and a little better documented.

dixon_’s picture

After the IRC discussion yesterday with heyrocker I decided to removed the experimental storage possibilities for authentication settings. Obviously because of the security implications this will have. So this patch is a little more lightweight.

I don't know how strict conventions are about breaking function naming during 1.x-dev development. But I've tried to introduce a little more consistent naming throughout the deployment flow.

Basically, the flow looks like this:

One module can declare one or more authentication methods. The declaration looks similar to the array in hook_menu().

  • deploy_plan_push_form() - From for selecting the server. Invokes the "form callback" for auth modules and appends server settings (username, password ord api key and domain name) via AHAH.
  • deploy_plan_init() - Is called during the form submission above. Initiates depoyment. Also invokes the "init callback" for the server's auth method. Session auth method will connect and log in to the server at this stage. The key auth method doesn't need to do anything.
  • The form redirects to the Batch API that invokes the "arguments callback" in every operation. This populates the XMLRPC request with appropriate arguments for the method call.
  • deploy_plan_cleanup() - Is the last function that will be called during batching. This also invokes the "cleanup callback" which lets the auth method cleanup after it self.

Because of lack of time, and the recent changes to the patch I haven't got the time to implement this for other sub modules than node_deploy.module. I will continue work on this next week!

dixon_’s picture

StatusFileSize
new27.01 KB

And here comes the patch...

dixon_’s picture

StatusFileSize
new28.75 KB

Here is the first complete patch! With this node, comment and taxonomy deployments should work with the new pluggable authentication system.

dixon_’s picture

StatusFileSize
new27.86 KB

Here is a new cleaner and better performing patch! I also added more flexibility to the system by making it possible to alter authentication types.

New hook introduced: hook_deploy_auth_type_alter()

dixon_’s picture

Status: Needs work » Needs review
StatusFileSize
new27.64 KB

Here is a new reroll that should apply with the latest code in DRUPAL-6--1. The important thing with these two latest patches, as mentioned in #15, is the introduction of hook_deploy_auth_type_alter() that let modules alter authentication types.

gdd’s picture

StatusFileSize
new27.97 KB

I would like to get this in so I took another look this weekend and rerolled the patch for head since there have been a few changes committed over the weekend. Can you give this a run through and let me know if it's working right for you?

It might be nice to put the auth info out into includes by auth type (deploy_auth_session.inc, deploy_auth_key.inc) but I'm not really going to sweat that right now.

dixon_’s picture

Wonderful! I been very busy the last few months but I'll give it a look before I head to Copenhagen this weekend!

dixon_’s picture

Status: Needs review » Fixed

I've done some extensive testing with this today, and can't find any problems with it.

I've tested deployment with...

  • both authentication methods (sessions and keys)
  • nodes (with various dependency modules like nodereferences etc)
  • users
  • comments
  • views
  • taxonomy (both as a dependency for nodes, but also single deployment of a vocabulary)
  • system settings
  • content types

I've also tested the upgrade path. And the auth_type column gets successfully applied and session authentication is set as the default for all existing servers... Man, we need SimpleTest for Deploy 2.x :)

So, everything seems to work as expected with your reroll from #16. Patch committed. Yay!

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.