Automated database migration from test to production

ashearer - July 18, 2007 - 00:35

I'm working to achieve a test server + production server setup with Drupal. With code, it's easy. Just make the changes in test, then copy to production when they're ready (via a Subversion repository for extra credit).

However, database changes are problematic, and just about any development that involves clicking the Administration menu consists of database changes. The best practice guidelines on drupal.org and other sources suggest either (1) keeping notes on all activity in order to repeat the exact same sequence again on prod, or (2) copying the whole prod database to test, making all the changes, then copying it back. The former technique is time-consuming and error-prone. The latter technique loses any comments, posts, user registrations, and other changes made in the interim. (Unless the site is brought down during the changes--but a major purpose of trying changes out on the test server in the first place is to keep them from bringing down the prod server.)

Has anyone thought about making a script that copies only the configuration-related tables of the site to production, while leaving the content alone? Doing so would made configuration changes as easy to replicate as code. Certain actions would need to be performed twice, such as adding new CCK fields, because the database schema of the content changes. But many changes including variables and menu items should be transferred.

I've actually written a first cut at this, after categorizing a lot of tables. But so I know better what I'm getting into, does anyone else have experience with this kind of setup? Do any problems come to mind?

See AutoPilot

Boris Mann - July 18, 2007 - 01:16

See http://workhabit.org/projects/autopilot -- code release forthcoming...

--
The future is Bryght.

subscribing http://drupal.org

ulfk - July 18, 2007 - 03:03

Subscribing

Quint - July 18, 2007 - 03:31

Subscribing

MacRonin - July 18, 2007 - 04:09

Subscribing

-------------------
http://www.PrivacyDigest.com/ News from the Privacy Front
http://www.SunflowerChildren.org/ Helping children around the world

subscribe

Chris Herberte - August 13, 2007 - 03:52

you say there is some code for this?
http://www.xweb.com.au

subscribing

Caleb G - oldacct - August 20, 2007 - 21:53

Project committed

Boris Mann - August 21, 2007 - 17:30

Boris, do you have 'inside

-Anti- - July 20, 2008 - 12:41

Boris, do you have 'inside information' about this project, then?

Because the last information update by Earnest Berry was June 2007, over a year ago. And there has only ever been one development release (Aug 2007) http://drupal.org/node/169057/release. So it appears like a completely dead project to anyone on the 'outside'.

If drupal developers found a way to make development updates to a live site without closing the site down for the weekend, or having to manually reproduce and apply dozens of changes to a live site, I think we'd all be very happy, but I think it is probably a bit of a pipe dream?

Deploy

Boris Mann - July 21, 2008 - 20:56

It doesn't like like Earnest / WorkHabit have had the time to finish this off. The Deploy module is a piece of the puzzle, but doesn't solve everything.

--
The future is Bryght at Raincity Studios

sub

Caleb G - oldacct - August 24, 2007 - 15:52

===

How do you do this manually?

Athaclena - July 20, 2008 - 11:19

Automatically would be great but I'm having enough trouble doing this manually - specifically any new nodes I need to migrate are overwriting anything my users have written in the forum.

Does anyone do this without losing info and if so how?

> Does anyone do this

-Anti- - July 21, 2008 - 21:49

> Does anyone do this without losing info and if so how?

As far as I'm aware, there is no smart way to continue radically developing a site once it is live. After you've tested the changes on a dev site, you simply have to close your site so no new information is added by users, manually apply your changes, and re-open the site.

It's a bit of a holy grail to be able to make substantial developments without shutting your site down. It would take really smart merge/sync software to do it without error.

Thank you for your reply

Athaclena - July 22, 2008 - 08:17

I have a bit of a special situation since the ONLY things users can do is write in the forum. A non-admin can write forum topics and comment on them - nothing else.
So it's probably a little simpler that most other cases would be...

I was thinking maybe to manually set sequences.node_nid to something like 10000 or more, which is likely to sort out most of my troubles... I'd just need to merge the testing in to the production somehow.

The idea of having to manually reapply changes makes me cringe... it's bad enough when import/export is a possibility (workflow-ng, CCK).

Long discussion by heyrocker about deploy

Boris Mann - July 22, 2008 - 18:38

http://heyrocker.com/drupal/content/deployment-and-change-management-wha...

He has a presentation somewhere talking about shifted sequence tables and various other potential options and their related pitfalls.

--
The future is Bryght at Raincity Studios

Thank you

Athaclena - July 23, 2008 - 06:53

I didn't come across that before. Very helpful!

 
 

Drupal is a registered trademark of Dries Buytaert.