Maintaining live (production) and development versions of large dynamic site
Dear all,
I've asked this in several other forums and thought I would bring it up here.
Our situation is that we are moving a large, multi-user social networking site into private beta. While we wish to have a "live" version of the site, with content creation etc etc etc, we wish to maintain active development on several aspects of functionality.
In general, developing functionality, modules, etc, will require modification of the attached database, creation of nodes, etc etc etc on the "production" version of the site--
at the same time that users are adding nodes, content, etc to the "separate" live version of the site.
This would seem to make the advice "record and replicate what you've done" a little impractical. If you design a series of modules and nodes, replicating the node/content is not necessarily trivial. If you design portions of the site based on the node structure (nids or tids etc), and those numbers are taken by the action of users on the "live site," replicating the work can clearly NOT be trivial in many situations.
How do people handle issues such as these in "live, dynamic production environments?"
One solution that comes to mind is to link both live and production versions to the same db, with very frequent automated backup of the db, such that "if anything breaks" because of db changes, the db can be quickly reverted to a previous state (and the broken db etc then inspected). Is this practical?
What other strategies are out there?
Thanks,
~kwt
SVN
On a current project we're using SVN.
The trunk is the development site and we have a branch for the live version. Changes are merged to the branch when stable. We have a seperate database for each site.
A fair amount of content was added to the dev site before it was branched and went live (enough to populate each site section). At times I've needed some additonal test content - that is no problem as I can just create it without worrying about what is going over on the live site.
We have used vids, tids & nids as you describe, but these were all set in stone before the site went live.
I have the same question
subscribing. I see this as a real problem.
Example:
Let's say you want to add a new CCK node type that uses a field type already in play for just one other CCK node type. The CCK module will split that content field out into its own table in your db. How do you manage this sort of change across a dev and a production server?
-s
Not sure if it could work
Not sure if it could work for you, but on my sites whenever we are developing new functionality/features/design we create a mirror of the site based on a snapshot - then set up a team (of users) that has access to it to help test.
The test install is nearly always on a new domain.
This gives us 100% stability on the production site (as it's untouched) and we get what we want, a safe testing platform.
So, your push process is manual?
That's great to hear (as that's my current practice). But it drives me batty that when I want to push the new modules to production, I have to go back through the Drupal site admin interface to duplicate the settings I have just adjusted on my dev instance.
Take a real world example: I just upgraded the gmap module for Drupal 5.0. The newest version of the module incorporates additional features that were available in 4.7 but didn't get released with the initial versions for Drupal 5.0.
Here's my process:
1. Mysqldump production db.
2. Over-write my development site's db with my production site msyql dump file.
3. Install the new module code on my dev server.
4. Reconfigure the module's admin settings on the dev server via the admin interface on the dev site.
5. Write down - by hand - the module setting changes I've made on the dev site (so that I can replicate them on the production site)
6. Push the module code changes to production.
7. Log into the production site, replicate the module setting changes I've just made on the dev site.
This just seems like a lot of work. And what if I don't configure the module settings just the same way on the production site as I did on my dev site?
I really love Drupal, but until I can convince my shop that there are solid ways to test and deploy modules from dev to production, my admittedly purest Perl and Rails cohorts see Drupal as sorta a quick hack.
-s
In the same boat
We're dealing with the exact same issue. Has there been any progress or at least flashes of insight between now and the start of this thread?
autopilot
check out the work on autopilot. It's not ready for production as far as I know, but when it is, it will rock.
....
quiet similar issue with me too!
There's a g.d.o. group all about this very topic
Check out http://groups.drupal.org/change-management-systems cause it was started by the guys who wrote AutoPilot. I'll be posting a followup thought about this thread over there too. I have an idea...
[/Senpai]
Same issue here
I currently have a site with blogs and forums, dynamic features which I have just added. My development (dev) site is on my laptop, which is running XAMPP. When I make changes to my dev site, I export the entire database from the dev server using PHPMyadmin, then import the database into the live server using PHPMyadmin. I transfer any files necessary, like new module files, images, tweaks to theme files, etc..., from the root directory of the dev server to the live server. If I am unsure that I know all the neccesary files that need to be transfered, then I just drop all the drupal files from my dev root directory into my live root directory. This way I can make changes to the dev site and then dump them live when ready without working on the live site, and making it unstable while live.
Now that I am adding dynamic content I see three problems with my production system. Users, forum content, and blog content that are added by users to the live site while I am developing the dev site will be overwritten when I update the live site with my dev version.
I have thought that I could not export from dev and import into live the forum and user tables, thereby updating the site info other then forums and users. Would this work, or will I encounter problems with other tables that interact with these tables. For example, it seems as if the node table interacts with the forum table to coordinate nid and vid entries. Also, I don't know where blog info is stored in the database, and how to deal with that content.
Finally, I would like to introduce a staging site into my workflow. This would be served from a password protected subdomain on my live domain, and would allow me to test changes on my live server over the internet before moving them to the live primary domain site. My strategy for this was to change my database name by version, tweak my settings.php to dial into the newly named db, test it on the subdomain, and when ready move the newly named db and all drupal directory files to the live primary domain, thus replacing the old site. Again, this method would overwrite dynamic content generated on the live site during dev and staging development and testing.
Any advice or strategy sharing would be greatly appreciated.
Thanks in advance,
code
Couldn't you use a diff / sync tool?
Have you guys tried looking at something like MySQL Sync from SpectralCore? http://www.spectralcore.com/syncdatabase/index.php It lets you do a diff between DBs and then select what you want to merge.
Here are some related
Here are some related threads:
http://www.dave-cohen.com/node/1779
http://drupal.org/node/181128
http://groups.drupal.org/node/786
The first details my personal approach.
This thread has inspired me
This thread has inspired me to suggest a BOF about this very topic at DrupalCon. If you're interested please vote and/or comment here:
http://boston2008.drupalcon.org/session/updating-and-upgrading-live-site...
Thanks.
Trackback:
Trackback: http://codebaboon.com/method-syncronising-database-changes
I propose a fairly complicated but workable solution on my blog. I haven't really tested it, but comments are welcome!
__________________________________________________________
Mark Theunissen
Code Baboon
Drupal based services
p.s. the solution uses Toad
p.s. the solution uses Toad for MySQL, and odd/even database id numbering.
__________________________________________________________
Mark Theunissen
Code Baboon
Drupal based services
This saturday a video broacasted session about staging
This saturday will be held a session about staging. France24.com team will explain there odd and even numbers solution.
That way we will catch up with BOF session at last Drupalcamp when we were not able to engage a dialogue across the ocean because the sound was not working.
More information on https://barcamp.pbwiki.com/DrupalCampParisStaging
Fell free to edit the page and tag the video.
We are considering around 3PM French time.
As there is time differences could you please tell me when at the earliest you would be available ? on the comment.
Subscribing...
Subscribing...
Tecito.com
The growing Hispanic community in Canada.