I've been using and developing Drupal sites for a year and a half now. Every time I need to do something like design a form, write a module, alter a theme, I am amazed with the simplicity of design and forethought that has gone into things like the theming system, hooks, form api, database api etc. There just seems to be one glaring deficiency that I can't seem to understand why more people aren't encountering: deploying data between a development environment and a live environment.
With Drupal, it is really easy to develop version 1.00 of a website. But once the site goes live and people start making product orders, posting comments, submitting webforms, posting forum topics, etc. 24 hours per day, it is no longer to feasible to take the site down for a few days to upgrade to version 1.01. It also is not feasible to do development on the live site, where, for example, a missing semicolon in a module you're developing causes the site to crash for everyone.
There just doesn't seem to be a good way to say "take this predefined set of system settings, nodes, content types, modules, and themes and deploy it to the live site." The only way to do this seems to be to bring the live server down and then have one screen with your development server and another with your live server and copy the settings over one click at a time, making sure to expand all the collapsible fieldsets so you don't miss a check mark. This is a time-consuming and error-prone process requiring extended downtimes.
Writing a script aided by drush helps, but is only part of the solution. The deploy and services modules are certainly a step in the right direction, but they seem only suited to simple content. To start with, both modules are still -dev versions, so they haven't really achieved stability. Deploy doesn't work at all if you have the upload module enabled. It is purported to work with fieldfields, but I have seen frequent duplication of content. I have yet to get it to work with nodereferences.
In the past, whenever I felt like I was pushing a boulder uphill, I would stop and review Drupal best practices. If something was exceedingly hard, it was because I had diverged from these best practices and was blazing my own path. I can't help thinking that I'm doing something wrong now.
So: What is the recommended best practice to develop version 1.01 of a web site, assuming that version 1.00 is operational and changing frequently, so you can't just take a snapshot and develop on that?
Comments
Hi, have you tried features
Hi, have you tried features (module) ?
M.
Freelancer Senior Drupal Developer -- http://www.ziobuddalabs.it
Thank you
I wasn't aware of the features module. I just looked at it and it's one more tool I can put into my arsenal. It still seems that there isn't one organized solution for dealing with this issue, but we're getting closer.
hm
I completely agree with the OP. It would also be convenient from a logistical point of view to have the ability to sync with another deployment of drupal. For those of us that develop locally and could benefit from being able to 'publish' from our local dev install.
It seems we are missing a small suite of tools which would allow us to manage Drupal deployments. Though I often think of these grand problems, then read a post a day later detailing how I really should be working/thinking about a 'problem' and the solution has been cleverly engineered in 2 years ago ;)
Configuration management in
Configuration management in Drupal is a big problem that lots of people are working on. But, in all my searching, I haven't found a good way yet. Fortunately, Drupal 8 looks much better. See http://buytaert.net/configuration-management-in-drupal-8
Oh, and Features is pretty cool, though it doesn't back up anything that could be considered content. Like if you are using taxonomy terms as a way to organize things, it won't back up your terms.
What kind of scripts have you written using Drush? I've considered doing that, but haven't yet. So it would be nice to see what you've done.
git, features, context, drush...
Features is great as it allows you to put all kind of things into code, which you can put into your git repository.
Context helps you to arrange all our blocks and other stuff and you can put it into features as well ;)
Drush helps you with the general drupal settings.
Git is very useful as you can have your development branch for your own server, a stagging branch for a stagging subdomain and of course a production branch or master where you can tag your releases. On their server then you can just clone the production branch for the first release. For future releases you can manually pull the changes or you can use one of the git hooks to kick off a shell script after you pushed your change to automate the process. (http://progit.org/book/).
For the module changes, I'd use drush.
I like it this way, but if that doesn't work for you, checkout getpantheon.com. It might be overkill for small projects, but they definitely have good solutions for deployment. It's in early stages, but keep an eye on them.
Cheers,
Johannes
You might also want to check
You might also want to check out the Deployment module. http://drupal.org/project/deploy
Danny Englander | Twitter | Instagram
I've had some success with
I've had some success with the "backup and migrate" module. It's a good way to copy database changes.
The backups are nice too of course.
Concurrent databases
It will be good if Drupal can persist multiple databases concurrently. i.e. live_db, and dev_db
This way, we can work on the development site (which is connected to dev_db), and at the same time allowing new content to come through from the live site, and indirectly avoiding messing/crashing the live site.
i.e. live site will be updating 2 databases - live_db, and dev_db
dev site will only be updating the dev_db
both sites can have a shared-code-base
Is this achievable at the moment?
If you're using MySQL - yes.
If you're using MySQL - yes. There is an option to set up a master-slave DB replication, exactly as you described. See e.g. http://dev.mysql.com/doc/refman/5.0/en/replication.html. All changes to master (live) are applied to slave (dev) at the same time, but not the other way around.
Another possible solution for MySQL is using phpMyAdmin synchronization feature (http://www.phpmyadmin.net/documentation/documentation-gsoc/Synchronizati...).
However, I haven't used it yet and I am not sure if it just mirrors the tables or *merges* the diffs like in SVN or Git.
Good idea is to exclude some tables like the ones with cache.
phpmyadmin synchronization
I had a quick look at the phpmyadmin synchronization. It says it can sync between local and remote, but what I discovered is that it will only sync local-local OR remote-remote. It was also 1 way as it would DROP the tables on the destination and then recreate them. There were NO changes made to the source database.
I also tried MySQL Workbench. It will sync 2 way 2 database tables but NOT THE DATA IN THE TABLES.
Of all the users Drupal says they have there must be someone that has figured out how to sync 2 websites...
MYSQL Workbench
Hi,
Has anyone looked at the FREE MYSql Workbench? On the website: http://www.mysql.com/products/workbench/features.html it says it can do sync from dev-db to live-db and vice versa.
It can be downloaded from http://www.mysql.com/downloads/workbench/ in various different versions, i.e. Linux, Apple or Windoze.
Have just started using it out so really don't any information other than the above.
Old question but unresolved problem too
We worked in this problem for at least a year. We are now , the next december 10th, presenting our results in Santiago de Chile, http://camp.drupalchile.org . Please, take a look to our tool, that allow drupal developers to push-pull content from dev to stage, stage to prod and viceversa. Please, meet us www.drumine.com