Hello,

I am in the need of a little guidance regarding version control for developing a site using Drupal6.

I have a small team of developers in the process of coding various areas of what is to be a fairly large site in Drupal6. (Actually, its an existing site running on an old, outdated CMS that we want to migrate to drupal so we can build on its core.)

The developers are coding custom modules from our existing code, expanding on already contributed modules, and setting some configuration options both in drupal core and for said modules. Each developer is using their own installations of drupal, as to not step on each other while they were learning, keeping in mind that one day (soon) all these elements will have to be combined to one master development installation.

I have read numerous posts on CVS and Subversion and Drupal, but I'm very confused as to what the best practices are.

I think one of my biggest concerns is the database and merging the configuration settings (and other values) each developer has set.
I realize that the migrating of content shouldn't be done until the basic infrastructure is sable, and it hasn't been. However, some "areas" of the site being developed rely on "container" nodes. Which, is technically "content". These needed to be created during development, so custom modules knew they existed and could talk to them.

Right now, I'm left with several sites that all have working elements of the whole. Various content types have been created on each installation, as their functionally called for.
What would be the best way to merge these databases together into one master site? Retaining all the content types, various config settings, and some "container" content.
Is this even possible? Do i have to redo these steps on a master installation?

And just in general, could someone shed some light onto some methods they have been using to manage development of a large site with multiple developers?
I'm starting to feel that Drupal wasn't the right choice for this project (although i love it and have come to know it intimately over the past few months)

Any insight is appreciated. Thanks!

Comments

Anonymous’s picture

I'm surprised that there hasn't been any comments. Is this a really tricky topic? This kind of makes Drupal not so attractive if there isn't a nice and clear way of using it in a team development environment.

I posted something recently here... http://drupal.org/node/430340.
Let's see if what I posted will spark any comments.

Scott McCabe’s picture

I stumbled on this post while researching the same subject, hoping to find a comprehensive resource on properly building a Drupal development environment (including step-by-step instructions on implementing version control, and common traps to avoid when building and maintaining a Drupal-based site).

There is an apparent complexity involved in educating newbies how to build a complete development environment from scratch, perhaps the reason for the lack of replies here.

Based on my immersion in Drupal and surrounding software applications over the past few months (I'm a Drupal newbie with some PHP/SQL/CSS/xHTML and UNIX skills cramming a lot of knowledge and experience daily), picking up bits and pieces here and there (even buying the Pro Drupal Development 2nd edition book and access to Lullabot's Do It With Drupal seminar videos, both containing useful information), there's apparently a lot to consider when creating a proper development environment, including a serious challenge with respect to ensuring the production site always has an accurate copy of the database (e.g. can be tough when updating your site with new features), and ensuring that all of the software (Drupal, OS, Web server and database apps, etc.) and hardware is consistently optimized to meet your site's traffic needs once you're up and running.

On the one hand, I understand the Drupal community's need to focus on Drupal alone. To create well-organized, detailed instructions on version control, Web and database server configuration and maintenance, hardware architecture, etc. would require a lot more resources to put it mildly, especially considering the many options.

On the other hand, Drupal usability requires an understanding of other software applications that can be intimidating to people unfamiliar with the UNIX culture apparently generally favored. If newcomers can't properly develop their Drupal applications, the result could be broken websites and a false perception against Drupal. The Drupal community must either basically say, "This is complicated stuff even for experts. Doing it right requires professional assistance. Do not attempt without that assistance." and continue to focus solely on Drupal, or become a community truly embracing the complete newbie, working with other relevant open source communities to create education materials for everyone's benefit (e.g. extract information from CVS and Drupal documentations and create a new hybrid piece) -- i.e. community modularity.

If you have the resources (I sadly don't at this time, though the hacker in me loves wading around up to my neck in this stuff), perhaps it would be best to hire someone to help you build a proper development environment and maintain the site (I also stumbled on this company in my research FWIW: http://www.workhabit.com/solutions/by-platform/drupal).

At a Drupal meetup in my area, people recommended Git for version control and Ubuntu for an operating system FWIW.

Hope this helps. Sorry I can't offer more. Maybe if I can ever free myself up from an overflowing plate of commitments (including developing my own Drupal-based site), I will contribute more on this interesting subject if there's a reasonable demand (I'm guessing there will be).

oscarpalacious’s picture

I always work with Subversion. I hadn't worked with Drupal, though. Versioning Drupal is a bit tricky because the configuration, which must be versioned to have a homogeneous development environment, is stored in MySQL. Now, I am a total newbie when it comes to Drupal, so take my advice with a grain of salt.

On my root directory, I have three folders,

documentacion
drupal
mysql

I installed everything related to Drupal in the drupal directory; all documentation in the documentacion directory (the name is in Spanish, although without accent); and, most important, all MySQL dumps (necessary for versioning) in the mysql directory. You can of course use the names you want, etc.

On the mysql directory I have the following contents:

completa.sql
dump.sh
esquema.sql
load.sh

This is the code in dump.sh (I changed the database username, password, etc):

#!/bin/bash
mysqldump -u mysqlUser -pmysqlPassword dataBaseName access actions actions_aid authmap batch blocks blocks_roles boxes comments content_node_field content_node_field_instance files filter_formats filters flood history languages locales_source locales_target menu_custom menu_links menu_router node node_access node_comment_statistics node_counter node_revisions node_type permission role system term_data term_hierarchy term_node term_relation term_synonym url_alias users users_roles variable vocabulary vocabulary_node_types > completa.sql

mysqldump --no-data --add-drop-table -u mysqlUser -pmysqlPassword dataBaseName > esquema.sql

After dataBaseName and just before > completa.sql are listed all "relevant" (for versioning purposes) tables on the MySQL database for my drupal installation. I read on several forum threads that I could safely discard (but you may wanna double check this) all cache* tables, along with the sessions and watchdog tables. On my MySQL I have the following cache* tables:

cache
cache_block
cache_content
cache_filter
cache_form
cache_menu
cache_page
cache_update

You may not have an identical list of cache* tables as I added some stuff.

After dumping the "relevant" tables' contents to completa.sql, I dump the entire schema (which must include all the tables) to esquema.sql. Note the --no-data --add-drop-table options on the second instruction.

Something very important to remember is that several drupal modules add tables to the database. Unless these tables are of the cache* kind, you must add them to the "relevant" tables list.

Now the contents for load.sh:

#!/bin/bash
mysql -u mysqlUser -pmysqlPassword -e "drop database dataBaseName"
mysql -u mysqlUser -pmysqlPassword -e "create database dataBaseName"
mysql -u mysqlUser -pmysqlPassword dataBaseName < esquema.sql
mysql -u mysqlUser -pmysqlPassword dataBaseName < completa.sql

There may be more efficient ways to do this, but this definitely worked for us.

This is how it works. I assume you already have a repository and a working copy of your drupal, but you haven't versioned your database yet. The mysql user (in this case, mysqlUser) must have the proper privileges.

1. Add the scripts to a directory you find convenient on your working copy.
2. Edit the first script (the one I named dump.sh), and check that you have all "relevant" tables in the list. Change the username, password, etc.
3. Run the script.
4. Edit the second script to have the proper names, etc.
5. Add both scripts and both dumps with svn add, and commit:

svn add completa.sql dump.sh esquema.sql load.sh
svn ci completa.sql dump.sh esquema.sql load.sh -m "Whatever message here"

6. Once your comrades make an update, they should have the scripts and the dumps on their working copies.
7. To have an identical database, they only have to run load.sh.

When other teammates add configuration changes, or modules, or whatever changes the database, they'll have to run dump.sh. They should remember to add the "relevant" tables to the script before running the script. I'm sure there's a way to do this more elegantly (anyone?). You can always run the script and check for any changes to esquema.sql with svn st and svn diff commands, though.

Now, what load.sh does may also be pretty barbarous. It drops the entire database, it then creates it again, loads the schema, and then loads the data.

The users table is also committed, so all usernames and passwords are also homogeneous. This may not be very safe, so you may want to omit the users table from the "relevant" tables list, but I don't know if this may have any negative repercussions on the system.

I hope this helps someone, and I also hope for some healthy criticism for this method to improve things.

retrolima’s picture

Thanks a lot Óscar for your detailed guideline. I do similar steps for applications in other frameworks.

What I try to figure out, and hope you can help me, is how deal with the changes in the database in a distributed team with several dev environments scenario.

Right now we are doing a manually diff between

  • sql data dump files of last main version commited
  • developer current version

After that add what is necessary to the main version and then do the push (commit)
This procedure takes place in every dev environment before any commit!

If you can give me any help on this I wold really grateful.