Get servers in place for Drupal 7 staging/dev environments

We need a round of staging sites for Drupal.org on Drupal 7, including as many supporting services as possible for Solr, Git, etc. Like the staging sites on scratchvm, these should be automated to deploy anything in BZR and do daily upgrades from snapshots of live sites. This will give us a good view of what would happen if we upgraded today.

The vhosts can probably fit on scratchvm as soon as we get files moved off, if we want them there. Everything else needs a bit of planning to fit in, especially for disk space.

Only one comprehensive set of staging sites needs to exist in the end, either this or *staging.devdrupal.org will eventually go.

Updated the issue summary, including completed work.

cool, great to see progress on this in advance of the sprint. this will help a lot.

minor nit: what's the point of an "infrastructure" issue tag in this issue queue? slightly confused by that...

thanks,
-derek

todo list

forgot vhost conf

Okay, fair enough. I could argue your last point, but it's not important. ;) I can play by whatever rules y'all prefer. Tags++, regardless. ;)

Cheers,
-Derek

vhosts done

Adding Solr.

We have a DB server now and things are starting to work. However, the core upgrade to Drupal.org takes 14 hours, and then breaks.

I filed #1536160: Make update_fix_d7_block_deltas faster and removed the old block variables to reduce the time needed to run an upgrade.

We are running into #966210: DB Case Sensitivity: system_update_7061() fails on inserting files with same name but different case which is blocked by #1237252: DB Case Sensitivity: Allow BINARY attribute in MySQL.

In Drupal 7, the uri column in the new file_managed table needs to be unique. The data is derived from filepath in Drupal 6's files table, which doesn't have this constraint. Drupal.org has 222 duplicates found with SELECT count(DISTINCT f.fid) c, f.filepath, group_concat(u.nid) FROM files f LEFT JOIN upload u ON u.fid = f.fid GROUP BY cast(f.filepath AS BINARY) HAVING c > 1; Either Drupal 7 should not have the unique constraint and/or we need to clean up our data.l

The files are a mess. For example, 484 files have filepath "files/issues/". An example is #629528: Error. The file actually does exist at http://drupal.org/files/issues/Error_3.JPG. There really isn't a good way to clean this up. Doing it well would mean matching up all the files, maybe using file size to help.

We are just going to have to remove the bad data and work on cleaning it up on the live site in the meantime. #1542666: Clean up duplicate files.filepaths

Ran into #1542854: system_update_7061() converts filepaths too aggressively too.

Success! update.php has completed once on http://7.devdrupal.org/. (That page WSODs because the home page code isn't D7 yet.)

Downgrading the priority and re-assigning to @nnewton for followup of any needed configuration or settings that still might need to be touched.

Closing this.

A couple things got done.