Problem/Motivation

Nesting contrib projects (e.g. modules, themes) inside other contrib projects (e.g. install profile) makes managing project in a site repo and contributing changes overly complicated. We can reduce friction for maintainers and contributors by NOT nesting contrib projects inside other contrib projects. This way, developers can work in a parent site repo, then use git-subtree, git submodules, or git subtree merge to break out work done in a site repo and push it out to drupal.org project repos.

Proposed resolution

Drush 7 now supports this contrib_destination property. Initially the solution proposed here was to get drupalorg_drush updated to run on Drush 7.x (master). This is now seeming overly complicated. A simpler, faster path forward is to backport Drush support for the contrib_destination property to Drush 5.x and 6.x. Then update drupalorg_drush (now on 5.x) to use a newer version.

Remaining tasks

  1. Drush 6.x support for contrib_destination property, https://github.com/drush-ops/drush/issues/834
  2. Drush 5.x support for contrib_destination property, https://github.com/drush-ops/drush/issues/836
  3. Update drupal.org server to use a version of drush including either of these patches ^^.
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

bryanhirsch’s picture

Issue summary: View changes

Drush Make seems solid for use as of commit 71305b7.

All automated tests are passing as of commit 71305b7 (HEAD):
https://travis-ci.org/drush-ops/drush/builds/26964694
https://github.com/drush-ops/drush/tree/master-fulltest

Anecdotally, I've been using the Drush master (7.x) branch for make builds for months and haven't had any problems.

bryanhirsch’s picture

Project: Drupal.org site moderators » Drupal.org customizations
Version: » 7.x-3.x-dev
Component: Project/Git problem » Code
bryanhirsch’s picture

Here's a patch for drupalorg.

dww’s picture

Status: Active » Postponed (maintainer needs more info)

I don't fully understand what this issue is proposing to change.

The only "nesting" going on is either caused by:

A) The packaging script creating full tarballs which are supposed to be easy for end users to download and install without knowing about package management tools, running drush make themselves, etc. These are a whole "site in a tarball", including the distro, core, all modules/themes, just as a tarball, without git anywhere to be seen.

B) Distribution maintainers Doing It Wrong(tm). See #2153139: Unpublish Distributions with Forks of Core < 7.32

If a developer/contributor wants to do things with a more fancy workflow involving site repos and git sub(trees|module)s, they can/should run drush make themselves, can use whatever version of drush and its features they want. But our packaging script definitely isn't setting up git at all, so I don't see why we need to do this.

Can you please explain?

Thanks!
-Derek

bryanhirsch’s picture

Issue summary: View changes
bryanhirsch’s picture

Issue summary: View changes
drumm’s picture

Regardless, upgrading the drush version used for packaging distros is good to do. This should either be merged into #1854444: Upgrade drush used for distro packaging with drush make, or close that in favor of this issue since it has more recent information.

The next step will be updating our Puppet module that installs drush & other packaging requirements. We will want both installed in parallel, while we try out staging/dev.

class drupalpackaging {
  vcsrepo { "/var/www/drupal.org/tools/drush5":
    ensure => present,
    provider => git,
    source => 'http://git.drupal.org/project/drush.git',
    revision => '6f30fec6a8a3b2091228fd71d569879563aa6023',
  }

  vcsrepo { "/var/www/drupal.org/tools/drupalorg_drush":
    ensure => present,
    provider => git,
    source => 'http://git.drupal.org/project/drupalorg_drush.git',
    revision => 'cd474d2e64aad71dcb6f479f9b63a819611b2777',
  }

  vcsrepo { "/var/www/drupal.org/tools/drush_make":
    ensure => present,
    provider => git,
    source => 'http://git.drupal.org/project/drush_make.git',
    revision => 'a945642c5c48b759ae4a935a1b1cc0943ed720dc',
  }
}
bryanhirsch’s picture

@dww, thanks for the question. I had posted a detailed response, but it's verbose and I think it makes this ticket inaccessible. I have moved the verbose answer to this gist for people interested in the user stories, options, and background on the conclusion:
https://gist.github.com/bryanhirsch/a3e29e945d087a02affe

Hoping we can get this issue moving forward again to get this issue unstuck.

bryanhirsch’s picture

Issue summary: View changes

It's now seeming to me like the path of least resistance is actually to backport support for contrib_destination to Drush 6.x and 5.x (a tiny, simple patch). Then update drupal.org to use Drush 5.x (the version currently being used) with support for contrib_destination.

Here are the related issues and pull requests I filed for Drush to backport support for the contrib_destination property:
6.x, https://github.com/drush-ops/drush/issues/834
5.x, https://github.com/drush-ops/drush/issues/836

drumm’s picture

Title: Update drupal.org distro builder so distros are not forced to nest contrib projects inside contrib projects. » Update Drupal.org's drush make for distro packaging so distros are not forced to nest contrib projects inside contrib projects
Priority: Normal » Major

Now that drush make files can be YML formatted, this is causing more friction for distribution maintainers.

drumm’s picture

Status: Postponed (maintainer needs more info) » Active

I tested drupalorg_drush out with Drush 9 and managed to get tests passing without substantive code changes.

drumm’s picture

We’ve been using Drush 8.1.3 to build & deploy *.Drupal.org sites for a couple weeks and everything is working well. There were some minor changes in info file formatting, but nothing major.

The remaining work here is to get drush with drupalorg_drush set up in Puppet, alongside the old drush in case we have to switch back; then switch the packaging script to use the new path.

drumm’s picture

Assigned: Unassigned » drumm

Finally going to do this.

drumm’s picture

Status: Active » Reviewed & tested by the community
Issue tags: +needs drupal.org deployment

All my testing shows this is ready to deploy, which should happen within the next hour.

  • drumm committed 7a8a575 on 7.x-3.x, dev
    Issue #2281721: Update distro packaging to use Drush 8
    
drumm’s picture

Status: Reviewed & tested by the community » Fixed
Issue tags: -needs drupal.org deployment

Done - Drupal.org packaging now has Drush 8.1.12

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.