Note: this has now been deployed on drupal.org. See the announcement post: Fully packaged Drupal distributions now deployed on drupal.org.

Since the 5.0 release, Drupal core has included an installer that supports installation profiles to setup and configure a site for a certain use-case. In theory this allows for people to create a better "out of the box" solution by configuring Drupal like a wiki, a conference, or a publishing site. If done right, installation profiles have the potential to help end-users get sites done faster and accelerate the adoption of Drupal. However, this promise has yet to be realized, and this core Drupal feature is currently being vastly underutilized.

As part of their grants from the John S. and James L. Knight Foundation, Deproduction and Quiddities want to help realize the benefit that install profiles have to offer. The Drupal Association, with the support of individual and corporate sponsors, has put forth a matching grant to help see that it meets its mission of supporting the Drupal project. The potential to help address the usability problems of people trying to get started with Drupal is truly enormous.

Read on for details on how installation profiles work, why they are important, how packaging them will help, and the technical decisions behind the way the packaging scripts will work. Now is the chance to provide feedback which might make it into the code before it goes live by the end of November.

Why do installation profiles need help?

The main problem holding back the full potential of installation profiles is the lack of a packaging tool for contributed modules. Unless the profile is built on core alone, using an installation profile involves: downloading core, downloading an installation profile, unpacking both and putting the installation profile into a very specific directory inside the core directory tree, trying to find all the required modules in the documentation of the installation profile (if that is even complete), going back to drupal.org to download some set of requirements, placing those in the right directories inside your core installation, and finally visiting the installer page to "start". This totally cuts off the "ease of use" that installation profiles aimed to address.

To fix this usability problem, we're going to provide a mechanism for the authors of installation profiles to include some metadata about the dependent modules and themes. The drupal.org packaging scripts would then do all of this work automatically and create a single file containing the latest copy of Drupal core, the installation profile itself, and all the required modules and themes that it depends on. This would give end-users a three-step process for getting started with a site that was configured specifically for their use-case: 1) find an installation profile that suits their needs, 2) download one file and unpack it in their web root, 3) visit install.php.

About the funders

Deproduction and Quiddities are contributing $2,500 each to develop a Packaged Install Profile in Drupal and the Drupal Association will match those contributions.

With support from the Knight Foundation, both organizations are working on projects that use Drupal to help community media organizations improve their services:

Quiddities is working with public radio stations to create a Drupal install, Radio Engage, with features that can support the online web presence of radio stations. The Radio Engage toolset consists of existing Drupal modules that will create dynamic station sites that integrate internal and external station content and provide a platform for user engagement. The platform will launch this fall at KUSP in Santa Cruz and KALW in San Francisco and then be available as a install profile.

Deproduction has been building up the Open Media Project for a year and has successfully installed the Open Media Project toolset in seven public access stations through their Knight-funded beta test. Those television stations include Amherst Community Television, Boston Neighborhood Network, channelAustin, Davis Media Access, Denver Open Media, Portland Community Television and Urbana Public Television. Pioneering an initiative to leverage Public Access TV stations to close the digital divide, the Open Media Project, openmediaproject.org, seeks to put the power of the media in the hands of the people by making it easier for all communities to share and distribute content online.

The Packaged Install Profile will make it easier to complete the install process for new users in public radio and public access TV. This project will create a sustainable solution on Drupal.org for updating the core features of the toolset used by the Open Media Project and Radio Engage.

Scope of work

To realize the full potential of installation profiles, the following tasks are necessary:

  1. Finalize the structure and contents of the packaging metadata files used by installation profiles. A lot of thought and community input has already gone into this problem via discussions on groups.drupal.org, the developer mailing list, and the issue queue.
  2. Modify the packaging script to parse the packaging metadata files, checkout all of the required code and assemble it into the right directory structure, package it into a single file, etc.
  3. Writing better error-propagation and recovery when maintainers incorrectly define their package. This problem already exists for translation packages, where bugs in the .po files cause the packaging script to refuse to create a translation package. However, there's no way for maintainers to fix these errors without creating a whole new release. We're going to face the same problem with installation profile maintainers that have bugs in their packaging meta data files, so we'll want a way for them to validate their install profile before creating the release, or a way to fix these sorts of bugs and let the packaging script re-try.
  4. Documentation on how installation profile metadata files are defined, and how to use the resulting bundled packages.
  5. Identifying and mentoring a subset of the Drupal community who will deeply understand this system, and train them to be able to support the rest of the installation profile maintainer community. Once the system goes live, there will inevitably be questions and problems with it, and it's vital that there be a pool of people who can support the system going forward.

Limitations of Installation Profiles and related work

Even once installation profiles are packaged into single distributions with all the resulting usability benefits, there are still limitations in their usefulness. For example, you can not run an installation profile on an existing site to enable a set of functionality. However, the proposal above would immediately (and retroactively) improve functionality already present in Drupal core dating all the way back to Drupal 5. There are a lot of efforts to address this other need among Drupal contributions (spaces + features, patterns, etc. However, something like this will not be included in Drupal 7 core before the code freeze to address this need, so we should improve on the usability of the technology we already have. None of this would prevent another solution (in addition to installation profiles) in Drupal 8 or later that bundled code and configuration for various feature sets which could be enabled on existing sites. In fact, most of the technical solutions (files containing packaging meta info) would be reused.

Where to provide feedback and follow progress

Most of the discussions around all of this have been taking place in the Distribution profiles group on groups.drupal.org. Although most of the technical questions have already been answered, there might be other discussions posted there, so that'd be the place to watch.

Comments

Jon Pugh’s picture

I don't think any of us can predict the impact this will have on the non-drupal-using community. If this can be done right, you'd only ever download the version of drupal that fits your needs already!

Setting up a simple Drupal site can be a real pain even for those of us with years of experience, simply because the modules we need are scattered over the place.

We're on our way to something amazing here.

__________________________
Jon Pugh
ThinkDrop Inc
https://devshop.support
https://thinkdrop.net
https://twitter.com/jonpugh

Summit’s picture

Hi,
Great news, I made a post about this some time..may be months ago, to build a install profile using an wysiwyg editor. hierarchical_select etc...

EDIT: Found the post: http://groups.drupal.org/node/19759#comment-75385
think drupal will benefit a lot having install-profiles which make drupal out-of-the-box copying wordpress functionality with fckeditor/imce and other modules standard enabled and configured or other CMS-like install-profiles (Joomla/Typo3) to get beginners using Drupal as tooling box without having to set up fckeditor and other not-core modules which are standard in other cms's.
Do you like the idea? this way Drupal 6 will get more beginners using Drupal I think.

Will look it up the coming days. Great news!

Jamie Holly’s picture

This is really great news. I've been wanting to see something like this on Drupal for a long time now. I think it will really help the newbie put together a site they need quicker instead of spending tons of time hunting down different modules.

---------------------

HollyIT - Grab the Netbeans Drupal Development Tool at GitHub.

nmridul’s picture

This would be one great addition to drupal. One of the major obstacles with drupal for a newbie is the difficulty in getting a simple site up. Ease of installation would bring in more folks to drupal.

AlanT’s picture

Sounds like you'll be creating a packaging system for Drupal much like the packaging systems for Linux.

The really nice thing I see here is the possibility to have "1-click" upgrades for the whole system, core & contributed modules.

- Alan Tutt

Exceptional Personal Development for Exceptional People
http://www.PowerKeysPub.com

batje’s picture

Would this also fold into the work that is being done on features? That would be really sweet: you could configure a site, then export as a feature and wrap it into an installation profile.

kreynen’s picture

The work on Features and Open Atrium is largely being done outside of Drupal.org. My understanding was Features exists outside of D.O. specifically to allow individual groups to maintain their own distributed feature servers. While running Features outside of Drupal.org has some advantages, there are downsides to allowing code from these servers to be distributed on D.O. as well. There is an active discussion about maintaining a whitelist of external repositories deemed safe. dww set a deadline for Oct 15 to either agree on how to roll external code into Drupal or move forward without this feature.

Our primary goal in contributing funding is to reduce the amount of staff time required to maintain an install profile of core and contrib modules. Personally having almost quit Drupal altogether when I failed to get permission to include a customized version of TinyMCE included with that module, I'm staying out of this discussion. If key folks who lead the groups volunteering their time on D.O.'s infrastructure and security can agree on a way to get external code like Features and TinyMCE rolled into an install profile, great. If not, the changes on D.O. will still be a big help in maintaining both the Open Media and Radio Engage profiles since we are both building on a large number of existing contrib modules.

For us, this is really a question of resources. Neither Deproduction nor the 6 other public access stations we've partnered with on the Open Media Project have the resources a group like Development Seed has to manage our own public version control, releases, issues queue, and developer community. We depend on both volunteers form the larger community and the Drupal Association to provide that framework so we can focus on working with our small niche of Drupal users who happen to be public access television stations. While it's great to see what groups like Development Seed can do with profiles outside of D.O., we've seen enough well intended groups implode or fade away after releasing an install profile to know what a resource drain maintaining the framework to support the profile can be.

kreynen’s picture

Have you looked at drush?

int’s picture

I think, that this efforts should be to Drupal 7.

Summit’s picture

Why only Drupal 7? I think lots and lots of sites will benefit from it already with Drupal6!!! +1 for Drupal 6!
Greetings, Martijn

kreynen’s picture

Just like the D6 version of the Project module running on D.O. already allows module maintainers to created and maintain releases for any current version of Drupal, install profile maintainers will have similar flexibility. Just because D.O. is running D6, doesn't stop you from using it to release a D7 module.

mlncn’s picture

That covers it.

benjamin, Agaric

hass’s picture

Many many times I planed to build such a module, but than I saw it was planed to add plugin manager to D7 and a download feature for profiles that will download all required files itself before starting the install. Not sure about the latest status of this... but it stopped me from taking over this task.

Elijah Lynn’s picture

This is fantastic!

I too thing this will have an amazing impact on the Drupal community! Go profile project!

-----------------------------------------------
The Future is Open!

Jogger1234’s picture

I too think that this is having a great impact on drupal community......

q0rban’s picture

We're going to provide a mechanism for the authors of installation profiles to include some metadata about the dependent modules and themes. The drupal.org packaging scripts would then do all of this work automatically and create a single file containing the latest copy of Drupal core, the installation profile itself, and all the required modules and themes that it depends on.

Yes, please.. :)

gusaus’s picture

Great to see this come to fruition after many years of hard work, technical issues, and debate. Being able to follow through with the promise of something like this will be a huge win for Drupal users and providers.

That said, it's still a bit unclear how people could get involved. Should people be weighing in on this discussion (http://drupal.org/node/594704) and offering their help? For the 1-2 (under scope of work) - Is this something Derek, Chad, and team have a handle on? What about #3? #4 and 5 definitely seem to be areas where the community can help.

In addition to visiting a somewhat disjointed Drupal group, I think programs like the Drupal Dojo, GHOP, and Drupal Kata (http://drupalkata.com/) can help get people involved in driving this effort forward and also building profiles themselves.

Thanks again to everyone involved in bringing the innovation this far. Let's be sure we can make rally around this effort and make sure it rocks!

---------------------------------------
Gus Austin

john.karahalis’s picture

We would also be interested in learning more about how groups can get involved. We are the developers of the Innovation News Installation Profile, so we are starting to wonder how (if at all) we can help in this effort.

Paul Natsuo Kishimoto’s picture

Installation profiles are underutilized, but have the potential to be a very powerful driver of Drupal adoption—much like tasksel has been for Debian and related Linux distributions. With apologies to the author, it would eliminate the need for users to read books like Drupal 6 Site Blueprints.

In fact, there are two ways to do custom "appliances" in Debian et al.:

  1. Create a custom CD/DVD image containing all required packages.
  2. Use tasksel to force the standard installer to download, install and configure a list of packages.

This is the age-old "push/pull" dichotomy. #2 is by far more flexible and popular, and I wonder whether it—supported by Plugin Manager, which is *fingers crossed* going into D7 core—might be a better model for install profiles than #1.

hardieas’s picture

I develop sites on a multisite installation and the ability to make a "deployment tarball" of a site from that, with all the relevant modules and themes, would be *very* useful.

I also have to say that Drupal's default behaviour of silently disabling modules that are enabled in the system table but not present on the system is a Very Bad Idea. In my view, the correct behaviour would be to NOT disable them immediately but instead to put out an error message that shows what's missing and allows you either to accept the disabling or to install the missing items before proceeding. (If this is in D7, I apologise...)

Anonymous’s picture

Also you could take a look also at the Slax project: http://www.slax.org/ and "build slax" menu option which utilizes this concept.

alcheme’s picture

I think that this is a great idea! One of my many drupal planned sites is going to be the back-end, "mid-end", and various front-ends of my business, and I would love to offer of the "setup" to fellow artisans in the business, as software in the field is exceptionally lacking (which is why I went the web route, well, that and I am also a web developer).

It is late, so my breezing through your post didn't quite make sense (coffee would be fab right now...). I think that a *managed* Firefox "collections" type setup would be great. It would allow people to install what they needed, and*hopefully* nothing more, but for those wanting to offer an more robust site, would allow them to install several different collections at the same time.

As I said, it is late, so I might have gone off the deep end and just started rambling. In that case, feel free to ignore me... :)

schock’s picture

This is going to be sweet. Any update?

greggles’s picture