The OSCMS conference in Vancouver allowed upwards of 250 people to gather together and collaborate, discuss, plan, and yes, code. The 3 days were an intense blur of presentations, hacking sessions, food (and drinks), socializing, skiing, and musical mayhem courtesy of the Quadruplicons. Meeting face-to-face allowed many important issues for Drupal to be brought to the forefront and see progress made, including:

  • Identifying important areas in which to focus, including the File API, Views, Actions, Workflows, and Relationships
  • Helping to streamline efforts going into Drupal community processes, through Drupal Enhancement Proposals (DEPs), working groups, and local user groups
  • Making serious headway in terms of increasing Drupal's usability
  • The Drupal 4.8 roadmap (or lack thereof ;))

Read more to find out the finer action points and pointers to write-ups of the various sessions. This post has been collaboratively written by the various attendees.

Session summaries

Drupal enhancement proposal

The Drupal Community Process session saw the introduction of the Drupal Enhancement Proposal (DEP) -- a document that encapsulates an idea that someone has to improve Drupal. The main motivation for DEPs is to have a means for coordinating efforts and discussion when deciding whether and how to bring proposed ideas into fruition, as well as preserving the record of the effort. The session introduced a rough proposal for a website that could be used to manage DEPs. The website also managed to highlight a number of emerging Drupal technologies used for rapid development (views, actions, workflow, relativity), but that was tangental to the session's goals. Finally, the attendees were split into two groups to discuss two different DEPs. The first DEP is the "DEP bootstrap DEP", aimed at providing a specification for how DEPs should work. Djun Kim and Angela Byron were active recording the outcomes of this, and Moshe Weitzman was the effective moderator of the group. The second group was formed to create a DEP proposing the inclusion of some sort of image management in Drupal core, and the discussion was led by Dries and recorded by Matt Westgate. Summaries of these two DEPs will be provided as soon as I've received all relevant materials.

Large Project Workflow Improvements

The group discussed how best to augment the current patch queue system to handle bigger coding changes such as the forms API. Chad Phillips gave a short presentation outlining how DEPs, a test site, and Bazaar-NG could be used to facilitate the process via small workgroups. The main outcome of the discussion was the decision to research how other OS projects were handling these challenges before implementing any new systems of our own, and also to take a look at the current tools available to developers, and see if these can be made to work better for larger coding projects.

Actions and workflows

Robert Douglass gave a compelling demo that used the Content Construction Kit, Views, Actions and Workflow to create a dynamic site for submitting DEP's.

Drupal Enterprise-wide

See the meeting notes.

Ad-hoc event discussion

This session focused on the various event-related modules available, outstanding issues with each and potential development ideas for improvements and better integration of events into Drupal sites. Here are some brief meeting notes.

Improving Drupal documentation

This session was held as a round-table discussion where people raised issues important to them, most of which surrounded making Drupal.org more easily accessible and identifying areas which were lacking in Drupal's documentation as a whole. You can see a write-up of the points discussed.

Theming system enhancements

Adrian led a session on how to make Drupal theming easier. Currently some of the difficulties in Drupal are: using one theme for user-facing and administration pages; A lot of HTML is generated in code and is not visible to designers; Multiple CSS files are difficult to trace through to find out exactly which rules are being applied to which parts of a page.

To become easier to theme, Drupal must become more transparent in the mark up it generates, and the CSS Classes and IDs that are used to apply style to that markup. An audit of Drupal's core HTML output is underway (by m3avrck?), and guidelines should be published for defining Classes and IDs in themes and modules as an easy reference for designers. Potentially, future versions of Drupal could be shipped with a default theme that is only a CSS file which would enable designers to focus more on the layout and graphics (something like CSS Zen Garden ease of theming).

Inter-site functionality

See the meeting notes.

File API and media support

Adrian Freed presented an overview of the state of Media handling in Drupal. Between these two sessions, it became apparent that Drupal is in desparate need of a more robust way of handling uploaded files in general, and multimedia file types in particular. File access control, mime type discovery, and metadata introspection are all problems that are waiting for a centralized solution.

Administration user experience

See the review of a new administration module.

URL aliases

here. We would like to make it possible to programatically deliver URL aliases instead of the current one-by-one aliasing. This would not add functionality that does not exist now through pathauto but make it much more easier to administer and scale better. Code readability is expected to rise a bit at a big number of places. An important side effect would be the increased reusability of Drupal as a PHP library.

Decisions module

decisions.module which will enable groups to take decisions online in a manner that replicates and augments what is possible in face-to-face meetings. After describing some of the problems that groups encounter when trying to collaborate together online he described the future direction that this tool could take and invited people to participate in its development.

Views

merlinofchaos presented Views. Views is a query builder with UI that allows site administrators to create custom views of multiple nodes. Future work for Views, such as separating the UI from the query builder, was discussed, along with the idea of researching RDF as a mechanism to describe the databases rather than the array it uses now.

The Content Construction Kit

JonBob presented the Content Construction Kit (CCK). CCK allows creation of arbitrary node types that can share data across nodes. A modified database schema that improves performance was considered for CCK. CCK now uses the new module install system, and a supported upgrade path will be provided from this point forward. The user interface was recently decoupled from the database layer as well, allowing the user to select from an assortment of "widgets" for each field. The module suite will be released alongside Drupal 4.7 as a contribution.

  • jjeff volunteered to write file and image field types.
  • chx volunteered to help with implementation of multiple-valued fields using the new form API.
  • grugnog volunteered to write a Flexinode-to-CCK migration script.
  • RobertDouglass volunteered to write a date field type, but had to recuse himself to work on the DEP system. A new volunteer is needed.

File API

James Walker lead a discussion on addressing the current issues with File API. Key factors are : file security, public vs. private files, general lack of understanding in how to use the current API, additional hooks and flexibility. There were several good suggestions surrounding ways to make the File API more extensible, including introducing hooks on file upload as well as making the storage backend flexible to allow for alternate storage methods. James has volunteered to lead development, but is looking for volunteers to help!

User experience issues

Following a series of user interviews and a survey of almost 900 Drupal users over the last 6 months we prioritized tasks to be completed to improve Drupal. We presented a summary of top tasks and most difficult tasks for users. We then broke up into 6 groups and focused on the following issues:

Install, update and dependencies

Neil Drumm broke down installation and update-related issues into various categories. Both Drupal and Drupal modules must worry about installation, initial configuration, and upgrading. Additionally, there is metadata storage, dependency management, package management, and error handling and avoidance which can be improved. In 4.7 we have improved Drupal upgrading and added new module installation and upgrading features. There is some code out there in need of review which has been enumerated on tthe session web page. A group of people worked on implementing install and update featues in their modules. A second group worked on making an installer for Drupal core. The third group did general brainstorming.

Some action points

  • Dries: coordinate the Drupal 4.7.0 release.
  • Dries: coordinate the Drupal.org website upgrade.
  • hunmonk: research how existing open source projects manage larger code changes.
  • hunmonk: review/upgrade existing developer tools to assist larger code changes.
  • robertDouglass: create a site on a subdomain of Drupal.org to manage Drupal working groups and Drupal Enhancement Proposals (DEPs).
  • webchick: Investigate into creating a standardized way to access remote files for file API
  • webchick: Look into ways of browsing through organic groups by location, category, etc.
  • webchick: Create 'module_builder.module' -- think RoR scaffolding, but for Drupal
  • Zacker: Help get fancy new Drupal theme made and in core (w/ Ted, Aaron, Jeff, Steve)
  • Zacker: Consultant directory (w/ Gregory Heller, Richard Orris, Kris Krug)
  • rkerr: Collaborate with other willing folks to improve media (and file) handling.
  • mattWestgate: Investigate ways further speed up path aliasing.
  • Nedjo: Document solution components for inter-site functionality, identify gaps, prototype/test networks of sites (w/ Allie / Chris Johnson)
  • chx: Remove all (or at least most) arg() calls.
  • Kieran Lal and Jeremy Andrews: Installer for Drupal core
  • merlinofchaos: Create a post (hopefully front page) to gather input on administration module and get admin menu rearranging into 4.7
  • puregin: (Blocks for Roles)
  • puregin: Enable import/export of books on Drupal.org to simplify documentation editing
  • puregin: Improve book module, to allow more open contribution/editing of documentation on Drupal.org

Some anecdotes

  • Drupal developers O.D'd on sushi. The use of chop sticks is a deterrant to "forking".
  • Morbus Iff's presence was astonishing. His talk was very popular (picture of Morbus effigy).
  • chx tried to remember a node number, but got only 2 digits correct. He may have been "distracted".
  • Adrian successfully washed his laundry.
  • Dries' pictures are available at http://buytaert.net/open-source-summit-2006-photos.
  • Allie posed the question, "Adrian, what does the word 'Relationship' mean to you?" and later confessed she was "all about multi-level relationships." ;)
  • The strength of the Drupal coding army is contagious, motivating Wordpress Matt to join in and work on his own code late into the night.
  • Vancouver locals were stunned by the re-appearance of the sun after 60 solid days of rain, attribute this to visit by powerful secret society

Comments

dman’s picture

Can I ask where, if anywhere, the thoughts on relationships got to?
My project's got to a certain point, but I'm now unsure whether (or how) to embrace CCK compatability.
I started putting a little demo/tutorial on my relationships implimentation together to submit when I heard about the conference, but didn't exactly tell anyone very loudly.

.dan.

http://www.coders.co.nz/

Boris Mann’s picture

Ideally, Dan, you would use CCK and also push relationships into the 4.7 cycle. Your RDF stuff is fantastic, but everyone is moving forward with 4.7 and Forms API.

merlinofchaos’s picture

Damn, such a thing would've been nice. Your name came up quite a bit during the relationships conference. Mostly as a "Whoa who is that guy, and wow!". I heard a lot of wow about your stuff.

I'm very intrigued. Also, have you done any Views integration? =)

-- Merlin

[Point the finger: Assign Blame!]
[Read my writing: ehalseymiles.com]
[Read my Coding blog: Angry Donuts]

-- Merlin

[Read my writing: ehalseymiles.com]
[Read my Coding blog: Angry Donuts]

robertDouglass’s picture

Making one of those for all of Drupal would rock.

- Robert Douglass

-----
My Drupal book: Building Online Communities with Drupal, phpBB and WordPress

dman’s picture

It's all scripted - it runs an array of actions which are themselves sub-actions, such as 'go to the div called 'title', 'Highlight it'

No, it's not hypercard ;-B just some fun javascript.

At the moment there is no exception handling or branching, but it could turn into an entire wizard.
I need to sort out some issues with the pause and restart process, and tart up the console a bit. I think I'll stop having the text follow the pointer around, it's hard to see.

But sometime I'll do a bit more of it for things that strike my fancy.
It's too much to do 'the whole of Drupal' but if you can suggest a little howto demo that would be useful I could have a go.

.dan.

http://www.coders.co.nz/

robertDouglass’s picture

How far out would it be to make a macro recorder that just "watched" what you did on a Drupal site and then was able to play it back using generated actions? Set up a roadmap, share the scripts you've got, get others on board. This could revolutionize the way we do Drupal documentation. I have no doubt you could find funding for it. I would never think of asking *you* to do an all-of-Drupal tutorial.... you need to keep porting Relations to 4.7! =)

- Robert Douglass

-----
My Drupal book: Building Online Communities with Drupal, phpBB and WordPress

dman’s picture

But macro recording doesn't, CAN'T pick up the semantics of what you are doing.

Are you moving to an x,y position, or towards a DIV, or an INPUT?

The script is pretty lightweight, but uses named elements as much as possible. This way it'll work even through different themes.

And the script encodes exactly what you are intending to do, not what you happened to do to get it done.

 howtoCreateAPredicate["Select Add Term"] = 'goClickElement(findLinkTo("admin/relationship/add/term"))';
 howtoCreateAPredicate["- Select Add Term"] = 'wait()';
// fill in the details
 howtoCreateAPredicate["Fill in the details, there's lots of explanations"] = 'movePointerToElement("edit-label")';
 howtoCreateAPredicate["Fill in the details, "] = 'setValue("edit-label","Cites");runScript()';
 howtoCreateAPredicate["Some bits are pretty standard"] = 'goClickElement("edit-name")';
 howtoCreateAPredicate["Just Click Them..."] = 'goClickElement("edit-description")';
 howtoCreateAPredicate["... And I'll take a Guess"] = 'goClickElement("edit-uri")';

That's all pretty easy to read, I think. Only the flow control (wait() and runScript()) are tricky.

Recording would come out much messier.
And would possibly take a while longer :-/.

http://www.coders.co.nz/

jenvirtual’s picture

Results of breakout group on DEP Workflow at DrupalCon 02.09.06

DEP – Drupal enhancement proposals as a vehicle for announcing ideas, direction and progress.

Brain Dump
- Drupal Community “random junk draw”
- open to everyone
- Author submits idea
- when someone ready to take the lead on the idea,
- Author specifies reviewer [panel] and scope
- Review Panel reviews for completeness and fit with roadmap
- Reviewers determine if Core track or Other
- Email members of track to notify
- Moving idea forward after confirmation by a certain number of
members and/or the review panel,
- comments always allowed, not considered an endorsement.
- idea moves to “In-Draft” stage

In-Draft
- Team lead/members further refinement of active idea
- As DEP created a SIG is created and integrated with forums

Proposed/Ready For Review
- Endorsement stage [by review panel]
- Community checks draft and provides feedback
- Feedback positive and constructive
- Review for acceptance
- Reviewer can send back to “in-draft” stage or move idea forward

Review accepted Or rejected
- If accepted, enhancement goes forward
- Activity is monitored for currency, i.e. if no activity for 6 months,
enhancement goes back to “In-Draft” stage

Archive
- Enhancement complete
- Implement new enhancement!

robertDouglass’s picture

What is a "random junk draw"?

SIG = Special Interest Group (could be called a working group instead).

I think the step about deciding whether something is core-track or other is important to have codified in the tool.

For those who weren't there, the tool I demonstrated created a new Organic Group for every DEP that was created. This simplistic approach has the advantage that the OG module then handles lots of issues related to "who is working on this?" and "who is the leader of this group?" and "where should I go for information about this idea?".

So the workflow above seems very detailed in the phase that happens before work begins. Did your group think about the role of the DEP and the SIG during development of the idea?

It's great that you included the archive step... it's important that visitors to the site can differentiate between what has been done and what hasn't. This is a problem with a lot of forum topics here at drupal.org... someone writes "I need XYZ" and three years later, long after XYZ has been completed (or the need removed), the forum post still says "I need XYZ" and the only clue about it is the date.

- Robert Douglass

-----
My Drupal book: Building Online Communities with Drupal, phpBB and WordPress

jenvirtual’s picture

Hi Robert,
Sorry for the typo.
Brain Dump is like the "random junk drawer" so many of us have, where anything goes. In Drupal terms, this means a space to put any idea for enhancements. The idea doesn't require refinement at that stage, it is simply to get it out of our heads and into the realm of possibility.

An idea moves forward if it "fits" with the roadmap and has someone to take the lead for it to create something of value. How well the idea fits with the roadmap is a matter for the appointed review panel and membership.

We had quality participants in this breakout group. If any have further to add, I hope we'll hear from them.
Peace! Jennifer