Portland UI-SIG cognitive walkthrough

The Portland Usability Special Interest Group did a cognitive walkthrough of Drupal 5.0 in January 2007. A group of usability practitioners tried doing usual first tasks of setting up a web site with Drupal.

We used screen sharing software and a conference call line. The following is the screencasts, about 1.5 hours total, notes for each section, and high-resolution screenshots of the pages we are talking about.

Introduction to Drupal

for usability practitioners

Watch 14 min 30 sec

Neil went over the basics of the Drupal project, focusing on how the project is developed and who our users are. The development of Drupal happens quite differently from the company-backed products which the usability-types might be familiar with. The ~492 person development team self-identifies by contributing and determines development goals without a formal structure.

Who uses Drupal is important to know when thinking about usability issues. We divided Drupal's users into four groups:

• Developers
• Webmasters
• Site editors
• Visitors

In all levels, except visitor, time commitment ranges from side-job for a small website with little time spent to full-time professional. One person may fulfill any number of roles. On many sites, one person does everything; on larger sites, tasks like development may be contracted out to a dedicated development shop.

• Introductory PDF

Review tasks and installation

Watch 7 min 5 sec

The following tasks were to be completed and evaluated by the participants:

1. A non-profit wants to set up a new site for a fundraising campaign. Give the site a name and customize the theme.
2. Publish the mission statement and a news item of the upcoming fundraising drive.
3. Add an image to the mission statement. Set up a new account for your colleague who will manage the content going forward.

While Frank went over these tasks, I used Drupal 5's new installer to set up a new site.

Creating the first account

Start of group review, the interesting part

Watch 12 min 20 sec

• Fantastico is mentioned as a convenient way to install Drupal. I responded by summarizing "Fantastico De Luxe: an insecure recipe for disaster"
• Creating the first account went smoothly, leading to the edit account screen.
• The password being set for user 1 was surprising.
• A dynamic password strength indicator would be a nice addition.
• The status and signature controls do not have enough context. Are these for the site, all users, or the specific user account?
• Deleting user 1 should not be offered.

• Front page, on first run
• Create first account
• Edit first account
• Saved first account

Customizing the theme

Watch 22 min 35 sec

• Administer doesn't seem like the best word.
• Initial configuration tasks are not provided up-front, but are spread throughout the administration section.
• On the 'Themes' administration page, enabled and default columns can be confusing. Nothing mnetions what enabled does.
• The indication of someone being logged in isn't obvious.
• Since users can choose their own theme, figuring out what each user will actually see becomes confusing.
• The 'Operations' column heading is a confusing word.
• 'Configure' is too generic and doesn't provide any context for what will be on the next page.
• The top links behave like tabs, but are not visually treated as tabs.
• The behavior of the lock controls on the color configuration is not obvious.
• The 'The directory files does not exist' error message is vaguely written. What happened, why it happened, and how to correct it are not answered.
• The error messages do not have enough contrast. It could be mistaken for a design element of the page.
• Customizing the page elements and the color scheme are different tasks and might want to be on separate pages.
• Placing 'Reset to defaults' next to 'Save configuration' are regressive and progressive actions. The buttons don't have much visual distinction.

• Administration front page
• Themes administration
• Garland theme administration

Creating a page

Watch 25 min 35 sec

• The "Content management >> Content" administration page doesn't link to the create content page.
• The navigation menu can make it hard to figure out what is in each section since all you get is the top-level links.
• The navigation menu has subtle indications for the current page and hierarchy.
• Required field indication doesn't obviously indicate its meaning.
• "Submit page" isn't the best title, "Create page" or "Create new page" might be better options.
• The text area resizing should save the size. An additional "resize" label might be helpful.
• The collapsed field set titles don't look clickable and can be mistaken for simple headings if there are other form elements directly below them.
• Authored on control requires knowledge of date formatting. A clock control would not have this problem.
• Authored on control could be either manually changing the created date or scheduling.
• The defaults in collapsed field sets are not visible.
• Previewing can make the page quite long. Tabbing between the preview and editing can be a good way to conserve space.
• HTML ability isn't immediately obvious since it is hidden in the input format collapsed field set.
• WYSIWYG editing would be a nice addition.
• The upload module might want to be enabled by default.

• Content administration
• Create content
• Create page
• Create page, field sets exanded
• Create page, preview

Ending comments

Watch 7 min 54 sec

• Mary: Labeling is the biggest concern. Many labels seem to be in developer-speak. Developers won't mind if things are in clear English.
• Donair: More descriptions, such as flyout descriptions for collapsed field sets.
• Sarah: More modules should be built in to make more things work off the shelf, but this can be a departure from the platform nature of Drupal.
• Jeff: Wizards to make themes can help people get people started more quickly.