University of Minnesota 2011

Summary

From May 17 - 19, 2011, in advance of DrupalCamp Twin Cities, a number of Drupal community members met at the University of Minnesota usability lab in Minneapolis to perform a round of formal usability testing on Drupal. This is the fourth such test we as a project have done, and the first targeting the new Drupal 7 release.

The great news is that most of the changes we made in Drupal 7 seemed to test very well; Drupal was no longer confusing participants with basic conceptual hurdles like where the front-end vs. back-end of their site was and how to create an "About us" page, and for the most part the places in the administrative interface that people needed to go to perform various tasks were very clear; much less blind clicking around.

The bad news is that now that some of these basics have been dealt with, we've uncovered a whole new layer of challenges that first-time Drupal site builders encounter, including some that were pretty surprising. Modules, blocks, and content types are really difficult, as are many of the obscure words that we use *(e.g. does anyone without an art degree know what a "Triptych" is? :)).

This report talks about why usability testing is important, goes over the methodology we used, reviews previous findings,

Why usability testing?

Usability testing allows us to uncover issues which otherwise get unnoticed. The participants of the usability test represent our "real" users doing "real" tasks. The intend is obvious and simple to make Drupal easier to use. Also, usability testing is the second most used evaluation method and has the most impact on making products better.

The learning curve with Drupal is a steep one and we need to work on making this a smoother road. Agree, this is easier said than done but we need to start trying to fix things one at a time for a more usable Drupal 8, the emerging Drupal market and for YOU. You could be a cool programmer, creative designer or a smart consultant working for Drupal, the list is endless. There is a benefit for each one of YOU in making this initiative a success.

Methodology and Participant Demographics

Whom did we test and why?

After a healthy discussion on Usability groups (http://groups.drupal.org/node/137109#comments), the team decided to focus the usability study on our primary persona Jeremy . The test was focused on the site builder who is curious about Drupal and inexperienced with Drupal. You can find a more details explanation of the test in the Summary Report.

Resources:

• Presentation (pdf): Covers results of testing, given at DrupalCamp Twin Cities.
• Summary report (odt | doc) - Contains overview of process, who participated, and what methodology was used.
• Appendix A - Issues Analysis Matrix ( ">xls): A detailed list of all issues found during testing, graded by severity per participant, and how much community involvement will be required to fix.
• Appendix B - Desirability Matrix Responses (xls): List of words about Drupal chosen by participants after their tests
• Appendix C - Scenarios (odt | doc): What participants were asked to do
• Observer Feedback Form (doc): Template for questionnaire that participants were given after their sessions.
• Mockup (png): "Target" wireframe of sites that participants were given to help guide them in their completion of tasks.
• Recordings: Full video (screen + voice) of all 8 participant sessions
• Issues: Issues that we identified were tagged with UMN 2011.