Usability tests on drupal installation * DROP task: http://drop.cwgordon.com/node/61 Drupal.org: http://drupal.org/node/211815 Mentor: Bevan@drupal.org * Users:3 evaluators Friend1(19 yrs) - Some Java programming knowledge, no idea what a CMS is. Friend2(29 yrs) - Some basic Pascal Programming Knowledge, no idea what a CMS is Friend3 (29 yrs) - No idea what a CMS is. Uses Internet to mail, hi5... Profile: Users with no Drupal experience but with some skills in internet navigation, applications installation and some English knowledge. System: Drupal version: 6.x-dev Server: Apache 2.2.8 (local) PHP: 5.2.5 Browser: IE7 Database: MySql *I created the database and gave each user the Database name and user/password access Before performing the tasks, users were informed that: 1-They were going to create a new website trough the installation of a CMS application 2-All tasks should be done alone 3-The study target weren't them but the Drupal's usability 4-During the tasks always to keep present a critic spirit and the will to explore everything *The tasks were supervised by me – I just intervened when some translation difficulties came across or when the users couldn’t end Drupal’s installation because they didn’t knew what to do next. While users were performing the tasks I registered their reactions using these guidelines: >Hesitates or it’s forced to think >Doesn’t understand something >Gets fustrated or annoyed >Gives up Right after the installation I asked some questions: ********************************************************************************************************************************************************** ***************************************************************** Inquiry ***************************************************************************** 1.In a 1-10 scale (1 being extremely easy and 10 extremely hard), how would you classify the Drupal Installation? user1 -3 user2 -3 user3 -7 2.In a 1-10 scale (1 being extremely intuitive and 10 extremely counter-intuitive), how would you classify the Drupal Installation? user1 -3 user2 -3 user3 -7 3.What were the major difficulties you came across? When you found problems (if you did) was the help supplied enough to get by this difficulties or did you feel the need to find help anywhere else? user 1 and user 2 -Database configuration, the help provided was not enough to understand the difference between database types user3 - Choose language, the help available wasn’t succinct enough to encourage the user to install other language beside English 4.What were the easiest parts of the process? user1 - Choose language user2 - Choose language user3 – Site Configuration 5.Did anything unexpected happen? (if it did) What was that? user1- Warning at the end of installation user2- No surprises! user3 – Wasn´t able to select other language /Have to press go back in browser to get back to installation (in update notification information) ********************************************************************************************************************************************************** ******************************************************************** Notes ******************************************************************************* All evaluator’s actions were recorded in a video and then reviewed by me and the evaluator. Review’s guidelines: >what the evaluator wants to do first >where the evaluator gets lost or confused >what the user expected >where did the evaluator spends their time in the first few seconds of each new UI / page >when and where they search for help >where they search for help Finally I asked the evaluators to give their opinion and suggestions to Drupal’s installation process; especially at the steps they felt more difficulties on. Notes taken during the tasks execution and after reviewing the video (by screens): Videos of each user exerience: user1 - http://blip.tv/file/852655 user2 - http://blip.tv/file/852741 user3 - ****************************************************************** Choose language *********************************************************************** - One user was trying to install drupal in portuguese but, even using Drupal’s help, he wasn’t able to figure out which languages were available for installation – He quitted, and continued installation in English - One of the users had some tendency to click on the left side bar (where it shows the installation step you are at) - The other 2 users just ignored the “other language installation..” and just clicked in “Install Drupal in English” Reactions: “…what are the languages I can install Drupal? Is portuguese available?” (in how to install alternative languages) - link “…like the minimalist and clean look”(at first screen) Suggestions: It should have some kind of scroll down selection to choose from the available languages ********************************************************************************************************************************************************** *************************************************************** Database configuration ******************************************************************* - Two evaluators weren’t able to understand the differences between databases, and didn’t found any help to make that clear. Reactions: “…MySqli what is this? Is it the same as MySql?..” (in selecting the database type) Suggestions: Adding some kind of description about the differences between database types with a link like “if you want to know more…” would be helpful for first time users ********************************************************************************************************************************************************** ***************************************************************** Site Configuration ********************************************************************* - One of the evaluators didn´t changed the name because the field was already filled. - All users reacted very well to “strength password”, and followed the given advices. - One user changed the time zone, detected Clean URL selection was deactivated and came across with an unexpected behavior after clicking the link update notification information – he was trying to understand how the was the updated was done – He became frustrated when he realize the link didn’t opened a new window and he had to retrocede using the browser to continue the installation. Reactions: “…I can´t select this field. What is this?” (in clean URL) “…how do I get back to installation?” (in update notification information page) Suggestions: Opening a new window after clicking a link would be more practical during installation process, because to get back to installation we just needed to close or minimize the link page. ********************************************************************************************************************************************************** ************************************************************** Drupal Instalation Complete *************************************************************** - Two users were stunned by the big warning of mail function thinking the installation was not complete. - One of this two couldn’t figure out what to do next because he didn’t perceived the enter new site link. - The other user just read the warning and thought the problem was because he entered a fake mail. But he had no problem finding the new site link Reactions: “Hey?! What went wrong?...” (in just after complete installation) “uoU! Didn’t installed?! AHh OK! What is this error about?” (in just after complete installation) “…giant red message…” (in just after complete installation) “What now?” (after he was warned the error didn’t hinder him to complete installation) Suggestions:The warning should be shorter and the “enter new site” and the congratulation message should be more evident. Maybe a button in “enter new site”? ********************************************************************************************************************************************************** Videos of each user experience: user1 - http://blip.tv/file/852655 user2 - http://blip.tv/file/852741 user3 -