Doing a usability test of Drupal means you will observe while a friend, family or anyone will attempt to perform a series of tasks with Drupal. This kind of user feedback is fundamental to improving the usability and user experience of Drupal. It will help you build a better understanding of users in general, allowing you to design better user centered systems future wise.
Beside reading the Usability test kit, there are a few quick steps to follow :
Recruitment
Test session (30 minutes)
Analysis
There is no particular skill required for doing usability testing, understanding of Drupal 7 is a plus but not required. Although this is a design task, no actual design work is involved its only about observing users and reporting back. We suggest trying to recruit up to 3 participants (more is always welcome).
Deliverable
The deliverable is a summary of your findings and possible recordings or notes(of individual sessions).
Resources
Primary contact
Bojhan
| Comment | File | Size | Author |
|---|---|---|---|
| #33 | usability.xls | 37 KB | Dan Silver |
| #1 | Usability testing kit.zip | 50.91 KB | Bojhan |
Comments
Comment #1
Bojhan commentedThe usability test kit documents in a zip.
Comment #2
Bojhan commentedThis task is medium-hard difficulty as it will take some time and can be hard to recruit participants.
Comment #3
webchickExxxxxcellent! This is one of those great tasks we could re-use a couple of times, too.
Comment #4
cwgordon7 commentedThis looks great, thanks!
Comment #5
bowersox commentedsub
Comment #7
trevjs commentedI'm jumping in here because this seems fascinating, and I'm hoping I can add something constructive. May I suggest changing "There is no particular skill required for doing usability testing, understanding of Drupal 7 is a plus but not required." To say, something to the effect of, that there are no technical skills required, an understanding of Drupal 7 is useful, but that people skills are very important in order for the participant to feel comfortable and to get them to share their experience openly. Qualitative analytical skills are also important for identifying patterns and outliers when constructing the final report.
I think stating it this way does justice to the task, because there is something to usability testing (though I've only conducted tests twice for classes we've been talking about doing it at work, and looking back making people comfortable was important). And we do want it to sound like it provides an interesting challenge.
Comment #8
Bojhan commentedI understand your point, but I believe there is enough challenge in it already - raising the bar on the required understanding only diminishes the chances we get a student working on this already fairly challenging task. However please consider this in perspective a student from 13-18 needs to be able to do this :
1. People skills : Its likely they ask people around them, this is fine and should be enough not to make the participant feel awkward.
2. Analytical skills : We do this in collaboration with the student, its unfair to expect this from someone who probably hasn't done it before.
Comment #9
cwgordon7 commentedHi Bojhan,
Can you please estimate the time, in hours, that it would take a student to complete this task once it is claimed? This estimate is required to input the task into the google system. Thanks!
Comment #10
Bojhan commentedThe maximum? 10 hours
Comment #11
dawehnerhttp://www.google-melange.com/gci/task/show/google/gci2010/drupal/t12904...
Comment #12
dawehnerwrong issue: http://www.google-melange.com/gci/task/show/google/gci2010/drupal/t12904...
Comment #13
dawehner.
Comment #14
webchickComment #15
dawehnerComment #16
Dan Silver commentedI'm not sure if I'm supposed to comment here or not, but I have applied to claim this task on the google-melange.com site. Am I supposed to follow it here (Drupal.org) or on the other site?
Thank you and I look forward to doing this task if I get it!
Dan Silver
Comment #17
cwgordon7 commentedYou need to upload your final work to the google site, however all work and review and stuff will be done on this issue queue. You can assign this to yourself. Thanks!
Comment #18
webchickYep, we like to have you do interaction with the community in this queue because then you get to see how a "real" open source community operates, which is not on google-melange.com :D
Comment #19
Dan Silver commentedAssigning this to myself. Thanks!!!
Comment #20
webchickwoohoo! Go Dan Go! :)
Comment #21
Bojhan commented@Dan Let me know if you have any questions, I will be on IRC or Skype if needed.
Comment #22
Dan Silver commentedAn update on my progress. I have found 2 people today that are willing to do the usability test. One of them is a college graduate who has a good amount of experience in Joomla. The other is in high school and knows WordPress very well. Tomorrow I'm going to continue searching for a third person and on the 27th I will start the testing. At last resort, can someone that I know who has used Drupal 6 for less than a year (basic user, non-programmer, never used Drupal 7) be my third person?
Comment #23
Bojhan commentedYes, for him(the one with experience of Drupal) you need to use the Intermediate test plan. Others the beginner.
Make sure you use the latest Drupal 7 beta, and run through the test script once yourself.
Comment #24
Dan Silver commentedIf I'm supposed to give them the database information so they can install Drupal, how am I supposed to set up the environment so the who's online on the right hand side before Drupal is installed? After they install Drupal should I pause the test and move it over?
Thanks
Comment #25
Bojhan commentedHey,
You are right, I am sorry! I think you can adjust that task a bit and say to place it under Navigation on the left side - the primary think is really making clear "where" to place it to the user, maybe even by pointing to the navigation menu.
We made the script was made with an install script in mind, but thats gone a bit outdated - so improvise where you see necessary but there should be very few occasions. Avoid any work around that will require you to take control though.
Comment #26
Dan Silver commentedAn update on my progress. I have finished testing 2/3 people so far. The third person is taking the usability test tomorrow night. I hope to have a video summary and detailed notes of each test done and uploaded on December 1st.
Dan
Comment #27
webchickThanks a lot for keeping in such good contact, Dan. :) Greatly looking forward to your results!
Comment #28
Dan Silver commentedI have uploaded a video summary containing short clips and descriptions of problems that the users had during the study. It's on my site since the maximum file size here is 1 mb. I apologize for the low video quality, I'm having trouble with my video editing software so I will try to have a better version up within the next few days.
http://www.futuremessage.org/usability.rar
And I will also upload my spreadsheet notes tomorrow.
Comment #29
Bojhan commentedGreat! I will review it later today.
Comment #30
magpie5212 commentedSubscribing...
After over 25 years working on usability in the industry I may be able to help with review.
Having read the test pack, I see this is a test for developers and installers. Is there a test being organised for end users? That would be for just site readers, and for non-developer contributers.
Comment #31
Bojhan commented@magpie5212 This test is for site builders. As much as we want to do it for content creators too, we need volunteers to get this going :) : to develop the test plans and do some initial runs.
Comment #32
Bojhan commented@Dan Silver Can you supply me the full videos? Possibly without the faces (not sure how you recorder that - but its because of privacy of the participant).
The highlight looks very intresting already!
Comment #33
Dan Silver commentedHere is a link to the last usability test (without his face), and I was able to get it in HD this time :).
http://www.futuremessage.org/usability_full_1.rar
Unfortunately, it appears that my video editing software (Pinnacle Studio) is doing more harm than good, as it appears that the first two participants screen captures have been partially corrupted. I'm not sure exactly what happened or why it has occurred. I can go through the video and pick out spots that are still ok and make that into a video, but that will take a day or two.
I have also attached notes that correspond to the issues in the summary video (from post number 28).
Comment #34
Dan Silver commentedI just looked at the Google page for this task and I have 23 hours left to complete it. Time has gone by fast! What else should I do to be able to complete this task? I will do the video edits described in my previous post immediately after school tomorrow. I'm also rendering a HD version of the video summary overnight tonight and will upload it in the morning.
Dan
Update - Here's the HD version of the same summary video:
http://www.futuremessage.org/usability_hd.rar
Comment #35
Bojhan commentedI will get you extra time, assume you will get another 7 days. Take your time to see if you can recover the movies, I suggest if you have raw material that works to use a diffrent video editing tool like Camtasia.
Did you make any notes during the test, or can you remember the problems? I would love to see more detailed problems of the sessions, the excell attached seems a little sparse(if you can partially/fully recover the movies I can help with this)
Let me know if its too much.
Comment #36
Dan Silver commentedThanks for getting me more time. I will start on the videos later today. Yes, I did take notes and I will type those up and expand on them.
Comment #37
Bojhan commentedCan you submit something for review in the google system? That way, we can extend it.
Comment #38
Dan Silver commentedSure. Unless I hear otherwise, I will just submit a link to the full video in post number 33.
Comment #39
Dan Silver commentedI submitted some work earlier but it still shows that the task is ending in 36 minutes!
Comment #40
Bojhan commentedYhea, we had to wait for something to be reviewed - before we could extend it. You got it though.
I have watched the HD clip, thanks for the summaries in between - really excited by this :) Its great work and clearly shows some of our flaws, looking forward to the notes + other videos.
Comment #41
Dan Silver commentedI'm working on some of the other clips right now and the notes. Should have it up within the next 2 hours.
Comment #42
Dan Silver commentedI have uploaded 2 more videos of the other 2 testers and more detailed notes. Both of these videos are 10 minutes long, so hopefully this will get us more data. I apologize that so much video was lost. I know someone that took web design courses in college and uses statistical software every day in a hospital, should I do another test with the new software that I'm using? I am more than willing to do this.
http://www.futuremessage.org/user_1.rar
http://www.futuremessage.org/user_2.rar
http://www.futuremessage.org/usability_notes.docx
(In the word doc, user 3 refers to the person who is in the previously uploaded, full length video.)
Comment #43
carlos8f commentedsubscribing
Comment #44
Bojhan commented@DanSilver Please put all these files in Google-melange for review so we can complete your task. Although sadly, much of the videos was lost I still think you have done a terrific job working on this task! I already seen quite a few issues that we need to work on in Drupal 8, some relatively minor labeling issue to more critical issues about users not understanding main concepts.
Its not neccairy to provide another participant, however if you like doing this task - I would be more than happy to create a new usability testing task in Google Melange (can you possibly do two participants then?).
Comment #45
Dan Silver commented@Bojhan I submitted all of the work to Google-melange for review. I might want to do this again, but not right now. I'm going to search for a task relating to php so I can get practice coding.
Comment #46
simohell commentedThis jumped in on my search as "active" - so for clarity I'll close this as outdated as the last activity was 12+ years ago.
We have made good progress since.