Defining project scope

Hey,

The Usability Testing Suite sounds really great to get usability research on-going in Drupal as usability tends to be more a process then a project you want to continuously monitor developed modules there improvements and behavior over-time.

Note : Test early and as much as possible, trying out different solutions to usability problems and keeping it a process instead of a phase of the development cycle. For us its best to have more testing, then the best method of testing, since obvious usability issues will always be recognized.

If I am correct the Usability Testing Suite will pretty much be a platform to plug usability methods into where a usability test where you run the person trough predefined task(s) and get feedback upon that is the current method we want to pursue.

Since I really suck at attacking each question you gave at once, I will also do a brain dump and try to get into some questions you have.

A quick word on remote usability testing, and how its applied in most of the world today. There are several suites some rather expensive to cheap ones that offer screen recording and facilitate in the research process. Most of the actual in browser ones require add-ons to be installed, in general I find most of this quite a hassle. To get the test person to install all kinds of client-side software. And work with it correctly. http://www.techsmith.com/morae.asp is quite great at all this and facilitates immensely in the remote usability testing but costs around 1.4k but the biggest drawback is that setting up takes quite some time – I haven’t been able to use Morae that much but most of what I saw was amazing. My advice on this is to be as in-sync with an actual task as possible, if this means hidden testing it’s even better and to make testing a quick process (avoiding requirements as much as possible), so it can be done often.

What we are testing?

Task path testing – The entire flow
As Bevan explained me you will be testing a given task, this task has either one or a couple set modules that are required to run trough in order to complete the task. As I understand the user will be free to move among the site, so there is a so called “happy path” the path the user should use to most effectively complete the task. And the user research will show all the other paths (the different pages the user visited) in order to complete the task, like going through a page that was completely not associated with the task but the person expected it to complete it in that way (mental model vs. implementation model)

Task Element testing – Individual parts of a flow
This is where a certain element of the interface of a module is tested, so for example a form or a combination of a few forms. In this you will test a small part of a task.

As far as I understood wont we be testing in the wild, so where no specific task is given or identified. A big note here is that the developer will have to do some kind of research to find out what task the user wants to complete, especially big modules as views2 will have the problem of finding out what the most prominent task of the user is, some form of feedback from the user as what do you want to achieve with this module in the initial questionnaire would be interesting information like what the user wants to do is actually totally not what the module does :’)
However the most obvious usability problems will most of the time be notified by any type of task, so there shouldn’t be too much worries around this, especially as we focus on testing early on the process most tasks can’t be completed fully we have to give extra thought to task element testing.

Are we going to make the distinction in task-testing between performing a task, and getting it? So for example letting a user stare at a screen for 20 seconds, and then asking if they got the purpose of the forms or will we only go for performing a task and with that task completion/failure?

Who are we testing (How many)?

It’s very important to know who you are testing, just as important as it is not to focus your testing too much. The general idea is to focus on the most prominent user tasks which should include about 80% of the preformed task, however with Drupal that last 20% is still a whole lot of people (like a couple thousand?) so only focusing the tests on the most prominent user will essentially hurt that other 20%. And sometimes people tend to focus their tests on diversity, trying to get the expert, intermediate and beginner in where they end up with a complete different group then who will be actually using it ( Ask the question : What attributes will cause one user to behave differently than another?)
So although an test designer would love to test a specific audience like someone who wants to build a community with minimal Drupal experience (or no, as done by the University staff) . In general, this is hard to achieve and more important it takes a lot of time and when you want to release your module in 3 weeks you can’t wait too long. So for the initial development of this module I advise to avoid focusing too much effort towards getting really specific users for testing, but more towards loosely (within limits) picking the users for the test.
Is it possible to identify the user by the modules he is running? For example a drupal wiki module who wants to setup a usability test, would want users who are also running drupal wiki tools. In cases where the module will be almost exclusively used by one type of user its far more important to get this one type of user then when your testing CCK or Views2.

For obvious usability problems you don’t usually need a lot of people , but they will miss underlying usability problems since they only see the big ones. Common sense will say, recruit more people, they will notice it. But the best way to go here, is to fix the prominent usability problems and let them test it again.
I would love to get participant profiles, but how much of a strain would this be on the user? Questionnaires can’t possibly cover all the different scenarios all the models on Drupal cover so is it possible to get some hidden facts about the user? Like modules they use, how long the site has been running, the scale of the data on the site? (For modules that you test on their admin interface this would probably be of great value, but public parts of a module this will be useless) Maybe the best participant profiles can be setup by contextual questions, when they are using the system.
A good profile will help us indentify users with a particular experience using modules differently users that don’t share this same experience.

What are the tasks?
As webchick how do you make task/scenario’s and with that descriptions without them being leading is crucial to the validity of the test. We could give tips on how to write good tasks and how to identify the most prominent ones amongst your users, but at the end of the day it’s in the hands of the test designers expertise so setting requirements to that wouldn’t be a good idea.
The best tasks, avoid us from thinking of the steps involved. And if we don’t talk about the steps involved, you can avoid using languages that they can use as directions.

Where will the user start his task?
So will we let the task start at the module, or does the user first has to indentify which module(s) to use?
What result means task completed?
Remote usability testing means a user can take all the time he wants to do a task, however when will we decide a task is failed or completed?

Will we have some kind of advise on the different tasks/scenario’s to choose, like the 37signals will advise the defensive tasks, can you easily get to know when you can’t do a task? Like misuse scenarios, exception scenario’s, what-if scenarios, elaborated scenarios.

Ohh, and onto having errors in your tasks it should be normal to dry-run them before you run usability research.

What data should we capture?
I am somewhat opposed to clickheat, because to me making the user click all the time doesn’t sound like a viable method, noting down that people tend to look really fast and think before they click (http://www.uie.com/articles/users_decide_first/) . It is fancy looking though, just like eye-tracking is.
Video feed, audio feed all of this to me sounds like traditional usability testing that is invaluable but hard to apply in remote field research that you want to do often and user-friendly. I am still thinking heavily on what we should capture as written feedback alone sounds not enough.

To me a combination of methods as multi-variation testing (different versions to see which one has the best task completion that fits with the users expectations), 5 second testing for ‘getting it’ tests and (I have no name for this method) a test where you let an more experienced Drupal user talk about the module and find out how he completes a task against a less/no experienced Drupal user. In this way you can benchmark, how long in general they take to complete a task and with testing out different solutions how to fill the gap for the beginner.

How do we present the data?
As you said, people didn’t see the usability report. Why? Well there is a whole bunch of reasons to this, and I would like to get more in-depth on this before I can provide viable solutions and actionable issues right-away. And field testing, should have some sort of presentation of the problem for others otherwise you won’t be able to convince them of how critical the issue is.

I will reply to this issue queue more with some better examples and more in-depth information on what I just said, and also more applicable functional stuff that we can work with straight away. Maby we should break this discussion up in parts that derive a function list (What are we testing? Who are we testing, and how will we get them? How do we get the “right” tasks? How will we capture the tasks? How will we present the usability test? What methods will we use in the test? What are our constraints?)

However I want to note that this is really a big project that has allot of key elements of it in order to work, so we do have to find out more about our constraints. We cant allow our self’s to obsess about numbers as much as most online usability analytics do (or Jakob Nielsen for that matter) so we have to look for solutions and usability methods that are outside of the box.

I apologize if I haven’t covered all the questions or if I am a bit vague at some points, I haven’t been able to proof read it or go as in depth as I want due to time issues.

Useful links
Morae Observing, Analyzing, Sharing ( I really advise looking at this one)
http://video.techsmith.com/morae/latest/demo/ondemand/index.html?movie=2
Web Apps Podcast 2
http://www.uie.com/brainsparks/2008/02/13/usability-tools-podcast-succes...
Web Apps Podcast 1
http://www.uie.com/brainsparks/2008/02/07/usability-tools-podcast-useful...
Online Usability testing
http://www.leotrace.com/leo_leistungen/index.html
Online Recruiting
http://ethnio.com/
Luke Wrobreski on Content and Form
http://www.boxesandarrows.com/files/banda/ia-summit-2008-day-1/Content_P...
Must have book : Dont make me Think (Steve Krug)

On the issue of scope, is it possible to clarify the difference between Master Server testing(or how it is called), and preforming testing on your own site.

Angie & others,

Here's my brain-dump in response to your brain-dump! We have some aligning to do of our ideas and the UTS requirements! :)

I also have a more formal brain-dump here: http://www.civicactions.com/blog/scaling_up_usability_testing_in_drupal

This module needs a strong documentation component with checklists and such about how to choose good tasks

I'm not sure that this should be part of the the UTS's interface. I think it would be better documented elsewhere.

I wonder too if we need some sort of centralized "usability test approval body" to sign off on things.

If Drupal community members write tests for the Drupal community or project, then this would be ideal -- in a similar way that handbook pages would ideally be proof-read by others before being published. However if a UX professional wants to use the UTS to test their client's Drupal website or application, then this doesn't make sense. In other words this sort of moderation should happen in the community, not in the UTS, and the UTS should focus on handling the technical details involved in doing remote informal UT (usability testing).

I have absolutely no idea how we're going to 'enforce' good participant selection across hundreds of these field research usability tests.

As above, I don't see this as being in scope for the UTS or the GSoC project. That is the responsibility of the UTE (UT engineer -- the test designer. Usually a designer, developer, UX professional of the website/application being tested -- probably Drupal community members if the subject of the tests is Drupal core or contrib).

I guess we'll need some sort of "participant profile" type of thing that people fill in, and will need to discuss what that will entail.

A part of the usability 'unit' test, a UTE can request or require that a participant/evaluator answers certain questions (as defined by the UTE). These questions can be inserted into the 'unit' test before, after, or in between tasks. The UTE could require that tasks change dynamically according to the participant's answers. Such the UTE could define that there are no tasks if the user has used Drupal before, or that task #3 is only part of the test if the participant has not used Views module, has used Drupal and completed Task #1 with some success criteria.

Technically, the UTS could possibly take advantage of existing modules, like survey, CCK node forms or webform module. I'm not sure how to manage this dependency though. What if someone wants to test Drupal 7 core, but CCK is not updated to d7, and the UTS has a dependency on CCK? Perhaps the UTS runs on a separate instance of drupal? If so then perhaps we may as well expand the scope of testing to include ANY website, whether built on Drupal or not?

IRT audio and video feeds, I think Jimmy mentioned that data capturing plugins for audio and video are likely to be out of scope for the GSoC portion of this project.

We'll need usability test leader to instruct participants that they should move the mouse wherever their eyes are moving, which is pretty ghetto, but it'd work.

Such a request may overload the participant and cause them to behave differently. So this would need to be optional at the UTE's discretion. This would be part of the 'unit' test that the UTE creates.

this module might be administered by a single admin at some local Drupal meetup

Further, it should be able to run remotely, so that a UTE can see the data from anywhere they have access to the server running the test, which is probably also the server running the instance of the application being tested. Test data analysis is assumed NOT to be done in real-time. This is the big differentiating factor between usability research with the UTS and other types of usability testing (both formal lab testing and informal 'over the shoulder' testing are done mostly in real-time). This is both an advantage and disadvantage, but is mostly just a different alternative. If the data capturers, hardware and network (think screen-casting and audio-streaming) are good enough then this might be possible.

It'll also need to be able to scale from someone's crappy 486 HP laptop

Agreed. The challenge here is data capturers like audio and video which are resource consuming. Capturers may be turned off by the UTS or the participant if they don't have the required soft/hardware, such as webcam or a compatible OS for the screen-capturer. The UTE should optionally be able to require certain data capturers for a 'unit' test. Such that if test data is only valuable with an audio feed, the participant's time is not wasted doing a test with no audio. The UTE would define this as part of the 'unit' test creation, as with the survey, requirement questions and feedback-soliciting questions.

we'll need to figure out what are the key pieces of feedback to capture that we get the most "bang for our buck" on.

If I understand correctly, this is the UTE's responsibility. These would be questions in the survey that are flagged as 'feedback' and can be analysed, as opposed to 'requirements' that are checked order to qualify a participant to do a certain task or test.

After each test, we'd go through our notes and write any issues into a big spreadsheet, along with checkboxes for which participants had which issues.

This is where in can see the UTS being really valuable. A lot of this can be automated in the analysis process by asking for the right sort of feedback in the questions. This will often mean asking for numbers or a selection, instead of writing text. Although some really smart algorithms could probably also analyze textual feedback!

Something we did NOT do very well was keeping up with logging issues to drupal.org about stuff that came up.

I agree. I think this will always be an issue (with findings from usability research) in open source projects. Ideally the UTE would be responsible for doing that or otherwise delegating that responsibility. But open source doesn't work like that.

An addon module could possibly be created that aids a UTE in matching or creating d.o issue nodes to results/issues found post-analysis of the tests? This shouldn't be part of the UTS proper because it wouldn't be useful for UTs on projects that aren't Drupal core or contrib.

Moving on to Bojhan's looooong comment...

http://www.techsmith.com/morae.asp is quite great at all this and facilitates immensely in the remote usability testing

Morae seems very similar to what I have in mind for the UTS. I have downloaded the trial software (Windows only!) and the video.

Angie and Jimmy, please checkout the "How it works" tab. Click through to the "Recorder" page too. Videos at http://video.techsmith.com/morae/latest/edu/tutorials/enu/mor_tutorials.... are good too, especially the "Modify the Configuration" and the "Record and Log Participants" videos.

I am still thinking heavily on what we should capture as written feedback alone sounds not enough.

I think what is captured depends on a number of things -- most importantly, what is being tested and what resources are available (e.g. Does the user have a webcam? Does the user have the screen-casting software installed?).

Some data capturers would likely require the remote machine have special software installed. These could, at best, simply be browser extensions. Allowing a UTE to require or request such data-capturers to be available is reasonable because participants may be technical enough to feel confident installing them. Or the UTE may facilitate a test (or coordinate with a test facilitator) and provide the participant with a computer that meets the data-capturing requirements, or assist the participant in installing the software before commencing the test.

In this way you can benchmark, how long in general they take to complete a task and with testing out different solutions how to fill the gap for the beginner.

Split testing with UTS will be possible by having two separate 'unit' tests, or two separate test environments and comparing the results, after or as part of post-test analysis.

On the issue of scope, is it possible to clarify the difference between Master Server testing(or how it is called), and preforming testing on your own site.

I'm not sure what you're referring to here.

Once Jimmy has checked in here and shared his thoughts or ideas, I'd like to schedule a group call or IRC session to bash these ideas out and try to come up with a more consistent concept of what the UTS is, it's scope, and requirements. We also need to work out how much of it is part of GSoC, and what isn't. Perhaps we could start defining entities and create some sort of glossary and relationships. E.g. is UTS a good name? UTE and 'unit' test are silly words that don't consistently communicate the concepts. Morae uses 'Study' for 'unit' test, which I like.

As for relationships, e.g. Project has at least one study, studies may have 0 or more sessions, sessions have exactly one participant (evaluator), studies have exactly one (albeit large and complex) analyzable dataset. Studies at least one required data capturer, and 0 or more optional data capturers. Each data capturer must do X, Y and Z...?

What times (UTC) are you available? I'm available (with notice) approximately from 7pm to 12pm most weekdays.

Bevan/

If I am correct UTC, is GMT-2. which means the upcoming days, till Thursday the 29'th of May I will be available from 5PM till 12PM and after that I will be available any time (working at home :) - gotta love field testing, hehe.

So the immediate problem with any type of capturing trough video is that the files are simply huge, Morae is famous for this which simply inst effective if we want a widespread adoption of usability testing. As far as I am aware are there no browser extensions on capturing a webbrowser. I am somewhat worried though, that you see UTS as what Morae pretty much is. Because, why not use Morae then?

Part of the scope, for who are we building this module? You gave a very vague description of who acctually, but its my impression we are building this for the programmers (which is the majority after all), is it not more important to take into account their current knowledge of Usability when go for scope? Because, it will means we have to educate and take away certain strains on the programmer trough the interface (ie, talks about if we have to find the usability participants)

On the issue of scope, is it possible to clarify the difference between Master Server testing(or how it is called), and preforming testing on your own site.

With this I meant what I understood from Webchick that UTS should test Drupal modules by the community, so tests that are centrally runned by the community. And then tests that are runned by someone with a Drupal site.

I want to point out even more usability testing should be done often. Morae is great tool for usability experts, you do need some knowledge on how to setup the stuff and interpret the data. I can see problems arise for programmers who have no idea how to find the participants, set the right tasks, choose the right data to be captured, facilitate in the whole testing process and then successfully interpreted data. Apart from that its just incredibly time intensive and that won't help anybody.

Upon presentation again, its acctually not really a open source problem. Even in those big companies, usability is usually treated as something that produces a document. And this document itself is just archived somewhere, however the real value is in the discussion about the document, where programmers learn about the user and implant changes (but still often not). Apart from just the issue que, it might be interesting to present them on a website? Maybe not just the Usability group but a centralized site (to broaden our reach and have more effective communication)?

I must say, if you can find algorithms in textual feedback, you will be able to work at any usability company out there. Although there will of course be some patterns (obvious ones) its still all about interpretation what the user (tries) to say. Although stuff like, was this module like you expected survey questions chose no / a bit / good / completely are all about what the user interpretation as no / a bit / good / completely so this kind of quantitative data is utterly useless, if we go for anything quantitative we have to re-think common usability approaches since (http://www.useit.com/alertbox/20040301.html) its either really hard to do or simply produces the wrong kind of numbers to work with (Jakob is famous for doing just that) ie determining margin of error (http://www.measuringusability.com/stats/) for the type of Usability testing we probably will be doing, I am not sure that anything as statistical significance is really useful for anything but sure will confine programmers of issues.

This time I tried to keep it as short as possible, Bevan :).

I think that one of the major priorities needs to be determining what the UTS will be able to do and what is part of the GSoC project. This project will most certainly need to be developed after the GSoC has ended and may even be continued in another GSoC project.

My understand of what needs to be done first and what the majority of the GSoC project will focus on is creating API and structure for later improvements. To complete this we need to figure out patterns of testing that can be put into an API and data structure. A major part of this is the way in which tests will be create, administered, and reviewed (or work-flow).

By the end of GSoC the essential APIs, interfaces, and data structures should be in place with several simple API implementations. In other words a few basic data collection and exportation plug-ins. These basic plug-ins can later be expanded on to provide more higher level plug-ins such as streaming audio and video.

I am a proponent of the "design as you go model" in other words don't try and design your entire project before you begin since I have found that you will never think of everything. Based on that I think we should take things one step at a time. There are a lot of good ideas here, but there is much to be done before many of them can be implemented.

So to start things out we should focus on the first item on the list.

Design and implement usability unit test creation and review interface, and provide database structure for usability unit tests.

This involves creating a set of screens, or wizard, that a UTE (I believe) will use to create the definition of a test, or study, and what data will need to be stored in the database.

We will still need to discuss some of the upper level things, but I think we should focus on parts that can be written and re-factor if we come into problems later.

We should probably make threads devoted to individual things to help break down the conversation.

Short answer to Jimmy, I am not so much of a programmer so I have some troubles understanding what you say, however the design as you go is great if you're working on a small project. Usability studies are quit broad, so we do have to define some scope early on, to me it sounds like lets go build a community module (and not defining on the forehand what kind of community module).

I already guessed you would be looking for a more drilled down function list, though not really sure where you are really asking for with the first item on the list (Can you care to explain that in noob terms?)

Ohh, Bevan about UTS, I would love it if you chose study instead of unit. I wanted to point out that "unit" is kind of a programmers term, however I seem to be the odd one out here constantly talking about how stuff is called just doesn't seem right.

And another link : http://www.keynote.com/products/customer_experience/web_ux_research_tool...

I'm sorry, I will try and keep my language in this topic friendly for all readers. I know what that is like since Bevan and I had terminology issues during our first discussions.

First of all, my understanding of what was to be accomplished was fairly scoped out as it is written in my proposal. Any discussion here can make changes to that proposal, but that is the basis for these discussions.

The first item on the list refers to what information needs to be collected from a UTE and how that information should be collected. By that I mean how to layout a set of screens, or a wizard, that will collect information from a UTE when creating a study.

This includes all the specific forms that define the task, what kind of participants should be found, what data to collect, etc. So from a higher level the basic definition of what a study will consist of so that I can create some sort of interface, wizard, to collect the information and store it.

I plan to stub something out based on everything here and other discussions and allow others to give me feedback on the interface.

I hope that helps.

Threaded replies to both Bojhan and Boombatower,

> If I am correct UTC, is GMT-2.

UTC == GMT. UTC is the same as GMT.

IRT audience of the UTS: I primarily see this as UX professionals, the "Drupal Usability Group", and possibly drupal developers that are interested in the usability of the intrfaces they are working on.

> I can see problems arise for programmers who have no idea how to find the participants, set the right tasks,

Of course there will be. No module can teach this, so it's not in scope for the UTS. Programmers wishing to learn how to do UT need to learn this elsewhere.

The UTS shouldn't focus on testing individual modules. A 'study' in the UTS would often be more general and, for example, be able to test the UX of a site visitor to The Onion, as well as a Drupal developer's UX with Views 2 betas.

> Ohh, Bevan about UTS, I would love it if you chose study instead of unit. I wanted to point out that "unit" is kind of a programmers term, however I seem to be the odd one out here constantly talking about how stuff is called just doesn't seem right.

I agree. So a UTE (How about Usability Expert" or "Test Designer"?) creates a Study, which has Tasks, Participant Requirement questions, Client soft/hardware requirements, Feedback questions, Required and Optional Capturers, Test sessions, Data, and Analyses. A Project has at least one Study with further cross-study analysis (out of of scope for GSoC?).

Bojhan, your suggestions for terminology and language are much appreciated since folk like you are a target audience for the UTS. The UTS needs to speak YOUR language :). We also need to consider folk that are learning about UT and need to learn the domain's jargon and parlance.

> First of all, my understanding of what was to be accomplished was fairly scoped out as it is written in my proposal. Any discussion here can make changes to that proposal, but that is the basis for these discussions.

I agree. I still feel some lack of clarity IRT to entities and their relationships. How do you feel about that? Should we sketch up some UML diagrams or something? WDYT?

It probably wouldn't hurt to sketch something up to make sure we are on the same page.

If you would like to take charge that would be great.

I graduated this last Thursday so I should be free most of the time. I have a few engagements and activities, but if we get a day set we should be able to work something out.

I am UTC - 6 (Central US) so anything that works for Bevan from afternoon til night should be fine.

Jimmy,

The 1st semester at my University is coming to an end in 10 dayst and I'm quite busy with a final assignment. I'll try make time this weekend for a draft "entities and relationships" diagram. "UML" is probably too formal for what I have in mind.

Is there anything holding you up with getting started on this? I don't see why we couldn't start sketching the primary UIs, like for creating a Study and it's requirements, questions etc..

All,

I'm not sure an IM chat would be very productive at this point. What do others think?

Also, I've just found out I will be travelling for 2 weeks at the beginning of July. I will be connected most of the time but may not be as available as normal. We don't need to do anything now, just advance warning...

B/

I will be traveling June 8th - 15th. That may push the milestones around a tad, but I should be able to make up for it.

I have begun coding the first stage: creating the UI for studies.
It currently supports:
* Basic study info
* Add/Edit/Delete operations
* Add/Edit/Delete tasks related to a study is almost finished.

Feel free to download a dev tarball and try it out. I plan to have a somewhat functioning interface for everyone interested to react to. That will give a starting point for further improvement and developement.

Jimmy,

Great to see you've started on this already!

I'm a bit concerned that you're implementing the interface before designing it. It would be great if we could roughly sketch out the interface on paper or in a drawing program and review it before putting too much effort into implementing it. That's what I understood you were going to do, by "Design and implement usability unit test creation and review interface". This approach will encourage a usable interface that makes sense for our users (UTEs) and their tasks. It would be a shame to have a badly usable interface for a usability testing interface! ;)

Paper prototypes (or simple drawing program prototypes) are cheap quick and easy to implement to try out different ideas and change the design as much as we need while we discuss and design the UI. Drupal, php and html interfaces are not. I don't want to put too much time into this part of the project, but I think two or more rounds of design and reviews of cheap prototypes is necessary.

Also, I had forgotten to mention earlier, feel free to take on the "Entities and relationships" diagram yourself.

I just installed and tried out the module. It looks good. However I still think designing the UI for these processes is necessary. For example, a "Task" probably needs it's own page, and add/edit forms, since it has it may have it's own Task-specific questions, requirements etc.. However an overview/summary on the study page is necessary. We probably also want to be able to re-use tasks in multiple studies.

If we establish these UI-requirements with cheap prototypes before implementing we don't need to redo the interface ten times over (or worse, NOT redo the interface and have a difficult to use UI) and will save time in the long run.

What do you think?

Before going any further on the UI I think we should figure out http://drupal.org/node/263968 and http://drupal.org/node/263972.

New thread http://drupal.org/node/264398.

First, Congratulations boombatower with your graduation.

I just got back from a short break, I will go over the threads and respond, I have a few more fundamental questions for which I will open a new thread(webchick advised).

Designing upfront is certainly a must-do, Bevan just use Omnigraffe or Photoshop or scan in drawings is the best way to approach this online, Though lots of usability software is hard to use, so just designing upfront isn't going to simply do it, it requires a lot of thought to get to usable UI so doing this to fast, will mean we may have beautiful code but a hard to use interface.

I have no idea what a dev tarball is, but if you can point me to a download link or so I would be glad.

@Bojhan: Sorry about the confusion. The dev tarball is just the compressed version of the latest code. Instead of a .zip file it is a .tar file. The file can be found by going to http://drupal.org/project/usability_suite and clicking the download link on the 6.x-1.x-dev release.

It will require you to have a Drupal install setup and install the module. I'm not sure if your comfortable doing that. Once we discuss the UI some more I will provide some wireframes or screenshots. That may be a better option.

Thanks for your interest and ideas.

Now that UTS is released, the project scope inst an issue any more. Anything related to usability tests, is within scope of this project.