There are several ways that eligibility requirements can be implemented, some of which would require enormous amounts of code and would be somewhat of an overkill.

  • Create eligibility requirements for each study. The requirements can be created dynamically and the desired answers specified.

    This would not only require a large amount of code, but a complex interface.

  • A base set of eligibility requirements can be created (hard-coded) that will be answered for every study. The desired answers can then be specified for each study.

    This would not only be easier to code, but would provide a standard set of eligibility requirements and could be used as a way to direct people to a study they are eligible for. A person could come to the site, fill out the eligibility requirements, and be directed to an appropriate study, if found. Users could also be directed to a specific study to see if they are eligible for that study. If not, recommended studies could be displayed.

  • Extend the proposal above to included a set of study defined questions to be answered in addition to the eligibility requirements that could be used by the UTE to further understand the test results. The questions would provide a text field and the results would be stored along with the rest of the test data.

I think that the first one provides the most flexibility, but at this point may be out of the scope of the project and somewhat unnecessary. I'm leaning towards one of the latter two.

Regardless of what option we choose above there are several ways we can interpret the results.

  • Simply match up the answers to the exact desired responses.
  • Provide a points system and a fudge factor. Each response could be worth a certain number of points, the points could then be summed and the result would be compared to the desired score.

    This idea could also be implemented by simply choosing how many or what questions can be answered differently.

  • Provide a way to make certain questions not count in relation to eligibility.

Depending on which interpretation method we choose will dictate the usefulness of storing the person's exact responses.

Comments

webchick’s picture

webchick
she/they
Primary language English
Location Vancouver 🇨🇦
commented

Summary from IRC of my 2 cents:

The idea of hard-coding the eligibility requirements is extremely unappealing to me. Because right off the top of my head, I can think of at least three separate use cases for this module, which all need completely different eligibility requirements:

1. Drupal core/contrib module developer wants to see if the interface I just built for this new admin feature makes any sense to new and existing Drupal users.
2. I built "Foo site" with Drupal, and I want to use this module to find out if the navigation on my site is easier to spot on the left or the top. None of the users for Foo site have any idea what Drupal is.
3. I am using Drupal as a platform to usability test something else entirely (an advertisement, for example, or a game). This test might not have anything whatsoever to do with websites, let alone Drupal.

At the same time, the idea of you spending thousands of hours coding up some sort of AI-powered requirements generator thing is equally, if not moreso, unappealing. I mean it'd be cool and stuff. But it'd also probably take the remainder of the summer. :P

So what I suggested was leveraging off an existing module that already does most of what you want: in this case, Webform. Add a hook to Webform module that mimicks hook_views_default_views() and allows modules (such as usability_testing_suite.module) to "suggest" one or more forms, which can then be overridden and customized by someone for their specific needs. So we can brainstorm a default set of questions that would apply in most instances, and then ship UTS with some "starter" content, without locking people into any of our design decisions.

Jimmy brought up the good point, "Well how do we know whether people meet the requirements if the questionnaires are open-ended?" And I think the answer is, we don't. Webform can export form results to CSV, table view, etc. Let the usability test runner (I don't know the term) apply their own logic to the results, and let them filter out the candidates that aren't a good match manually. That's the only sure way to make this module flexible and useful throughout the ages, imo.

boombatower’s picture

boombatower commented

I'm all for using webform.

It allows for dynamic eligibility requirements without allot of code. The results can be looked through by the UTE.

People who participate should be directed to the site after they UTE knows they fit the basic criteria.

The eligibility requirements can be expanded on later, but I think try to do something more complicate will take too long for GSoC.

Bevan’s picture

Bevan commented

I would be inclined to use CCK rather than webform for the eligibility survey and background info questions, however that's implementation detail and up to you.

Eligibility requirement questions should be answerable with checkboxed radio buttons or select lists, and the correct answer/s must be answered by all questions.

Having the UTE manually check answers then 'approving' a participant is too long-winded and will result in less participants completing a test session.

webchick’s picture

webchick
she/they
Primary language English
Location Vancouver 🇨🇦
commented

Hm. In that case, we're talking about something closer to Quiz module, which isn't ported to Drupal 6 yet.

CCK works great for the input form, but how do you do "the correct answer"?

And actually, what /is/ "the correct answer"? :) Bevan, could you kind of walk us through where this step fits into the larger "I wanna do a usability test about $blah" and how you envision this data being captured and used? and/or point me to where you've already talked about this; I might be spacing a previous conversation.

boombatower’s picture

boombatower commented

There are several implementation issues with defining correct answers.

It will either be very cryptic, something really primitive like a text field asking for the result value from the input field.

Or very complex, trying to figure out what datatype the field is and display the appropriate desired answer field.

There may be something already written and if so that would be great, otherwise we need to look at how complicated we want this to get.

Bojhan’s picture

Bojhan commented

Hey,

Boombatower great to see you having so much interest into this part of the module. I just put up some wireframes on the other page, what I used there was this scenario :

    Create eligibility requirements for each study. The requirements can be created dynamically and the desired answers specified.
    A base set of eligibility requirements can be created (hard-coded) that will be answered for every study. The desired answers can then be specified for each study.

Normally you would have a participant database that you could ask specific questions and they would respond with either a desired or not desired answer. However in the case of Drupal the scenario where its most likely to use is eligibility requirements :

I have a module that is specific for a certain type of user, for example a developer who is also using cck, views2 and some other modules. It is highly unlikely that a user without these will use it, and past experience of the developer with cck and views2 interface has strong influence on the expectation and behaviour within my module.

In this case you would want eligibility requirements to be applied to the participants. I would always love to know more about the participants, however I rarely make it a requirement for my study (unless its a very specific audience or if I want to see the difference between certain types of users).

Knowing how other systems do this, participant selection is mostly done by hand. Since they already have a database of users to select from, however hardly anyone keeps his profile up-to-date. I will contact a recruiting agency I know, to see if they can provide us with some answers on how they approach a systematic recruiting process.

Bevan’s picture

Bevan commented

> Bevan, could you kind of walk us through where this step fits into the larger "I wanna do a usability test about $blah" and how you envision this data being captured and used?

Webchick, Did Bojahn's response answer your question?

boombatower’s picture

boombatower commented
Status: Active » Postponed

Conclusion we would like this, but too much for initial implementation.

Or perhaps this is not really necessary/feasible at all?