We're live testing Quiz extensively nowadays, and it's clear to me that quiz revisioning still causes usability issues. I fully understand why it is so, but never the less it does create problems. I've had several questions about wether a teacher could correct spelling errors in questions, emphasize certain words in certain questions, adjust the feedback text on a quiz etc, and even if none of these requests change the meaning of the quiz I don't feel it's safe for me to let teachers fix this themselves, as I know how easy it is for Quiz to create a new revision.

The "Auto revisioning" checkbox helps here, but it is still *very* easy to forget to change the standard setting that a revision is made. I've even done it myself a few times, even though I constantly keep this in the back of my head. See screenshot.

So I propose to expand this from a on/off check box on "Auto revisioning" to three radio buttons:

[* ] Auto revisioning (default)
[ ] Manual revisioning
[ ] No revisioning

Choosing "no revisioning" (and maybe "manual revisioning" too) would be accompanied with an always showing message in the quiz admin that revisioning is off, and that quiz creators must not change values that affect the scoring of a quiz (like adding new question alternatives, pass scores etc.). For me it would be much safer to explain and train our quiz creators this, than trusting them to always uncheck the new revision button when they make a minor change.

What makes this issue more serious is that if you have created a revision by accident, and users continue to take the quiz, there is no way you can fix this. This is extra important for us now as we have started to work on a Quiz statistics module (see #501858: Dashboard - Statistics on test results) - if we get unwanted revisions here we will bork our statistics report, with no way to merge results from two revisions into one report.

So, will you maintainers accept a patch for this, or do you have any strong objections (or suggestions!) to the problem or proposed solution? :)

CommentFileSizeAuthor
revisioning2.png145.45 KBvegardjo
revisioning.png93.59 KBvegardjo
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

falcon’s picture

I understand the problem.

I don't have the time to think this through right now. We should do something about this problem in quiz 4, and we have to make sure we get it right this time.

Have we considered all the options here? What about a "merge revisions" feature instead or in addition? That way we can make sure that the people who merges the revisions knows what they are doing. (Just an idea. I'll think about this some more and post a new comment later this week)

borgewarvik’s picture

Has anyone tried to define what operations on a Quiz should result in a new revision being created? I personally believe in a auto-revisioning system, a "pool" of events that would initiate a new revision. But this approach needs careful planning and testing for many scenarios. This, of course, can be very time consuming. But if we get it right, it will be very user-friendly for the end user.

All events can be turned on/off globally and individually for a quiz. Example events:

- Adding/removing questions
- Changing question score
- Changing pass/fail options
- Changing typos in question and description texts
- Changing availability options
- Changing feedback time
- Re-ordering of questions
- Changing multiple takes settings

There will be scenarios where quiz will not create a new revision and the user wants to. In this case a mechanism to force new revision should be properly displayed to the user, probably under the "manage questions" tab.

falcon’s picture

yes, this was what I was aiming for in the beginning. After some thinking I came to two conclusions:

The philosophical conclusion:
- If a change doesn't affect scoring there is no need to make the change. Hence all changes requires a new revision. (A corrected spelling error might make the user understand the question better, and thus make the question easier. It will affect scoring.)

The practical conclusion:
- It is impossible to know if a text change is (significantly)affecting scoring or not. All other changes can be evaluated automatically.

An approach where we let the user decide whether to revise or not on text changes and decide automatically for all other changes is interesting! We could present a dialog like

"You have changed the question, do you want to create a new revision of this question?"
"No, I only made some minor adjustments. My changes won't affect the scoring of this question."
"Yes, my changes might affect scoring"

vegardjo’s picture

Aight, so we have:

  1. Like today, manual and automatic revisioning for all
  2. New ability to also never create revisions
  3. Creating a feature to merge revisions
  4. Creating a system for what do be automatically revised and what the user must decide on
  5. Creating a system and an UI where the quiz admin could select revise / not revise per field in their Quiz setup.

I guess all of these have issues, typically along the lines of:

  1. Has the issues I lined out in my original post here
  2. Will probably be misused, so someone will change important details at a point and users will get different tests
  3. Might be a very good thing to have in any case, but kind of solves the consequence of the issue rather than the issue itself. And how to decide wether it is ok to merge or not, if the one who can merge is not the one who has done the changes?
  4. Will we be able to find a good agreement along the lines from "philosophical" to "practical"? :) There are some no-brainers on both, but also several fields where you easilly can argue both ways.
  5. Very flexible, but there are lots of fields, and new question types and quiz features will add more, so it will be a very big UI, increase code complexity, and increase the learning curve for quiz. On the other hand it would force new quiz admins to reflect upon the revisioning issues..

One simpler adjustment we could do to probably prevent a few unwanted revisions is to *always* show a message to the admin if manual revisioning is on, typically along the lines of "You have changed something, so Quiz will make a revision. If you have only made minor adjustments you might want to unchech the "new revision" box". Or even better, as falcon indicate, have this in the node save functionality, like "save changes and make new revision" and "save without making new revision".

Are there more ways we could go to fix it? I agree with falcon that it's important that we get it right this time!

falcon’s picture

6. Block editing of Quiz after someone has started to take it, and unblock when all results are deleted. (This is how most LMS do it afaik)

6. Forces the quiz creator to make sure that the Quiz is correct from the beginning, but won't work well with our reuse of questions across multiple quizzes, and it is also frustrating having to delete all results in order to change a quiz...

7. Don't do anything about revisioning and let it be up to the creator to make sure that he doesn't ruin quiz results

fuzzy76’s picture

Component: Code » Code - Import/Export

I would actually go further and suggest a special permission needed in order to edit a quiz after someone has taken it. This would solve a lot of problems for us, because our course instructors could make and edit quizzes, but if they need to make changes after it has been made available to students, they have to go through us (administrators).

Anuprita’s picture

Hi Guys,

I am trying to merge the revisions as one of my client wants all the results of all the revisions in one report.

Could you please let me know if there is any way i can customize it for time being till these features are incorporated.

Any help would be appreciated.

djdevin’s picture

Version: 6.x-4.x-dev » 7.x-5.x-dev
Component: Code - Import/Export » Code - Quiz core

This is an old issue but I agree.

A Quiz admin should not be able to unknowingly create a new revision.

Moving to 5.x

djdevin’s picture

djdevin’s picture

Status: Active » Closed (fixed)

Almost positive this was fixed in #2470253: saving a quiz with attempts creates revisions even if auto-revisioning is not set

Possible to just not have any revisions now (uncheck auto revisions, and either grant/don't grant "manual quiz revisioning").

Covered by testcase testQuizNoRevisioning