We're live testing Quiz extensively nowadays, and it's clear to me that quiz revisioning still causes usability issues. I fully understand why it is so, but never the less it does create problems. I've had several questions about wether a teacher could correct spelling errors in questions, emphasize certain words in certain questions, adjust the feedback text on a quiz etc, and even if none of these requests change the meaning of the quiz I don't feel it's safe for me to let teachers fix this themselves, as I know how easy it is for Quiz to create a new revision.
The "Auto revisioning" checkbox helps here, but it is still *very* easy to forget to change the standard setting that a revision is made. I've even done it myself a few times, even though I constantly keep this in the back of my head. See screenshot.
So I propose to expand this from a on/off check box on "Auto revisioning" to three radio buttons:
[* ] Auto revisioning (default)
[ ] Manual revisioning
[ ] No revisioning
Choosing "no revisioning" (and maybe "manual revisioning" too) would be accompanied with an always showing message in the quiz admin that revisioning is off, and that quiz creators must not change values that affect the scoring of a quiz (like adding new question alternatives, pass scores etc.). For me it would be much safer to explain and train our quiz creators this, than trusting them to always uncheck the new revision button when they make a minor change.
What makes this issue more serious is that if you have created a revision by accident, and users continue to take the quiz, there is no way you can fix this. This is extra important for us now as we have started to work on a Quiz statistics module (see #501858: Dashboard - Statistics on test results) - if we get unwanted revisions here we will bork our statistics report, with no way to merge results from two revisions into one report.
So, will you maintainers accept a patch for this, or do you have any strong objections (or suggestions!) to the problem or proposed solution? :)
Comment | File | Size | Author |
---|---|---|---|
revisioning2.png | 145.45 KB | vegardjo | |
revisioning.png | 93.59 KB | vegardjo |
Comments
Comment #1
falcon CreditAttribution: falcon commentedI understand the problem.
I don't have the time to think this through right now. We should do something about this problem in quiz 4, and we have to make sure we get it right this time.
Have we considered all the options here? What about a "merge revisions" feature instead or in addition? That way we can make sure that the people who merges the revisions knows what they are doing. (Just an idea. I'll think about this some more and post a new comment later this week)
Comment #2
borgewarvik CreditAttribution: borgewarvik commentedHas anyone tried to define what operations on a Quiz should result in a new revision being created? I personally believe in a auto-revisioning system, a "pool" of events that would initiate a new revision. But this approach needs careful planning and testing for many scenarios. This, of course, can be very time consuming. But if we get it right, it will be very user-friendly for the end user.
All events can be turned on/off globally and individually for a quiz. Example events:
- Adding/removing questions
- Changing question score
- Changing pass/fail options
- Changing typos in question and description texts
- Changing availability options
- Changing feedback time
- Re-ordering of questions
- Changing multiple takes settings
There will be scenarios where quiz will not create a new revision and the user wants to. In this case a mechanism to force new revision should be properly displayed to the user, probably under the "manage questions" tab.
Comment #3
falcon CreditAttribution: falcon commentedyes, this was what I was aiming for in the beginning. After some thinking I came to two conclusions:
The philosophical conclusion:
- If a change doesn't affect scoring there is no need to make the change. Hence all changes requires a new revision. (A corrected spelling error might make the user understand the question better, and thus make the question easier. It will affect scoring.)
The practical conclusion:
- It is impossible to know if a text change is (significantly)affecting scoring or not. All other changes can be evaluated automatically.
An approach where we let the user decide whether to revise or not on text changes and decide automatically for all other changes is interesting! We could present a dialog like
"You have changed the question, do you want to create a new revision of this question?"
"No, I only made some minor adjustments. My changes won't affect the scoring of this question."
"Yes, my changes might affect scoring"
Comment #4
vegardjo CreditAttribution: vegardjo commentedAight, so we have:
I guess all of these have issues, typically along the lines of:
One simpler adjustment we could do to probably prevent a few unwanted revisions is to *always* show a message to the admin if manual revisioning is on, typically along the lines of "You have changed something, so Quiz will make a revision. If you have only made minor adjustments you might want to unchech the "new revision" box". Or even better, as falcon indicate, have this in the node save functionality, like "save changes and make new revision" and "save without making new revision".
Are there more ways we could go to fix it? I agree with falcon that it's important that we get it right this time!
Comment #5
falcon CreditAttribution: falcon commented6. Block editing of Quiz after someone has started to take it, and unblock when all results are deleted. (This is how most LMS do it afaik)
6. Forces the quiz creator to make sure that the Quiz is correct from the beginning, but won't work well with our reuse of questions across multiple quizzes, and it is also frustrating having to delete all results in order to change a quiz...
7. Don't do anything about revisioning and let it be up to the creator to make sure that he doesn't ruin quiz results
Comment #6
fuzzy76 CreditAttribution: fuzzy76 commentedI would actually go further and suggest a special permission needed in order to edit a quiz after someone has taken it. This would solve a lot of problems for us, because our course instructors could make and edit quizzes, but if they need to make changes after it has been made available to students, they have to go through us (administrators).
Comment #7
Anuprita CreditAttribution: Anuprita commentedHi Guys,
I am trying to merge the revisions as one of my client wants all the results of all the revisions in one report.
Could you please let me know if there is any way i can customize it for time being till these features are incorporated.
Any help would be appreciated.
Comment #8
djdevinThis is an old issue but I agree.
A Quiz admin should not be able to unknowingly create a new revision.
Moving to 5.x
Comment #9
djdevinComment #10
djdevinAlmost positive this was fixed in #2470253: saving a quiz with attempts creates revisions even if auto-revisioning is not set
Possible to just not have any revisions now (uncheck auto revisions, and either grant/don't grant "manual quiz revisioning").
Covered by testcase testQuizNoRevisioning