Incomplete patch attached. Just uploading as a placeholder.

mollom-config.jpg

Files: 
CommentFileSizeAuthor
#20 20110318-jfkgtn2n2ixu3329cg97a4hr3d.jpg42.88 KBsun
#11 mollom-HEAD.checks.11.patch6.27 KBsun
#10 mollom-check-captcha.png5.67 KBsun
#10 mollom-check-content.png10.55 KBsun
#10 mollom-check-content-captcha.png46.68 KBsun
#9 mollom-HEAD.checks.9.patch16.99 KBsun
#7 mollom-edits.png227.82 KBNoyz
#6 mollom.png226.63 KBNoyz
#2 mollom-HEAD.checks.2.patch16.71 KBsun
FAILED: [[SimpleTest]]: [MySQL] 1,823 pass(es), 340 fail(s), and 66 exception(es).
[ View ]
#2 mollom.checks.2.png12 KBsun
mollom-config.jpg514.69 KBDries
help-text.patch1.93 KBDries
PASSED: [[SimpleTest]]: [MySQL] 2,188 pass(es).
[ View ]

Comments

Triggering details for this issue:

1) The user did not expect to see a CAPTCHA when text analysis is enabled. We don't explain that there are 'backup' CAPTCHAs upon unsure.

2) The user did not understand why he'd want to use Mollom's CAPTCHA only mode. He believes it is easier to use the captcha.module in that case.

Status:Active» Needs review
StatusFileSize
new12 KB
new16.71 KB
FAILED: [[SimpleTest]]: [MySQL] 1,823 pass(es), 340 fail(s), and 66 exception(es).
[ View ]

mollom.checks.2.png

Status:Needs review» Needs work

The last submitted patch, mollom-HEAD.checks.2.patch, failed testing.

1. Having one checkbox for spam, and one checkbox for CAPTCHA is confusing IMO. The top of the UI in #2 doesn't work for me. Spam and profanity are things you want to block, CAPTCHA is a tool to block spam. I don't think users will be able to make sense of that.

2. I also think we need to better explain the different between 'strictness' and 'interval'. If I want to be more strict, I can tweak either the strictness or the rate limit. We don't explain what gets more strict with the strictness. Does it also affect the rate limit? It would be interesting to do some quick user testing -- I'd be surprised if people understood these settings.

3. It is not clear that the 'interval' is across all sites in the Mollom network. This is important to understand because it changes the way you need to think about the values. If explained well, it also makes people understand that it can be very effective against spambots.

I didn't spent much time with the Strictness option, because it is yet unclear whether we're going to expose it in the first place; needs internal testing first.

1. Having one checkbox for spam, and one checkbox for CAPTCHA is confusing IMO. The top of the UI in #2 doesn't work for me. Spam and profanity are things you want to block, CAPTCHA is a tool to block spam. I don't think users will be able to make sense of that.

Sounds like users might have a different perception for these options? To me, these options feel more logical and natural, since a CAPTCHA check is a hidden (and now configurable) feature of the spam check — if I enable the CAPTCHA option, then there will be a CAPTCHA; if I enable the Spam and CAPTCHA checks, then there might be a CAPTCHA; if I enable Spam only, then there won't be a CAPTCHA.

In short: I don't think that users are making a differentiation between "things I want to block", and "tools I want to use to block". They are not (and don't want to become) experts in fighting spam. They just simply want to enable some checks to prevent unwanted content on their site.

3. It is not clear that the 'interval' is across all sites in the Mollom network. This is important to understand because it changes the way you need to think about the values. If explained well, it also makes people understand that it can be very effective against spambots.

Right before I created the patch, the label for Interval still had a "across all sites" suffix, but the label looked too lengthy and felt incomplete, since the "in the Mollom network" part was missing, so I removed it. Seems like we need to add a #description along the lines of:

Author information is tracked across all sites in the Mollom network.

Not sure whether we specifically want to mention bots/spambots/machines in the description.

StatusFileSize
new226.63 KB

I'd like to help here, but Im having trouble following the conversation. Points of confusion:

1. Is the visual shown in the original post a revised "settings" page? None of the tabs are in an active state, so it's hard to understand where this is. If it's to replace the existing settings, are you planning to do away with the settings shown here: https://skitch.com/jeff.noyes/rkjh8/dreamweaver
2. The original post is about educating the user about Text Analysis and Intelligent Captcha's, but intervals and strictness was introduced. Is it related. If it is, I need more knowledge about the goal/intentions of these settings.
3. Based on the language (but not position), I would think these settings (https://skitch.com/jeff.noyes/rkkya/dreamweaver), only apply if "Text Analysis" is selected. Is that true?

If you're strictly to focus on educating the user about Text Analysis and Intelligent Captcha's. and if I'm correct about #3, then I'd say you're kind of on the right track in the original post but A) it's way to much to read B) it's not in context to the decision. For example, I'd do something like this...

StatusFileSize
new227.82 KB

1. Is the visual shown in the original post a revised "settings" page?

Nope, this is all about the page that allows to protect a form with Mollom. Dries' screenshot in the OP merely adds a help text to the page.

Mine in #2 revamps most of the form to take new configuration parameters into account, which should allow us to simplify this form. The most important change is the replacement of the separate Protection mode and Analyze text for settings into a single Checks setting.

2. The original post is about educating the user about Text Analysis and Intelligent Captcha's, but intervals and strictness was introduced.

Yes, the aforementioned replacement/merging of settings is not obvious to see. We have three new parameters:

  1. unsure: When checking posts for spam, this setting allows to customize whether a CAPTCHA should be shown when Mollom is unsure. Previously, or right now, when checking posts for spam and Mollom is unsure about whether a post is spam or not, the user who tries to submit the post is presented a CAPTCHA to solve. For some site administrators, it is not clear and perhaps even undesired that checking a post for spam may result in a CAPTCHA being displayed, which (partially) is why we introduced a setting to configure whether CAPTCHA should be displayed. This setting allows to configure whether Mollom should require to solve a CAPTCHA when it is unsure, or whether it should directly respond with either spam or ham.

    It is important to understand that a form can also be protected with a CAPTCHA only; i.e., without text analysis for spam. In that case, the CAPTCHA is shown directly when accessing a form, and has to be solved correctly to submit the form.

    Overall, we have the following possible options:

    • Text analysis for spam
    • Text analysis for spam + CAPTCHA
    • Text analysis for profanity
    • CAPTCHA

    Note: If a post is profane, no CAPTCHA will appear.

    The idea is to simplify all of the options into a set of checks to perform for a post:

    • Spam
    • CAPTCHA
    • Profanity

    The user just enables the desired checks; i.e., if I enable the CAPTCHA option, then there will be a CAPTCHA; if I enable the Spam and CAPTCHA checks, then there may be a CAPTCHA if the spam check is unsure; if I enable Spam only, then there won't be a CAPTCHA.

    The only suboptimal part I personally see with this is that when enabling the Profanity and CAPTCHA checks, then a CAPTCHA will be displayed right from the beginning. The text analysis for Profanity is performed additionally. That is, because only Spam+CAPTCHA can be intelligently combined. But in the end, there's nothing wrong with enabling the Profanity and CAPTCHA options, and someone might even have a valid use-case for doing so.

  2. rate_limit ("Minimum interval..."): A configurable time interval (in seconds) that needs to elapse since the last form submission attempt that has been checked by Mollom (across all sites in the Mollom network), before the same author can submit the form. For example, it is unlikely that someone registers more than one user account on different sites within, say, an hour. Thus, you could configure a one hour treshold here, to prevent a spambot from registering on your site, in case that bot attempted to register user accounts on other sites in the Mollom network within the past hour.
  3. strictness (high, medium [default], low): Allows to tune the strictness of the different checks that Mollom is performing (eg. the reputation of the author, the URLs, ...) For trusted users we could eg. use the “low” value.

3. Based on the language (but not position), I would think these settings, only apply if "Text Analysis" is selected. Is that true?

Yes, that is true for "Protection mode"/"Text analysis" in the current UI. We are already using #states to hide those additional settings in case text analysis is disabled. In #2, they're conditionally displayed too, but based on whether one of the "Spam" or "Profanity" checks are enabled.

StatusFileSize
new16.99 KB

Just a backup of my latest and greatest code.

StatusFileSize
new46.68 KB
new10.55 KB
new5.67 KB

I can only guess that Dries thought of something along the lines of the following screenshots:

Note: Still focusing on the new CAPTCHA/unsure option, since that seems to be the most problematic issue.

mollom-check-content.pngmollom-check-captcha.pngmollom-check-content-captcha.png

For me, it's the excessive amount of conditionals that makes it hard to understand.

They translate into: I can have a CAPTCHA only, but I can also have a CAPTCHA when doing text analysis, but only when checking for spam. I cannot have a CAPTCHA when checking for profanity.

The earlier proposed simplification: I can have a CAPTCHA or not.

StatusFileSize
new6.27 KB

Just to backup the mockup-only code that served for the screenshots in #10.

I like the propose mockups in #10 but I might know too much about the Mollom internals. I'm going to ask some other people to comment too. Might be helpful ...

Here is some of my feedback though:

  1. What is not clear is that the "Minimal interval" is across websites. That might be important to know, and is particularly powerful when locking down the user registration form from fake spam bots.
  2. It makes more sense to show the 'Mollom CAPTCHAs are intelligent' link at the top of the configuration screen where people need to decide between text analysis and CAPTCHA.
  3. A possible better way to introduce and emphasis the intelligent CAPTCHAs is by making it a setting. When CAPTCHAs are used, we could allow people to explicitly enable/disable the use of visitor reputation to prevent spammers from filling out CAPTCHAs. By making it a separate setting, not only allow we people to disable this behavior, but it also gives us some space to properly explain what this feature does. It's a real differentiator so it might be worth to make it a setting. I'm pretty excited about that idea, but would love to hear your thoughts. :-)
  4. I'd leave out the 'Show a CAPTCHA when Mollom is 'unsure' setting for the time being. If we keep it, we need to explain what happens when you disable this setting and Mollom is 'unsure'. Will the post be rejected or accepted? I'd make this an API feature without a UI. By making it part of the UI, the Trackback module can use a binary classifier and we allow developers to change it. I don't think this needs to be exposed to regular users.

I think we should have the following
For a content moderator or site owner that wants to protect his/her site against spam, we don't want them to be, or become, Mollom experts. They should be able to configure Mollom without prior knowledge in less than 5 minutes. Thus, I think the UI should be very intuitive -- meaning some settings should probably live in code and only be exposed to developers.

It might be a good idea, to compile a list of all new settings, and try to highlight the ones that are most frequently requested. Only those should be exposed through the UI, the others should be exposed through the API.

Points I struggle with, and mind you, I don't know much about this so I'm a fairly good test subject...

It would be easier for me to map sub questions, if they were children of the original question. For example:

Protection mode
[x] Text analysis
Analyze fields
[ ] Subject
[ ] Comment
Analyze text for
[x]Spam
[x] show a capthca when...
When Text analysis identifies spam...
[ ] automatically discard the post
[ ] retain the post for manual moderation
[ ]CAPTCHA

Which alternatively would look like this:

Protection mode
[ ] Text analysis
[x] CAPTCHA

Regarding the minimum intervals thing... I'm not sure I get what's being asked of me (or the user) here. I don't think I could answer this part of the UI, but it could partially be because I don't know what "default' means, nor do I know what's underneath the combo box. That said, there's probably some more work to do here.

I think the UI is trying to say... If text analysis is enabled, and if the same author posts [every minute] (just a guess as to what's in the combo), be [strict]. But I don't know what strict means. Does that mean, be strict about showing the CAPTCHA? So in short, I think Default needs to actually state the value instead of default, and strictness need to be better defined.

shit, I hate how D.) always removes spacing. Gets me all the time. Try this instead: https://skitch.com/jeff.noyes/rikcd/dreamweaver

#15 is the most natural way to do it -- it is how I've seen it done in other UIs. It might get pretty complex though as there might be 2 or 3 levels of sub-questions.

I assume you mean complex in terms of engineering. In terms of user experience, the level of clicks is irrelevant so long as each question is clear.

FYI: I've extracted the configuration and implementation of rate_limit and strictness parameters into #1111378: Minimum post interval and text analysis strictness options — which allows us to focus on the user interface challenge for configurable CAPTCHAs (in combination with text analysis) in this issue.

Would it be possible to have an option that would show the CAPTCHA even when Mollom's text analysis is "sure" it is spam? Lately, I've been getting a TON of false positives from valid users, who are being blocked without even being shown a CAPTCHA.

Title:Educate user about text analysis and intelligent CAPTCHAsRethink/simplify UI to protect/configure a form
Version:7.x-1.x-dev» 7.x-2.x-dev
Status:Needs work» Active
Issue tags:+Usability
StatusFileSize
new42.88 KB
  1. Re-reading the full history of this issue again, I still object to putting marketing/yada content/help text into the module. That's what the http://mollom.com website is for. Duplicating that information will not be maintainable. We should improve the UI options and UX instead, which actually is what we've been discussing in the majority of comments here. :)
  2. Clarifying issue title.
  3. Backing up #15 as attachment, since Skitch is EOL:

20110318-jfkgtn2n2ixu3329cg97a4hr3d.jpg

That said, there are more text analysis options in the meantime. Some/most of them are conditional; i.e., they do not apply to all forms that can be protected with text analysis.

In the past, I considered to potentially introduce vertical tabs, as a means to expose those options as "skippable" controls, but always got stuck in the question of how to group them.