Support alternative spam rejection methods than a CAPTCHA (e.g. unpublishing)

Marked #414962: Handling false positives as a duplicate..

I'm in the same boat.

Though I'm not really specifically looking for a moderation queue solution, but rather any solution which can guarantee that 0% false positives are blocked/lost. Mollom already tries to do that with the uncertainty/captcha challenge, but evidently it's doesn't catch 100% of the false positives.

I've had few ideas in mind for few weeks but never got into thinking them through, I will hopefully post an update on that soon.

Marked #492294: Option for CAPTCHA comments to be dumped in approval queue and #401092: Valid submission lost: can we store spam messages in database for human review? as duplicates of this issue.

Sites are permanently losing valid form submissions daily because of this. Marking as critical. Watchdog isn't a viable solution since it only saves a couple days (or so) of denied submissions and even if you find a false positive in there, there's obviously no way to mark it as approved.

I agree with this. I 'm looking at Mollum as it's a high usage module but I'd rather have human checking on possible false positives, with ability to accept/reject, rather than risk blocking/loosing them

Marked the following issues as duplicates:
#575350: Add Fallback strategy: Leave all forms unprotected and place submissions into approval queue
#371187: Mollom Unpublish all nodes fallback
#575092: fall back to registered users only if mollom inaccessible

It would be very good to provide alternative fallback strategies, but with the impending use-on-any-form code going in, this is going to be a difficult task. We might have to implement something like a fallback strategory per-form. For nodes, comments, we know we can do certain things, but we don't want to limit ourselves like we're currently doing.

Marking this postponed until after #245682: Enable use of Mollom for any form lands.

Subscribe; would love this feature.

I like Mollom and the way it works, but I'm running a very low volume site and I'd rather manually review rejected entries rather than lose a single false positive.

For now I check admin/reports/dblog and that works decently but when I find a false positive I only see the body of the comment, not author etc.

Subscribe.
I have a very low volume site, but I know that at least twice legitimate posts were blocked by Mollom, with no recourse, and no way to send feedback to Mollom to let them know a mistake was made.

I'd like the option of never having a message rejected without having offered a captcha. At the moment, I'm considering other spam-blocker options.

I do not understand the actual problem here. People stated that "content is lost" and "Mollom deletes valid posts", but the current Mollom module does not delete posts on its own.

If forms are protected by text analysis, then there always is a fallback to a CAPTCHA if Mollom is unsure about the submitted content.

Can we clarify what the actual problem is, and what is actually asked for here, please?

The problem is when Mollom blocks the post outright and doesn't allow the user to fill in a CAPTCHA. User cannot post, gets frustrated, gives up. Hence data is lost.

Thanks! Now I understand. Quite complex, but possibly doable. Basically equals to introduce a configurable plug-in system for protection modes.

I already posted on this issue but perhaps this thread is more appropriate. I would question whether it is best to put Mollom messages in Watchdog with all the rest since it does not seem possible to include an "action" button on Watchdog lists. Given that when I have a comment that has gotten through incorrectly, I can "report it to Mollom", surely it would be logical to do the reverse and report to Mollom things that have incorrectly been identified as spam. I have one user (who I know to be bona fide) who has reported to me that his comment has been blocked (all comment posters are anonymous on my site) - what do I do about this if Mollom now has his IP address and refuses to allow him through at all?
As a first thought, I would like to have Mollom post to its own log (on Drupal) and be able to filter the log by Mollom rating (spam, ham, or "unsure" for example), by user, or by IP address perhaps. Then I should be able for any message or group of messages (check-box for example) to "report to Mollom" that the message is in fact OK.
It would also be nice to have message-specific (ie form-specific) actions, eg for comments "post comment to published queue".
I presume that there is no chance of having this in D5 - what about D6?

subscribing

Hey folks,

I'm confused about where this issue stands and how false positives are treated within mollom as of the current release. Does anyone have an update on the sitch?

I agree that a pluggable action system would be the best case scenario for arbitrary forms with built-in options for node and comment submissions to place them into moderation queues.

I believe there is a little confusion in this issue.

There are two different concepts at play here: fallbacks and actions.

• 'Fallbacks' are the behavior followed if the Mollom servers are unreachable or if you account is over quota.
• 'Actions' describe additional or alternative behaviors to follow based on what type of reponse comes back from the Mollom servers.

These two desired features would have completely different implementations and should probably be separated into two issues.

I have taken a first stab at solving the fallbacks issue. Attached is a patch that adds the concept of 'fallbacks' to hook_molllom_form_info(). 'Fallbacks' have a title, optional validate callback, and optional submit callback. In the patch is some basic logic to make this work and a sample implementation in a fallback that allows comments to be published to the moderation queue.

Looking forward to your thoughts, review, etc. on this general strategy.

P.S. This patch applies to Mollom-6.13 as this is the version we are working with. Once we agree on an approach someone could roll a patch against HEAD and then back-port it.

Working on a new version of the patch that exposes 'fallbacks' in a seperate info hook, that can optionally be limited to specific form ids. This way the core fallbacks will be implemented in the same manner as contributed fallbacks, simplifying the logic and providing better examples.

I'm also working on a similar patch that exposes 'actions' in a similar manner. I'll probably make a new issue to post that.

Interesting concept/patch.

I support moving these settings to the individual forms, but I recommend we get rid of the site-wide default. That seems unnecessary.

I need to look at this patch in more detail -- only skimmed it now.

We'll certainly want to have tests for this.

Dries, You might as well hold off for my next version. It should remove some currently kinda hacky things (the way the custom fallbacks combine with two core fallbacks).

I am ok with removing the site defaults. The key to this will be an appropriate upgrade path for all existing configured forms.

Alright, apparently I was full of it in #16. I said that 'fallbacks' and 'actions' were two different concepts and that they would require different implementations. I was wrong.

Attached is a patch that solves both problems by introducing 'actions' and 'triggers'.

Actions can be defined by any module for any combination of forms (or all forms). They are defined by a title, callback, forms array, and a triggers array (which triggers the action can respond to).

Triggers are hard-coded into the mollom form processing logic and include:

• The Mollom servers are down or otherwise unreachable
• Text analysis response: HAM
• Text analysis response: SPAM
• Text analysis response: UNSURE
• Passed captcha

You configure the actions on a form-by-form basis and the default "Fallback strategy" setting has been removed. Also, attached is a screenshot of the comment form settings (which includes an unpublish action).

This implementation also addresses a @todo that was in the code:

// @todo mollom() can be invoked outside of form processing. _mollom_fallback()
//   unconditionally invokes form_set_error(), which always displays the
//   fallback error message. Ideally, we would pass a $verbose argument to
//   _mollom_fallback(), but for that, we'd have to know here already.
//   Consequently, mollom() would need that $verbose argument. In the end, we
//   likely want to either embed the fallback handling into form processing,
//   or introduce a new helper function that is invoked instead of mollom()
//   during form processing.

The above problem is solved because the watchdog logging remains in _mollom_fallback(), but the form_set_error is now in mollom_action_block().

The one thing I'm not 100% happy about with his patch is the terminology. The hooks and helper functions for these actions and triggers are suspiciously similar to the core action and trigger systems (especially for D7, where the hooks are hook_action_info and hook_trigger_info respectively). I'm up for suggestions on an alternative naming scheme.

This patch should apply against 6.13 (not HEAD). This is the version in production on our sites so its what I am developing it for. We can work out rolling a patch against HEAD once we figure everything out.

Fixed a typo in the mollom_actions() function that was causing the static caching to never be used.

Oops. Disregard #21. Rolled against the wrong repo. This is the real one.

One more little fix. Last for the night :)

Just a note.. I accidentally left in a debug line that causes communication failure every time:

mollom.module - line 1416

$result = NULL;

Very sloppy patch making on my part. Sorry :)

1. We can and need to remove the constants; use the mapped strings instead.
2. "block" and "reject" are not very descriptive. "block" is "fallback" (unavailable) in reality. "reject" may be renamed to "deny" (being more direct opposite of "accept").
3. Overall, this patch is more or less trying to replicate D7's trigger/actions system. I'm not sure whether it makes sense to duplicate that system, instead of implementing and using it. I'm aware that this patch was purposively done against D6, but I would not recommend to do such a large change for D6 now, especially because these changes are touching the fundamentally flawed form integration of D6.
4. Instead of introducing new actions that do exactly the same as already existing actions of Drupal core, we should re-use those existing actions. The great design question of course is: how to do that properly.

   To some extent, this (again) duplicates the goals of the Form project. Additionally, I've just recently learned that the Rules module seems to contain a sub-module to apply rules for forms, which most probably has quite some overlap to what this patch is trying to do.

   In the end, the goal of this issue needs to be to introduce a (partially recursive) trigger and actions (or: rules) system for form submission attempts. It gets "recursive" for the case when Mollom servers are unavailable - several other issues in this queue are already asking for various additional fallback mechanisms, including support for third-party modules, which may or may not be applied to certain forms.

5. Contrary to Dries, I'd recommend to keep global defaults. They allow us to output a "Actions" header with a summary value "Default" or "Custom" in the administrative overview (list) of configured/protected forms. If I want or need to protect 20 forms with Mollom, then I most probably want to use globally configured defaults for actions, and only change those once for all that use the defaults.
6. Before going ahead and writing more code, I'd recommend to outline and discuss all intended changes first. For example, I'd call it highly debatable whether it makes sense to implement actions that alter form values (btw, I think that the concrete example of comment_mollom_action_unpublish() does not work for D6).

Marking as needs review, just to see what the testbot thinks of those patches.

Patch isn't against HEAD, hence the failure to apply.

1. Not sure which constants you are referencing. The TRIGGER constants?

2. Sounds fine.

3. Agreed, but as you say actions and triggers are not so pretty in D6.

4. My first thought was to use the core Actions and Triggers. This would work great in most situations.. but.. the problem is that we would expect people to administer how Mollom behaves in a separate interface. We could, of course, link to the appropriate settings forms (which are different between D6 and D7).

There is some recursion, because a typical user may find themselves hitting several different triggers during a form submission. The fact that all of this occurs during validation makes this 'recursion' pretty easy to handle though.

As far as certain actions applying to only some forms, this patch addresses that through the 'forms' key in the action info. Not sure how this would be handled utilizing the core Actions.

5. I can see the value of Defaults. I'm on the fence and so just went with Dries. It doesn't require much code to add Defaults back in.

6. In my fairly standard install, the unpusblish action does indeed work. But, you are right that it isn't very clean. I would have prefered to set a flag and handle it in hook_comment, but I don't believe things are properly passed around by reference in D6.

I agree we need to talk about and agree on a final solution before anything will be even close to committing. Code is gold, as they say, so I'd wanted to create something to get the ball rolling :)

Overall, my dilema is that I need to solve this problem in D6 and I can spend time doing so for D6. My play-time for D7 is quite limited and does not satisfy my client's immediate needs. I recognize that the implementations for D6 and D7 could/should differ wildly, but I'm not sure how to deal with that.

If I have time, I will look into the viability of using core actions/triggers, specifically in D6 (if it works there, it will work in D7).

@sun Also, I hope my naming convention (action , trigger) is not prompting any over-complication. If I had called them "protection modes" or "reactions" or something else would your response have differed?

1. At this point, I don't want to introduce dependency on other contributed modules, like Rules module.

2. I do think we could expose this functionality to our users -- many have asked for it. However, I'm quite nervous about forcing people to use the Triggers and Action UI. I'm OK with the direction this patch takes, as long we can build a streamlined UI for it as part of the Mollom module. In an ideal world, we'd start with the UI first.

As far as the interface goes.. I just need some ideas for improvements. As it stands, for each form the user sees a list of "triggers" and selects the desired "action". The copy for these choices could probably be improved.

I am starting to side with sun on defaults. I think I will add them back in and add a column to the forms tables that indicates whether you are using the defaults or not. It could save a lot of configuration for sites with lots of protected forms.

I still may look into actions and triggers (the core variety).. if they can serve as the backbone of the implementation, we could always try to make an 'easy-mode' UI as part of Mollom. However, my previous patch does solve the general problem of allowing more 'fallbacks' and contrib can add more as it sees fit. What will really need to be put to the test is if one callback function in the validate stage accommodates what people want to do. Most situations should be able to be appeased by playing with form_state, form_altering in a submit handler, etc.

I'm posting here as I can't tell if the thread over in GDO is still active: I've experienced the 'unexplained blocking' problem on a site 'protected' by mollom and I'm having trouble finding a conclusive answer to what is being done, or proposed, to deal with false positives. Perhaps more at the philosophical level, I'd like to find some real documentation of exactly what mollom is doing. From this post and some comments I've seen elsewhere (about fallout from using your real identity to do testing, for example) it seems that Mollom is logging and tracking IP addresses, for instance. Is that true? What provisions exist for someone remedy the situation if, god forbid, there's been a mistake? Do you have an ombudsman or some public, easily discovered way for problems to be mediated? My experience was that as a registered user on a community site my posts started to be blocked outright, no captcha, no explanation, no recourse. This was in a community I'd been active in, on and off, for 5, 6 maybe more years. WTF? Are we free and open source here or not? I'm really confused. If I don't know what's happening, there's no explanation, no way to provide feedback if I think there's been a mistake, and my posts are blocked, how exactly would I do anything about it and how would I know what to do? Most commercial sites would freak out if they thought they were losing visitors and didn't even know they were losing them. Much less registered users. Sure, spam is a real pain. And I'm sure the Proprietary Textual Analysis algorithms are quite robust, and clever, and totally correct. Unfortunately, there seem to be unintended consequences. And it's not like the issues have not been raised already. Of course, there are all those comments we don't know that we don't know about . . .
Cheers.
Howard J

Just came across this bit of info on Mollom and the author IP address question - http://mollom.com/api/handling-author-ip-addresses so that clarifies it a little. Interestingly, it makes the statement that having the author's IP address "can drastically improve Mollom's results," and goes on to say that "Sending Mollom the right IP address is key, and often non-trivial. Care needs to be taken to do it right." Is it just me, or is that a little disingenuous? Lord knows the spammers have all kinds of tools and tricks for things like spoofing IP addresses. It would seem there are some serious security issues here, but it's out of my league. Still would like a way to check what's being done to me, though.
Howard J

Thanks for the rant, Howard. ;) Do you think you get hold of the Mollom session IDs of the incorrectly blocked comments? On D6, they are in the watchdog messages. If not, would you mind sharing a screenshot so we can look at the data that you entered in order to get blocked? That would help us investigate the problem.

@rcrusoe, did some digging and found the problem. I believe it should be fixed now. Give it a try -- sorry for the inconvenience caused.

@Dries, do you mind my asking what you changed? Was it something specific to rcrusoe's account or something on a more general scale? We've had Mollom protection disabled until some of these issues are sorted out, and I'm curious if what you changed would have a positive impact on our situation.

This needs a lot more brainstorming and considerations to be done properly. I've therefore worked on a very simplified version of this feature request, which just accounts for the unpublish-if-spam case, over in #881534: Allow to unpublish spam posts instead of rejecting

I'd like to keep this issue open and would love to continue working on it with others, but we should focus on doing this for D7 only.

@sun Agreed. Once I realized it was going to be a long-haul, major endeavor, I had to step back and just get something working for the client. After D7 launches and we migrate our codebase, I'll be able to take another look at this.

Subscribe.
Might work around by showing reCAPTCHA for anonymous and Mollom text analysis for authenticated.
Look forward to the eventual all Mollom solution. Sounds like it's quite a re-factor.

At this point, the 2.x series provides a few additional options that allow to control how unsure and spam responses from Mollom are handled, but without turning the configuration into a cockpit/sea of very advanced options.

Therefore, I consider this issue as complete. There are other feature requests in the queue, which are asking for Actions/Rules support and integration, and which might be of your interest. That said, those will be not exactly trivial to implement, so any architectural ideas are highly appreciated!

Thanks all,
sun