Blocked posts by registered users whose name equals common spam authors

Subscribing. Every submission now fails.

Subscribing, I get this message "Your submission has triggered the installed spam filter and will not be accepted." on a perfectly valid anonymous comment. I set the comment form to "text analysis and CAPTCHA backup", but the CAPTCHA backup is never shown and the comment is simply rejected. Subject and text field are enabled for analyzing.

I think Mollom should present the CAPTCHA in all spam cases and should never reject comments.

Steps to reproduce:
Got to http://fsinf.at/bachelor-info-ss2010
Fill out the comment from with:
* Your Name: "Dries Buytaert"
* Homepage: "http://www.mollom.com"
* Comment: "Gibt es bis September eine weitere dieser Infoveranstaltungen?i"

And here comes the patch to fall back to CAPTCHA in any case. This one is already live on fsinf.at, so the steps to reproduce in #4 will not work anymore.

Before I look into fixing the tests I want a review from a human whether this approach is acceptable or not.

Maybe this could be exposed as an checkbox in the interface: [x] Still allow textual-analysis spam to still use a CAPTCHA.

Of course that text could easily be improved.

Why do we have to expose that as checkbox option? Mollom should protect as from spam, but Mollom should not make it impossible to post a comment (which it does right now). Don't we trust the CAPTCHA to solve a possible spam attempt?

klausi, there are actually a lot of manual spammers.

OK, this has never been a problem for us, but i can imagine that this is true for large sites. So we have several use cases:
* small site, few manual comments to review for site admins. Blocking manual comments is not necessary nor acceptable.
* large site, many manual comments. Loosing valid manual comments is acceptable.
* large site, many manual comments. Loosing valid manual comments is not acceptable (needs comment review squad).

So maybe the suggestion from #8 would be a solution, to let the users choose from the interface whether they want to possibly loose comments or not. #364575: Support alternative spam rejection methods than a CAPTCHA (e.g. unpublishing) would also come in handy to avoid comment loss, with a message like "your comment looks like spam and will be examined by a human moderator. If it is not spam, it will be published."

+1 for optional captcha (always), small sites can live with the threat of manual spammers rather then loosing a commenter.

I've got serious manual spam problems: http://drupal.org/node/632288

But in the last month I have had several accounts that are prevented from posting any new nodes because of the Mollom spam filter. I can confirm that they are known and trusted users and yet they are prevented from completing tasks on the site. They complete the node creation form and then receive the following message WITHOUT the CAPTCHA form:

"Our submission has triggered the installed spam filter and will not be accepted."

I will cast an additional vote for comment #8's approach of always presenting a CAPTCHA form when spam submissions are blocked. Right now, its just frustrating my users. For the two that have taken the trouble to call me, there must be dozens more that have given up.

Surprisingly, we have almost no comment spam even though we allow anonymous users.

Could you share the Mollom session IDs of the incorrectly blocked posts? You can find them in the watchdog messages. Thanks.

I had just flushed my watchdog table, but I recreated the issue and here is the entire watchdog message of the blocked node including the session ID. The user account posting this node is trusted and legit:

Message Spam:

Array
(
[post_title] => This is a test
[post_body] => At 1:00PM on Tuesday, June 29, 2010, at the Cambridge Senior Center, CCTV will be hosting a screening of personal narrative videos produced by seniors in the digital storytelling workshop, “The Stories of Our Lives.” The screening will be held in the ballroom on the first floor of the Cambridge Senior Center. Light refreshments will be served. After the screening, all digital stories will air.
[author_name] => Some User
[author_mail] => someone@somesite.com
[author_id] => 1222

[author_ip] => 11.11.11.111
)

Result:

Array
(
[spam] => 2
[profanity] => 0
[quality] => 0.799
[session_id] => 1006297d0a03ddcb2d
)

Odd, I can't find a session with session ID 1006297d0a03ddcb2d. Do you have any other sessions IDs?

Here are a bunch of comment spams blocked by Mollom, the session IDs are:

100629cfe027201b46
1006298c652c27906a
10062931139b10ecf1

But those aren't the issue. Here is the session IF of Mollom blocking a trusted user and not providing a captcha backup:

10062908ff579f11c6

Odd, we can't find 10062908ff579f11c6 either ... the other sessions I did find, but they are all obviously spam.

Here is another node attempt:
100629942d4bb79fd5

Here is a comment attempt:
100629b6d17bfb3d97

I found a message in the log from some earlier post from this same account:
"Non-existent session id 10062908ff579f11c6."

I think the message was because I had walked away from the unfinished node for an hour or two, then resubmitting the node forced a new session to open with Mollom.

Those 2 sessions actually exist. Both the node and the comment were blocked because the person entered 'Career Source' as its author name. It is considered to be very spammy -- there is a lot of career spam ... Normally, a regular personal name is expected instead of a company name.

OK, that one case makes sense. But should they not get a captcha back up so they can prove they are human?

I know and trust this user and I know this person represents a non-profit career training organization in the US. All of their posts will contain career fair announcements:

http://www.cctvcambridge.org/user/career_source

I can advise the user to change their name, but I know of some other accounts that are not so spammy and have the same issues.

You could give him the 'by-pass mollom' permission?

Sure, I could do that. But then our admins have to take responsibility for approving accounts and reviewing nodes and they don't have time to moderate. This is the reason we chose Mollom, actually.

Is there no way to diagnose why these couple of reported users are not getting a catcha backup?

#5: 463026-mollom.patch queued for re-testing.

Whew, quite an issue. That's indeed an interesting edge-case -- being: Mollom is not really triggered on the post, but on the user name. Although the user is a (locally) registered site member, a name like that is used to spam countless of other sites.

I'm not sure whether Dries likes it, but here's the compromise I'd propose to fix this issue:

- Anonymous users won't ever submit a spam post (like now).
- Registered users may submit a spam post if they additionally answer the CAPTCHA correctly. (new)

Improvement: Registered users are able to submit a post at all. If the user managed to register on the site (hopefully also Mollom-protected), then there is chance that a post identified as spam may not actually be spam.

Downside: Such registered users will have to permanently enter CAPTCHAs for every post. Only granting the "bypass mollom protection" permission for such users would eliminate it completely. Also, manually submitted spam posts (which perhaps happens on larger sites only) can get through this.

Effectively, I'm not sure whether this change makes sense. However, I think it makes more sense than the previous proposals (entirely stopping to block posts, or adding a checkbox to all form configurations).

Slightly better title?

It is better if we tweak this in the backend. Putting it in the module could be a recipe for problems once spammers figured this out.

hm, I guess that effectively makes this issue "by design" (for the Drupal module) then?

Mollom should do this automatically in the backend already, so yes.

Dries,

I have a very trusted user who is now blocked on my site from posting any comments.. for the past 1 week..
When trying to post a very genuine trusted comment on my site, he got the message "Your submission has triggered the spam filter and will not be accepted.".. and was unable to continue any further after that...

I would like to request you to please see if the user's reputation can be made "trusted". I do not want to lose him from coming back to my site..
Site URL: forum-network.org
User's username: drtobywatson

Appeciate a response.

Thanks!!

I'm having a similar problem that Mollom thinks almost all comments are spam. I won't open a new issue for my problem because I have a strong feeling that the problem is related to usernames.

Site: http://dev.curlingcalendar.com

Description of the problem and notes from my tests

Users that are logged with their Facebook accounts are not able to post comments to (tournament or wiki) nodes because Mollom thinks they are all spam. They don't even see a CAPTCHA.

I use Drupal for Facebook -module to provide the Facebook Connect functionality and the Drupal usernames are of form "fb-profile-id@facebook", for example 123456789@facebook. I believe this is a reason why Mollom blocks the comments.

What makes me think so?

Test 1: I've been logged in with my FB-user account on my destkop computer and try post comments. Mollom says no-go.

Spam:
Array
(
    [post_body] => Thanks for sharing this schedule. 

We used it on a 4 sheet rink and it worked like a charm!

Thanks again.

-Markus
    [author_name] => 100000xxxxxx@facebook
    [author_id] => 14
    [author_ip] => xxx.xxx.xxx.xxx
)

Result:

Array
(
    [spam] => 2
    [profanity] => 0
    [quality] => 0.518
    [session_id] => 1007287f636f97c3e9
)

Test 2: I created a local drupal user "test2" with my mobile phone (thus getting a different IP than my home desktop). Works without problems.

Ham:

Array
(
    [post_body] => Turku is a very nice place for a curling tournament. i look forward for this and I'm so exited about it. we will win, by the way.
    [author_name] => test2
    [author_mail] => foo@example.com
    [author_id] => 16
    [author_ip] => yy.yyy.yyy.yyy
)

Result:

Array
(
    [spam] => 1
    [profanity] => 0
    [quality] => 0.488
    [session_id] => 100729ba9c7b5a5130
)

Test 3: Logged in with the same "test2" account on my desktop computer and again everything works. It looks like the problem is not in the reputation of my desktop computer's IP.

Ham:

Array
(
    [post_body] => Thanks for sharing this schedule. 

We used it on a 4 sheet rink and it worked like a charm!

Thanks again.

-Markus

    [author_name] => test2
    [author_mail] => foo@example.com
    [author_id] => 16
    [author_ip] => xxx.xxx.xxx.xxx
)

Result:

Array
(
    [spam] => 1
    [profanity] => 0
    [quality] => 0.518
    [session_id] => 100729a85822b957e0
)

Test 4: I asked my friend to log in with his FB account and try to post comment. Again, Mollom blocks the comment.

Spam:

Array
(
    [post_body] => Sending a longish English comment to check if it works this way.
    [author_name] => 787xxxxx@facebook
    [author_id] => 10
    [author_ip] => zz.zzz.zzz.zzz
    [session_id] => 100729809b2f9e6346
)

Result:

Array
(
    [spam] => 2
    [profanity] => 0
    [quality] => 0.401
    [session_id] => 100729809b2f9e6346
)

Is there any news about this?

I run a large scale website, and even we wouldn't mind having the option of displaying the captcha if mollom thinks the content is pure spam. As other users have mentioned, we have trusted users who are getting blocked for apparently no reason. We'd rather they got a captcha than force us to add them to a special trusted role. They are 'trusted', but we'd still rather not let them completely bypass spam protection. Of course, the mollom API specifically warns against showing the captcha when content is considered spam:
http://mollom.com/api/checkContent

I have not had the chance to do as much testing of the issue as others, but it's possible we are running into the spammy name issue. Mollom keeps track of a user's reputation over time. Is this not on a per site basis? Is it really a global name comparison? Is there no way for us to check a users reputation for our site? It looks like the mollom module used to store the quality, reputation, and languages in the mollom table, but it appears it's only calling the 'spam' check these days. If it was still storing these values it would definitely help with this type of investigation.

I also interested in the reputation and classifier parameters in the checkContent method, but the API documentation on this is lacking. I'm curious of that is something that could help with these types of issues.

follow-up patch for the approach in #5 for people that want to have a captcha in any case and are not targeted by manual spammers very often.

Any news on this? I think I'm running into the same problem - authenticated users on my site should be able to bypass Mollom checking; however, I've got a few that aren't able to get their comments through, and I'm guessing it might have to do with their usernames.

I'm using Drupal 6.19 and Mollom 6.x.1.14.

We get a lot of spam traffic on our site (manual and automated), though, so the patch in #34 might cause us some problems.

Session IDs: 101105eeab29e8764a, 101105c4a06748ea20, 101105d50bfe511370, 101104ebd1d17c0381

I've flipped the status on this because I have users who are getting blocked by Mollom, and there have been a couple good pitches here for resolving the problem. Admins can't give bypass-mollom privileges to users just because users aren't given a manual captcha option, that's mad amounts of micromanaging that captchas are supposed to help with.

I think giving Mollom some gradient of protection is an excellent idea, and put the control of that gradient into the hands of the admins. But soon, because our organization is losing users because this spam protection tool is protecting us from non-spammers. I would help in any way I can, but its been suggested that this issue be resolved on the Mollom backend, am I right?

Hmm, on our site user register usually with "artists names" not real names. So there are some "pirate of ..." e.g. Unfortunately many of them are blocked by mollom - not for the comment, but for their names. I think this is not a good idea to block by user names. Therefore I bump here. The issue isn't fixed yet?! Thanks for your affords.

I'm planning to implement Facebook integration to another site (see my comment #31 where I debugged this issue with my site www.curlingcalendar.com).

I really like both Mollom and Drupal for Facebook modules. Unfortunately it looks like nothing has happened to this issue that usernames like NNNN@facebook (created by Drupal for Facebook) are identified as spammers by Mollom.

If this remains the case, the only workaround that I can think of is to modify this module so that all data sent to Mollom servers uses Anonymous as the username instead of Drupal usernames. I checked the code of this module and it looks like this could be easily changed in function mollom_form_get_values around line 1128 (in 6.x-1.16 release).

Before starting to hack I would like to ask a couple of questions from module maintainers:

1. Do you think something will be done in the Mollom end to help the issues with problematic usernames?

• I totally understand that the username is an important piece of information when making the spam analysis but still... Drupal is a great platform for community websites and Facebook integration is a must for many, many community sites today

If you think that no changes will be done in Mollom end to this, I would like to ask a second question:

2. How do you feel if a feature would be added to this Drupal module that allows us to always send "Anonymous" as the username that will be sent to Mollom? If the community creates this patch, would you commit this feature?

If the question to the second question is that you would commit the patch I am willing to contribute to this patch. If you don't like the idea and won't commit the patch then I will only make a quick & dirty patch that modifies the username and won't spend any time adding the configuration option...

I also have this problem. I thought I could get around this by setting the Facebook permissions to allow access to the email address, but what gets set instead looks like this:

Spam: 'I would answer, except that I can\'t ente'
Data:

  post_body = 'I would answer, except that I can\'t enter a proper equation here.'
  author_name = '100002870625907@facebook'
  author_mail = 'app+3tvbbjlcbh.2quetp91jj.c5bc9bd8f3dcc12361719ad400b59b87@proxymail.facebook.com'
  author_id = '134'
  author_ip = '192.168.40.1'
  session_id = ''
  checks = 'spam'

Result:

  spam = 2
  session_id = '110926d6379702e1cd'

I understand that this email address looks awful spammy, but I'm not sure what else to do.

The site I am building falls under COPPA laws, and for users under the age of 13, I cannot store their email addresses. When a child posts, no email address is sent, and Mollom responds with a CAPTCHA form and flags the content as unsure regarding whether or not it is spam.

I feel like the best option is to prepend my own validation handler to my forms to process the Facebook email address before the form data gets sent to Mollom. If I can't get an email address, all of my Facebook users will need to fill out the CAPTCHA on every submit. As a workaround, it's better than nothing.

Something like this:

  function MYMODULE_form_alter(&$form, &$form_state, &$form_id) {
    switch($form_id) {
      case 'comment_form':
        // Prepend our own form validation handler
        array_unshift($form['#validate'], '_mollom_facebook_email_preprocess');
        break;
    }
  }

function _mollom_facebook_email_preprocess(&$form, &$form_state) {
  if (strpos($form_state['values']['author'], '@facebook') !== false) {
  	$info =_get_fb_user_info();
  	if(!empty($info['name'])) {
  		$form_state['values']['name'] = $info['name'];	
  	}
  	if(!empty($info['email'])) {
  		$form_state['values']['mail'] = $info['email'];
  	}
  }
}

function _get_fb_user_info() {
  global $_fb, $user;
  $fbu = fb_get_fbu($user->uid);
  if (isset($_fb) && $fbu) {
    try {
      $info = $_fb->api($fbu);
      if (!empty($info)) {
        return $info;
      }
    }
    catch (Exception $e) {
      if (fb_verbose()) {
        fb_log_exception($e, t('Failed to get facebook user email.'));
      }
    }
  }
  return FALSE;
}

When I implement this code, my form submissions appear to pass Mollom's validation.

@ebeyrent: Thank you for providing that session_id! I've passed along that information to Mollom's backend engineering team and they were able to identify an inappropriate classification result in one of Mollom's classifiers, and adjusted it. Can you try whether you see an improvement now?

Sure - here's another example of something that should have passed, but didn't:

Type	mollom
Date	Tuesday, September 27, 2011 - 09:58
User	100000924679465...
Location	http://cert.answerplease.infoplease.com/comment/reply/33418
Referrer	http://cert.answerplease.infoplease.com/comment/reply/33418
Message	Unsure: 'No. Albany is the capital of NY.'
Data:

  post_body = 'No. Albany is the capital of NY.'
  author_name = '<hidden to protect the innocent>'
  author_mail = '<hidden to protect the innocent>'
  author_ip = '192.251.134.5'
  session_id = '1109273e5682cc403c'
  checks = 'spam'

Result:

  spam = 3
  session_id = '1109273e5682cc403c'

Severity	info
Hostname	192.251.134.5
Operations	

Please note that the original post had a valid username and email address.

@ebeyrent: Sorry, but I don't see how that is related to this issue. Mollom's final spam classification was "unsure" (3), and so the user was asked to solve a CAPTCHA (which she solved). The topic of this issue is users getting blocked due to their user/author name.

Did you test the facebook case once more?

@sun - I felt it was related because it happened when I implemented the code in #40. Facebook users, even when providing a valid user and email, were still being mis-identified.

In response to #41, I removed the extra validation code from the form and resubmitted with another Facebook user account. This time, the post was not blocked by Mollom. Here's the log entry:

Ham: I'm curious as to why you want to know t
Data:

  post_body = 'I\'m curious as to why you want to know this.'
  author_name = '100002870625907@facebook'
  author_mail = 'app+3tvbbjlcbh.2quetp91jj.c5bc9bd8f3dcc12361719ad400b59b87@proxymail.facebook.com'
  author_id = '134'
  author_ip = '192.168.40.1'
  session_id = ''
  checks = 'spam'
  strictness = 'normal'

Result:

  spam = 1
  session_id = '110927a5ebf2d6756c'

Alright, thanks! A "ham" response sounds like a definitive improvement.

Since user/author names related to Facebook integration kinda dominated this issue, I think it's safe to call this issue fixed now.

Note to everyone: Issues like this can be resolved a lot faster when filing a support request directly with Mollom support.

As mentioned on http://drupal.org/project/mollom :

Use the Mollom support ticket system or send an e-mail in case of issues pertaining to the Mollom service but not the module; i.e., inappropriately blocked posts (false-positives), false-negatives, etc. Make sure to include the Mollom session IDs of the concerned posts (which can be found in your site's Recent log messages).

I have entered a ticket in the Mollom ticket system - while this is great that my Facebook users can post without being blocked, many of them are reporting that Mollom is unsure whether or not the post is spam, so they always have to fill out the CAPTCHA form.

Thanks for your help @sun!

It is perfectly possible that Mollom's classifiers slowly need to "retrain" themselves on this kind of Facebook author information. The "unsure" response seems to be logical for me -- it probably means that Mollom engineers made the classifier "forget" about this pattern entirely, and thus, there's neither positive nor negative data for it currently. As long as that kind of author information (pattern) is not reported as spam from sites in the Mollom network, the classifiers will slowly learn that it is ham.

The original issue here of users getting entirely blocked, however, seems to be resolved at this point.

I'd like to keep this issue closed, because there is no bug in the Drupal module, and so there's not really something that can be fixed here.

I agree - thanks again @sun.