Mollom incompatible with Pressflow's (and Drupal 7's) session handling -- causes CAPTCHAs to fail

I am getting some similar problems. I also have enabled CAPTCHA on the user registration form (which has been aliased to '/join'). In my case I can eventually get the CAPTCHA to pass after a few tries but it's very inconsistent.

Similar to the OP, our server configuration is non-standard. We are using memcache for our sessions and we are also in a load balanced environment.

Please let me know if any other information will be helpful.

I'm experiencing the same issue on two sites, one of which is running pressflow and one which is not. None of them are in a load-balanced environment.

Mollom was working fine until about two weeks ago, when my client reported that all text analysis were reporting failures, so we disabled the text analysis. Then yesterday all captchas were failing for the authenticated comment form, so we had to disable that as well. The registration form is still protected, but somehow this one still works fine.

That message can mean at least three things:

1. your environment is setup properly and spammers are trying to do record-replay attacks by reusing Mollom session IDs,
2. your installation's session handling is not working as expected, or
3. you are caching pages that shouldn't be cached.

By the way, I see (1) happening on my sites. Typically always from the same IP address. That is, spammers are using record-replay attacks so it is a legitimate message on my site.

It could be a legitimate message on your site as well -- but (probably) not if it happens (almost) all the time.

Talked with DamZ about this, and it is indeed a problem with PressFlow (and Drupal 7). To enable more aggressive caching, PressFlow and Drupal 7 no longer keep a session if there is nothing to save in that session. As a result, session IDs are not maintained across page requests. Mollom uses the session ID to identify record-replay attacks across page requests.

A quick and easy solution would be to have Mollom create a permanent session ID by storing some dummy data in the session, but that would affect cache-ability of those requests.

Moved the session matching into $_SESSION, that should fix the issue with the new session manager of D7 / D6 Pressflow.

Would be great to get some test reports. :)

The problem here is that requests for these pages won't hit Drupal/Pressflow in the first place if the page is being served via a CDN or reverse proxy cache. There's no easy solution to this because Mollom is designed to prevent replay attacks, and proper page caching induces essentially identical behavior.

It would be possible to fix this with an uncached edge-side include with Varnish and some CDNs, but that's non-portable to Squid and cheap CDNs. Javascript could do something similar in, again, a not-totally-portable way across JS- and non-JS-supporting browsers.

A really nice solution would be using lack of the has_js cookie to bypass the reverse proxy and using JS on supporting clients. The Mollom ID would be stored in a cookie that the reverse proxy would also filter out. That would work for everyone and allow the vast majority of anonymous users to load Mollom-powered pages from a cache.

The suggested $_SESSION-based fix above is a fix, but it will cause users to bypass the reverse proxy as soon as they've even *viewed* a page where Mollom works its magic. A slightly less bad quick fix would be Mollom setting a cookie that expires quickly (far faster than a typical session); this would allow users to quickly return to loading fully cached pages.

In the mean time, I've filed this as a compatibility bug on the Pressflow side:
https://bugs.launchpad.net/pressflow/+bug/432090

The problem here is that requests for these pages won't hit Drupal/Pressflow in the first place if the page is being served via a CDN or reverse proxy cache. There's no easy solution to this because Mollom is designed to prevent replay attacks, and proper page caching induces essentially identical behavior.

This is not an issue. Mollom will specifically prevent a page from being cached when a CAPTCHA is displayed. All the pages that needs to hit Drupal will hit it.

The suggested $_SESSION-based fix above is a fix, but it will cause users to bypass the reverse proxy as soon as they've even *viewed* a page where Mollom works its magic. A slightly less bad quick fix would be Mollom setting a cookie that expires quickly (far faster than a typical session); this would allow users to quickly return to loading fully cached pages.

Actually, there are two mitigating factors:

* first, a session will only be started when displaying a CAPTCHA-protected forms. Merely displaying a form will not start a session.
* we expire the Mollom session when it has been idle for one hour, and as soon as a Mollom session has been used (when a form is successfully submitted)

subscribing.

Similar, if not same, issue.

I have a load balanced website with 2x web servers and 1x database. I'm not sure what's in front of it to load balance though, as the host deals with this and I don't have access to that info.

Is there anything in Drupal's "Performance" caching if it's set to "Normal" and the Lifetime is set to "NOne" that could cause this.

@j0rd: That is not a similar issue. It is a support request. Please take it elsewhere (new support issue, IRC, forums).

@Damien Tournoud: That's comforting. Anything I can do to help this along?

@j0rd: Sorry, just realized you were asking about the effect of the cache configuration, not how to configure the cache on your host. To answer your question, a "normal" cache configuration will not cause this problem on Drupal or Pressflow, regardless of downstream proxies and CDNs (assuming they're configured to respect HTTP cache header RFCs).

@j0rd: Wait, still investigating further about the "normal" option on Pressflow. I believe sessions are still lazy, and hence would cause this issue. Standard Drupal 6 should be fine.

I am using Drupal 6, not PressFlow.

I noticed when I turned my cache off, I was unable to re-duplicate the issue. I am leaving my cache off for the weekend to see what it does to my Spam vs. Ham ratio. Should my "spam" (failed because captcha doesn't show up) ratio go down, I'll assume it's an issue with caching and I'll get back to you guys on that.

I'm using Durpal 6.14 and but no PressFlow. Mollom is reporting failure everytime. This started for me last weekend (approximately) and i've been watching for updates to make it work all week. My caching has been disabled the entire time.

I also upgraded recently (from a site that had been on v5), and the captcha was failing all the time (blocked even though entered correctly), despite everything looking right in settings. I tried turning caching off, and still failed.

I turned off the module, ran the uninstall routine, then re-installed and and configured, and now it's working fine.

I'd also be open to adding a drupal_session_id() function to PF6 and D7 that returns a session ID and starts a session if one isn't already in progress. Would that make this easier?

My spam rate is still hovering around 90%. I worry that this isn't all spam and that some stuff is not getting past mollum.

Is there a way to log which failed requests are blocked by mollom?

j0rd et al, have you read http://mollom.com/blog/important-note-to-drupal-5-and-drupal-6-users ?

I'm still looking for some more success stories with the patch in #7. I haven't run the Mollom 6 tests yet with this patch applied but I will.

Thanks for the heads up Dries. I will update Drupal with my next stage roll out and see if that resolves the issue.

Marking fixed because these issues should be fixed now.

Has this been fixed for users of reverse proxies? We are using the reverse proxy patch and Mollom on g.d.o, and I'd like to know if we should expect any issues.

Sorry, this issue isn't fixed yet.

Sorry, I'm way late to the party on replying to this. Way back when I posted #1 we were doing lots of debugging and determined that our the mollom cache was not persistent enough across the particular page requests that the mollom issue was cropping up on. For most loads of the page, the cache was being either not properly saved to memcache, or memcache was being dumped altogether. Occasionally it would be properly cached and loaded and the mollom logic would pass.

Our solution at the time, (and its still working) was giving the mollom cache it's own personal memcache bin. This allowed those entries to have the appropriate amount of persistence.

I'll have to run this thread by someone more knowledgeable about the current server configuration of the site to give any meaningful feedback. I'll also see if we can try out the patch and report any findings. As for now, things are running smoothly with mollom happily storing to its own bin.

One more note, my setup does not fall into the reverse proxy category that I believe you guys want testing feedback on anyhow.

I'm having mollom failures on my site, too. If text + captcha are enabled, comment posters get a "we're sorry, looks like spam, fill out the captcha", but there is no captcha to fill out. If I turn off the text and just use captcha, I get the image but it won't accept the input. It just returns the red warning "Captcha is a required field". I'm using drupal 6.14, so the September issue should be updated.

This is happening since this morning.

subscribe

subscribe (using pressflow + mollom doesn't work)

As #25 mentioned, that was our solution as well. Give mollom its own bin. O ya subscribe :-D

I can confirm that the CAPTCHA on http://groups.drupal.org/node/add/account-synch works when I'm logged in, but not when I'm logged out. So it must be related to this problem as well.

The patch in #7 doesn't apply to DRUPAL-6--1-10, but it applies to DRUPAL-6--1. The patch allows CAPTCHAs to work on Pressflow, without CDN/Varnish. Not tested with.

I am using pressflow (with current bzr update) and the changes in the patch on #7 fix this problem for me. I had to adjust the patch to fit the latest mollom module -- the modified patch is attached.

Correction: the patch from #7 sometimes causes an error in (patched) line 166 if $_SESSION['mollom_sessions'] is not set. So the attached revision of the patch, again against the latest mollom release, checks this first. Still corrects the problem for me.

Getting Pressflow set up shortly to test...

Patch #34 works well in pressflow

This will need a re-roll and re-test with the latest code.

#34 works for me also

CVS code is pretty different. It's not a straight re-roll since the code that's altered has been refactored; I'll make a run at this later if I get the time, but the patch works good vs 6.x-1.10 for anyone who need sit.

I am experiencing the same issue with the Captcha Consistently returning failure on both Image and Audio captcaha. We are using d6 on a dedicated host, no load balancing atm. We are not using the afformentioned press module.

Any updates on steps to fix this would be good.

Re-rolled against DRUPAL-6--1.

Also fixed a small bug in the form submit handler, where the wrong key was tried to be unset.

Aside from that,

+++ mollom.module	(working copy)
@@ -157,6 +157,25 @@
+function mollom_init() {
+  // Expire the mollom session IDs as soon as possible.

Shouldn't this happen in hook_exit() already, so a subsequent request won't needlessly start a session?

This review is powered by Dreditor.

Fixed a PHP notice in mollom_init() when trying to unset a key from non-existing $_SESSION variable.

All tests pass for me. Additionally, this patch allows more tests to pass in #663246: Port to Drupal 7.

I reviewed this patch and it looks good. Committed to the DRUPAL-6 branch so this fix will be part of the next Mollom release. Didn't commit it to the HEAD branch where Drupal 7 gets backed. Thanks all.

Hmmm. We should still investigate whether hook_init() is the right location to kill session data.

Technically, I'd say that we would already know in hook_exit() whether the next request will still need session data. Hence, by removing the session data at the end of the current request, the next one possibly wouldn't have to start a session at all...?

Thoughts?

I think you're right. Should be relatively easy to try given that you have the Drupal 7 port of Mollom working -- we can change it and simply run the tests. I could give that a try tomorrow evening.

Tested the simple change from hook_init() to hook_exit() in D7 and tests passed. However, postponing this issue until #663246: Port to Drupal 7 is "sorted out" and committed, so this change is actually tested.

I took the DRUPAL-6--1 branch of the Mollom module and tried it. ( I have a site that I recently switched to Pressflow to fix a different problem, and I think that Mollom quit working when I did the switch.) Using that version of Mollom appeared to cause captchas to start working.

However, when I did the Mollom Simpletest, it gave "433 passes, 525 fails, and 0 exceptions." That was on a Pressflow 6.14 that has many modules and custom modules. I then tried it on a "clean" Pressflow 6.15 ( only a couple standard modules enabled, suchs as Blog ) and then on a "clean" Drupal 6.15 (only the Mollom and SimpleTest modules were added) with about the same results -- "433 passes, 525 fails, and 10 exceptions."

Are the large number of fails in the SimpleTest results OK ?

I am also posting this on https://bugs.launchpad.net/pressflow/+bug/432090 .

Thanks,

--Rob

My initial reaction would that these fails are not OK and that further investigation might be required. Hence, I'm setting this to back 'active'.

With sun's help, we recently completed upgrading the Mollom module to Drupal 7 and all tests pass on CVS HEAD. To my knowledge, Pressflow's improved session management was backported from Drupal 7, and the patch in this issue was also integrated in the Drupal 7 version of the Mollom module. In other words, we had more reasons to believe it would work on Pressflow. Maybe Pressflow diverged at other points causing these tests to fail -- someone will have to look into the results. In general, it would be great if more of Pressflow's changes were merged into Drupal 7 so we don't have to go through these pain points and we have much better predictability and compatibility ...

Given that the tests fail on a clean Drupal 6 install, maybe something is wrong with your hosting environment. Care to share more information about the failures?

@Dries I'll check with Robert tomorrow, but he's probably either testing on a CentOS 5 VM or our Rackspace server. Nothing strange.

Just to make sure, on a recently updated Ubuntu 8.10 install in a VMWare I did a clean Drupal 6.15 install.

I checked the mollom-DRUPAL-6--1 out of CVS, and copied the directory to sites/all/modules/mollom. I also install the simpletest module there, and I applied the patch that is in that simpletest module so it would install.

I then activated the Mollom, MollomTest, and SimpleTest modules, put in a public / private key in the Mollom settings, and went to the admin/build/testing and checked the Mollom box and ran the tests.

This time the results were "894 passes, 41 fails, and 0 exceptions", which is better than what I got on the results I posted above. The main difference with the other tests would have been that more of the core modules were enabled, but they were also not on Ubuntu.

The CVS 6--1 version now has fewer simpletest errors under the latest versions of both Drupal and Pressflow. It is interesting that they still don't have exactly the same results, however, Drupal giving "886 passes, 49 fails, and 0 exceptions" and Pressflow "897 passes, 26 fails, and 0 exceptions".

I am using this version of Mollom on a testing site and have not had any problems.

This is very strange. On regular Drupal 6, all tests are passing for me without any fails. I even double-checked this by running them on two separate systems/sites, one on Windows (local dev machine), and Linux.

In any case, we need more info here. It's also possible that those test failures are caused by communication errors. Did you look at the actual results?

(On a related note, I now have to wonder how test results could be provided in a sane way in case like this.)

Perhaps - the most helpful debug info would be if you'd copy the source code of the testing results page, save that into a .txt file, and attach that file here.

subscribing

I apologize for the delay in getting back to this.

I did a "cvs up" on my 6--1 checkout of mollom this morning, and then ran simpletest on mollom under both Drupal 6.15 and Pressflow 6.15.

I have attached two files, they are the "view source" for the two test results.

Let me know if I am doing something wrong in the way I run simpletest, or something else wrong. If it is necessary I could probably move these test environments to a server that I can give outside access.

For any reason, I'm getting the following XML-RPC error for all xmlrpc requests when running tests on Pressflow - but not when running the same tests/code on Drupal 6:

http://drupal.test/pressflow/xmlrpc.php?version=, last error: -32700 - Parse error. Not well formed

I already skimmed the differences in includes/*, but couldn't any important ones. Does Pressflow not support running a site in a sub-directory or something like that? Or are any tweaks to SimpleTest required?

The error in @rgristroph's testing output looks like a Apache modSecurity issue or similar:

http://192.168.245.132:8000/xmlrpc.php?version=, last error: 403 - Forbidden

I tested increasing the permissions on the xmlrpc.php file, and I still get that particular error in the testing output. If I go to that URL directly, I don't get an error from apache -- it says that xmlrpc only accepts POST requests.

I will set up a clean Drupal install, and a clean Pressflow install, on a internet-accessible server. These tests are running on a VMWare on my laptop, maybe some part of that setup is not right.

I don't think anything in Pressflow would not work with regards to being installed in a sub-directory.

Due to a follow-up patch in #674230: Broken mass-reporting, session storage, node title mapping, watchdog messages, administrative user creation form, all tests pass for me on Pressflow now, and manually testing on it worked flawlessly, too. I'm therefore marking this fixed.

I dont understand. Do we need to appy the patch or does the new 6.1-10 version for Drupal 6.15 have the patch already applied? Does this fix the CAPTCHA issue where it denies all new registrants on high traffic sites or does this fix something else?

I am not real clear on what the solution or next steps should be. Can someone clarify?

Here is my problem:

I have a very high traffic site using Drupal 6. The CAPTCHA check for new registrations work for a very short time then all new registrants are denied for having the wrong CAPTCHA entry. I have tried this numerous times for my own new account and get the "Mollom Id: xxx has been tampered with" effor in my logs for a VERIFIED ip address source. (MY MACHINE). I know it is not an attacker or spamer. I am watching the transaction via tcpdump to make sure.

Thanks for any help

6.1-10 is not compatible with Pressflow. You'll have to wait for 6.1-11 (hopefully released later this week) or download the latest development snapshot.

Hi Dries,
We are currently thinking of moving to PressFlow, this thread seems to have been abandoned...
has it been confirmed that Mollom 6.x-1.15 is now compatible with PressFlow 6?
Thanks

@gofair: Yes, it has been reported to work now. However, a follow-up issue has been created last week, which may or may not affect you, too, depending on your caching configuration: #962534: Support CACHE_EXTERNAL in Pressflow