Hello Everybody,
I keep getting "Front controller: X-Authcache header is missing on request" errors in my drupal logs.
PHP code:
if ($check_header && empty($_SERVER['HTTP_X_AUTHCACHE'])) {
watchdog('authcache_p13n', 'Front controller: X-Authcache header is missing on request', array(), WATCHDOG_ERROR);
throw new AuthcacheP13nRequestInvalidInput();
}
According to the php snippet the error is thrown when the authcache header is not set.
I checked server logs
109.169.29.30 - - [30/Jan/2014:09:29:34 +0100] "GET /nan-nue HTTP/1.1" 200 20883 "-" "Mozilla/5.0 (compatible; aiHitBot/2.8; +http://endb-consolidated.aihit.com/)"
109.169.29.30 - - [30/Jan/2014:09:29:40 +0100] "
GET /sites/all/modules/drupal.org/authcache/modules/authcache_p13n/frontcontroller/index.php?r=frag/form-token&a=commerce_cart_add_to_cart_form_17780&q=&v=null HTTP/1.1" 400 5
"-" "Mozilla/5.0 (compatible; aiHitBot/2.8; +http://endb-consolidated.aihit.com/)"
109.169.29.30 - - [30/Jan/2014:09:29:44 +0100] "GET /sites/all/modules/drupal.org/authcache/modules/authcache_p13n/frontcontroller/index.php?r=frag/form-token&a=commerce_cart_add_to_cart_form_17780_197976&q=&v=null HTTP/1.1" 400 5 "-" "Mozilla/5.0 (compatible; aiHitBot/2.8; +http://endb-consolidated.aihit.com/)"
109.169.29.30 - - [30/Jan/2014:09:29:48 +0100] "GET /sites/all/modules/drupal.org/authcache/modules/authcache_p13n/frontcontroller/index.php?r=frag/form-token&a=commerce_cart_add_to_cart_form_17780_197969&q=&v=null HTTP/1.1" 400 5 "-" "Mozilla/5.0 (compatible; aiHitBot/2.8; +http://endb-consolidated.aihit.com/)"
The error happens when a search robot follows the links like /sites/all/modules/drupal.org/authcache/modules/authcache_p13n/frontcontroller/index.php?r=frag/form-token&a=commerce_cart_add_to_cart_form_17780_197969&q=&v=null. Such links are used to replace html snippet with the current user sensitive data using ajax.
Do you have any solutions to prevent such links from being followed by search robots.
Comments
Comment #1
znerol CreditAttribution: znerol commentedThanks for your detailed report. This is indeed a very interesting case. In fact Authcache Ajax injects URLs into data-attributes of
span
tags. Obviously some robots follow everything which looks like an URL, regardless of whether it appears in ahref
attribute or somewhere else on the page.I wonder whether it would help when not the absolute URL, but only the path would be embedded into the markup. I.e. instead of writing
<span data-authcache-ajax-src="http://example.com/path/to/frontcontroller.php?some=param"></span>
just output the relative URL, e.g.<span data-authcache-ajax-src="path/to/frontcontroller.php?some=param"></span>
.Like this it wouldn't be very likely that a robot will extract and follow those links. I will try to come up with a patch soon.
Comment #2
znerol CreditAttribution: znerol commentedThis patch introduces a new configuration setting which allows to specify whether or not URLs to the frontcontroller should contain the
$root_url
($conf['authcache_p13n_frontcontroller_absolute']).The setting defaults to
FALSE
. If no value is specified, a relative URL is generated.Comment #4
znerol CreditAttribution: znerol commented2: 2185299-frontcontroller-use-relative-urls-by-default.patch queued for re-testing.
Comment #6
znerol CreditAttribution: znerol commentedComment #7
znerol CreditAttribution: znerol commentedComment #9
znerol CreditAttribution: znerol commentedI don't know why the tests fail on the testbot. They pass on my machine. Add a
debug
message to the flag test, and hope it will show us what is going on inside the testbot.Comment #11
znerol CreditAttribution: znerol commentedOk, seems like the relative-URL approach does not work when clean URLs is disabled. Therefore we need to generate the url with the
external
option set toTRUE
.Comment #12
znerol CreditAttribution: znerol commentedPerfect, tests turned green at last. So this is ready for a manual test-drive now. @Esculap: It would be great if you could apply this patch and test whether it resolves the problem.
Comment #13
Esculap CreditAttribution: Esculap commentedHello,
Thank you for your quick response. I will apply the patch on my website and will let you know about results
Comment #14
znerol CreditAttribution: znerol commented@Esculap: Do you happen to have a chance to report whether the approach works?
Comment #15
Esculap CreditAttribution: Esculap commentedHello, Znerol!
I am sorry I did not reply to you sooner. I have applied your patch manually and have made some testing. Your changes look great and I am about to deploy them on the acceptance server.
Just one notice, as you propose to change AuthcacheP13nDefaultRequestUrlGenerator property from $frontControllerPath to $frontControllerURL I think it makes sense to rebuild registry by calling authcache_p13n_request_router_rebuild(). What do you think?
Comment #16
znerol CreditAttribution: znerol commentedYes indeed, this change requires a rebuild of the request router (note to self: should add an update hook for that). You can trigger a rebuild by visiting Administration » Configuration » System » Authcache » Personalization » Frontcontroller and click the Rebuild routes button on the bottom of the page.
Comment #17
znerol CreditAttribution: znerol commentedCommitted and pushed: 71efdc9