The problem with Boost module is that if someone types an URL similar to 'sitename/thisismyurl/' (i.e. with slash at the end), Drupal + Boost will generate a 'Page not found' cache and save it under 'thisismyurl.html' file. However, URL 'sitename/thisismyurl' (i.e. without end slash) could be perfectly working redirect. And once 'Page not found' cache is generated, Boost starts serving it from there instead of redirecting it properly, thus breaking the site.

Any ideas on how to fix this issue are appreciated. It can be easily fixed for singular occurences (by adding redirect with / at the end), but it cannot be done for hundreds of automatically generated redirects.

Regards,
Alexander

CommentFileSizeAuthor
#4 cache_only_200_d5.patch1.71 KBAlexander Ufimtsev

Comments

Arto’s picture

Title: Non-existing URLs ending with something/ generate cache file something.html » Prevent caching 404 Not Found responses
Version: 5.x-1.0 » 6.x-1.x-dev
Component: Apache integration » Caching logic
Assigned: Unassigned » Arto
Status: Active » Fixed

Thanks for the bug report, Alexander.

I've fixed this in 6.x-1.x-dev as of changeset [148738], by simply preventing the caching of HTTP responses other than the normal "200 OK". Hence "404 Not Found" responses will never be cached in this newest version of Boost, eliminating this issue.

I will not be backporting this to the 5.x-1.x-dev branch myself, but in case you want to do that and post the results as a patch here, I will certainly commit it. Feel free to reopen this issue should you want to proceed with the 5.x patch.

Arto’s picture

Linking to an earlier related issue for future reference: #175633: 404 Not Found response on front page

Anonymous’s picture

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.

Alexander Ufimtsev’s picture

Version: 6.x-1.x-dev » 5.x-1.x-dev
Status: Closed (fixed) » Needs review
StatusFileSize
new1.71 KB

indeed, porting this patch was literally copy and paste. Thanks again, Arto. This works for me, please test:

Index: boost.module
===================================================================
--- boost.module        (revision 197)
+++ boost.module        (working copy)
@@ -300,7 +300,7 @@
   chdir(dirname($_SERVER['SCRIPT_FILENAME']));

   // Check the currently set content type; at present we can't deal with anything else than HTML.
-  if (_boost_get_content_type() == 'text/html') {
+  if (_boost_get_content_type() == 'text/html' && _boost_get_http_status() == 200) {
     if (strlen($buffer) > 0) { // Sanity check
       boost_cache_set($GLOBALS['_boost_path'], $buffer);
     }
@@ -318,14 +318,27 @@
  * has overridden the content type.
  */
 function _boost_get_content_type($default = NULL) {
-  static $regex = '/^Content-Type:\s*([\w\d\/\-]+)/i';
+  static $regex = '!^Content-Type:\s*([\w\d\/\-]+)!i';
+  return _boost_get_http_header($regex, $default);
+}

-  // The last Content-Type header is the one that counts:
+/**
+ * Determines the HTTP response code that the current page request will be
+ * returning by examining the HTTP headers that have been output so far.
+ */
+function _boost_get_http_status($default = 200) {
+  static $regex = '!^HTTP/1.1\s+(\d+)!';
+  return (int)_boost_get_http_header($regex, $default);
+}
+
+function _boost_get_http_header($regex, $default = NULL) {
+  // The last header is the one that counts:
   $headers = preg_grep($regex, explode("\n", drupal_set_header()));
-  if (!empty($headers) && preg_match($regex, array_pop($headers), $matches))
+  if (!empty($headers) && preg_match($regex, array_pop($headers), $matches)) {
     return $matches[1]; // found it
-
-  return $default;
+  }
+  return $default; // no such luck
 }

+
 //////////////////////////////////////////////////////////////////////////////

mcarbone’s picture

d5 patch worked great on a production site. Thanks!

joshk’s picture

@marco: rad!

I think this will also solve my other issue #345484

I'll test and see if I can get this committed to 5.x branch today.

joshk’s picture

Well, no such luck there. #345484: 404 hits to /files directory cached as homepage with broken form actions remains in the wild. This is still good code though. Works great. I will commit to 5.x-dev shortly.

joshk’s picture

Status: Needs review » Fixed

Committed to 5.x-dev

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.

Anonymous’s picture

Version: 5.x-1.x-dev » 6.x-1.18

I'm not sure if this is appropriate to rise the issue here. I think prevent Boost from caching "Page Not Found" urls might bring a Drupal-based website crashing down.

This is because it's a quite heavy load for Drupal to deal with a "Page Not Found 404" or "Rediection 301 302" request. A hacker can take advantage of this to generate some "Page Not Found" requests (around 5 requests per second) to make a Drupal-based website out of order. I think this is a quite serious problem.

I was wondering if there is anyone knows how to cope with the case that I just mentioned.

Thanks.
Ian

Anonymous’s picture

I was thinking to modify the following code to allow Drupal to cache the page not found urls.
- if (_boost_get_content_type() == 'text/html' && _boost_get_http_status() == 200) {
+ if (_boost_get_content_type() == 'text/html' && ((_boost_get_http_status() == 200) || (_boost_get_http_status() == 404))) {
if (strlen($buffer) > 0) { // Sanity check
boost_cache_set($GLOBALS['_boost_path'], $buffer);
}

And using .htaccess to rewirte all ULRs with "/" at the end to make this kind of URLs run without the last /.
Of course, Apache will take care of the case that if there is a .html file under the path.

Not sure if this could work and any side effect?

Thanks.