By escoles on

I've been pointing 404/403 hits to my index.php page as a convenience to users; however, this is murdering my bandwidth, as something over 85% of my site traffic right now (over 100K pages so far this month) is losers trying to trackback- or comment-spam my site by passing Drupal URLs. With trackback disabled and comments only available to registered users, this means I've gotten over a hundred thousand 404 errors in teh last few weeks. That means lots of database traffic -- or at least, lots of page code going down the pipe.

SO -- I desperately need to make custom 404/403 pages work. But I can't seem to make it happen.

Here's what I've done so far:

* created pages to act as targets for 400/403/404/500 errors.
* Set the value on the Drupal Setup page for the 403 and 404 error pages to the appropriate files.
* Set ErrorDocument parameters in .htaccess (e.g., "ErrorDocument 404 /404.shtml").

And yet still, requests for non-existent pages are directing to index.php. Am I missing something?

Comments

escoles’s picture

escoles commented

Sorry, Drupal version is 4.5, hosted on a Linux system.

jo1ene’s picture

jo1ene commented

You don't need anything for errors in .htaccess to use the Drupal 404/403 feature. Just take it all out. Create a page for each error - through Drupal. Then specify "node/##" or whatever in administer > settings in the error handling section.

Advanced Web Design

escoles’s picture

escoles commented

It's CRUCIAL that these pages NOT BE SERVED BY DRUPAL.

I'm pushing over 6GB of bandwidth usage a month and a large part of that is because I'm having to serve the entire page layout of a Drupal node page every time I have to serve a 404. And my ISP is threatening to shut me down because of the CPU usage. There's no point in driving these pages out via Drupal when I can just serve them statically with Apache.

I do not want to replace Apache functionality with Drupal functionality. That's just silly. And wasteful.

This feature should work. Why doesn't it? What am I missing?

jo1ene’s picture

jo1ene commented

...I was missing your point. The reason is you were referenceing drupal's error pages... nevermind. I just tried a couple of tests on my site and you're correct. Drupal completely takes over error page handling.

If you look in index.php, you'll see the function calls for 404 and 403 errors.

switch ($status) {
  case MENU_NOT_FOUND:
    drupal_not_found();
    break;
  case MENU_ACCESS_DENIED:
    drupal_access_denied();
   break;
}

Those functions are in common.inc. I tried commenting out the function calls in index.php to see what would happen but you get a blank page. Nothing. I guess you would have to alter the functions in common.inc. I wouldn't know how off the top of my head. Good luck.

Advanced Web Design

escoles’s picture

escoles commented

I was hoping not to have to hack core to do something as simple and obvious as this, but here's what I've done: I've modified drupal_not_found() and drupal_access_denied() so that they redirect to the hard-coded address of a real 404/403 page if it's a 404/403.

Since there's no need for such a page to be more than a few bytes, and since something over 80% of my 6GB of bandwidth usage for the first 18 days of this month was 404 pages, this step alone should reduce my bandwidth usage by around 7.5GB per month.

Here are the code changes I made: 

/**
 * Generates a 404 error if the request can not be handled.
 */
function drupal_not_found() {
  header('HTTP/1.0 404 Not Found');
  watchdog('page not found', t('%page not found.', array('%page' => theme('placeholder', $_GET['q']))), WATCHDOG_WARNING);

  $path = drupal_get_normal_path(variable_get('site_404', ''));
  $status = MENU_NOT_FOUND;
  if ($path) {
    menu_set_active_item($path);
    $status = menu_execute_active_handler();
  }

  if ($status != MENU_FOUND) {
    // begin ES 2005-08-19: ADDED TO REDIRECT TO REAL 404 PAGE
	header("Location: http://antikoan.net/404.shtml");
	exit;
	// end ES 2005-08-19
    drupal_set_title(t('Page not found'));
    print theme('page', '');
  }
}

/**
 * Generates a 403 error if the request is not allowed.
 */
function drupal_access_denied() {
  header('HTTP/1.0 403 Forbidden');
  watchdog('access denied', t('%page denied access.', array('%page' => theme('placeholder', $_GET['q']))), WATCHDOG_WARNING, l(t('view'), $_GET['q']));

  $path = drupal_get_normal_path(variable_get('site_403', ''));
  $status = MENU_NOT_FOUND;
  if ($path) {
    menu_set_active_item($path);
    $status = menu_execute_active_handler();
  }

  if ($status != MENU_FOUND) {
    // begin ES 2005-08-19: ADDED TO REDIRECT TO REAL 403 PAGE
	header("Location: http://antikoan.net/403.shtml");
	exit;
	// end ES 2005-08-19
    drupal_set_title(t('Access denied'));
    print theme('page', message_access());
  }
}

Now, none of this should have been necessary. It should be possible to let Apache do the work of sending users to the right 404 page. In fact, I expect that I'll learn when I dig into this further (which I will, since I have loathe operating with a forked core codebase) that the real demon is in the mod_rewrite rules that implement clean URLs. I.e., I should be able to fix this defect by hacking .htaccess after all.

ADDENDUM: I upgraded to Drupal 4.6.3 last night. Also, changed code above to point to a common error.shtml page that will display conditionally.

styro’s picture

styro commented

that you might want to look at...

There's a patch for the customerror module to allow non themed pages as error pages like the way front_page module works.

http://drupal.org/node/28648

It wouldn't completely cut out all Drupal processing, but should help (I think) by avoiding all the theme templating and block generation etc. And it would allow you to use a very light error page to reduce traffic.

It would save you having to hack core code, and if you still want redirects, you could add them to it the same way front_page does them as well.

IamPter’s picture

IamPter commented

Are you using gzip compression on your site?

escoles’s picture

escoles commented

Not that I'm aware of. Why?

IamPter’s picture

IamPter commented

Using gzip compression should reduce your bandwidth issues.

Add this to the .htaccess file:

php_flag zlib.output_compression On

oNyx’s picture

oNyx commented

And my ISP is threatening to shut me down because of the CPU usage.

Caching helps. But CPU usage should be lower anyways, because you upgraded.