By Ben E on

Hi all,

I have a site that I have recently converted to Drupal, so many of the old urls on the site are no longer valid. However, because Drupal generates its own 404 errors, these pages are not being deleted from search engine indices, and still show up when I search Google etc. Is there any way to get Drupal to send a real 404 for these old urls, so that the crawlers delete them?

cheers
Ben

⋅ Categories: Drupal 4.6.x

Comments

drubeedoo’s picture

drubeedoo commented

If you want to permanently delete pages from crawlers, you need to look at "Redirect 410" rather than 404.

Locate the following in your .htaccess file:

# Customized error messages.
ErrorDocument 404 /index.php

Add the following directly above it:

# Old pages to 410 error
Redirect 410 /myoldpage.php
Redirect 410 /someotherpage.php

Redirect 410 is a permanent "page gone" and should signal Google & other crawlers to remove the page. In case I've messed up the syntax, Google for "htaccess" and "redirect 410".

That should get you on the right track...

Ben E’s picture

Ben E commented

Thanks, that's definitely the appropriate code. As usual Mark Pilgrim has a good page on the topic:

http://diveintomark.org/archives/2003/03/27/http_error_410_gone

I've got dozens of these pages, so I can't stick them all in .htaccess (or don't want to!), but I can use mod_rewrite to check for the old-style url - testing for the .html suffix should do it - then send the 410.

I'll give it a go, cheers

Ben