Enhancement request to search404 providing ability to set meta noindex tag to "noindex, nofollow"
| Project: | Search 404 |
| Version: | 5.x-1.x-dev |
| Component: | Code |
| Category: | feature request |
| Priority: | normal |
| Assigned: | zyxware |
| Status: | closed |
Jump to:
Currently, we use Google Webmaster Tools to manage URL removals and Google's policy for removal states:
To remove content from the Google index, do one of the following:
Ensure requests for the page return an HTTP status code of either 404 or 410.
Block the page using a robots.txt file.
Block the page using a meta noindex tag.
http://www.google.com/webmasters/sitemaps/
The issue with search 404 is that it does not return a 404 message nor does it contain a meta noindex tag set to "noindex, nofollow" and defining all the pages in the robots.txt for removal would be tedious.
Currently, a search for a page will redirect Google's bot to the search404 results and return a successful search results page (with no 404 error) with a meta noindex tag set to index, follow. The web page removal is then denied.
If the search404 page could substitute the existing meta noindex tag with:
Then Google's bot should obey the rules and mark the page for removal. This functionality would need to be tested.
If there is SEO benefit to indexing search404 pages then consider making the meta noindex tag editable so users can specify which tags they would like to set. Maybe it is possible to integrate search404 module with node words module (aka Meta tags).
http://drupal.org/project/nodewords
Hope this makes logical sense,
E&E
#1
Forgot to add code tags around meta tags. The following should read:
If the search404 page could substitute the existing meta noindex tag with:
<meta name="robots" content="noindex,nofollow" />#2
My apologies for the delay.
This has been bugging for a while, but I can't come up with any solution that doesn't increase the page load on every page (i.e. hook_init). I suppose you use the "jump"-feature, 'cause the search results page should generate a proper 404 header. One temporary solution could be to disable jumping until the pages are removed from the index.
The root to the problem is that 404 is not a valid redirecting code. Right now it passes a 302/Temporary redirect instead of the 404. http://api.drupal.org/api/function/drupal_goto/5
#3
Hello,
Google bot and other search engines boats still visit hundreds of pages that have been created by a translation script even though I disabled the script and deleted the translated pages on the server,
do you think it is because of search 404 that the bots still visits the translated pages, causing high load on the server ?
I do not use the jump feature.
Thanks.
#4
Any help ? I still have the same problem. Thanks.
#5
This feature seems to be working fine and search404 is returning a 404 error it is going to be assumed that this problem has been fixed. Also please upgrade to the latest version of the module. Regarding the meta noindex nofollow - since the 404 error code is returned there is no need for meta tags.
#6
Automatically closed -- issue fixed for two weeks with no activity.