I just installed this module and wonder if anyone knows where I can find information on how to make the robots tag display as noindex of the 403 & 404 pages?

Thanks.

Comments

wiredescape’s picture

wiredescape commented

Bump...

Also noticed this problem.

When pages are un-published it would be better if search engines got a 'noindex'.

wiredescape’s picture

wiredescape commented

Of course when there is no robots tag for the node it defaults to "INDEX,FOLLOW":

Metatags (Nodewords) D6x-1.11 does provide for setting robots tag on 403 & 404 pages. Unfortunately, Nodewords 5x version does not so defaults to "INDEX,FOLLOW".

Would be better if Custom Error set robots tag to "NOINDEX" as a default.

gisle’s picture

gisle
he/him
Primary language Norwegian Bokmål
Location Norway
commented
Status: Active » Closed (works as designed)

The HTTP server responses returned for the 403 and 404 error pages are:

HTTP/1.1 403 Forbidden
HTTP/1.1 404 Not Found

Search engines are smart enough to figure out that such pages shall not be indexed, so there is no need to provide specific robots metatags for such pages.