When I get a 404 and/or 403 page the sourcecode gives me this metadata (see below). Resulting in Google indexing my 404 and 403 pages.

<meta name="robots" content="index,follow" />

Can this be changed? Or is there a workaround?

Comments

BioALIEN’s picture

Priority: Normal » Critical

This is certainly a major bug. Raising this on the radar!

xl-network’s picture

Anything yet...? I guess it should be easy to fix. I am a webdeveloper, just not in PHP / Drupal.

xl-network’s picture

I guess I'm talking into space... this looks like a serious bug... Google is indexing my 404 pages, but wait... that's not right...

Robrecht Jacques’s picture

This is not easily fixable. At least I don't find a solution.

As a work-around: set the default 404 and 403 pages to a node and set the ROBOTS meta tags on that node to not index or follow.

xl-network’s picture

Ahh, but I'm using customerror (so the 403/404 pages won't show up in search (for instance)

BioALIEN’s picture

Robert, I think maybe you can look at existing implementations in other CMS systems and see how they've overcome this?

Two noticable ones I've come across are MODx and iJoomla's methods but I am not sure how they handle 404/403 errors:
http://modxcms.com/assets/images/metaContentCreate.png
http://www.ijoomla.com/MetaTag_Generator/iJoomla_Metatag_Generator/iJoom...

Robrecht Jacques’s picture

The (technical) problem is that at the point where Meta tags inserts the metatags (hook_menu() - which is the only place I know of to reliable insert them), I can't yet know if the page will result in a 403/404 yet. So the Meta tags (the <head>-section) are already generated and only later in the process Drupal will notice that the page is inaccessible. At that point, there is no way to "cancel" the already inserted Meta tags (afaik).

xl-network: I'll check whether customerror module adds something I could use. I'll get back to you asap.

BioALIEN: the design of Drupal is significantly different from Joomla etc. So it won't help me looking at their code. I've spent quite some time looking at some other possible insertion points for the Meta tags in the callback stack of Drupal, but hook_menu() is unfortunately the only place to do it (unless I instruct the user to change their themes, which means it won't work out-of-the-box).

I do appreciate the feedback on how Joomla etc handles setting of Meta tags though - I will need to check that a bit closer for resolving the "custom user-defined meta tags" feature requests.

Robrecht Jacques’s picture

Version: 5.x-1.0 »
Status: Active » Postponed
smitty’s picture

Status: Postponed » Fixed

To get rid of this problem I put the function phptemplate_modify_header into my template.php:

function phptemplate_modify_header($head) {
  if (arg(0) == 'customerror') {
    if (module_exists(customerror) && module_exists(nodewords)) {
      $head = str_replace('follow', 'nofollow', $head);
      $head = str_replace('index', 'noindex', $head);
    }
  }
  return $head;  
}

and changed in my page.tpl.php the line

<?php print $head ?>

to

<?php print phptemplate_modify_header($head) ?>

I know: that's not very smart. But it works for me.

Please feel free to post a better solution.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.