Add content="noindex" meta element on Register and Login pages

Yes, I just was trying to fix this in D7 and there are similar links in comment module that cannot be altered except by hacking core.

Here's a quick 1st pass at a patch.

Instead or in addition we could add to the HEAD on all these pages:

<meta name="robots" content="noindex">

Note that the way the code is built in comment module is stupidly annoying since this attribute change would come through as a change to a translatable string, making this much less likely to be backportable.

If we're going to patch core, why not add these paths to robots.txt directly?

You're right @eule, looks like robots.txt doesn't prevent pages from appearing in the search engines index, I wasn't aware of this subtlety. From Google webmaster documentation:

To entirely prevent a page's contents from being listed in the Google web index even if other sites link to it, use a noindex meta tag or x-robots-tag. As long as Googlebot fetches the page, it will see the noindex meta tag and prevent that page from showing up in the web index.

So it looks like what we need is an explicit:

<meta name="robots" content="noindex">

in the HTML, as I don't think that adding a nofollow will be sufficient either. This is particularly true if there are other links pointing to these pages, including external sites.

updating title. Note that this is what wordpress does for example: http://wordpress.com/wp-login.php

Either (1) the user module could implement HOOK_preprocess_html() and execute drupal_add_html_head() if the path is '/user/login' or '/user/register'. OR (2) you could add a drupal_add_html_head() inside the respective form builders '\Drupal\user\Form\UserLoginForm' and 'user.register'. I'm not sure which one is kosher in D8.

Some sites might want to have their user/register form indexed I think.

Since this breaks strings, it's not eligible for 8.0.x at this point. However, a site-specific workaround in that case might be to provide a custom translation of the string that adds the nofollow.

@eule, your comment is not helpful. You can help this issue make progress by testing the actual patch provided and confirming whether that works. See https://www.drupal.org/patch/apply for help on how to apply patches.

It sounds like we should also remove the user login block change from this patch according to @scor's review. https://www.drupal.org/node/707484 explains how you can create a new patch, and https://www.drupal.org/node/1488712 explains how you can create an interdiff to show a change from a previous patch. You can also come to the Drupal core mentoring hours to get help with contributing to a core issue. See https://www.drupal.org/core-mentoring for more information.

Finally, @eule, since we may not make the user login block change in core, I think a good way to provide a fix for the user login block would be to create a contributed or custom module that provided an alternate user login block with the nofollow option. Then sites that wanted the nofollow could place that block instead.

Thanks!

I accomplished it this way by using a preprocess_page hook in template.php. It checks requested URL to see if it has /user/ at the beginning.

function MY_THEME_preprocess_page(&$variables) {

  $robots_noindex = array (
    '#type' => 'html_tag',
    '#tag' => 'meta',
    '#attributes' => array(
      'name' =>  'robots',
      'content' => 'noindex',
    ),
  );

$url_parts = explode('/', request_uri());
  
  if($url_parts[1] == 'user') {
    drupal_add_html_head($robots_noindex, 'no_index_rsvp');

  }

}

Usefull piece of code @AndraeRay
You can modify it this way to handle sites in subfolders as well :

$url_parts = explode ( '/', request_path () );
	
if ($url_parts [0] == 'user') {
	drupal_add_html_head ( array (
			'#type' => 'html_tag',
			'#tag' => 'meta',
			'#attributes' => array (
					'name' => 'robots',
					'content' => 'noindex' 
			) 
	), 'no_index_rsvp' );

You can use this piece of code in html.html.twig

{% set url = url('') %}
{% if 'user' in url|render|render %}
meta name="robots" content="noindex" (add <> since those are hidden by drupal.org editor)
{% endif %}

What I suggest is to install drupal/rename_admin_paths module and change /user/* url. After that configure this code to check for new keyword in url string.
Be aware that in order for this to work, your url should not be in robots.txt: https://developers.google.com/search/reference/robots_meta_tag

None of the code suggestions in the thread above worked for me. I solved it using metatag_routes.

In D8 you can also do it through a hook_page_attachments_alter and use path.matcher service to add to all /user pages like this:

/**
 * Implements hook_page_attachments_alter.
 */
function theme_page_attachments_alter(array &$attachments) {

  $current_path = \Drupal::service('path.current')->getPath();
  $patterns = '/user/*';
  $match = \Drupal::service('path.matcher')->matchPath($current_path, $patterns);

  $noindex = [
     '#tag' => 'meta',
     '#attributes' => [
        'name' => 'robots',
        'content' => 'noindex',
      ],
  ];

  if($match) {
    $attachments['#attached']['html_head'][] = [$noindex, 'description'];
  }
}

https://www.drupal.org/project/metatag_routes mentioned in #27 by @caspervoogt works as expected, and I do not think we should bring this into the core. So let's close this. Thanks all for your efforts!