Hi,

I have a question on Google indexing pages. I have noticed that google has indexed the user/register page and password page as example below:

mysite.com/user/
mysite.com/user/password?page=1
mysite.com/user/register

Is this normal or is there anything to prevent it?
Please advise in as detailed as possible so I can understand what to do to prevent this if possible.

Thanks a lot :)

Comments

Helen-kh’s picture

I just wanted to add that I checked the robots.txt and I found this scripts:

# Paths (clean URLs)
Disallow: /admin/
Disallow: /comment/reply/
Disallow: /filter/tips/
Disallow: /node/add/
Disallow: /search/
Disallow: /user/register/
Disallow: /user/password/
Disallow: /user/login/
Disallow: /user/logout/
# Paths (no clean URLs)
Disallow: /?q=admin/
Disallow: /?q=comment/reply/
Disallow: /?q=filter/tips/
Disallow: /?q=node/add/
Disallow: /?q=search/
Disallow: /?q=user/password/
Disallow: /?q=user/register/
Disallow: /?q=user/login/
Disallow: /?q=user/logout/

Does this mean that these pages are disallowed to show on search engines? And if so, why it is not effective in my case?
(The robots.txt is located on the same directory as other pages and folders of drupal)

sheckay’s picture

I'm having the same problem. I don't have any need for that login page to be accessed by anyone but me. But Google keeps indexing it. My robots.txt disallows it. Don't understand why.

PeteDavidson’s picture

The robots.txt is the best method, you could also use rel="noindex" on your login (& register if you have one) link(s).

tzed’s picture

It seems the problem is caused by the trailing slashes in Drupal's robots.txt. Now I'm playing with my robots.txt in webmaster robots testing tool and when I add the URLs like this:

Disallow: /user/register
Disallow: /user/password
Disallow: /user/login
Disallow: /user/logout

It seems to work. Now I'm waiting for the real googlebot to come, I hope it will help.