As per the title, links to my user profiles are included in the sitemap which is causing crawl errors with Google.
I haven't added the paths as custom links or anything.

Comments

Anonymous’s picture

Anonymous (not verified) commented
Category: bug » support
Status: Active » Postponed (maintainer needs more info)

What modules do you have enabled?

hixster’s picture

hixster commented

sitemap specific or in general??

of the XML sitemap ones, these are enabled:

XML sitemap custom 6.x-2.x-dev Adds user configurable links to the sitemap.
Depends on: XML sitemap (enabled)

XML sitemap engines 6.x-2.x-dev Submit the sitemap to search engines.
Depends on: XML sitemap (enabled)

XML sitemap menu 6.x-2.x-dev Adds menu item links to the sitemap.
Depends on: XML sitemap (enabled), Menu (enabled)

XML sitemap modal UI 6.x-2.x-dev Provides an AJAX modal UI for common XML sitemap tasks.
Depends on: XML sitemap (enabled), Chaos tools (enabled)

XML sitemap node 6.x-2.x-dev Adds content links to the sitemap.
Depends on: XML sitemap (enabled)

XML sitemap taxonomy

Anonymous’s picture

Anonymous (not verified) commented

In general, what other modules do you have active? My guess for this is that one of the modules is creating nodes out of user data. If that is the case you can go to the content type settings for that data and exclude them from the sitemap.xml file.

hixster’s picture

hixster commented

Hi Earnie, thanks for following up- i'm not do anything fancy with the user accounts on the site - its a fairly vanilla Drupal set up - I don't have content profile or anything else like that installed?

Anonymous’s picture

Anonymous (not verified) commented

Ok, so what permissions do you have set for the profile module? Xmlsitemap only considers links given it from the standard Drupal API where the user is the anonymous user. If user profiles are getting included it would be because somehow the standard API or someone added it via the custom links module.

I don't use the AJax module or the Menu module, perhaps one of them is the issue. Do you need to use them?

hixster’s picture

hixster commented

That's the thing - both the user paths are 'access denied' when visiting them - there's no way they should be in the XML sitemap.
as per #1 I don't have custom links enabled.

Anonymous’s picture

Anonymous (not verified) commented

Repeating myself: I don't use the AJax module or the Menu module, perhaps one of them is the issue. Do you need to use them?

hixster’s picture

hixster commented

Hi Earnie, thanks for the continuing support on this.
I'm not using the Ajax module - With reference to the menu module, are you referring to the XML sitemap menu module?
it's not really a practical proposition to not use the core menu module to get the sitemap to work, if that's what you mean?
Many thanks

Anonymous’s picture

Anonymous (not verified) commented

XML sitemap modal UI 6.x-2.x-dev Provides an AJAX modal UI for common
XML sitemap tasks.
Depends on: XML sitemap (enabled), Chaos tools (enabled)

XML sitemap menu 6.x-2.x-dev Adds menu item links to the sitemap.
Depends on: XML sitemap (enabled), Menu (enabled)

I'm talking about these two; in #2 you stated that these were enabled. Why? Do you need them?

hixster’s picture

hixster commented

Sorry Ernie - my bad - I see what you mean. Thanks for your continuing patience.

I have disabled both of those modules and rebuild the sitemap - I still have the same problem.

Anonymous’s picture

Anonymous (not verified) commented

Sent you private email.

dave reid’s picture

dave reid
he/him
Primary language English
Location Nebraska USA
commented

@hixster: Hrm, are you using a module like Content profile for your user profiles? Is it really user pages that are showing up in your xmlsitemap? If this is a live site, you can e-mail me via my profile form as I'd like to be able to look at the raw sitemap XML on your site.

dave reid’s picture

dave reid
he/him
Primary language English
Location Nebraska USA
commented

Hrm, you said you don't have content profile installed, nevermind. You also might want to look in your {xmlsitemap} table on your site - look at where those records are coming from (what are the type and IDs). Could try deleting any records you don't want to show up because they couldn't get deleted before for some reason. If you do a rebuild after deleting the records you don't want and they come back, that has to mean something is responsible for creating those link records.

hixster’s picture

hixster commented

O.k. quick update on this. I checked the XML sitemap table in the DB and there were 2 user entries which I deleted.
I flushed the caches on my site and the users were still in the sitemap.
I then rebuilt the sitemap from the admin page and voila! the links have now gone.
So looks like some data was there that might have been included when I had the XML sitemap user module enabled which didn't get cleaned out.

Anonymous’s picture

Anonymous (not verified) commented
Component: xmlsitemap.module » xmlsitemap_user.module
Category: support » bug
Status: Postponed (maintainer needs more info) » Active

Based on Hixster's last update I'm switching this to a bug report for the xmlsitemap_user module. I don't use it so I've never tested it. Dave do you have a clue as to why this module would include paths that the anonymous user cannot see?