See here: http://www.ysearchblog.com/archives/000437.html

This seems to indicate that all major search engines will now obey the "Sitemap:" tag in a robots.txt file. Is this something that XML Sitemap can manage?

Thanks
Adam

CommentFileSizeAuthor
#7 gsitemap.module-149134.patch10.56 KBdarren oh
#3 robotstxt_hook_0.patch699 byteshass
#2 hook_robotstxt_0.patch822 byteshass

Comments

hass’s picture

hass commented
Status: Active » Postponed

There are plans to add a hook_robotstxt for modules like xmlsitemap to add this line automatically... today you should add this by hand to your current robots.txt file.

hass’s picture

hass commented
Status: Postponed » Needs review
StatusFileSize
new hook_robotstxt_0.patch822 bytes

Here is a patch for this feature in reference to http://drupal.org/node/138434

hass’s picture

hass commented
StatusFileSize
new robotstxt_hook_0.patch699 bytes

Wrong patch... take *this* one :-(

hass’s picture

hass commented

This patch can go in. Robotstxt module have this hook_robotstxt implemented.

darren oh’s picture

darren oh
Primary language English
Location Lakeland, Florida
commented
Status: Needs review » Fixed

Fixed in CVS commit 83645.

SubZero5’s picture

SubZero5 commented

Sitemap and i18n makes result as site.com/en/sitemap.xml :(

darren oh’s picture

darren oh
Primary language English
Location Lakeland, Florida
commented
StatusFileSize
new gsitemap.module-149134.patch10.56 KB

Try this patch.

darren oh’s picture

darren oh
Primary language English
Location Lakeland, Florida
commented

Fixed in CVS commit 84193.

SubZero5’s picture

SubZero5 commented

Sorry for the delay. I have been kind of busy. :(

can it be better if you just say:

 function xmlsitemap_robotstxt() {
  return "Sitemap: /sitemap.xml";
}

and with the patch, the sitemap prints correctly with /sitemap.xml BUT the sitemap0.xml still does not have the i18n paths like /en/, /fr/, /it/, /de/, /tr/ etc..

darren oh’s picture

darren oh
Primary language English
Location Lakeland, Florida
commented

Unfortunately, we cannot be sure that everyone has clear URLs enabled.

It will take a significant amount of work to enable XML Sitemap to include links in more than one language. Previously it used whatever language prefix was in use when the site map was generated. For consistency and compatibility with Google, it currently does not use language prefixes. Further development is being discussed in issue 182442.

Anonymous’s picture

(not verified) commented
Status: Fixed » Closed (fixed)