By Bob4613 on

I would like to know if it is a good idea to allow comment/reply paths to be crawled by search engines. Currently, my robot.txt disallows this and it seems you would want the search engines to crawl this so they can index what has been added. Am I missing something here?

⋅ Categories: Drupal 6.x

Comments

vm’s picture

vm commented

comment/reply is the comment form as such there is no need for spiders to crawl this path.

comments after they have been sent by the form are picked up by spiders through the node path

_____________________________________________________________________
My posts & comments are usually dripping with sarcasm.
If you ask nicely I'll give you a towel : )

Bob4613’s picture

Bob4613 commented

Ok, I'm still new to all of this, so your info helps a lot.

Thanks for the quick reply,

couturier’s picture

couturier commented

Thanks for this tip. Google Webmaster Tools shows so many pages with comment/reply/100 or another number at the end. After reading this post, I discovered that each node on my site that allows comments also has its own comment form, so that is why there are so many. You can count on Drupal core to do the right thing, so its good to know that its just the comment form itself and not the comments that are being restricted. Thanks again.