Index: robots.txt
===================================================================
RCS file: /cvs/drupal/drupal/robots.txt,v
retrieving revision 1.9
diff -r1.9 robots.txt
21c21,23
< Crawl-delay: 10
---
> # Uncomment the following line if your server is getting hit too hard by robots
> # Crawl-delay: 10
> 
45a48,49
> # If you want your aggregator pages to be indexed, comment out the following line
> Disallow: /aggregator
47,48c51,53
< Disallow: /contact/
< Disallow: /logout/
---
> Disallow: /contact$
> Disallow: /logout$
> Disallow: /node$
49a55
> Disallow: /search$
51,53c57,60
< Disallow: /user/register/
< Disallow: /user/password/
< Disallow: /user/login/
---
> Disallow: /user$
> Disallow: /user/register
> Disallow: /user/password
> Disallow: /user/login
55a63,64
> # If you want your aggregator pages to be indexed, comment out the following line
> Disallow: /?q=aggregator
57,58c66,68
< Disallow: /?q=contact/
< Disallow: /?q=logout/
---
> Disallow: /?q=contact
> Disallow: /?q=logout
> Disallow: /?q=node$
59a70
> Disallow: /?q=search$
61,63c72,81
< Disallow: /?q=user/password/
< Disallow: /?q=user/register/
< Disallow: /?q=user/login/
---
> Disallow: /?q=user$
> Disallow: /?q=user/password
> Disallow: /?q=user/register
> Disallow: /?q=user/login
> # Miscellaneous
> Disallow: /*sort=
> Disallow: /*/feed$
> Disallow: /*/track$
> Disallow: /tracker?
> Disallow: /tracker/