There are a lot of search engine bots on my site crawling on biblio links: authors, bibtex, and ... . How can I remove these links for unauthenticated users?

Comments

rjerome’s picture

rjerome commented

The export links have a permission associated with them however the author links do not.

On the up side, all the cross linking really drives your site up the top of a Google search :-)

Perhaps some well crafted "robots.txt" file would help. If you add any path that looks line "biblio/author/" and "biblio/export/" and "biblio/keyword/" that would eliminate a lot of the traffic. See http://www.robotstxt.org/robotstxt.html

Ron.

liam morland’s picture

liam morland
Primary language English
Location Ontario, CA 🇨🇦
commented
Issue summary: View changes
Status: Active » Closed (outdated)

This version is no longer maintained. If this issue is still relevant to the Drupal 7 version, please re-open and provide details.