Posted by Peter Apokotos on July 28, 2005 at 7:41pm
Jump to:
| Project: | Troll |
| Version: | master |
| Component: | Miscellaneous |
| Category: | feature request |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | closed (won't fix) |
Issue Summary
block user agents
| Project: | Troll |
| Version: | master |
| Component: | Miscellaneous |
| Category: | feature request |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | closed (won't fix) |
block user agents
Comments
#1
Not really interested in this. Apache is way better for doing a sweeping block by user agent. If spiders are what you are after spiders should be blocked with robots.txt. If not, its very easy to spoof user agent anyway, so someone who wants to play with you will still be able to.
#2
Only is the spider respects the robots.txt file. And I know this can also be done with htaccess file.
But with a large enough list and on a shared server. It can really slow things down.
Peter Apockotos
http://www.apockotos.com
http://www.knightrider.org
http://www.macmariner.com
#3
#4
#5
...and I'm locking the thread.