Active
Project:
SEO Watcher
Version:
7.x-1.0-beta1
Component:
Code
Priority:
Normal
Category:
Feature request
Assigned:
Unassigned
Reporter:
Created:
1 Mar 2011 at 14:10 UTC
Updated:
1 Mar 2011 at 14:30 UTC
Hi,
I really like the idea of this module (just trying it out now). I read your warning on the project homepage about getting an IP address banned from Google. It seems to me that even with 10 search queries and a depth of 100, that still equates to only 100 page views (assuming we don't increase the number of results per page, which would make it fewer page views). That doesn't seem like a lot to me for a 24 hour period - do you have any references for getting banned? I'd be happy to supply patches to stagger the requests via cron, or even just to put a small pause between requests.
Does that sound dangerous to you, or is that acceptable?!
Thanks for your work on the module!
A.
Comments
Comment #1
andyf commentedOr maybe it would be better to rework the google crawler to use the custom search API?