Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
I'm fairly certain that Google will reindex the same page, trying every single value from a drop down list if you have one as an exposed filter. I'm suggesting that
# URL Variables
disallow: /*list
disallow: /*of
disallow: /*drop
disallow: /*down
disallow: /*url
disallow: /*variables
be added to robots.txt, or at least a warning somewhere so web admins know how to handle the issue. Spiders might try other values in other url variables as well? Guessing on the robots.txt syntax, it's correct right?
Similar yet different issue
#280281: Views pager duplicates content
Comments
Comment #1
mikeytown2 CreditAttribution: mikeytown2 commentedor would you do something like this
Or this
using clean urls.
The option of having views insert a "noindex" meta tag might be the answer.
EDIT:
http://www.google.com/support/webmasters/bin/answer.py?answer=76329&hl=en
Comment #2
mikeytown2 CreditAttribution: mikeytown2 commentedActually now that I think about it, I'm disabling anything that makes a url variable until we get robots.txt figured out. Duplicate content is a killer.
?page=
is fine, everything else isn't.Comment #3
mikeytown2 CreditAttribution: mikeytown2 commentedhttps://www.google.com/webmasters/tools/dashboard
Once I get some feedback on this, I'll add it to the handbook page
This is what I came up with
Comment #4
mikeytown2 CreditAttribution: mikeytown2 commentedHere's the handbook page...
SEO (Search Engine Optimization) Guidelines When Using Views
Comment #5
halisemre CreditAttribution: halisemre commentedI have a question?
If i use
Disallow: /*?
Allow: /*?page=
Disallow: /*?page=*&*
http://www.mysite.com/?page=1 is ok
http://www.mysite.com/?page=2 is ok
but what about
http://www.mysite.com/?page=0
it is the same as http://www.mysite.com/ so it is kind of you are duplicating the frontpage.
Is there a way to eliminate this problem
Comment #6
mikeytown2 CreditAttribution: mikeytown2 commentedI updated the handbook page
http://drupal.org/node/345620
Another way is to not link to page=0; or 301 it.
Comment #7
merlinofchaos CreditAttribution: merlinofchaos commentedHow is this a bug? Views has no control over the robots.txt. Looks like you guys have it sorted, anyhow.