Hi,

I just disovered that you can define adsense settings in robots.txt, so adsense can be displayed despite Googlebot robots.txt settings.

This will allow adsense bot on entire site
User-agent: Mediapartners-Google
Disallow:
Allow: /*

But webmaster tools don't recognise robots.txt with:
User-agent: Mediapartners-Google*
Disallow:
Allow: /*

found here:
http://www.askapache.com/seo/seo-with-robotstxt.html

Comments

jcnventura’s picture

jcnventura CreditAttribution: jcnventura commented
Status: Active » Closed (fixed)

Nice to know. However this has nothing to do with the adsense module.

FlemmingLeer’s picture

FlemmingLeer CreditAttribution: FlemmingLeer commented

Oh yes,

You are absolutely right :)

I added it to Drupal 6.13 issue here:
http://drupal.org/node/573008

SGhosh’s picture

SGhosh CreditAttribution: SGhosh commented

Thanks a lot. This is exactly what I was looking for.

I had google adsense ads implemented on the search page, but since /search/ is blocked for all bots under robots.txt ads were coming up inconsistently. However, adding this seems to have solved the problem.

Thumbs up for community help! :)

freelylw’s picture

freelylw CreditAttribution: freelylw commented

Hello

I am using google adsense ad on my site, I just paste the code into the block but not using the adsense module.
From the research on google I have found , I need to paste the code:

"User-agent: Mediapartners-Google
Disallow:
Allow: /*"

in the robot.txt to make the adsense works.

The question is : the code below in robot.txt already: should I paste the "User-agent: Mediapartners-Google..." above them or should I remove the code which already in the robot.txt first,and then paste the new code ? Thank you very much!

User-agent: *
Crawl-delay: 10
# Directories
Disallow: /includes/
Disallow: /misc/
Disallow: /modules/
Disallow: /profiles/
Disallow: /scripts/
Disallow: /themes/
# Files
Disallow: /CHANGELOG.txt
Disallow: /cron.php
Disallow: /INSTALL.mysql.txt
Disallow: /INSTALL.pgsql.txt
Disallow: /install.php
Disallow: /INSTALL.txt
Disallow: /LICENSE.txt
Disallow: /MAINTAINERS.txt
Disallow: /update.php
Disallow: /UPGRADE.txt
Disallow: /xmlrpc.php
# Paths (clean URLs)
Disallow: /admin/
Disallow: /comment/reply/
Disallow: /filter/tips/
Disallow: /logout/
Disallow: /node/add/
Disallow: /search/
Disallow: /user/register/
Disallow: /user/password/
Disallow: /user/login/
# Paths (no clean URLs)
Disallow: /?q=admin/
Disallow: /?q=comment/reply/
Disallow: /?q=filter/tips/
Disallow: /?q=logout/
Disallow: /?q=node/add/
Disallow: /?q=search/
Disallow: /?q=user/password/
Disallow: /?q=user/register/
Disallow: /?q=user/login/