Google News can't index Drupal sites!

Posted by Tom Corgan@www.... on April 12, 2005 at 8:23pm

Hi, I tried to submit my site to Google News and recieved this comment back from them:

Thank you for your inquiry regarding inclusion in Google News. After some
investigation, we've found that our system cannot crawl your articles
because of the format of their URLs. In order to have your articles
crawled by Google News, their URLs must contain a number consisting of at
least three digits.

For example, our news crawler would not crawl articles with the following
URLs:
www.google.com/news/article23.html
www.google.com/lemurs_in_the_mist.html

It would crawl these pages:
www.google.com/news/08112003/article.html
www.google.com/news/lemurs_in_the_mist/23467.html

An example of a site that we are able to crawl successfully is
http://english.chosun.com. Please note that each article on this site has
a highly unique URL.

Please note also that, at this time, we don't accept RSS feeds for
inclusion in Google News.

We apologize for the limitations of our system. If you are able to make
changes on your end to allow us to crawl your content, please let us know.

Regards,
The Google Team

Anyone have any idea how I can get my site to do this? One thing I thought of was maybe aliasing each page...but that would just be unmanageable in the long run.

Regards,

Tom
http://www.friendsofaljazeera.org

Comments

i dont get this?

Posted by bertboerland on April 12, 2005 at 8:51pm

why would google exclude urls based on the content of the urkl? are they trying to ban blogs? is there some logic behind it? seems really weird and ungoogle-ish.

stupid trick, put your starting nid to 100? willt that work?
--
groets
bertb

--
groets
bert boerland

I got this email also. I

Posted by ddegner on March 15, 2006 at 11:30pm

I got this email also. I have set up my site already and have a few stories up. How do I set my starting nid?
-----------------------------------------------------
www.fulleffectmagazine.com
Jump through a few hoops, get a free iPod.
http://ipods.freepay.com/?r=8480908

-----------------------------------------------------
www.fulleffectmagazine.com
Jump through a few hoops, get a free iPod.
http://ipods.freepay.com/?r=8480908

Regular Google does index Drupal sites.

Posted by brainshrub on April 12, 2005 at 8:52pm

I've looked at your site, and I don't see the problem since all your news stories seem to have three digits anyways.

While I am relativly new to Drupal, I do know quite a bit about how Google decides what to index. The reply that they gave you does not sound like them: It is not logical that Google News would not index URLs without three digits.

Is it possible that Google is giving you a hard time b/c the name of your URL is friendsofaljazeera.org? (Not to malign Google News... but it is a possiblity.)

In any case, I think if you just added a three-digit code you would be fine.

Dreams are liquid reality.

There is a more complete discussion...

Posted by ddegner on March 29, 2006 at 6:13am

There is a more complete discussion of this issue on the civic-space website.

http://civicspacelabs.org/home/node/12210

-----------------------------------------------------
www.fulleffectmagazine.com
Jump through a few hoops, get a free iPod.
http://ipods.freepay.com/?r=8480908