By modul on

Hi all,

I'm restructuring my site. Google, however, has already found the old site and has already added a couple of thousand links to its database. These links won't be used any more, however, and there's no point redirecting them. The simplest way would be to eliminate all traces of my site from Google. I know robots.txt with

User-agent: Googlebot
Disallow: /

will keep Google out, but what about the old links? Is there a handy way to not only keep Google away from my site, but to also remove my site from Google? I found things on removing individual URLs in the Google Webmaster Tools, but I couldn't figure out how to get rid of the whole site. Does anyone have a clue on how to remove all traces of my site from Google?

⋅ Categories: Drupal 5.x

Comments

Lityi’s picture

Lityi commented

subscribing

modul’s picture

modul commented

Oops... Was this a tough one? There are so many tricks and advices to get a site IN Google that I would have imagined that it shouldn't be too difficult to get a site OUT of Google? Any hints on how to remove all traces of my site from Google? Thanks!

themphill’s picture

themphill commented

If the pages which Google has links to all turn into "404 Not Found" errors, it shouldn't take long for Google to remove them. You should confirm that your server is properly sending "404 Not Found" HTTP headers. There are various tools which allow you to view the server response headers, one is the Web Developer plug-in to Firefox.

modul’s picture

modul commented

Thanks for your information, Themphill. Two questions:

1) When my site is http://www.mysite.com, and when I want to make sure that Google does Not include http://www.mysite.com/blog/etc. ("blog" is not a folder, mind you), what would be the .htaccess code to make sure that all http://www.mysite.com/blog/*** urls would return a 404 header?

2) If I make all urls of my site return a 404 Not Found header, would I myself still be able to access my site?? If that is the case, the 404 method would not be the proper way to choose.

Tiburón’s picture

Tiburón commented

Hi,

I am writing this in general terms:

Register your site with the Google Webmaster Tools.

Go to the overview of the domain in question and open the "Tools" menu in the left menu. The (currently) second last submenu is "Remove URLs". Click it and follow the instructions given on how to make a "New Removal Request".

Currently the wizard lets you remove:

  • Individual URLs: web pages, images, or other files
  • A directory and all subdirectories on your site
  • Your entire site
  • Cached copy of a Google search result

@modul: Guess it is the third option you will need to use. For obvious reasons I have not followed the wizard further ;-).

The removal is not instantaneous but happens faster than only using the robots.txt to inform the search engine since the file is cached between visists.

NB: If you want to omit content from Google and friends but still have it online on you site then remember to block it using robots.txt and maybe even meta elements on the pages them selves. Since robots.txt is cached meta elements will work faster in some situations. Google have a periode of 90 days from removal to reinclusion unless the content is blocked by other means like robots.txt or meta elements.

Hope this helps...

Regards,

Christian Larsen

modul’s picture

modul commented

Thanks guys, that should set me on the way. Now let's hope Google is willing to cooperate :-)

userofdrupal’s picture

userofdrupal commented

Personally I wouldn't mess with any option that involved removing my ENTIRE site from google. I agree with the earlier poster who said just serve up the 404's. It won't take long for google to drop your old content and find your new content.

modul’s picture

modul commented

Why would it be "messy" to remove my entire site from Google?? It's an option Google itself is offering, and I think it even looks "cleaner" than working with 404's. So, why messy?

userofdrupal’s picture

userofdrupal commented

Sorry. I'm not sure if you're English is your native language. My usage of the word "mess" is not really related to the word "messy". It's an English idiom. It basically means don't change something that is potentially fragile or dangerous to change. What I meant was:

1. Google pagerank is important to many people (I think you are in this category)
2. Telling google you want to remove your entire site from its index is obviously harmful to your pagerank (at least in the short term)
3. Therefore, I would not do something that involved removing your entire site from google.

Just my two cents.

modul’s picture

modul commented

Thanks, Overflowing. Pagerank is indeed important to me, but not in the short run (say, a couple of months). So, I don't really mind if Google would 'punish' me for wanting to erase my site from its database.

voipfc’s picture

voipfc commented

subscribing

===============================
Quintessentially | Supremely | American
Krishna is the God star

byanfu’s picture

byanfu commented

Sorry didn't read all the post so apologize if someone already answered. Here is a link on Google to remove sites.

http://www.google.com/support/webmasters/bin/answer.py?answer=35301