Community Documentation

Prevent Duplicate Indexing from Search Engines When Using URL Aliasing

Last updated February 14, 2009. Created by silverwing on January 18, 2006.
Edited by ezraw, varunvnair. Log in to edit this page.

Drupal can create friendly, easily understood URLS by using the Path and Pathauto modules.

By default, Drupal automatically creates web address like www.example.com/?q=node/67, which many people feel are not user-friendly.

Path and Pathauto allow for human-friendly URLs, for example, www.example.com/?q=all-about-tarantulas.

And with Drupal's "clean URLs" feature you can get rid of the "?q=": www.example.com/all-about-tarantulas.

However, if you are using Path and/or Pathauto, then you may want to add Disallow: /node/ to your robots.txt.

This is because nodes with aliases are accessible from 2 URLs: the default URL and the aliased URL. Search engines will discover both over a period of time, and some users have reported a penalization for duplicate content. However, Google states that there is no penalty.

It may also result in twice as much crawling as is necessary, which may effect your site's performance and bandwidth use.

There is more information on users' experience with duplicate content and URL aliasing and search engine bandwith issues.

Google has also described how webmasters can set a canonical URL for each page.

Site Building Guide

Drupal’s online documentation is © 2000-2012 by the individual contributors and can be used in accordance with the Creative Commons License, Attribution-ShareAlike 2.0. PHP code is distributed under the GNU General Public License.