I don't were '&from=' is appearing in Drupal, but googlebot has caught quite some from my Drupal 5.10 site:

domain/?page=16&from=1289
domain/?page=16&from=1357

The above shows the exact same page but is being listed as duplicate content by Google Webmaster Tools.

Until a fix, you can block
Disallow: *from=
in your robots.txt to stop indexing of these urls.

Comments

FlemmingLeer’s picture

Title: &from= produces multiple content in google » ?page= produces fictional pages in Drupal

Similar to the above at present these are generated by Drupal:
this
domain/node?page=929
doesn't exist.

Currently
domain/?page=568
is the last page.

At the time being Googlebot has indexed these non-existent pages:
domain/node?page=550
domain/node?page=551
domain/node?page=552
domain/node?page=553
domain/node?page=554
domain/node?page=555
domain/node?page=556
domain/node?page=557
domain/node?page=560
domain/node?page=563
domain/node?page=564
domain/node?page=565
domain/node?page=751
domain/node?page=759
domain/node?page=787&%24Version=0&%24Path=/&%24Domain=.domainname.xx
domain/node?page=817
domain/node?page=819
domain/node?page=832
domain/node?page=861
domain/node?page=862
domain/node?page=872
domain/node?page=905
domain/node?page=920
domain/node?page=921
domain/node?page=923
domain/node?page=925
domain/node?page=926
domain/node?page=927
domain/node?page=929

Please fix.

To avoid indexing block
Disallow: *?page
in robots.txt

newbii’s picture

How to make these pages as clean URL's ?
i.e. : domain/page/22

Rhino’s picture

I have found similar problems, like domain/nodename?page=5 - as I noted here: http://drupal.org/node/475256

Rhino’s picture

One can remove the ? from robots.txt - but this doesn't prevent situations where one gets

domain/HEADLINEOFPOST?page=5
&
domain/NAMEOFTOPIC?page=7890009

The top one being the issue. It's possible that the topic pages may have 7890009 pages returned as result (or at least had in one point in time) but a single post itself never ever ever has more pages. So why does the "?page=" thing even go there? How can i remove the "?page=" part (301 redirect so that searchengines quit indexing them)? How it ended up in searchengines such as google - which has collected many many "?page=1 through ?page=16" on the same exact post/node - is a mystery to me. Nothing points to these URL's, is googlebot guessing?

Disallowing "?" or "?page" from robots.txt may prevent search engines from linking content further down in the site, which is a little bad (though sitemap may help them find the individual pages anyway, which is probably better for search results rather than a bunch of "topicname?page=7" results whose content changes as you post more)

I'm using the following modules: Pathauto and Global Redirect as well as clean URL's. I have yet to figure out which modules is causing this behaviour, any clues welcome.

Rhino’s picture

Nobody has any idea why this is happening?

dman’s picture

yes, this is HTTP.
Not sure where the links are COMING from though.
And if you worry about Google-juice because of amateur SEO speculation, A suggested solution could be 301 redirect or cannonical url that are intended to flatten these multiple hits.

FlemmingLeer’s picture

Issue tags: +SEO, +Drupal 5.x, +Drupal 6.x, +paging, +indexing, +robots

Hi,

The nodewords module now has added noindex, follow to ?page= so that these pages are no longer indexed.

The bots now reads the paging but do not index them.

Described here:
http://drupal.org/node/294996#comment-1241947

Get nodewords module here:
http://drupal.org/project/nodewords

There are both drupal 5.x and drupal 6.x versions.

__Sander__’s picture

Priority: Minor » Normal

Let's raise the priority of this issue a little bit.
I will explain, why this is important.

For example we have a node with 30 comments and 20 comments per page setting.
let node be our first page
node?page=1 is our second comments page, and we want it to be indexed, therefore we can't block it in robots.txt
node?page=2 ... what's that? actually, a copy of node?page=1
where did this link come from? I do not know.
But the search engines keep finding the pages with ?page=3 and so on
And as I said earlier, I do not want to use robots.txt page blocking because that will block existing comment pages.

In fact, I've found how i happens that Drupal returns those pages as existing. Please have a look at this code from in;cudes/pager.inc in function pager_query

$pager_page_array[$element] = max(0, min((int)$pager_page_array[$element], ((int)$pager_total[$element]) - 1));

Here $pager_page_array[$element] is the page being requested, $pager_total[$element] is the number of pages we have.
As a result, requesting for pages with greater numbers results in the last page as well.

My suggestion is to modify the Drupal core and to insert before this line:
if ($pager_page_array[$element]>=((int)$pager_total[$element])) drupal_not_found();

What do you think?

__Sander__’s picture

Version: 5.10 » 6.20

Status: Active » Closed (outdated)

Automatically closed because Drupal 6 is no longer supported. If the issue verifiably applies to later versions, please reopen with details and update the version.