Using this module I've found an issue with finding next page url (using the XPath query that points to the 'Next' button). Speaking shortly it works if site is using path offset to point next site, i.e.:
example.com/path/1
example.com/path/2
example.com/path/3

but it does not work with sites that are using query arguments for paging:
example.com/some/path?pager=1
example.com/some/path?pager=2
example.com/some/path?pager=3

The reason is that url for next page is build as example.com?pager=3, part /some/path is ommited.

Given patch adds checkbox for Feeds Crawler admin form that removes such behavior.

CommentFileSizeAuthor
feeds.crawler.patch1.69 KBpawel_r

Comments

twistor’s picture

Version: 6.x-2.0-beta1 » 7.x-1.x-dev
Status: Patch (to be ported) » Fixed

This should be fixed in 7.x. 6.x is not longer supported.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.