By jockox3@drupal.org on

Brethren,

I've been trying to make trackback.module "autodiscover" trackbacks (see http://drupal.org/node/175801 ) through visits to my site and their referrers. This question requires no knowledge of trackback.module so feel free to assist me even if you don't know about it!

Here's the basic framework of what I'm trying to do:

1. START processing:
2. When cron runs,
3. For each row in the access_log table that accesses a Drupal node and has a referrer URL since the last time cron ran
4. Try to fetch an RSS type feed url from the referrer site.
5. If the site doesn't provide a feed url, then it's likely it's not a blog or news type site and we can stop processing.
6. If it does provide a feed url, fetch the feed and parse it.
7. For each article in that feed check whether it contains a reference to the Drupal node referred to in the access_log record.
8. If it does not contain a reference to said Drupal node we can stop processing.
9. If it does, extract an excerpt and title from the article in the feed and save the whole lot into the trackbacks_received table. It should then appear under the node on my site when it is viewed.
10. STOP processing.

It's painfully slow for me! I'm no great shakes as a programmer. But I am specifically stuck on step 4.. above. What I have been trying to do is to make a drupal_http_request to the referrer url from the accesslog page. This seems to work fine, but then I need to go through it and look for <link rel="alternate" type="application/atom+xml" type lines to try to discover a feed from the referrer page. I tried loading the $data output from drupal_http_request into a new DOMDocument using "loadHTML($data)" but it does not seem to work. Additionally, though drupal seems to be able to search $data itself for the existence of a link back to me, a device used in trackback.module already where it does:

    elseif (stristr($reply->data, $GLOBALS['base_url'] .'/') === FALSE) {
      $trackback->error = t('The sender page does not refer to recipient site.');
    }

...when I look at the contents of $data in my debugger (Eclipse, PDT, XDebug) it seems truncated - has anyone seen this behaviour previous - maybe it's a dbugger setting that will only show the first x characters or something?

Anyway - I guess my big question is does drupal_http_request() return, in the $data, variable, the whole of the requested page - I was watching what it was tring to do when it seemed to be trying to put together "chunks" and could see the end of the document being requested in one of the chunks, but not in the final version of $data passed back from the function.

Can anyone suggest a better way of discovering whether a given URL offers an RSS/Atom feed and if so to parse it for references to my page?

TIA,

Jock

⋅ Categories: Drupal 5.x