Google News feeds causing duplicate aggregator entries

i've been trying to hack the code in the aggregator.module to compare the current aggregator_item title with other titles in an effort to avoid duplicate titles (regardless of url, link), but i know very little php or mysql basically, so it's muddle and bumble along and test.

i had thought this was the section of code to change, around line 894

    /*
    ** Save this item.  Try to avoid duplicate entries as much as
    ** possible.  If we find a duplicate entry, we resolve it and
    ** pass along it's ID such that we can update it if needed.
    */

    if ($link && $link != $feed['link'] && $link != $feed['url']) {
      $entry = db_fetch_object(db_query("SELECT iid FROM {aggregator_item} WHERE fid = %d AND link = '%s'", $feed['fid'], $link));
    }
    else {
      $entry = db_fetch_object(db_query("SELECT iid FROM {aggregator_item} WHERE fid = %d AND title = '%s'", $feed['fid'], $title));
    }

    aggregator_save_item(array('iid' => $entry->iid, 'fid' => $feed['fid'], 'timestamp' => $timestamp, 'title' => $title, 'link' => $link, 'author' => $item['AUTHOR'], 'description' => $item['DESCRIPTION']));
  }

i had thought a check for $title = $feed['title'] around the save might avoid duplicates but i didn't get it to work.

another option would be a "clean aggregator_items and related aggregator_category_items" routine that would work like the timestamp flush routine. around line 839, something like...

for each row of aggregator_item with a title = to the currently being parsed title {
  delete all aggregator_category_item entries with that iid 
  delete all aggregator_item rows with that iid
}

if it were in that section of the code, it MIGHT be something VAGUELY like...but i'm sure this is not right...and it would be timexpensive to run for every line of the aggregator data...

  $result = db_query('SELECT iid, title FROM {aggregator_item} WHERE aggregator_item.title = $title}');
  while ($feed = db_fetch_array($result)) {
    db_query('DELETE FROM {aggregator_category_item} WHERE iid IN ('. implode(', ', $items) .')');
    db_query('DELETE FROM {aggregator_item} WHERE fid = %d AND timestamp < %d', $feed['fid'], 
  }
}

again, i'm groping in the dark, but still groping. i'd love help.

it strikes me that perhaps a aggcleaner module might be a better approach, running off poormanscron or cron, and certainly if this title compare were to become part of the aggregator.module there should be an option to set the value on or off for those who don't need it.

the best place for the routine is probably outside of the iterative loops if possible, perhaps after a new feed has come in and been processed but then there'd need to be a loop anyway to process it all. ok i'm rambling.

thanks to anyone who takes an interest in this.
blake

Sorry for the question but not sure if I add it to the aggregator module, or what??
Jim