The parsing stage of aggregator produces a normalized data structure in the form of an associative array that describes the feed.

The specific format of this data structure has been internal-use-only until #303930: Pluggable architecture for aggregator.module was committed. #303930 is exposing an API for fetching, parsing and processing feeds and therefore makes this data structure a de-facto standard for exchanging data between the parsing and processing stage of feed aggregation.

Start reading in aggregator.parser.inc aggregator_parse_feed().

We should clean up the house and at least make Array key strings in $feed->items lower case instead of upper case.

Comments

alex_b’s picture

After this is done, calls to aggregator_save_item() can be simplified.

foxtrotcharlie’s picture

To clarify - what needs to be done is to change all the keys of the $item array to lowercase?

So, for example, change

    if (!empty($item['TITLE'])) {
      $item['TITLE'] = $item['TITLE'];
    }

to

    if (!empty($item['title'])) {
      $item['title'] = $item['title'];
    }

Is that all?

alex_b’s picture

right, that's all :-)

foxtrotcharlie’s picture

Status: Active » Needs review
StatusFileSize
new3.1 KB

This patch makes all the uppercase keys in function aggregator_parse_feed() in aggregator.parse.inc lowercase. The goal is to then be able to simplify the aggregator_save_item() function.

foxtrotcharlie’s picture

StatusFileSize
new18.32 KB

I wasn't sure of a couple of uppercase items which I changed - see attached screenshot for the two lines I'm talking about. Let me know if I shouldn't have changed these.

Status: Needs review » Needs work

The last submitted patch failed testing.

alex_b’s picture

#5 - You'll have to change all of these uppercase keys. Also don't forget to change depending code in the rest of the module - OTOH, there is a bunch of uppercase keys work in aggregator.processor.inc. This is the reason why this patch failed testing.

foxtrotcharlie’s picture

Status: Needs work » Needs review
StatusFileSize
new10.52 KB

thanks alex_b

I've rerolled the patch to remove uppercase keys from aggregator.parser.inc and now also from aggregator.processor.inc too.

I did find this a bit tricky, and it's my first multiple file patch.

There are some keys in a comment at the top of function hook_aggregator_parse() in aggregator.api.php which I did not change as I wasn't sure:

 *   Recognized keys:
 *   TITLE (string) - the title of a feed item
 *   DESCRIPTION (string) - the description (body text) of a feed item
 *   TIMESTAMP (UNIX timestamp) - the feed item's published time as UNIX timestamp
 *   AUTHOR (string) - the feed item's author
 *   GUID (string) - RSS/Atom global unique identifier
 *   LINK (string) - the feed item's URL
 *
 * @see hook_aggregator_parse_info()
 * @see hook_aggregator_fetch()
 * @see hook_aggregator_process()
 *
 * @ingroup aggregator
 */
function hook_aggregator_parse($feed) {

I also found and left the following, because I wasn't sure, in aggregator.admin.inc function _aggregator_parse_opml()

the keys: OUTLINE, XMLURL, TEXT and OUTLINE (not sure if this one's a key?)

function _aggregator_parse_opml($opml) {
  $feeds = array();
  $xml_parser = drupal_xml_parser_create($opml);
  if (xml_parse_into_struct($xml_parser, $opml, $values)) {
    foreach ($values as $entry) {
      if ($entry['tag'] == 'OUTLINE' && isset($entry['attributes'])) {
        $item = $entry['attributes'];
        if (!empty($item['XMLURL']) && !empty($item['TEXT'])) {
          $feeds[] = array('title' => $item['TEXT'], 'url' => $item['XMLURL']);
        }
      }
    }
  }
  xml_parser_free($xml_parser);

  return $feeds;
}
alex_b’s picture

#8 All of these keys need to be changed.

Status: Needs review » Needs work

The last submitted patch failed testing.

aron novak’s picture

Status: Needs work » Needs review
StatusFileSize
new10.98 KB

Nice work!
Well, only one line was needed to get the things work.
In fact you need to convert the uppercase string to lowercase before examining.

spuky’s picture

I thought this patch is a easy patch so it could be one to get my feet wet with patch reviewing...

- aplyed the patch run simple tests on the agreator module (no error)
- added some feeds to my site which got parsed without error
- I read the patch and it is indeed just changing case of the keys
- got nothing from coder module

I don't know if that is sufficent for a "reviewed & tested by the community.

so I'd love to get feedback how I did reviewing this patch

spuky

spuky’s picture

Status: Needs review » Reviewed & tested by the community

since I got no feedback i am setting this one to rtbc

still not shure if my review is sufficent

spuky

dries’s picture

Status: Reviewed & tested by the community » Fixed

Committed to CVS HEAD. Thanks!

Status: Fixed » Closed (fixed)
Issue tags: -Novice

Automatically closed -- issue fixed for 2 weeks with no activity.