Feed Element Mapper = FEMP

boris, you're right. I was only having feed-specific fields in mind. Of course you will want to define things like "author value should always go into By - CCK field" on a global level.

Aron, how could a processor parse a feed on the feed settings stage to analyze the items the feed contains? Is there a function for it?

Working on this. There is a first version in my sandbox: mapping for taxonomy, mappings on a per feed basis.

A screen cast of the feed element mapper went up on the Development Seed blog today: http://www.developmentseed.org/blog/2007/oct/30/pick-it-feed-stick-it-node

Status quo of the sandbox version (http://cvs.drupal.org/viewvc.py/drupal/contributions/sandbox/alex_b/feed...) is:

* Currently, mapping to taxonomy is fully functional.
* I have not tested this module extensively.

This is the concept:

Defining the map: feedapi_mapper_map_form()

* When loading the feed's map form, FEMP scans the node type specified as feed item node type for fields on the top level
* For each field it calls a feedapi_mapper_map_[field_name] function with the argument 'list'
* The function returns the sub fields it exposes (see feedapi_mapper_map_taxonomy())
* FEMP parses the feed and merges all feed items on it into one feed item, scans the feed item and returns a list of all paths to all elements of the feed item (see _feedapi_mapper_get_feed_elements())
* FEMP uses the return values of all feedapi_mapper_map_[field_name] to build the form elements.
* Each form element is assigned a drop down of all feed elements in feed items found.
* When user submits form, the mapping is being stored with the feed's node id as key.

Mapping: feedapi_nodeapi()

* When a feed item is being processed and a mapping for its feed node exists, the mapping is loaded and executed.
* For each node field that has been assigned a mapping, feedapi_mapper_map_[field_name] is being called again, this time with the arguments 'map', the feed item node $node, the feed element mapped to the node field, and the subfield it was mapped to.

Todo:

* The feed item elements should be the form elements and the node fields should fill the drop down per element. Now it is the other way round. There are 2 reasons why the feed item elements should constitute the form elements: 1) it is more intuitive - you map FROM the feed item element TO a field on the node. 2) it allows you to map different elements of the feed into the same node field - e. g. the same taxonomy vocabulary. (The reason why I haven't done this from the get go is because I had imagined the list of paths to all feed elements to be a huge messy list, but it turns out that it is not.)
* nodeapi('prepare') has to go to nodeapi('submit') - depends on feedapi_item - or is there a better pre nodeapi('save') hook to use in this case?
* feedapi_mapper_map_[field_name] has to be come a unique name - only mapper implementation functions should start with feedapi_mapper_map
* Do general mappings on a per node type basis - this is a small step away. A simple definition of standard feed items in the format that is returned by _feedapi_mapper_get_feed_elements() plus a way of storing and loading mappings on a content type basis would do the job.

Go ahead and check it out - don't be shy :)

Update

* DONE: The feed item elements should be the form elements and the node fields should fill the drop down per element. Now it is the other way round. There are 2 reasons why the feed item elements should constitute the form elements: 1) it is more intuitive - you map FROM the feed item element TO a field on the node. 2) it allows you to map different elements of the feed into the same node field - e. g. the same taxonomy vocabulary. (The reason why I haven't done this from the get go is because I had imagined the list of paths to all feed elements to be a huge messy list, but it turns out that it is not.)
* NOT SURE: nodeapi('prepare') has to go to nodeapi('submit') - depends on feedapi_item - or is there a better pre nodeapi('save') hook to use in this case?
* DONE: feedapi_mapper_map_[field_name] has to be come a unique name - only mapper implementation functions should start with feedapi_mapper_map
* DONE: Implementations of mappers live now in their own subdirectory.

STILL TO BE DONE:

* Do general mappings on a per node type basis - this is a small step away. A simple definition of standard feed items in the format that is returned by _feedapi_mapper_get_feed_elements() plus a way of storing and loading mappings on a content type basis would do the job.
* Not sure wether mapper hook format feedapi_mapper_map_hook ("hook" to replaced by fieldname - e. g. "status") is optimal. The particularity with feedapi_mapper is that field names on nodes don't have to be consistent with module names (e. g. field_* names for CCK fields). So a hook_feedapi_mapper_map where hook is to be replaced by module name doesn't make sense. On the other hand, feedapi_mapper_map_hook is not really the Drupal way and could lead to namespace conflicts (?) or at least to the oddity of feedapi_mapper_map* functions in .module files.

DONE: mappings on a per content type basis are possible now.
PLANNED: feedapi_mapper_map_hook will be refactored to hook_feedapi_mapper_map($field). This has the advantage that there can be more than one mapper for a field at once. Check out commentary on feedapi_mapper_map_taxonomy() function.

Latest version in sand box.

This is a first patch against the sandbox version that changes the

feedapi_mapper_map_[field_name]($op) hook

to

hook_feedapi_mapper($field_name, $op)

(sandbox version: http://cvs.drupal.org/viewvc.py/drupal/contributions/sandbox/alex_b/feed... )

Check out how the mapper implementation of feedapi_mapper_taxonomy.inc changed now.

@Richard Eriksson: this also solves the problem on how to write mappers for CCK fields. In the mapper implementation, you can kick in on any field_* argument coming in on $field.

This patch also adds help text for users (descriptions). After consolidating this patch, whats missing to roll out the module, is cleaning up the UI on admin/content/types/node-type pages.

More to come next week,

Alex

@Richard: thanks for checking out the API. Helps me a lot, because I didn't come around to test the element mapper over the weekend (too much turkey :)

Did you have any problems with the patch in #19 ?

@John: Thank you.

I just committed the patch from #19 and #23.

Important: The order of arguments changed.

Before:
function taxonomy_feedapi_mapper_map($op, $node, $feed_element = array(), $sub_field ='');
In patch:
function taxonomy_feedapi_mapper($field_name, $op, $node, $feed_element = array(), $sub_field ='');
Now:
function taxonomy_feedapi_mapper($op, $node, $field_name, $feed_element = array(), $sub_field ='');

http://cvs.drupal.org/viewvc.py/drupal/contributions/sandbox/alex_b/feed...

@mikeryan: There is some problem with feed retrieval at the moment when you try to map. FEMP bails out on 312 because there are no items on the feed. I fixed this by adding an is_array() test.

http://cvs.drupal.org/viewvc.py/drupal/contributions/sandbox/alex_b/feed...

Now FEMP should fall back to standard feed item elements if it cant retrieve the feed. The standard elements are defined in _feedapi_mapper_get_standard_elements() - they are not complete. You might want to add your own elements or submit a patch that adds more standard elements to this function.

There is a working .inc example that comes with feedapi_mapper.

When feedapi_node saves nodes, prepare and update or insert hooks are called.

A mapper should be easy to write though.

In your module (yourmodule) create an implementation of hook_feedapi_mapper:

// I haven't tested or used this code, just written off the top of my head.
function yourmodule_feedapi_mapper($op, $node, $field_name, $feed_element = array(), $sub_field = '') {
  if (strpos($field_name, 'field_') === 0) {
   
    if ($op == 'describe') {
      return t('Maps anything to a CCK field.');
    }
    else if ($op == 'list') {
      return TRUE; // If you return true here, this mapping will be available for any node or node type.
    }
    else if ($op == 'map') {
      // Do here with the feed element what you need to do and stick it on the node - 
      // a little trickery with the actual format will be necessary here,
      // because the feed element that is passed in can be a string or an array. 
      // Keep also in mind, that the $feed_element is not an entire 
      // feed item, but a particular element of the feed item that's been mapped by the user to this field.
      $node->$field_name = $feed_element;
      return $node;
    }
  }
}

Or if it is not your module you would like to write the mapper for, you can create an on-behalf implementation in an .inc file, like the taxonomy_feedapi_mapper() hook implementation is an implementation on behalf of taxonomy.module in mappers/feedapi_mapper_taxonomy.inc - this approach follows the pattern how views module organizes its on behalf implementations.

Released :)

http://drupal.org/project/feedapi_mapper