Mapper for Node Reference fields

pkej - March 2, 2008 - 13:25
Project:Feed Element Mapper
Version:6.x-1.0
Component:Code
Category:bug report
Priority:normal
Assigned:Unassigned
Status:needs review
Description

It would be fantastic if it was possible to map incomming data to Node Reference fields. It is my only hurdle for importing a lot of data which I have customized into feeds from an export script I have.

The nodes are related, all the imported books are related to authors. First I import the author feeds, then I import the book feeds. The books should then use Node Reference to reference authors.

Using taxonmy for this might be possible by letting the Author nodes be sticky, but I would get a listing of all the books on the same page as the author, and not in the way I would like them; since I have views for the book listings already.

#1

wmclark - March 27, 2008 - 18:02

+1 for this - I have a similar need. CCK Reference Fields are commonly used for most sites.

#2

wmclark - March 28, 2008 - 06:41

I went ahead and gave this a shot. It currently only handles the $feed_element as a string so nid's won't work in the patch I wrote. The string must match exactly to the referenced node title.

It seems to be working on my test site.

Please test and report.

AttachmentSize
feedapi_mapper_nodereference.inc_.txt 1.92 KB

#3

wmclark - March 28, 2008 - 06:47
Status:active» needs review

Forgot to mark as "needs review".

Also, I wanted to mention that even if the field being mapped to is a required field, if no valid nodes are found, then the node is created anyway.

Is this a bug in FeedAPI not using the validate step of nodeapi or is this just something that isn't supported yet?

#4

pkej - April 14, 2008 - 22:28

Also, I wanted to mention that even if the field being mapped to is a required field, if no valid nodes are found, then the node is created anyway.

I think that is the way it usually works. I'll give your patch a run around asap.

#5

pkej - April 18, 2008 - 12:49

I tried this yesterday, and I had problems getting it to work. I will try on a clean install tonight (I hope) and detail what I do on the website after registering the first account on the site.

I have high hopes for this functionality.

#6

alex_b - April 18, 2008 - 15:27

Waiting for pkejs findings here before I test myself.

#7

pkej - April 18, 2008 - 23:08

These steps were performed on a Drupal 5.7 fresh installation.

  1. Go to ?q=admin/build/modules
  2. Activated CCK modules: "Content", "Node Reference", "Text" and "Number" all version 5.x.1.6-1
  3. Activated FeedAPI Default modules: "FeedAPI" and "FeedAPI Node" modules, both version 5.x.1.2
  4. Activated FeedAPI Add On modules: "Common syndication", "FeedAPI Aggregator" and "FeedAPI Inherit" all version 5.x.1.2
  5. Activated FeedAPI Add On module: "FeedAPI Mapper", version 5.x.1.0-beta6

That started up the initial modules; had to bump up the memory_limit of PHP to 16 MB at this point before continuing.

  1. Went to ?q=admin/content/types
  2. Selected "Add content type" tab
  3. Gave the new type a name "Author", left the rest of the page unchanged
  4. Saved
  5. Repeated the above steps and created a content type "Feed Story"
  6. Then I selected "Edit" in "Operations" on the "Feed Story" content type.
  7. Selected "Add Field" tab
  8. Created a field named "Author" of type "Node Reference"->"Autocomplete Text Field"
  9. Checked the "Required" check box in the "Data settings" field set.
  10. Selected "Author" as "Content types that can be referenced:"
  11. Saved the field settings

This should give me two content types where one should be able to reference the other. Now I needed to add an "Author" and a "Feed Story" just to see that I can reference an "Author" from the "Feed Story".

  1. Went to ?q=node/add/author
  2. Submitted an "Author" with the Title "Paul K Egell-Johnsen" and the Body "A thorough tester"
  3. Went to ?q=node/add/feed-story
  4. Filled in Title: "A New Story", started writing "Pa" into the Author: field and selected the suggestion from the autocomplete field "Paul K Egell-Johnsen [nid:1] and in the Body I wrote: "Lorem ipsum sic amet."
  5. Then I submitted the "News Story".
  6. Went to the front page to see my two newly created nodes, both functioning as expected. The "Feed Story" node shows a link to "Paul K Egell-Johnsen" as expected.

Now it is time to create the Feed

  1. Went to ?q=node/add/feedapi-node
  2. Title: "Cool Feed"
  3. Body: "A cool feed testing the node reference addition to the Feed API Mapper."
  4. In the Feed field set
  5. Feed URL: "http://drupal.org/node/feed"
  6. Checked: "Refresh feed on creation"
  7. Checked: "Update existing feed items"
  8. Opened the "Processors" field set
  9. Changed the "Node type of feed items:" to "Feed Story"
  10. Changed the "Promoted items:" to "3"
  11. Left the rest as is.
  12. Submitted

Next step is to create the mapping.

  1. Selected the "Map" tab
  2. Mapped "options->original_author:" to "Map to field_author (nodereference module)"
  3. Updated
  4. Selected the "Refresh" tab
  5. Went to "Home"
  6. At the bottom there are three stories from Drupal, when I tested those where "Businessweek picks Dries Buytaert as a 2008 top innovator", "Drupal 6.2 released, fixing security issues" and "Announcing Drupalcon Szeged 2008"
  7. None of them had the "Author:" rendered, as opposed to the "A New Story", which shows it on the same page.
  8. I went into one of the stories from Drupal by clicking on its title.
  9. Selected the "Edit" tab
  10. Verified that there is in fact an required "Author" field in the edit form, and that it is empty.

So, there it is. I am able to select the mapping for the feed, but the mapping never happens in reality.

To verify that it is the autonodereference at fault, I created a required text field called "Original Author" in the "Feed Story" node type. That field showed up after I deleted the old entries and refreshed the feed.

I also tried turning on Clean URLs, for some modules in the past it has been an issue wether it has been on or off.

Best regards,
Paul

#8

pkej - April 18, 2008 - 23:30

OK

I have misunderstood how it works.

I added an Author named "drupal.org - Community plumbing", and now it automatically appeared. I was under the impression from Comment #3 in this thread that the referred node would be created if it didn't exist.

Upon rereading with the knowledge of how it works I now understand that wmclark meant that the feed node was created regardless.

So, no problems, it works here.

Best regards,
Paul

#9

wmclark - April 19, 2008 - 03:51

Sorry for the confusion!

These are limitations because of other modules:

1. nodereference.module will not create new nodes if the title does not exist
2. FeedAPI Node does not validate fields before creating nodes. (So nodes might be created with invalid data)
3. FeedAPI Node does not allow you to moderate which feed items should be created as nodes. (Nodes with blank required fields can be created and published. Unwanted/Unrelated feed items can be created.)

I've created feature requests for some of these limitations.

#10

Summit - July 12, 2008 - 19:58

Subscribing, somewhat the same feature request asked here:http://drupal.org/node/241226#comment-792378, right?
If not sorry to interrupt.

Greetings,
Martijn

#11

pkej - July 16, 2008 - 15:02

Works here.

#12

kehan - December 3, 2008 - 15:45

subscribing

#13

vanderlip - December 31, 2008 - 04:27

Thanks for this node reference mapper!

I am using the Node Reference Mapper in #2 with Feed Element Mapper 5.x.1.x-dev. The first time I run a feed refresh after creating (or after deleting all feed items), the mapper works flawlessly, but on each subsequent refresh (either manual or cron), new node references are not mapped to the feed items. The new feed items (nodes) are created correctly and other Feed Mapper Mapped items, CCK Date, and links, are mapped correctly, but the node reference is not mapped.

The node references that are not being mapped in subsequent refreshes are the same as those that were successfully mapped in the first refresh, so I don't suspect there there is a problem with invalid node reference data.

Any assistance would be greatly appreciated.

#14

vanderlip - December 31, 2008 - 08:11

Update to #13

Upon closer examination, node reference mapper appears to work sporadically after the first refresh (which works 100% correctly). For instance, in a resent refresh 1 of the 4 new feed item's node reference values were mapped, the other three feed items were not mapped, even though the value to be mapped was mapped correctly in a previous refresh. I have checked the feed and all the data to map correctly is there.

Thanks again for any ideas.

#15

alex_b - January 26, 2009 - 14:57
Status:needs review» needs work

Changing status as a response to #14

#16

scottrigby - January 31, 2009 - 16:12
Version:5.x-1.x-dev» 6.x-1.x-dev
Status:needs work» needs review

Updated patch for 6.x-1.x-dev. I haven't been able to reproduce the problem from #14 in D6 with this patch.

  • The main change I made was to _nodereference_potential_references()
    D5: _nodereference_potential_references($field, $return_full_nodes = FALSE, $string = '', $exact_string = false)
    D6: _nodereference_potential_references($field, $string = '', $exact_string = FALSE)
    See http://api.freestylesystems.co.uk/api/function/_nodereference_potential_....
  • The only other changes are comments, etc, to match feed_mapper_content.inc.

Any hesitations for including this in 6.x?

AttachmentSize
feedapi_mapper_229040_nodereference.patch 2.37 KB

#17

jesss - February 27, 2009 - 15:58

I just tested this patch and it didn't work for me. This may be because I'm using 6.x.-1.0-beta4, not 6.x-1.x-dev, but just in case it's helpful, I thought I would post my results.

However, it did work when I replaced the else if ($op == 'map') { section with the matching section from feedapi_mapper_content.inc, replacing $field[0]['value'] with $field[0]['nid'].

The original code from the patch

<?php
 
else if ($op == 'map') {
     
// Here is where the actual mapping happens.
      // When we are called at this point, $field_name contains the name of the field the user has
      // decided to map to and $field_element is the feed item element the user has decided to map.
      // We just need to put the two things together. The data structure here depends a lot on
      // CCK. We stick the value in $feed_element into $node->$field_name[0]['value'].
     
if (!is_array($feed_element)) {
       
$cckfield = content_fields($field_name);
       
// We are checking for nodes that exactly match the string mapped to $feed_element. We
        // could have checked to see if it was numeric first and do something else
       
$nids = _nodereference_potential_references($cckfield, $feed_element, TRUE);
       
$field[0]['nid'] = (!empty($nids)) ? array_shift(array_keys($nids)) : 0;
       
$node->$field_name = $field;
        return
$node;
      }
?>

The code that worked for me

<?php
else if ($op == 'map') {
     
// Here is where the actual mapping happens.
      // When we are called at this point, $field_name contains the name of the field the user has
      // decided to map to and $field_element is the feed item element the user has decided to map.
      // We just need to put the two things together. The data structure here depends a lot on
      // CCK. We stick the value in $feed_element into $node->$field_name[0]['nid'].
     
if (!is_array($feed_element)) {
       
$field = isset($node->$field_name) ? $node->$field_name : array();
       
$field[0]['nid'] = $feed_element;
       
$node->$field_name = $field;
      }
      return
$node;
    }
?>

#18

alex_b - March 3, 2009 - 13:26
Status:needs review» needs work

Needs a patch.

#19

jesss - March 18, 2009 - 18:23

Here you go. This is my very first patch -- please be gentle. :-)

AttachmentSize
feedapi_mapper_229040_nodereference_1.patch 1.49 KB

#20

patchak - April 14, 2009 - 21:41

This does not work for me, I tried with node titles as the reference and not the nid's maybe that's the reason.

Using beta 7 patched version from no. 19

Patchak

#21

jesss - April 15, 2009 - 15:36

The patch matches on nids, not titles. Since the title isn't necessarily unique, matching on title could open the door to a lot of potential mis-matches.

#22

alex_b - April 15, 2009 - 15:44

@jesss - I only see this now, but you've patched the patch in #19.

Can you roll a new patch that includes the entire mapper? Hint: You will need cvsdo for adding a new file to the module.

#23

patchak - April 15, 2009 - 20:55

Well In my use case I have only use for titles, so I think that would be a choice for a user...

I reverted back to the original mapper for d6 in no.16 and that one works fine. I think we should offer a choice to use either titles or nid's for the mapping, since most clients would have titles and not nid's in their legacy data exports...

Patchak

#24

alex_b - May 19, 2009 - 13:16
Title:Node Reference fields...» Mapper for Node Reference fields

#25

omjn - July 14, 2009 - 09:29

+1

#26

ndm - September 10, 2009 - 14:41
Version:6.x-1.x-dev» 6.x-1.0
Status:needs work» needs review

I can't understand interest to map to node id. How can a feed know your nodeid???

So i have take code from #2 and adapt with standards of 6.x-1.0. Now link is made with node title, if doesn't exist even if you have a autocreate, nothing happens (more sure, i think for miss orthographic problem).

So for me that's work, and for you?

AttachmentSize
feedapi_mapper_nodereference.inc_.patch 2.15 KB

#27

SocialNicheGuru - October 16, 2009 - 14:20

subscribing

#28

MikMik - November 9, 2009 - 11:24
Category:feature request» bug report

Hi there,

I tried the patch in #26, but it didn't work at all! I don't see a difference in the mapping-screen. Actually, this counts for all the patches in this thread.
What I noticed in the #26-patch is that the function it declares is "content_feedapi_mapper", but this function has already been declared in another file, so php throws an error (cannot redeclare function etc).

Are there any updates for this highly anticipated feature?

#29

Jax - November 16, 2009 - 21:23

@26:
I can't understand interest to map to node id. How can a feed know your nodeid???
I'm writing my own eparser which can retrieve the correct related node id's throught the guids.

Say I have a first feed with authors:

<authors>
  <author id="1">name #1</author>
  <author id="2">name #2</author>
</authors>

I've written an eparser to turn those ids into guids for feedapi. Then I have a second feed with books:
<books>
  <book id="1">
    <name>book name</name>
    <authors>
      <author id="1"/>
   </authors>
  </book>
</book>

I've written a second eparser that can take the id from the author and fetch the corresponding nid from the feedapi_node_item table. In this case I need to be able to map the nid.

 
 

Drupal is a registered trademark of Dries Buytaert.