Port emvideo, emimage mappers from Feed Element Mapper.
Port tests for it.
Document it.

AlexUA has done some preliminary work on this. To make emfield mapping fully functional, we may need to add a media parser.

Comments

As an example for the API, take a look at mappers/content.inc should be used. For documenting the Mapping API, I opened an issue #623466: Document mapping API.

I wish i knew more about writing drupal modules and emfield. I'd jump in and help, I really love Feeds but need emfield in the worst way.

First volunteer to help test once we get something to alpha.

I spent a little time working on this over the weekend and ran into some difficulties, hoping someone can help. I'm trying to import a feed of products from Zazzle (http://zazzle.com/melodev/rss), and grab the Media enclosure and stick it into an Emfield so I can present different sizes of the image inside Drupal.

Borrowing from mappers/content.inc was very easy, and I had a new target & mapping setup in no time at all. Unfortunately, once my callback was actually executed, I could not find a way to get at the Media information from the value that was provided to me.

Here's what I did:

function melodev_feeds_node_processor_targets_alter(&$targets, $content_type) {
  $info = content_types($content_type);
  $fields = array();
  if (isset($info['fields']) && count($info['fields'])) {
    foreach ($info['fields'] as $field_name => $field) {
      if (in_array($field['type'], array('emimage'))) {
        $fields[$field_name] = isset($field['widget']['label']) ? $field['widget']['label'] : $field_name;
      }
    }
  }
  foreach ($fields as $k => $name) {
    $targets[$k] = array(
      'name' => $name,
      'callback' => 'melodev_feeds_set_target',
      'description' => t('The Emimage !name field of the node.', array('!name' => $name)),
    );
  }
}

This function successfully exposes Emimage fields to the mapper on the Feeds configuration page. I chose to map "Enclosures" to my "Emimage" field.

And then a very simple:

function melodev_feeds_set_target($node, $target, $value) {
  echo "<pre>value:\n";
  print_r($value);
  exit();
}

Unfortunately no matter what I do I cannot get any information to come through on "$value". If I exit out of the SimplePie parser and print_r on the $feed it's returning, the "enclosures" value is quite happily filled with all sorts of great information about the media file.

So my question is, how do I get that enclosures array from my parser to successfully arrive in my set target function?

Thanks in advance for any insights anyone can provide. I was a big fan of FeedAPI + Mapper so I'm looking forward to Feeds. I am doing my own forced inheritance on it for now, but getting access to media fields would be a huge plus.

Don't you have to map the URL of the item to the emfield and not the enclosure? There are no enclosures on the feed you're using.

This is calling for a good description of the mapping target :)

StatusFileSize
new20.28 KB

Well to be honest I expected something like "Media" to show up in the "Select a source" box, but Enclosures is the closest thing to what I was expecting. You mentioned in the description of this ticket that "we may need to add a media parser"? If that is something I can dive into let me know, I'm not sure where to start in parsing out Media so it appears in the "Select a source".

And if I'm 100% off on my interpretations here please point me in the right direction! Attaching a screenshot of my available sources based on the feed above.

Please let me know: Does Item URL not work?

To my knowledge item URL is the input that emvideo/emimage is looking for. If this is the case, we need to add a better legend to the emfield/emvideo targets.

The media parser *might* be necessary for retrieving meta information about media. E. g. blip.tv and youtube.com feeds include namespaced meta information on the media they're carrying. I haven't thought about this more than that.

Well that sounded promising, but "Item URL" is actually the URL to the Feed Item itself, not the URL to the Media file attached to the Feed Item.

$value contained this:

http://www.zazzle.com/the_welcome_home_shirt_mens-235060908460787096?gl=melodev

But I want it to contain this:
http://rlv.zcache.com/the_welcome_home_shirt_mens-p235060908460787096tmn7_500.jpg

I'm pretty sure I see what the problem is, though I haven't had a lot of time to spend on figuring out what the cause is, which is that it is inserting the item url value into the wrong part of the emfield field array. It should be getting inserted into $field_MyFieldName[0]['embed'], but it is getting inserted into $field_MyFieldName[0]['embed']['value']. That's why you get the following two strange behaviors:
1- if you edit a post you'll note that the embed string isn't in the field, yet there's a link to it below the field.
2- the link to that field doesn't actually go anyplace (because it's expecting the value, not the embed).

Emfield takes the embed string, parses it, and then it creates the value (i.e. the video, image, or audio id) as well as the accompanying $data array, that stores all of the info emfield needs.

Anyway, hope this helps...

Subscribing.

StatusFileSize
new1.71 KB

Attached is a simple provider that I was using to test. I changed both mentions of 'value' to 'embed', but it's still putting the url into the wrong place.

StatusFileSize
new1.7 KB

Oops, fixed some mistakes in the last one...

Status:Active» Needs review

And we have a winner! I removed and readded the mapping on the feeds mapper page and this is now working as expected. W00t!

Woot!!! One down!!!!

Status:Needs review» Needs work
StatusFileSize
new2.06 KB

Awesome. Thanks Alex UA.

- added CVS $Id$ string
- made sure comments aren't spilling the 80 character column
- rolled an actual patch

Mapper is RTBC.

The only thing missing now is a test (can be ported from FeedAPI Mapper, too).

Okay, first of all this patch requires "emvideo". Applying the patch gave this error immediately:
Fatal error: Call to undefined function emvideo_field() in sites/all/modules/feeds/mappers/emfield.inc on line 57

I'm not using emvideo, I'm using emimage, and the feeds module should use the appropriate function for the field that was mapped to.

Secondly, I think this patch is trying to solve the problem of mapping a value into an emimage field, when the actual problem I'm having is that there is no value being parsed out of the field in order for me to map.

I'm going to hack up the parser and make it work for myself and then share how I did it so you guys can tell me how to do it better. :)

It seems like simplepie.inc get_real_type() does not support image formats. This seems to be why I can't get images out of my enclosures.

The data I need is in $item->raw, so I did the following.

On line 79 in FeedsProcessor.inc I changed the following line:

        $callback($target_item, $mapping['target'], $value, $source_item); // added $source_item to custom target callbacks

So that a custom callback written by a user is also passed the entire original source item so they can futz with it.
From there I was able to pull the values I wanted out of enclosures and write a completely custom target mapper.

  // Grab the value manually
  $value = $source_item['raw']['enclosures'][0]->link;

Still struggling with saving emimage cck data appropriately but that is at least a normal problem I can tackle. I believe that passing the full original source item to custom target mappings is a great way to give people more customization in the Feeds module -- please let me know if this completely breaks some standards or something.

Finally, the emfield patch above needs some attention. It is calling variables $items and $field_name that were never declared, it only supports videos, etc.

Sarah-

I think you're getting a little mixed up here. Feeds should only grab the original url and then emimage should take care of the rest. I would start by just changing out "emvideo_field" with "emimage_field" and see if that gets emfield to work correctly (you'll note a populated emfield data array once you are). Once that is working you can select the image (or whatever else is attached to the returned XML) you want from emfield's data array (which is the appropriate place for this to happen).

Alex,

In your function emfield_feeds_set_target($node, $target, $value), $value is always empty when I choose "enclosures". If I choose "Item URL" as suggested above, $value is populated with the URL to the product node, and *not* the URL to the product image that is provided in enclosures.

Changing emvideo to emimage makes no difference if the $value parameter is empty.

I've tried mapping every available source to an emfield and just echoing $value & exiting --- *none* of the available sources contain any information about media files.

For now I have solved the problem for myself, but the fact is that simplepie.inc does not appear to support image files in enclosures.

Sara

Sara-
After re-reading this thread I think you're using emfield in a fundamentally different way than the norm. Usually, users of emfield post the url or embed code for a piece of media, for example:

http://www.youtube.com/watch?v=_4s2H9cH7Sw
or
<object width="560" height="340"><param name="movie" value="http://www.youtube.com/v/_4s2H9cH7Sw&hl=en_US&fs=1&"></param><param name="allowFullScreen" value="true"></param><param name="allowscriptaccess" value="always"></param><embed src="http://www.youtube.com/v/_4s2H9cH7Sw&hl=en_US&fs=1&" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="560" height="340"></embed></object>

That code is parsed, and a request is sent to the provider (YouTube in this example). Emfield then retrieve all of the information available for that video/audio/image file, and (via the provider file) it is then compiled it into the emfield data array, which is used to give you things like thumbnails, width, height, etc.

In this case you are not using a provider file, so while emfield does sort of support you posting images from zazzle via the custom_url provider, it's not meant to grab anything automatically (i.e. you have to paste in the actual path to the image), which is why it isn't populating any of the other info.

I honestly don't care whether I use emfield or a plain CCK text field to store the image URL. I am having no problems with emfield. I am only having problems with retrieving the URL to an image out of an enclosures array in a feed item.

I will spend some time this weekend writing my Simplepie enclosures parsing findings up for you so maybe it will help. Like I said, when simplepie.inc is parsing through the enclosures array, it runs get_real_type() on every item it finds to try and identify the mime type for the file.

In simplepie.inc between lines 7171 and 7176, there is no support for an image mime type.

// Mime-types by handler.
$types_flash = array('application/x-shockwave-flash', 'application/futuresplash'); // Flash
$types_fmedia = array('video/flv', 'video/x-flv','flv-application/octet-stream'); // Flash Media Player
$types_quicktime = array('audio/3gpp', 'audio/3gpp2', 'audio/aac', 'audio/x-aac', 'audio/aiff', 'audio/x-aiff', 'audio/mid', 'audio/midi', 'audio/x-midi', 'audio/mp4', 'audio/m4a', 'audio/x-m4a', 'audio/wav', 'audio/x-wav', 'video/3gpp', 'video/3gpp2', 'video/m4v', 'video/x-m4v', 'video/mp4', 'video/mpeg', 'video/x-mpeg', 'video/quicktime', 'video/sd-video'); // QuickTime
$types_wmedia = array('application/asx', 'application/x-mplayer2', 'audio/x-ms-wma', 'audio/x-ms-wax', 'video/x-ms-asf-plugin', 'video/x-ms-asf', 'video/x-ms-wm', 'video/x-ms-wmv', 'video/x-ms-wvx'); // Windows Media
$types_mp3 = array('audio/mp3', 'audio/x-mp3', 'audio/mpeg', 'audio/x-mpeg'); // MP3

Perhaps this means that Zazzle's RSS format is non-standard and Simplepie does not know how to deal with image files that show up in the enclosures array.

Sara-

I understand your frustration, but this thread is about porting the mappers for emfield, not about helping you figure out how to post images in general using feeds, nor about zazzle's rss format. Please start a separate support request for either of those topics instead of continuing to push this thread off topic.

Thanks

this code generates the following errors when populating the fields:

Invalid argument supplied for foreach() in /sites/all/modules/emfield/contrib/emvideo/emvideo.module on line 275.
Invalid argument supplied for foreach() in /sites/all/modules/emfield/emfield.cck.inc on line 65.

these errors are resolved by initializing the $items variable as an array in the emfield_feeds_set_target function

  $items = array();
  emvideo_field('presave', $node, $field_name, $items, FALSE, FALSE);
  $node->$target = $field;

Thanks for your support Alex. Please let me know if I can help with any additional testing in the future.

If someone else gets this new Feeds module to work with mapping to an emimage field (not emvideo) please contact me directly so I can pick your brain about how you had to set it up.

Thanks!

As you mentioned, you aren't trying to use emfield at all, so emimage won't be able to help you (and thus you aren't really testing this mapper, or emfield itself).

I think this is probably more along the lines of what you're looking for:
http://drupal.org/project/feeds_imagegrabber

I would like to embed youtube videos for example

http://gdata.youtube.com/feeds/base/users/youtube/uploads

Right now on my site only image is embedded and in the mapper I can see only the following options

Title Title

Description Body

Published date Published date

Item URL (link) URL

Item GUID GUID

Categories Taxonomy: Categories

there is no embed field to map???

StatusFileSize
new48.96 KB

here is the screeshot

use Item URL (link)

well i am using it and still get only the picture of the video, not in fact an embedded video on my site

Subscribing....

Is there now a sollution to map the importer to a custom cck field like emfield smewhere?

I creatd 3 Fields in CCK (Author, Emfield, Source)
none of them are displayed in the "target dropdown" on the mapping edit page for any of the Imprters.
Do i miss something. Ithink feeds is not suppoting cck custom fields at all ? Do i miss something?

Status:Needs work» Needs review
StatusFileSize
new1.99 KB

Here is a slightly changed patch from #14. What I've done is to remove the call to emvideo_field() in emfield_feeds_set_target(). That call isn't necessary as CCK will handle that by it self.

Everything works as expected for me, with this patch.

subscribe

WHY NOBODY HELPS HERE

i maean there are newbies here that need help and you just ignore them, we dont even know how to apply a patch for god sake

Fender,
This is code in development. It is not meant to be used by 'newbies'. Have you tried using the feedAPI module http://drupal.org/project/feedapi , which is more mature? Come back to this module after it is past 'alpha', or preferably 'beta'.

In answer to your question, I found these instructions: http://drupal.org/patch/apply . I don't think that I am any the wiser after reading this, but if you understand it, go to it.

Scott Jackson

So why you suggest anybody should switch from feedapi to feeds if it is not proper time, i never tried feed api

The patch works for me with latest from HEAD. The file that the patch creates has to go into /mappers. The "Item URL (link)" field then maps to the emfield for video. Didn't test images. I'm not familiar enough with the architecture to mark RTBC, but am glad to have the patch since this is one of the prime uses of feed_api that I can now replace.

i am really confused here, isnt it enough to add the file called emfield.inc and copy the content of the patch inseide feeds/mappers?
Or is the process much more complex?

Please help newbise, at least somebody should fix this to support the youtube and vimeo at least, now the plugin is useless

I have added that new file and I get this error

Parse error: syntax error, unexpected T_FUNCTION in /home3/altermed/public_html/modules/feeds/mappers/emfield.inc on line 22

@fender, if you mean do you copy the file http://drupal.org/files/issues/623432-32_emfield_mapper.patch into feeds/mappers, no, that's not it. Actually it's the first step, but then you have to go in there on the command line and type:
patch < 623432-32_emfield_mapper.patch
Or alternatively, use some other program that can translate patches.

where is the command line, is it in mysql?

Why nobody repairs or updates the whole plugin, please we need it, now my site is amess, at least fix youtube or vimeo

@fender - the nature of things here is that you have to use tools like google to answer basic questions like "what is the command line", and you have to stop pestering developers to do their job faster.

Status:Needs review» Reviewed & tested by the community

this thing is so ready. works like a charm! (just set it up on the new dojo, and i nearly cried because it was so easy...)

the code is clean & simple as well. perfect!

ok i am learning to patch right now, not easy, but I'll be bak :)

@fender: http://drupal.org/patch/apply is where you should start (edit: sorry, @swj, didn't see that you posted that earlier...)

Status:Reviewed & tested by the community» Needs work

Great - we're getting there.

2 things are missing still:

- Tests
- Better instructions

The inline description says "The Embedded !name field of the node" - could this be clearer? I. e. shouldn't it say something to the effect of: "Map a URL of a page containing media to this field." ?

"Map a URL of a page containing media to this field." might be technically correct, but possibly still confusing to the end user, since the end effect is not the URL but the embedded media itself.

Perhaps something like "Map a URL of a page to embed its media to this field."

Or like @alex_b wrote in irc but I almost missed: "The Embedded !name field of the node. Map a URL of a page containing media to this field." That might even be better.

#46:

"The Embedded !name field of the node. Map a URL of a page containing media to this field."

Let's use this line then. I summarize:

What's missing is:

1) Write tests.
2) Change existing description of target to "The Embedded !name field of the node. Map a URL of a page containing media to this field."

Has something changed ?
Because i need to urldecode $value before everything works correct

That patch in #32 worked great for me.

Patch 32 is working fine for me with Flickr photo links. Great work - thanks! I will test later with YouTube videos.

Seems to be working just fine with YouTube videos as well. Thanks again!

Any of the happy users up for locking down this patch by writing tests and adjusting UI per #47? :-)

StatusFileSize
new1.99 KB

Here's a patch with the description changed. I'm afraid I can't be too helpful with the test though.

Subscribing

Status:Needs work» Needs review
StatusFileSize
new5.57 KB

I added a unit test to #53 that includes a CSV with two video fields. The import works nicely.

However, as here I have the same problem that $this->assertCCKFieldValue('video', 'http://www.youtube.com/watch?v=gpkhANg919Y'); doesn't work, while values are correctly inserted in the database.

Patch contains #53 + unit test + sample csv file.

FYI, test files here:

Found form field field_video[0][value] for video with the expected value. Browser feeds_mapper_emfield.test 86 FeedsMapperEmfieldTestCase->test()

Great work folks! Patch in #53 applied cleanly. It's working fine to import from Flickr Atom Feed with the latest required modules (I am running almost all dev versions on my test site) on drupal 6.16. I am using Common Syndication Parser. I have not tested it with anything else.

Sorry, no help with the unit test issue in #55 (which is why I patched with #53).

I have tried the patch and am unable to get it to show images whic are embedded in the feed in emfield_image.... I used this patch and created the inc file which I put into the mapper directory and can see the mappings for the enclosure to the emfield.

I meant to say I tried patch # 53

+1

Subscribing

Status:Needs review» Needs work

#55: awesome.

Use $this->show() or $this->show('node/[nid]/edit') to figure out what the UI displays. show() will create a warning that prints the test site's current screen or the screen of the path you specified to the test log.

This doesn't seem to work with EMField 6.x-2.x-dev

I take that back. It works fine with emfield 6.x-2.x-dev.

I had some issues with the underlying folder structure.

Status:Needs work» Needs review

works as promised :) thanks

StatusFileSize
new1.7 KB

Hope this helps those that are new and are looking for a quick setup.

obviously you need to have the following modules
1) feeds
2) emfield

add a cck field type (embed video) to content type > feed item, as this is what the feed mapper is updating by default.

Now, lets just apply the patch / modify the feeds module to work with emfield

I have taken the patch form # 53 and converted it to emfield.inc

Now all you have to do is place the attached file in the folder below and then remove the .txt

\sites\all\modules\feeds\mappers

so in the end you have \mappers\emfield.inc

Thats it.

now you can see the cck fields for mapping under the target section.
Just remove the old source Item URL (link) and add a new to map to your cck embed video field.

awesome! Will try now!

So would the best strategy be to create a custom field for
1) embedded video
2)embedded audio
3)embedded image

and then we can create a

map enclosure--->embedded video field
map enclosure--->embedded audio field
map inclosure---->embedded video field

and teh mapped wnclosure , even though there is only one option, will know the difference and chose the right one of the three custom fields you created to map too?

#55 worked for me

#55 worked for me as well. I did not test the test, however. What more, if anything, is necessary before this can be set to RTBC?

Status:Needs review» Reviewed & tested by the community

Moving back to RTBC- this thing is ready for prime time!

Status:Reviewed & tested by the community» Needs work
Issue tags:+Needs tests

I hate to be harping on this, but we do need tests to commit the mapper to Feeds. Tests can be pillaged from FeedAPI mapper's emfield mappers.

Anyone able to get this working with audio too? I'm trying to import podcasts with emaudio fields but I couldn't get them to show up. Video works fine though.

Seems to be broken with the newest emvideo 1.23.
Can anyone confirm this?

Edit : Yes its def. not working for emvideo 1.23. It works fine with 1.20.

Yes for me to from a fresh install had to downgrade emvideo to 1.20 to get videos import

StatusFileSize
new74.88 KB

#72 & #73 according to emfield project page there was an error in the 1.21-23 releases. Please try 1.24 and see if that helps.

@alex_b & @pvhee I have used the patch in #55 and get the same results with the SimpleTests as mentioned in #55. So I ran show() as alex suggested and it all seems to be working. scrnsht attached. I also reviewed the code in the patch and can not see anything I would change. The test is cut/paste from feeds.content tests. Is assertCCKFieldValue a viable option for non-native CCK field types/widgets? Re #70 I had a look at FeedAPI mappers tests and they do not seem to be too complete.

I've followed instructions in #64 to no avail.

I tried attaching the importer to "Feed" and "Feed item" and added cck embed video field to both content types.

I don't get the new cck embed video field as an option when trying to map source to target.

I also was using emfield 1.24 and downgraded to 1.20...

Any suggestions?

Thanks for that. I've upgraded to 1.24 and all seems to be working OK

StatusFileSize
new6.55 KB

EDIT: DISREGARD THIS PATCH. It's not right at all. Will fix now.

Status:Needs work» Needs review
StatusFileSize
new5.51 KB

This patch should be complete. I do not use CVS so it's diff'd from the sites/all/modules root. You will need to apply it from there.

I used emfield (current, working version) with feeds (the last patch provided by cglusky) to create feed item nodes from youtube RSS - videos showed up perfectly. Thanks a ton for all of your hard work!

Status:Needs review» Needs work

Needs tests for getting committed. Can be modeled after some of the existing mapper tests.

Status:Needs work» Needs review

@alex_b the patch in #78 has tests

subscribing

cglusky: sorry, overlooked that. Will review as soon as I get some air.

Hi!

I'am new to feeds.

Is there any planning to deliver this mapper as a module?

How can I help to test? I have only a hosted drupal installation without possibilities to run patch scripts.

Issue tags:-Needs tests+beta5

paulap: I'd actually like to get this into beta5, need to review still.

Subscribing. This used to work really well in Feed API, I'd really rather not have to revert back to that just to get video import working.

Tested this patch against latest Feeds stable, works like a charm for grabbing YouTube videos (as well as their thumbnails, titles descriptions, and even thumbnails and other YouTube data). Thanks for your work in getting this patched!

I use patch #78 since it has been released. And I'm satisfied with it.
I map Flickr pictures into emfield image field.

patch #78 on Feeds release : 6.x-1.x-dev (2010-Jul-29)

subscribe

Status:Needs review» Reviewed & tested by the community
Issue tags:-beta5
StatusFileSize
new5.73 KB

Tests were broken, fixed. This is RTBC now.

Title:Port FeedAPI Mappers: emvideo, emimageMapper for emfield
Status:Reviewed & tested by the community» Fixed

This is committed now, thank you.

http://drupal.org/cvs?commit=423262

Im having a problem, can you give me the csv file? its not in the newest patch, tthanks

its ok I got it

so, If you want to map an enclosure to em audio, em video, and additionally to filefield, because youiu cant know ahead of time which type of media is in the enclosure, and you want to be ready for all types of media , using simplepie, do you just have multiple enclosure mappings? I have tried this and get so many errors/unable to import when I map the Enclosure to more than one target .

FYI:

http://drupal.org/cvs?commit=425688
http://drupal.org/cvs?commit=425678

The latter kept enclosures from working.

Yhanks so much! Applying patches now..

Status:Fixed» Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.