Images fail to import with the latest dev version of Feeds (2/11/14). In rolling back to Feeds alpha8, behavior returned to normal.

I had installed the dev version of Feeds to try to solve an issue; as it turned out, alpha8 was fine. So I guess this is more of a heads-up in case somebody else runs into this, or if there are code changes in the dev branch of Feeds that might be relevant for future production releases.

Thank you for this terrific module!

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

rooby’s picture

Can you give more information regarding "Images fail to import with the latest dev version of Feeds"?

Does it throw any error messages or just silently fail?

If there are error messages what are they?

bcobin’s picture

Gosh - it's hard to remember now, although if there was an error message, I undoubtedly would have posted it. So I'd have to say it just silently failed.

It's all working brilliantly on http://usgbc-li.org - all those nice LEED articles in the front page LEED block and on the News page are being pulled in from the national site with no user intervention. Suh-weet!

Thank you very much for the module - if I still had the dev site up, I'd be happy to verify, but I archived it some time ago.

Thanks for looking into this and for the great work you've done here... I wish I could be more helpful...

ibraaheem’s picture

I am getting this error - Warning: rawurlencode() expects parameter 1 to be string, array given in encode_url() (line 447 of /Applications/XAMPP/xamppfiles/htdocs/drupal-7.27/sites/all/modules/feeds_imagegrabber/libraries/url_to_absolute.inc).

GerZah’s picture

*bump* I also had to install the latest dev of Feeds 7.x-2.0-alpha8+56-dev today, and image grabber does not load images anymore. I don't see any error message, though.

GerZah’s picture

Uh-oh – I'm sorry, all clear:

Works fine, in this case Feeds 7.x-2.0-alpha8+56-dev and Feeds Image Grabber 7.x-1.0-alpha2+2-dev.

… Move along folks, nothing to see here! ;-)

anou’s picture

I'm sorry GerZah, but with both dev versions of Feeds and FIG, I didn't manage to make it work. Finally Feeds sable version (7.x-2.0-alpha8) with last FIG dev (7.x-1.0-alpha2+2-dev) did work fine.

rooby’s picture

Yeah since this module hasn't had many updates in a while it may need work for latest feeds dev.

Hopefully I will get a chance to try it out and make it work soon.

cyxair’s picture

Fixed with latest dev versions by adding the following to libraries/url_to_absolute.inc :

After line 424, at the begining of function encode_url

  if(is_array($url)) {
    $url = array_pop($url);
  }
Melissamcewen’s picture

cyxair's edit clears up this error for me, but it still doesn't work for me with Feeds latest dev

MegaChriz’s picture

The signature for the target callback changed. See the change record.

The function feeds_imagegrabber_feeds_set_target() should be adjusted. $page_url is now passed as an array. Furthermore, the empty check for that parameter should be removed to support emptying fields. See https://www.drupal.org/node/2301993

kdechant’s picture

Here is a patch that updates the callback signature and reads the URL from $entity->feeds_item. With this patch, I have FIG working with Feeds v2.0-beta1.

DarrellHQ’s picture

I wasn't able to get it working with the provided patch

mrvinceo’s picture

I'm having trouble with this too. I've tried the various version combinations, with and without patches, with no luck, and when I try the stable versions of each module I get the following error:

http://devsite.com|1449229051|php|66.249.84.240|http://devsite.com/feeds/importer/rss/65691||0||Warning: rawurlencode() expects parameter 1 to be string, array given in encode_url() (line 447 of /var/www/sites/all/modules/feeds_imagegrabber/libraries/url_to_absolute.inc).

I've tried the fix in #8, which gets rid of the error, but images are still not being pulled in.

Can someone please check if my mappings are correct? Please see image below:

Mappings screenshot

Many thanks,
Paul

mrvinceo’s picture

terminology’s picture

after applied the patch on #11,it still not works,
it says Failed importing 25 nodes.
Invalid enclosure https://s-media-cache-ak0.pinimg.com/236x/ee/1f/ab/ee1fab0f42173af2916bf...
Invalid enclosure https://s-media-cache-ak0.pinimg.com/236x/82/b1/96/82b196fe1b2e958a073e7...

Saoirse1916’s picture

Also doesn't work with the current beta version of Fields.

Saoirse1916’s picture

Here's what I did, though ultimately I think I (or someone else) needs to see about working this into the FIG settings area. At function feeds_imagegrabber_feeds_set_target($source, $entity, $target) I did a print_r($entity) to see what was actually getting sent to the function.

The problem is that $entity->feeds_item->url; is blank, presumably because of an issue with Feeds itself (which I asked about here). As a workaround, I changed the $page_url variable like so: $page_url = $entity->field_article_link['und']['0']['url'];. field_article_link is the machine name of the custom URL field I created for my custom Feed Data content type that holds the RSS data as nodes.

It then works to extract the necessary images as it's supposed to but it does throw the following error which I'm still trying to sort out:

Notice: Undefined index: cardinality in feeds_imagegrabber_scrape_images() (line 554 of /var/www/mysite.dev/public_html/sites/all/modules/feeds_imagegrabber/feeds_imagegrabber.module).

Ideally, I think it would be nice if there were an option in the FIG settings to specify the URL field in the node's content type.

Saoirse1916’s picture

Ok, sorted out the cardinality issue. Line 554 is if ($settings['feeling_lucky'] && count($images) == $settings['cardinality']) { and it should be if ($settings['feeling_lucky'] && count($images) == $options['cardinality']) {, as cardinality is a part of the $options array, not $settings. Same goes for line 566.

MegaChriz’s picture

Status: Active » Needs work
+++ b/feeds_imagegrabber.module
@@ -152,10 +152,9 @@ function feeds_imagegrabber_feeds_processor_targets_alter(&$targets, $entity_typ
+  $page_url = $entity->feeds_item->url;

The passed url should be seeked inside the $values array, not taken from the feeds item. The 'url' value is not guaranteed to be available there. This would imply that 1) the user also maps to "URL (url)" and 2) that the user sets up the mapping in a specific order.

Also, the implementation should take into account that the $values array could have multiple urls (which would be useful for multivalue fields).

Lastly, since Feeds now support multilingual fields, the option $mapping['language'] should be respected (using LANGUAGE_NONE or und is discouraged). See the change record.

terminology’s picture

so do we got an solution for this issue now,thanks

wOOge’s picture

I had similar issues but resolved it by:

1) Upgrading to Feed DEV, and Feeds Image Grabber DEV
2) Applying the code referenced in comment #8
3) Ensuring that I had set the option Search for the images between the tag which is identified by: set to ID
4) Verified that the HTML ID was correct (I was using the class accidentally once)

Hope this helps someone.

bkosborne’s picture

Priority: Normal » Critical
Status: Needs work » Needs review
FileSize
609 bytes

Here's an updated patch that works for us. If someone can confirm and set this to Reviewed and Tested by the Community that would be great. Then we can see about getting the module maintainer to cut a new release to include this patch. Without it, the module is just completely broken, so I'm setting this to critical.

smartsystems160’s picture

patch #22 didnt work for me. After applying the patch and retrying the import, i get the error: An AJAX HTTP error occurred. HTTP Result Code: 500 Debugging information follows. Path: /(my-website-folder)/batch?id=84&op=do StatusText: Internal Server Error ResponseText:

Using:

1. latest dev version of feeds (2016-Aug-05). Also tested with version 7.x-2.0-beta2 (2016-Feb-21)
2. Latest dev version of Feeds image grabber 7.x-1.x-dev (2013-Sep-30)

Notes:

1. I am using whm and i increased my php memory settings and rebooted my server: cPanel PHP max execution time (9000s), cPanel PHP max POST size (512MB)

2. the destination website is running as a subdirectory of the source (wondering if this matters but i do not think so since images are being downloaded from source site

3. If image was uploaded using the image field of the node, it is downloaded successfully using the Media Feeds Module, so the problem lies with importing images embedded on the body of the node using the Media and Wysiwyg Modules.

smartsystems160’s picture

Status: Needs review » Needs work

Updating issue status

smartsystems160’s picture

Did anyone get this working yet? Please post your solution if you dont mind, i really need this functionality right now. Thank you

ugintl’s picture

Solution #8 and #22 working for me, but not for youtube. It is showing a heading "Image:", but no image showing. I tried to put ID and classes as well. Copied from youtube source

class="feed-item-container yt-section-hover-container browse-list-item-container branded-page-box vve-check "

id=browse-items-primary

class="channels-content-item yt-shelf-grid-item yt-uix-shelfslider-item "

pyxio’s picture

@itpromanager - did you ever get this working? i have the same ajax error as you did.

specificleaf’s picture

Feeds version 7.x-2.0-beta3 in combination with 7.x-1.0-alpha2 does not import any images, but no error messages either.

GerZah's configuration in comment #5 (Feeds 7.x-2.0-alpha8/Feeds Image Grabber 7.x-1.0-alpha2) imports the images, but it also returns two errors (both on submission as in the background, using cron):

  1. Undefined index: cardinality in feeds_imagegrabber_scrape_images() (line 555);
  2. Fatal error: Allowed memory size of n bytes exhausted (...) in /includes/database/database.inc on line 2227.

Edit: Issue with cardinality has already been reported and patched (but not released..?).

rooby’s picture

Assigned: Unassigned » rooby

I'm looking into this now.

rooby’s picture

Title: Image Grabber doesn't work with current dev version of Feeds » Image Grabber doesn't work with latest version of Feeds
Component: Miscellaneous » Code
Status: Needs work » Needs review
FileSize
7.85 KB

Here is a patch that does a bit of refactoring of the feeds_imagegrabber_feeds_set_target() function and makes it adhere to these feeds changes:

If the source used in the field mapping returns multiple URLs then all the URLs will be scraped.

rooby’s picture

Slightly tweaked version, which will do less processing if there are no $values to process.

swhitters’s picture

The patch functions... but when Image Grabber is enabled I can't delete any nodes! When I disable it, I am then able to delete again. I get the "temporarily unavailable" error message.

sillo’s picture

This module is broken. Doesn't work. I used to have great success with it but now it just wont work.

rooby’s picture

@swhitters:
Thanks for the test, I will see if I can replicate that issue.

rooby’s picture

@sillo:

Have you tried the patch in #31?

If so, can you give more information on what isn't working?

sano’s picture

@swhitters, check the "Delete items from feeds" permission item on the Permissions page. There you can grant users with the desired role permission to delete "own" (imported) content.

swhitters’s picture

I checked the permissions and it was already checked off (I'm a super-admin anyway and I've deleted them before). Besides, I can't delete ANY node (of any content type) when image grabber is enabled. Once disabled, I can then delete again. I get a message that just says "The website encountered an unexpected error".

rooby’s picture

@swhitters:

After you get that message is there anything of note in the Drupal watchdog log or the PHP error log?

swhitters’s picture

Sorry, yeah I turned on logging and got this:

TypeError: Argument 2 passed to db_query() must be of the type array, string given, called in /mnt/www/html/xxx/docroot/sites/all/modules/feeds_imagegrabber/feeds_imagegrabber.module on line 127 in db_query() (line 2401 of /mnt/www/html/xxx/docroot/includes/database/database.inc).

rooby’s picture

Awesome, thanks. I'll have a look into it.

swhitters’s picture

Any progress on that error message? Thanks.

nst37’s picture

I had the same db_query() and a small change fixed the issue. Here is the patch.

swhitters’s picture

Thanks! That worked for me!

RgnYLDZ’s picture

What is the current status of this issue? the dev versions of both modules don't seem to work.

So this module is still broken as I understand ?

ericras’s picture

Status: Needs review » Reviewed & tested by the community
FileSize
8.57 KB

#42 was fixed in #2748459: Argument 2 passed to db_query() must be of the type array, string given

I've updated the patch in #31 with a minor REGEX change that was needed in url_to_absolute() for PHP 7.3. Appears at the bottom of the patch.

This is working great for me with latest Feeds as of today and PHP 7.3.

dasginganinja’s picture

The last patch is missing the db_query fix introduced in #42. I had to modify the patch in #42 to fix path offsets but with these two patches combined I am having success importing and also now deleting nodes.