Hello,
i'm developing a site that uses external pictures on other servers. pictures come from different sources and have different sizes.
i would like to pull the images and use imagecache and then safe a local copy of the image for next time the file is requested.
any idea how to handle remote files with imagecache?

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

dopry’s picture

Status: Active » Closed (fixed)

not a feature of imagecache. you'll have to do some coding to get that working.

taqwa’s picture

Were you able to figure out how to do this?

Jomel’s picture

This interests me too - I'm aggregating external feeds, and it would be really nice if I could just output resized copies of their images using the same theme('imagecache',...) I use for local images.

burgs’s picture

Version: 5.x-1.5 » 5.x-1.6
Status: Closed (fixed) » Needs work
FileSize
2.77 KB

This is a small patch, that doesn't respect a whole lot of the imagecache functionality: It doesn't deal with private files, and if the external image doesn't exist it will WSOD, i think.
It does however, cache the external image locally, then cache the preset version. So, it should only call externally once per image.

Patch is on version 5.1.6

To use, just use

<?php
print theme('imagecache', 'preset_name', 'http://www.example.com/image.jpg',  ...  );
?>
burgs’s picture

Category: support » feature
FileSize
3.41 KB

Ok, here's a better patch. It still needs some work.
Same instructions as above.

Eidolon Night’s picture

Is this feature available in 6.x-2.0-beta5 ?

drewish’s picture

Version: 5.x-1.6 » 6.x-2.x-dev

totally not a fan of this approach. i'd be into a separate module that handled the fetching and storage and then handed them to imagecache but i don't see any reason to complicate imagecache.module with that functionality.

and it'd have to happen in the latest version.

burgs’s picture

I agree, the patch i did screams of hack, and it breaks with urls with query strings etc. I am using it on a site though, with some fixes for query strings, and it's some of the best functionality ever. It's really handy.

Simon

nedim.hadzimahmutovic’s picture

Is there a way to get http://drupal.org/project/get_image and http://drupal.org/project/remote_file to work together? :)

nedim.hadzimahmutovic’s picture

This could be done with remote_file and cck - http://drupal.org/node/375549

drupalninja99’s picture

Title: Use remote images and cache them localy with image cache » Use remote images and cache them localy with image cache - 1 decent solution

One solution as a "bridge" is to have your own function to fetch the image locally and then use imagecache's existing functionality.

I have an MLS search with all remote images and I want to use them with imagecache. So I make my custom module fetch the image if it doesn't exist. If it does exist it returns the filepath for my imagecache call and voila we've got it. That means the first time an image is downloaded its a bigger hit. It also means I have a file check every time I look at an image. And it means I'm going to download alot of these images on my file system. So performance may be an issue here but for now I like it. Here's my function:

function nh_search_fetch_remote_image($url) {
	if (!$url) return FALSE;//fail
	
	if (preg_match('/([^\/]+)$/', $url, $matches))//get filename
		$filename = $matches[1];
	else
		return FALSE;
	
	$filepath = file_directory_path().'/mls/'.$filename;
	
	/**
	 * See if file exists first, if it does return
	 */
	if (file_exists($filepath))
		return $filepath;
	
	$ch = curl_init();

	//Set curl to return the data instead of printing it to the browser.
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
	//Set the URL
	curl_setopt($ch, CURLOPT_URL, $url);
	//Execute the fetch
	$data = curl_exec($ch);
	//Close the connection
	curl_close($ch);
	
	/**
	 * If data returned save to mls_images folder and return filename
	 */
	if ($data) {
		if (file_save_data($data, 'mls/'.$filename, TRUE))//replace
			return $filepath;
	}
}
OnkelTem’s picture

Thank you for the patch, burgs!

I was looking for such functionality to get emfield videos thumbnails to go through imagecache. I created a simple module which brings imagecache display options to video_cck and applied your patch. Now remote images work fine.

After investigations I found two drawbacks in your patch:

1) error in function imagecache_create_url($presetname, $path):
if (stripos($path, 'http://') === 0){ -- correct variant, with three '='s, not two

2) paths to imagecached images should be urlencoded too, otherwise all image links are processing through imagecache's transfer instead of simply be retrieved via direct link.

Quick solution: $path = 'http/'.urlencode(str_ireplace('http://', '', $path)) in imagecache_create_url.

Full function code (DRUPAL 5):


function imagecache_create_url($presetname, $path) {
  $path = _imagecache_strip_file_directory($path);
  if (stripos($path, 'http://') === 0){
  	$path = 'http/'.urlencode(str_ireplace('http://', '', $path));
  }
  switch (variable_get('file_downloads', FILE_DOWNLOADS_PUBLIC)) {
    case FILE_DOWNLOADS_PUBLIC:
      return url(file_directory_path() .'/imagecache/'. $presetname .'/'. $path, NULL, NULL, TRUE);
    case FILE_DOWNLOADS_PRIVATE:
      return url('system/files/imagecache/'. $presetname .'/'. $path, NULL, NULL, TRUE);
  }
}

Needs testing though

swill’s picture

In general I hate hacking modules or core. I ran into this same issue, so I wrote the following function in one of my modules to take care of the issue. Enjoy...

// This module takes the path to a remote image and downloads it to the files directory so that you can use imagecache on the image later...
function MYMODULE_get_image($path) {
	$return = "";
	$base_path = file_directory_path()."/my_folder";
	if (!file_exists($base_path)) {
	  mkdir($base_path, 0777);
	}

        // The next two lines are up to you.  Basically, change the input path to what you want the image file name to be.
        // Maybe explode on '/' and then grab the last element in the array???  Whatever works for you...
        $cut_path = str_ireplace("http://www.example.com/", "", $path); 
	$image_name = str_ireplace("/", "-", $cut_path);

	$image_path = $base_path."/".$image_name;
	if (file_exists($image_path)) { // you already downloaded that image, just return it...
		$return = $image_path;
	} else { // download the image...
		$remote_image = file_get_contents($path) or die('Could not grab the file');
		$local_image  = fopen($image_path, 'w+') or die('Could not create the file');

		fputs($local_image, $remote_image) or die('Could not write to the file');
		fclose($local_image);
		unset($remote_image);
		
		$return = $image_path;
	}
	
	return $return;
}

// Later, somewhere in code...
$output = theme('imagecache', 'present_name', MYMODULE_get_image("http://www.example.com/projects/image1235.jpg"));
drupalina’s picture

Subscribing.
I have a Digg-like site powered by Drupal 5.12 and Drigg_5x_1.36. Users submit links. If they want they can upload an image with Imagefield, which is handled by Imagecache, but this is a very "slow" solution (users need to download an external image, and then upload it, and then there are whole lot of copyright issues). What would be absolutely awesome is when users submit a link and while they're still on the "submit" form, a query goes out to that particular URL page and dynamically brings a set of images from that external URL, which are resized by Imagefield and immediately presented for the user to choose from. We see this kind of functionality in action on Digg.com and also on Facebook when we attach a link. My question: will any of the above patches and codes provide this kind of solution???

I really appreciate your help!

Finn Lewis’s picture

Thanks swill - Just what I was looking for. The code in #13 worked for me (on Drupal 6.10, Imagecache 6.x.2.0-beta8).

swill’s picture

ecofinn, thanks for confirming the code in #13 works for you. :)

mkalbere’s picture

Hi ...
just to make it a bit more generic :

.....
        // The next two lines are up to you.  Basically, change the input path to what you want the image file name to be.
        // Maybe explode on '/' and then grab the last element in the array???  Whatever works for you...
        $cut_path = str_ireplace("http://", "", $path);
        $cut_path = preg_replace("/^[^\/]*\//", "", $path);
        $image_name = str_ireplace("/", "-", $cut_path);

       $image_path = $base_path."/".$image_name;
.....
parrottvision’s picture

subscribing

sonictruth’s picture

Check out my blog post about doing this to pull in images from Flickr and run them through Imagecache.

http://marmaladesoul.com/using-imagecache-external-images

I'm thinking about making it more generic and turning it into a module.

mark_r’s picture

Subscribe. That is exactly what i'm looking for! :) Now, reading the thread.

swill’s picture

The code in #13 with the update in #17 is probably your best bet...

ChrisRut’s picture

Subscribe

zeezhao’s picture

subscribing

drupalninja99’s picture

One thing to note, I have found caching remote images is too slow for my site which downloads a lot of MLS listings for TN. I ended up resizing the images in CSS bc the time it takes to download the image and cache it was too long. It might be worth it for sites where image content isn't constantly updated like it is on an MLS site.

swill’s picture

That is a very good point. The function that I wrote in #13 (with the improvement in #17) is going to be quite slow when it gets the images for the first time (especially if you are getting lots of them). If the images that you are getting well be changing all the time and you are not reusing the saved images more than 75% of the time, this approach is probably not going to work for you.

In my case, I had to find a way to get the images initially and then reuse the images that I got from then on. I needed to make sure that if an image was changed remotely, that my application what dynamic enough to see that the image had changed and update itself.

Garrett Albright’s picture

Interesting to see this issue - I am currently suffering from Geek Ego (you know, that feeling you get when you've done something amazing but very few other people in the would have the capacity of understanding its greatness) because I've managed to pull this off successfully, and without hacking ImageCache or anything else. It's a kludge, and not for the feint of heart, but it works. Here's some basics on my plan of attack:

The first problem is that I wanted my images to be attached to nodes via a CCK ImageField/FileField, but FileField cannot attach files to nodes if the files don't actually exist yet. So my code (which programmatically creates nodes) attaches a blank placeholder JPEG; one white square pixel, 304 bytes. (Pre-fetching the images wasn't really an option since this code will sometimes create 20,000 nodes at a time, and each listing could have dozens of images… The ideal solution is to download them as the user tries to view them, so that the fetching is spread out over time.)

The second problem is getting the new image when it's time to view it. To do this, I created a new ImageCache action (see the imagecache.api.php and imagecache_actions.inc files) which, when triggered, checks to see if the image it's being asked to act upon is one of those placeholder JPEGs; it runs a regex on the filename and checks if $image->info['file_size'] === 304. If it's a match, it downloads the new image, replaces the placeholder image with the new image, and updates the filesize of the image in the {files} table. It then tweaks the $image object as mentioned in #623770: _imagecache_apply_action(): Explicitly receive $image by ref?.

Hopefully circumstances will let me release this and the other code involved with this module soon…

reformatt’s picture

its quite the hack burgs!
but this remote imagecache patch was exactly what I was looking for.

My webserver was full and I had to move my images to another file server and this was a great quick fix!
Thanks again for the patch!

giorgio79’s picture

This seems to be promising:
http://drupal.org/project/get_image

It seems this wont happen in Imagecache and it wont happen in Filefield sources either:
#712866: Periodical refresh of remote file

scotjam’s picture

FYI...

Garret's posted his code referred to in #26 here

http://drupal.org/project/pirets

swill’s picture

FileSize
3.95 KB

I get a bunch of emails about this post because of the code I wrote in #13, so I cleaned it up and added the code by mkalbere from #17 and put it together as a little drupal 6 module (changing it to drupal 5 would just include changing the .info file).

The function I wrote is meant to be used in code, but it could also be used in the UI if you allowed PHP code entered in the UI (only do this if you are the only person you give access too).

Usage example if you use the PHP Code input format would be:

    echo theme('imagecache', 'present_name', get_remote_image_from_url("http://www.example.com/projects/image1235.jpg"));

(NOTE: Be sure to change 'present_name' to the name of an imagecache present name you have setup on your site.)
(NOTE: You should only ever reference the url, if you already have the image, it will serve the local version.)

Thats it. I hope this help some people out.

Take note of #25. This function is great if you need to get the image once and then just use the local copy from then on. If you will be getting new images each time, this may not be the best option, but a good option is going to be difficult to come by. (You could just run this function in a cron job over all of the images you wanted to get so you would just reference the local image when you did your actual display to the user. I will leave that as an exercise if you need to make the download process faster because you are always referencing new images.)

NicolasH’s picture

subscribe

Cyberwolf’s picture

Subscribing.

alberto56’s picture

For those interested, here is a post on using imagecache with flickr images:

http://marmaladesoul.com/using-imagecache-external-images

Cheers,

Albert.

Stalski’s picture

@alberto56: Your page is not working. I would like to see your solution.

alberto56’s picture

@Stalski looks like the site's owner removed the content. Here is a cached version from Google, but it might not last long.

sovarn’s picture

I did a hacked solution to this problem for my site:

http://drupal.org/node/871144

using the flickr module and imagecache I was able to download the image then use imagecache to resize it.

NicolasH’s picture

Status: Needs work » Closed (fixed)

I think the official solution to this is http://drupal.org/project/filefield_sources.

Re-open if you disagree.

zilverdistel’s picture

Status: Closed (fixed) » Needs work

@NicolasH: I can't imagine how you would do something like

    echo theme('imagecache', 'present_name', get_remote_image_from_url("http://www.example.com/projects/image1235.jpg"));

with the module you mention (filefield_sources). Also it depends on the module filefields and only works if you use cck.

Reopening ...

NicolasH’s picture

Is there a good reason why you would not want to use filefield (and in fact Drupal's core file system)? And use CCK formatters to configure your presets, then have a simple variable available that contains the output in the tpl file, like $my_image_rendered? No further theme layer logic required and no hardcoded presets.

Sure, short-circuiting/bypassing all those great systems works, but what is the advantage? You're also missing out on a lot of opportunities to keep your file system sane...

zilverdistel’s picture

I'm writing a views 3 query plugin that pulls and displays data from a REST-api. I want to re-use the imagecache theming for displaying fields with images. Since I'm not working with nodes, I certainly can't use cck.

glass.dimly’s picture

I've used the module image resize filter in certain contexts to solve this problem, but it'd be nice if this were handled by imagecache.

dan_lennox’s picture

Dumping a basic D7 version of #13

function get_remote_image_from_url($url) {  
  $external_image_path = file_default_scheme() . '://external';
  
  $image = file_get_contents($url);

  $cut_url = str_ireplace("http://", "", $url);
  $cut_url = preg_replace("/^[^\/]*\//", "", $cut_url);
  $image_name = str_ireplace("/", "-", $cut_url);

  $image_path = $external_image_path . '/' . $image_name;
  
  return file_unmanaged_save_data($image, $image_path, FILE_EXISTS_REPLACE);
}

// Later
$image = array(
  'path' => get_remote_image_from_url($remote_url),
  'style_name' => 'thumbnail'
);
$output = theme('image_style', $image);

alexharries’s picture

@swill: thank you; this function has saved me an awful lot of time, and helped me a great deal!

Many thanks!

/al

mr.j’s picture

fizk’s picture

Status: Needs work » Closed (won't fix)

Use the ImageCache External module: http://drupal.org/project/imagecache_external