Use remote images and cache them localy with image cache - 1 decent solution
bara.munchies - April 25, 2008 - 04:46
| Project: | ImageCache |
| Version: | 6.x-2.x-dev |
| Component: | Miscellaneous |
| Category: | feature request |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | needs work |
Description
Hello,
i'm developing a site that uses external pictures on other servers. pictures come from different sources and have different sizes.
i would like to pull the images and use imagecache and then safe a local copy of the image for next time the file is requested.
any idea how to handle remote files with imagecache?
#1
not a feature of imagecache. you'll have to do some coding to get that working.
#2
Were you able to figure out how to do this?
#3
This interests me too - I'm aggregating external feeds, and it would be really nice if I could just output resized copies of their images using the same theme('imagecache',...) I use for local images.
#4
This is a small patch, that doesn't respect a whole lot of the imagecache functionality: It doesn't deal with private files, and if the external image doesn't exist it will WSOD, i think.
It does however, cache the external image locally, then cache the preset version. So, it should only call externally once per image.
Patch is on version 5.1.6
To use, just use
<?phpprint theme('imagecache', 'preset_name', 'http://www.example.com/image.jpg', ... );
?>
#5
Ok, here's a better patch. It still needs some work.
Same instructions as above.
#6
Is this feature available in 6.x-2.0-beta5 ?
#7
totally not a fan of this approach. i'd be into a separate module that handled the fetching and storage and then handed them to imagecache but i don't see any reason to complicate imagecache.module with that functionality.
and it'd have to happen in the latest version.
#8
I agree, the patch i did screams of hack, and it breaks with urls with query strings etc. I am using it on a site though, with some fixes for query strings, and it's some of the best functionality ever. It's really handy.
Simon
#9
Is there a way to get http://drupal.org/project/get_image and http://drupal.org/project/remote_file to work together? :)
#10
This could be done with remote_file and cck - http://drupal.org/node/375549
#11
One solution as a "bridge" is to have your own function to fetch the image locally and then use imagecache's existing functionality.
I have an MLS search with all remote images and I want to use them with imagecache. So I make my custom module fetch the image if it doesn't exist. If it does exist it returns the filepath for my imagecache call and voila we've got it. That means the first time an image is downloaded its a bigger hit. It also means I have a file check every time I look at an image. And it means I'm going to download alot of these images on my file system. So performance may be an issue here but for now I like it. Here's my function:
function nh_search_fetch_remote_image($url) {
if (!$url) return FALSE;//fail
if (preg_match('/([^\/]+)$/', $url, $matches))//get filename
$filename = $matches[1];
else
return FALSE;
$filepath = file_directory_path().'/mls/'.$filename;
/**
* See if file exists first, if it does return
*/
if (file_exists($filepath))
return $filepath;
$ch = curl_init();
//Set curl to return the data instead of printing it to the browser.
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
//Set the URL
curl_setopt($ch, CURLOPT_URL, $url);
//Execute the fetch
$data = curl_exec($ch);
//Close the connection
curl_close($ch);
/**
* If data returned save to mls_images folder and return filename
*/
if ($data) {
if (file_save_data($data, 'mls/'.$filename, TRUE))//replace
return $filepath;
}
}
#12
Thank you for the patch, burgs!
I was looking for such functionality to get emfield videos thumbnails to go through imagecache. I created a simple module which brings imagecache display options to video_cck and applied your patch. Now remote images work fine.
After investigations I found two drawbacks in your patch:
1) error in function imagecache_create_url($presetname, $path):
if (stripos($path, 'http://') === 0){ -- correct variant, with three '='s, not two
2) paths to imagecached images should be urlencoded too, otherwise all image links are processing through imagecache's transfer instead of simply be retrieved via direct link.
Quick solution: $path = 'http/'.urlencode(str_ireplace('http://', '', $path)) in imagecache_create_url.
Full function code (DRUPAL 5):
<?phpfunction imagecache_create_url($presetname, $path) {
$path = _imagecache_strip_file_directory($path);
if (stripos($path, 'http://') === 0){
$path = 'http/'.urlencode(str_ireplace('http://', '', $path));
}
switch (variable_get('file_downloads', FILE_DOWNLOADS_PUBLIC)) {
case FILE_DOWNLOADS_PUBLIC:
return url(file_directory_path() .'/imagecache/'. $presetname .'/'. $path, NULL, NULL, TRUE);
case FILE_DOWNLOADS_PRIVATE:
return url('system/files/imagecache/'. $presetname .'/'. $path, NULL, NULL, TRUE);
}
}
?>
Needs testing though
#13
In general I hate hacking modules or core. I ran into this same issue, so I wrote the following function in one of my modules to take care of the issue. Enjoy...
<?php
// This module takes the path to a remote image and downloads it to the files directory so that you can use imagecache on the image later...
function MYMODULE_get_image($path) {
$return = "";
$base_path = file_directory_path()."/my_folder";
if (!file_exists($base_path)) {
mkdir($base_path, 0777);
}
// The next two lines are up to you. Basically, change the input path to what you want the image file name to be.
// Maybe explode on '/' and then grab the last element in the array??? Whatever works for you...
$cut_path = str_ireplace("http://www.example.com/", "", $path);
$image_name = str_ireplace("/", "-", $cut_path);
$image_path = $base_path."/".$image_name;
if (file_exists($image_path)) { // you already downloaded that image, just return it...
$return = $image_path;
} else { // download the image...
$remote_image = file_get_contents($path) or die('Could not grab the file');
$local_image = fopen($image_path, 'w+') or die('Could not create the file');
fputs($local_image, $remote_image) or die('Could not write to the file');
fclose($local_image);
unset($remote_image);
$return = $image_path;
}
return $return;
}
// Later, somewhere in code...
$output = theme('imagecache', 'present_name', MYMODULE_get_image("http://www.example.com/projects/image1235.jpg"));
?>
#14
Subscribing.
I have a Digg-like site powered by Drupal 5.12 and Drigg_5x_1.36. Users submit links. If they want they can upload an image with Imagefield, which is handled by Imagecache, but this is a very "slow" solution (users need to download an external image, and then upload it, and then there are whole lot of copyright issues). What would be absolutely awesome is when users submit a link and while they're still on the "submit" form, a query goes out to that particular URL page and dynamically brings a set of images from that external URL, which are resized by Imagefield and immediately presented for the user to choose from. We see this kind of functionality in action on Digg.com and also on Facebook when we attach a link. My question: will any of the above patches and codes provide this kind of solution???
I really appreciate your help!
#15
Thanks swill - Just what I was looking for. The code in #13 worked for me (on Drupal 6.10, Imagecache 6.x.2.0-beta8).
#16
ecofinn, thanks for confirming the code in #13 works for you. :)
#17
Hi ...
just to make it a bit more generic :
.....
// The next two lines are up to you. Basically, change the input path to what you want the image file name to be.
// Maybe explode on '/' and then grab the last element in the array??? Whatever works for you...
$cut_path = str_ireplace("http://", "", $path);
$cut_path = preg_replace("/^[^\/]*\//", "", $path);
$image_name = str_ireplace("/", "-", $cut_path);
$image_path = $base_path."/".$image_name;
.....
#18
subscribing
#19
Check out my blog post about doing this to pull in images from Flickr and run them through Imagecache.
http://marmaladesoul.com/using-imagecache-external-images
I'm thinking about making it more generic and turning it into a module.
#20
Subscribe. That is exactly what i'm looking for! :) Now, reading the thread.
#21
The code in #13 with the update in #17 is probably your best bet...
#22
Subscribe
#23
subscribing
#24
One thing to note, I have found caching remote images is too slow for my site which downloads a lot of MLS listings for TN. I ended up resizing the images in CSS bc the time it takes to download the image and cache it was too long. It might be worth it for sites where image content isn't constantly updated like it is on an MLS site.
#25
That is a very good point. The function that I wrote in #13 (with the improvement in #17) is going to be quite slow when it gets the images for the first time (especially if you are getting lots of them). If the images that you are getting well be changing all the time and you are not reusing the saved images more than 75% of the time, this approach is probably not going to work for you.
In my case, I had to find a way to get the images initially and then reuse the images that I got from then on. I needed to make sure that if an image was changed remotely, that my application what dynamic enough to see that the image had changed and update itself.
#26
Interesting to see this issue - I am currently suffering from Geek Ego (you know, that feeling you get when you've done something amazing but very few other people in the would have the capacity of understanding its greatness) because I've managed to pull this off successfully, and without hacking ImageCache or anything else. It's a kludge, and not for the feint of heart, but it works. Here's some basics on my plan of attack:
The first problem is that I wanted my images to be attached to nodes via a CCK ImageField/FileField, but FileField cannot attach files to nodes if the files don't actually exist yet. So my code (which programmatically creates nodes) attaches a blank placeholder JPEG; one white square pixel, 304 bytes. (Pre-fetching the images wasn't really an option since this code will sometimes create 20,000 nodes at a time, and each listing could have dozens of images… The ideal solution is to download them as the user tries to view them, so that the fetching is spread out over time.)
The second problem is getting the new image when it's time to view it. To do this, I created a new ImageCache action (see the imagecache.api.php and imagecache_actions.inc files) which, when triggered, checks to see if the image it's being asked to act upon is one of those placeholder JPEGs; it runs a regex on the filename and checks if $image->info['file_size'] === 304. If it's a match, it downloads the new image, replaces the placeholder image with the new image, and updates the filesize of the image in the {files} table. It then tweaks the $image object as mentioned in #623770: _imagecache_apply_action(): Explicitly receive $image by ref?.
Hopefully circumstances will let me release this and the other code involved with this module soon…