Update emthumb to work with Drupal 6 / CCK 2

thanks, that's an excellent idea. when i first wrote emthumb, i was looking for some kind of image api. filefield/imagefield might be the best solution for that.

This sounds like a great idea.

aaron- given the current problems that emthumb is causing do you think I should mark it "not supported for Drupal 6.x?" in the info file?

alex, that's probably a good idea for now. would help people know until emfield is ready for an rc

Done.

I'm going to refactor emthumb to simply grab the image from the provider if available, and allow editors to override that. It will also take care of the feature request at #187555: Create local copy of image for Imagecache and ThickBox integration. Working on it today.

To integrate with FileField/ImageField, looks like for starters we can reuse the 'imagefield_widget' element defined by imagefield_elements(). That should knock out a lot of the code duplication we have going on.

I'm going to have one of our new developers work on this in the upcoming week. I'm hoping to have this taken care of by DrupalCamp Philly...

I've done a little bit of work in this direction, but it seems to be a big job to integrate it with imagefield. Not sure if this is the best direction.

I'm going to commit what I have so far, but I think we should start a wiki to plan this out better. Would be nice to have hook_file; this would be much easier in d7.

Things that will need to happen:

1) Respect data from d5, either using the same data structure, or migrating old data.
2) Create the image upload widget.
3) Save the file info from the uploaded image.
4) Allow fields to be displayed with imagecache: imagefield gives this, but may be too tied to cck (needing its own field) to be useful.

Extra credit:
5) Fetch the provider's thumbnail to store here, UNLESS/UNTIL it's been overrided with emthumb. If the image from emthumb is deleted (and not replaced manually), replace with the provider's fetched thumbnail again.
6) Whether step 5 happens or not would be controlled by the admin.

note i added a test widget in the dev version to begin imagefield integration. not sure if this will work; may need to roll back changes. if you're testing & following along, you need to change emthumb.info appropriately (included dependency of imagefield).

I'd actually like to see this get developed in slightly different direction. What I'd like to see first is for emfield to cache (i.e. download) thumbnails + emimage images and plugin to imagecache (which I suppose doesn't require imagefield), then later add in the ability to add ones own images. We've done work with imagecache that doesn't require imagefield, so I don't think that should be an issue, but one issue will be whether or not we utilize filefield's api. My inclination is that we should, since I'm assuming there will be an upload path from filefield to hook_file (maybe not the best assumption). I'm personally not against requiring filefield, since we could keep the old thumbnail functions in place for a lighter weight version of the module, if this seems like the right direction to move in.

As far as 1 goes, for non emthumb images this obviously won't be a problem, since we've created that refresh embedded media button, but the ability to upgrade will definitely be a sticky issue.

There's currently a feature request for FileField Upload files from a URL (Transloading/External Upload). Drewish marked it as "won't fix" and recommended the mediamover module but I was pushing for the feature because I think it's very useful. I think it'd be really helpful if we could pass the images URL and filefield would do the rest?

there are a bunch of new image api functions built into drupal that we need to tap into, such as file_validate_is_image, file_validate_image_resolution, and maybe even image_scale_and_crop for the preview thumbnail.

owen, extra credit if you want to type in some of the notes you took at drupalcamp philly about the emthumb roadmap here, so we can all follow along ;) a lot of it was stream of consciousness for me...

thanks,
aaron

first off, the hook_menu callback for preview needs to be revamped. as hook_menu is no longer called on any page callback, we'll need to move most of that simply to the callback.

probably something like:

$items['emthumb'] = array(
  'page callback' => '_emthumb_preview',
  'access arguments' => array('TRUE'),
  'type' => MENU_CALLBACK,
);

and then

function _emthumb_preview() {
  if (!is_array($_SESSION['emthumb'])) {
    drupal_not_found();
  }
  foreach ($_SESSION['emthumb'] as $fieldname => $files) {
    foreach ($files as $delta => $file) {
      if ($file['preview'] == $_GET['q']) {
        file_transfer($file['filepath'], array('Content-Type: '. mime_header_encode($file['filemime']),
                                           'Content-Length: '. $file['filesize']));
        exit();
      }
    }
  }
}

This still doesn't address the multiple thumbnails issue that still isn't resolved in 5, but at least it should get us started.

$_GET['q'] is the entire alias. we'll probably want to just change that to be emthumb/[md5hash] or something, and use arg(1) instead. just thinking off the top of my head. i don't remember right now how $file['preview'] is checked anyway.

Aaron,
Some things we talked about yesterday. correct / clarify these points since they might be vague:

Create an implementation of hook_elements() that specifies callbacks to handle the functionality for creating thumbnail upload -- including checking for existing file, file validation, etc. (we looked at some validation functions in the drupal API that might be useful)

This will change several functions in emthumb module: stuff in emthumb_emfield_field_extra() and emthumb_emfield_widget_extra() will be changed to hook_element() callbacks. Also _emthumb_widget_form() that actually builds the form can be called from hook_elements() , no ?

Using hook_elements() also means the theming function will need to be changed (or renamed at least).

I worked a little at this while still learning how all the pieces interact.....

-owen

subscribe

subscribe, assuming this is to enable "local copy" functionality.

Note I've posted a similar request to Flickr module: http://drupal.org/node/325599

Personally don't care about sync'ing photo sets, one time download's fine IMO

+1, I like the idea of using FileField so users can set a custom thumbnail if the default one sucks.

Hi Aaron ,
I've been trying to use filefield to implement emthumb functionality, similar to imagefiled. Bascially trying to delegate as much as possible to filefield, overriding with emthumb stuff where necessary. trying to do something like this:

/**
 * Implementation of hook_elements().
 */
 
function emthumb_elements() {
  $elements = array();
  $elements['emthumb_widget'] =  array(
    // Indicate to FormAPI that this element needs processing and is not simply a render element.
    '#input' => TRUE,
    // specify the return structure of the element (experimental, unused in releases).
    '#returns' => array('array' => array('fid' => 'int', 'title' => 'string', 'alt' => 'string')),
    // Delegate element processing to filefield. Filefield will include
    '#process' => array('filefield_widget_process', 'emthumb_widget_widget_process'),
    // See emthumb_widget[#process] documentation.
    '#value_callback' => 'filefield_widget_value',
    // delegate to filefield...
    '#element_validate' => array('filefield_widget_validate', 'emthumb_widget_validate'),
    '#description' => t('Changes made to the attachments are not permanent until you save this post.'),
  );
  return $elements;
}

/**
 *  Custom Thumnail, add settings, delegate to file field
 */
function emthumb_emfield_field_extra($op, $node, $field, &$items, $teaser, $page){
  
  $element = filefield_field($op, $node, $field, $items, $teaser, $page);
  
  $element += .....;
    
  return element;
}  
 
/**
 *  This provides extra widget settings to emfields.
 *  A checkbox to allow custom thumbnails, max resolution, image path, allow custom alt tags, allow custom title tags.
 *  add emthumb widget settings, then delegate to file field 
 */
function emthumb_emfield_widget_settings_extra($op, $widget){
  
  $element = filefield_widget_settings($op, $widget);
  
  $element += .....

  return element;
}

/**
 *  when editing a node with an emfield, this will add our custom thumbnail upload form if allowed.
 *  Assign default properties and delegate to filefield.
 *  emfield.cck.inc invokes this function, but does not yet pass it the right parameters,
 *  so it will have to be changed too ? 
 */
function emthumb_emfield_widget_extra(&$form, &$form_state, $field, $items, $delta = 0){
  // add default values to items.
  if (empty($items[$delta])) {
    $items[$delta] = array('alt' => '', 'title' => '');
  }
  
  $element = filefield_widget($form, $form_state, $field, $items, $delta);  
  
  return $element;
}

Does this seem like the right direction ? obviously missing parts... still a little confused by usage of hook_elements()

actually, taking the imagefield approach, it works a little differently.
for example with emthumb_emfield_widget_settings_extra Everything should be delegated to filefield, and filefield then invokes callbacks in a new file emthumb_widget.inc that add needed functionality....

@HansBKK - it will only refresh your images if you click on the new "refresh embedded media data" button. Think of this like a locally cached version. I think we're going to try and give an option to refresh thumbnails as well, since I could see a scenario where the data needs to be updated but you don't want to remove your custom thumbnail. Also- we'll be doing the same thing with emimage as well so that you can treat flickr images as if they are on your site (well, they are on your site, so...), meaning you can have imagecache support and the like.

Also- oweno has started committing his changes to the dev version, so definitely take a look and lend a hand if you can.

I probably don't understand this whole situation well enough, but as I'm reading this thread and the one at http://drupal.org/node/187555, I keep thinking that a solution that integrates emfield with imagefield would be a more manageable solution.

Many modules integrate into ImageField, not just imagecache. Lightbox2 jumps to mind, as well as imagefield_crop, imagefield_gallery, and more... This means that the bridge module would not have to handle any of this integration code itself, making the module much leaner. This would leave more time to focus on advanced features like syncing.

Perhaps it would look a little clunky to have both and imagefield and an emfield on the same node. The bridge module could theoretically hide the imagefield on the node editing form. ...? But I also Like the idea of having an Image node type that can either be used to pull images from a provider like Flickr OR upload a new image through the use of an imagefield, and not have an associated emfield entry.

emfield_2_imagefield (the bridge module) would be responsible for determining the url of the highest resolution source image (original/large/etc.) that could be imported from the provider website (Flickr) and it would save it to the web server in the appropriate directory (each instance of an imagefield lets the admin specify where the uploaded files will be saved. Couldn't we just use this directory as the location into which to store these files?).

Wouldn't this be much less work to build and to maintain? What am I missing?

just in case it's relevant, linking #345234: Illegal Offset Type Error here. i'm looking at the current status of emthumb again right now.

MikeyLikesIt, the issue I've run to conceptually and in practice is that ImageField/FileField creates a separate field. We could simply require adding a field and bridging them, which adds an extra step in configuration by the administrator, which is far from ideal. Additionally, we then also run into the problem of how to synch multiple fields together, adding another layer of complexity for the editors, and likely dozens of repeated support issues in the queue.

I do, absolutely want to tie in as much functionality from ImageField/FileField as possible into the module. However, perhaps Drupal 6 is too early for this functionality. Once we have fields in core and PHP stream wrappers, we can simply store the provider data as a stream/file object, and act on it with hook_file to display a [EDIT] local image. Or maybe even have a field on a field, if that's possible. Also, there's a planned media code sprint next month, which has as one of its stated sub-goals to port the hook_file patch from d7 into d6. Although we still wouldn't have stream wrappers or fields on files, we may be able to tap into that new module as well.

here's a proper patch from the start in #20 to get things rolling. i hope to work on it tomorrow. thanks owen!

btw, there's a planned media code sprint next month, with one of the possible goals to backport hook_file to d6 (as a new contributed module). that might be a better candidate than filefield in the long run.

here's my latest iteration. rolling imagefield into the mix as well.

starting on the #element stuff for field forms

Made some more progress. Using the new emthumb form element, and added a theme function. the forms all show proper values now. next is to make sure everything works and gets tied in properly.

Added the field & delta info to the element. The file is now uploading properly, according to debug messages; just need to make sure it stores the info in the proper place.

I've been wrangling this thing for awhile now. A big issue that I've only realized today is that filefield hijacks and discards the field's data entirely. Definitely not a play-nice API. Still working on the issue; might want to scrap filefield dependency.

should have this done very soon. refactoring to use File (hook_file). hook_file_save_upload makes things much, much easier for us.

Done! Committed to the dev version.

I'm a little worried about multiple fields, and about the hackish bit (legacy) from about line 23. Please try out the module and report any bugs to new issues.

Took out all dependencies from filefield and File (hook_file). Seems to work fine for me, at least with single fields. Haven't tested w/ multiple fields yet.

Thanks,
Aaron

For anyone following here, emfield's been released as a beta now too.

Could this technique be used with emvideo to save a local copy of the video providers thumbnail, like youtube. Currently the thumbnail is requested on every page load which can hang a page that has lots of video thumbs... I'm sure youtube would appreciate the reduction in unnecessary requests also.