Processing Asynchronously

I think there is a modification that we can make to media mover api to make this work better which is to add a "in process" state to files which can be returned from the media_mover_api($verb) function. Basically, right now you can return true, false, or the new filepath. We can add a third option which might be MMA_FILE_STATUS_PROCESSING. By default, if this is set, I will try to catch the module, verb, and action id that is currently fired and store this so the file can be "awoken".

On your end, you should setup a notification URL which is hit by your transcoding server when the job is done. You should be able to just reference the $file['mmfid'] and do any file moving that you need to (hint, php's ssh library can make file moving for this kind of stuff really easy). I will need to make an API call that you can use with the $file['mmfid'] which will push the file into the correct state for media mover.

Let's call this function: media_mover_api_file_processing_status($mmfid, $status = true) where $mmfid is an int and $status is boolean: either error condition or complete.

Do you think that's enough to handle this?

Ok, so the way I see this is the following:

* File is harvested via some mechanism
* File goes to be processed.
* You notify your transcoding server to process the file
* You return MMA_FILE_STATUS_PROCESSING in your process function
* MM leaves your file in the processing state

* your transcoding server completes its job and pings a url or fires media_mover_api_file_processing_status()
* MM unlocks your file and pushes it to the next state.

Does that work for you?

Well the issue is the following- media mover has four defined steps, three of which files can be either in ready or running. While media mover 2 has more flexibility here, it's not going to be ready any time soon. Why this is important is the way that media mover selects files for operations is based on the file status. For example, when your configuration runs, media mover runs your harvest configuration. New files are placed into the db set to MMA_FILE_STATUS_HARVESTED. Media mover then runs the process verb, selecting all the files from the db which match MMA_FILE_STATUS_HARVESTED and your configuration ID. This happens for status = MMA_FILE_STATUS_PROCESS_COMPLETE, MMA_FILE_STATUS_STORAGE_COMPLETE as each verb is run separately.

The reason why this is important is that the only time that the process verb will select your file for processing is when it is status = MMA_FILE_STATUS_HARVESTED. While it could be altered to include additional statuses, I would need to seriously think through some of the assumptions that the API makes to see if this would even be possible.

I guess I'm unclear on why you need media mover to be checking the status of your transcoding? Why not just use hook_cron if you can't create a callback url?

Here is the modification needed for the 1.x branch to handle alternate statuses. This will allow a different module to pull the file out of the DB and manipulate it. You would just get your files with: media_mover_get_files($status)

/**
 * lowest level of the configuration run process, actually operates on the file
 * and calls all the media mover modules based on the current verb
 * @param array $file
 *   media mover file array
 * @param object $config
 *   media mover configuration
 * @param string $verb
 * @param int $status
 * @return array $file
 *   the file that has been acted upon
 */
function media_mover_api_run_config_verb_file($file, $config, $verb) {
  // set the status for the file
  $status = media_mover_api_verb_base_status($verb);
  // make sure no one else is trying to operate on this file
  if (media_mover_api_file_lock($file['mmfid'], $status, $status + 1) )  {
    // now we run the actual verb on this file
    // @NOTE not using module_invoke here b/c it doesn't seem to pass by reference correctly ($file)
    $filepath = call_user_func_array($config->{$verb}->module .'_media_mover', array($verb, $config->{$verb}->action, $config->{$verb}->configuration, &$file, $config));
    // Action was successful
    if ($filepath) {

      // If the action modified the status, we do not change the status.
      // By default, advance the status.
      if ($file['status'] == $status) {
        // Increase the file status
        $new_status = $status + 2;
      }
      // add the new file path to the file array. Some modules may return
      // true instead of a file path. If this is the case, get the current file
      // from this file
      if ($filepath === true) {
        $filepath == media_mover_api_config_current_file($file);
      }
      // we do this here to keep the api simple for module developers
      $file[$verb .'_file'] = $filepath;
    }
    else {
      // failed
      $new_status = MMA_FILE_ERROR;
      // @TODO trigger email and log alert here
    }
    // update file record with the new data and new status
    media_mover_api_update_file($file, $new_status);
    // call the event triger
    media_mover_api_event_trigger('config', 'run', $verb, $file);
  }
  return $file;
}

The status fix was committed, closing this issue