After receiving a postback from Zencoder the encoded video is deleted by drupal from my bucket on S3 - and it shouldn't be!

To confirm this, I identified the zencoder job number for this issue, logged into app.zencoder.com and grabbed the incoming API request (from drupal to zencoder) which created the job. Then I removed the following lines and ran the request again manually through curl (i.e. keeping all other values in the API request exactly the same):

    "notifications": {
      "format": "json",
      "url": "http://www.example.com/postback/jobs"
    },

Then I took the new job number and checked on Zencoder that the encoding was done, and looked again on my S3 bucket - and the encoded video file was now there.

I also spoke with Chris W. in the Zencoder campfire chatroom and gave him my original job number. He confirmed that Zencoder had got a response back from Amazon saying that the video was successfully uploaded there - but when I checked, it wasn't there. He suggested that the file was deleted after my site received the postback. Running the above check proved him right.

This is odd!

There are 2 entries in watchdog regarding the postback:

"Postback received from Zencoder for fid: 250, Zencoder job id: 30330531." and "Set Video Testing S3 to published."

Then I changed my AWS key and secret key values at admin/config/media/awssdk to nonsense values, and submitted the video to Zencoder again for re-encoding (via drupal), and lo and behold, the videos are now showing up on S3! Hooray! So it's definitely drupal that's deleting the videos on a postback.

I'm using Zencoder API client interface version 2.0.2 from http://github.com/zencoder/zencoder-php (because zencoderapi doesn't seem to recognise the latest 2.1.1 version), version 7.x-1.0-beta6 of amazons3 module (7.x-1.0-beta7 is latest), awssdk 7.x-4.8 with version 1.4.8 of the Official PHP SDK for Amazon Web Services.

The reason for the slightly outdated versions is that this is what the Octopus drupal distribution ships with, and they claim that everything on their disto works (and in using it, everything does - except for this one thing!). I'm open to updating them, except that I think the issue is somewhere in TranscoderAbstractionFactoryZencoder.inc as opposed to one of those modules (since none of their code gets called on a postback).

I've looked through function processPostback() in TranscoderAbstractionFactoryZencoder.inc and don't see anything that could be doing this. Where else could I look?

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

burningdog’s picture

Title: Video immediately deleted from s3 when receiving postback from Zencoder » Drupal deletes video from Amazon S3 on receiving postback from Zencoder

Better issue title.

burningdog’s picture

Status: Active » Closed (works as designed)
FileSize
37.89 KB

I'm happily feeling stupid. This is a feature, not a bug, and it's because I didn't understand the following workflow.

You can set a "final destination" for the encoded file, so that the workflow is:

1. Ask Zencoder to encode the file and put it on, say, S3.
2. Receive the Zencoder postback and then move the file to its final destination.

So S3 can be used either as an intermediate location (in which case the final destination is the drupal files folder) or a final destination. These settings are now per-field (thanks to #1621600: Where did the 'Output file base' field go? I should've read the release notes for the 7.x-2.6 release), like so:

Video settings field

Jorrit’s picture

I am sorry that this caused confusion for you. I have noted on the Video module homepage that I am at this moment very busy, and that is the reason that I didn't reply before.

It is good to know that the module works as intended, although it could be a little more clear to the administrator. I hope other people with this problem will find this issue before they start pulling hairs out of their head.

burningdog’s picture

Status: Closed (works as designed) » Needs review
FileSize
1.42 KB

Thanks Jorrit. One thing that could have been clearer is an explanation of "final destination" in the "Location for Zencoder output" section in admin/config/media/video/transcoders - I hadn't come across that term before and didn't know what it meant. If I did, I would've known to edit the settings in my video field.

I've attached a patch that simply adds the following line to the description of that field:

"The final destination is set per video field and defaults to the files folder."

Jorrit’s picture

Status: Needs review » Fixed

Thanks, I have committed the patch.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.