Honor access limits when serving files through AmazonS3

I'm not sure the best way to handle it. I can think of a couple options:

1. Run the link-creation process through an "access check" function that does just that; you'd probably have to keep a separate table of records for each user/link combination, which gets checked with every click. This way, before a link is even built (and displayed) it makes sure that the user is allowed to see them.

2. Create links without access check, but those links go to an internal URL (like example.com/download-s3/filename.ext). This is how the Ubercart+UC File S3 modules handled it; the intermediary code would check the current user and their access limits, and block the user from downloading if those limits were reached. The "on success" function was just a redirect to a newly-generated and authenticated URL.

I'm also not certain if this is a feature request for Amazon S3 or Commerce File; I believe that Commerce File does have rules to delegate access control, but I believe those are handled through Rules. It's probably up to the Wrapper Handler (in this case AmazonS3) to handle access on a per-wrapper basis, since file systems and storage/delivery systems might need different requirements.

I'll leave it in AmazonS3 for now because I could see this being used with other systems and configurations besides just Commerce sites.

Based on our discussion at #1277152: Integration with Filefield Sources I'm re-titling this.

I'm also moving this to the Commerce File queue instead of AmazonS3. My thinking is that Commerce File probably needs a helper module or some glue code that works with AmazonS3, or even more abstract than that if possible.

Since AmazonS3 module provides merely a wrapper for handling streams, we should probably consider finding a way to honor the access control even when serving files through an external provider like S3.

+1

@destinationsound:

Yeah, because it's an external link (as the links that are created are now) the link just goes to your S3 bucket (or its cloudfront/CDN link if configured). That means Drupal is taken out of the equation.

The additional code would have to be either a conditional change to the url-creation code that first checks a user's access to those particular files - which IMO is the best solution - or you change the created URLs to be internal links which check the user's access and then redirects to the external S3 URL upon success.

It should probably force a download of the file as well rather than simply redirecting them to the S3 URL right? If somebody gets the URL doesn't it mean that they can share it with others?

@deggertson:

I think that's something that would be part of the Amazon S3 configuration. The URL that's generated includes header overrides such as a force-download.

The trick would be to give the generated URLs a short livespan (30 seconds or a minute) to prevent leeching. This is another value that's already configurable through the Amazon module's backend.

This feature would very much help me. +1

There is no issue with commerce file module here. To get this done i have created a patch for amazons3 (7.x-1.0-beta7) module for destinationsound (creator of this thread). Also sharing with everyone if it can help.

Will #11 work with files already purchased? Or does it only work with newly purchased files? I tried using this patch with amazons3 dev (2013-Mar-03) as well as the patch in #1277152: Integration with Filefield Sources and so far it doesn't seem to work. This error is showing up in the logs:

Notice: Undefined variable: bucket in amazons3_url_outbound_alter() (line 238 of /sites/default/modules/amazons3/amazons3.module).

Shouldn't all patches be applied to dev?
Should this issue be moved to the amazons3 issue queue if the patch needs to be applied there?
Should this patch be rolled together with the patch in #1277152: Integration with Filefield Sources to create a new separate module for those who need amazons3 to handle files with commerce file? Seems like we probably shouldn't be throwing all this functionality into S3 if it is only for the use case of those using both commerce file and amazons3.

I will keep testing over the next few days.

Thanks for the patch!

@deggertsen, Yes #11 patch will work for already purchased file too. I have mentioned that i applied it to beta version of this module not on dev version. But this is totally separate code so shouldn't be any issue even using with dev version. This can be in separate module, this is a good idea. Developer of amazon3 module can verify it if it could be separate module or need to be patched current module, as he can have plan to extend this code within amazon3 module.

#11 patch can be used along-with https://drupal.org/node/1277152 patch.

Thank you,
Tanzeel

I am changing module to "amazons3".

I had to make a change in one of the functions in patch #11 in order to not get the error I mentioned in #13.

I had to move $bucket = variable_get('amazons3_bucket', ''); to come before if (variable_get('amazons3_cname', 0)) {.

<?php
/**
 * Implements hook_url_outbound_alter.
 */
function amazons3_url_outbound_alter(&$path, &$options, $original_path) {
  if ($options['external'] == 1 && module_exists('amazons3')) {
    $bucket = variable_get('amazons3_bucket', '');
	if (variable_get('amazons3_cname', 0)) {
      $amazon_domain = variable_get('amazons3_domain', '');
      if (strlen($amazon_domain) > 0) {
        $amazon_url = 'http://' . $amazon_domain;
      }
      else {
        $amazon_url = 'http://' . $bucket;
      }
    }
    else {
      $amazon_url = 'http://' . $bucket . '.s3.amazonaws.com';
    }
    if (strstr($path, $amazon_url)) {
      $path_length = strlen($path) - strlen($amazon_url) - 1;
      $relative_path = substr($path, -$path_length);
      $path = base_path() . 'system/amazon/files/' . base64_encode($relative_path);
    }
  }
}
?>

commerce_file 7.x-1.x is in maintenance mode (no further changes unless a co-maintainer steps up).
There is a new 7.x-2.x branch and I'm very interested in having full S3 support in there.
In my opinion, code like the one in the patch should live in commerce_file.

I have opened #2049481: Ensure S3 compatibility and posted a patch. I believe this issue will be closable once the other one is done.
Feedback more than appreciated.

https://drupal.org/node/1395488#comment-7658191 has some of my reasoning around designing the 7.x-2.x branch.
The missing limitation methods are easily implementable with a single hook, and I will re-evaluate them as I get more feedback for the new branch.

That said, I did not understand what you mean about "because of how it is checking for the files". Can you clarify?

It's time to mark this as fixed.
commerce_file 2.x has full S3 support (redirecting to s3 right away instead of downloading to server first, hiding the bucket name, honoring download limits).
A beta1 will be tagged in the next 24h, and the only RC1 blocker is the upgrade path from 1.x.

So if you're starting a new project, we have everything you need.