Jump to:
| Project: | Ubercart CloudFront |
| Version: | 6.x-6.x-dev |
| Component: | Code |
| Category: | bug report |
| Priority: | critical |
| Assigned: | Unassigned |
| Status: | active |
Issue Summary
I'm very close to having this working now. I'm hoping someone can shed some light. Here is what I've done so far:
- Signed up for S3 and CloudFront
- Installed CloudBerry
- Created a Bucket and created the same directory structure in that bucket as I have on my site currently
- Uploaded my files to the bucket
- Using CloudBerry. Created a distribution. Enabled the distribution. Made the distribution private.
- Edited the ACL to allow owner all access and the Origin Access Identity Read.
- Tested file access using the CloudFront URL. Success!
- At this point the files are being served by CloudFront only and cannot be accessed through S3 directly. Perfect!
- Using Cloudberry, edited the distribution to allow myself as a trusted signer.
- Using Cloudberry, created a canned policy
- Tested file access without a signed URL from CloudBerry. Didn't work. Success!
- Tested file access WITH a signed URL from CloudBerry. Works. Success!
- Install Media_Mover
- Install uc_cloudfront
- Enabled the Media Mover API, the S3 Media Mover module, and the Cloud Front Module.
- Changed permissions for Media Mover and added site administrator to all media mover tasks.
- Changed permissions for CloudFront and added site administrator to all media mover tasks.
- Entered our CloudFront settings in the Amazon CloudFront settings, admin/store/settings/cloudfront
- Tested URL generation using admin/store/products/cloudfront. Didn't work. Received "Access Denied" error.
- Discovered that spaces in the file names cause the"Generate CloudFront URLs" to FAIL.
- Changed my filenames to not have spaces.
- Tested URL generation using admin/store/products/cloudfront. These work now. Success!
- Do a test purchase in the store and try to download the file via the Files tab and get a XML page pretty much displaying a listing, including dirs and sub dirs, of my entire distribution. Didn't work. FAILED.
Any help would be greatly appreciated.
Comments
#1
#2
Can you verify something for me. I don't believe new files added to S3 through media_mover are getting the correct ACL for the CloudFront user associated with your S3 account. While using CloudBerry, go to the new file that has been added, Right click and select "ACL". This should display a list of all users in your account, CloudFront should be one of them. Give this user "Read" permission on that selected file and then go back to your store and see if the Generated URL works now.
I'm trying to figure out how to automate that extra step of setting CloudFront user permission when generating the URL. Not sure if it needs to be done when media_mover moves the file over or simply when the Signed URL is generated.
#3
Dave,
The Media Mover module, as I am sure you know, only has the ACL options of "Private" "Public Read" and "Public Read and Write" for options when moving a file to S3. There is no "Inherit" which would make more sense.
To test...
- I have a separate directory for Media Mover files. I placed a file in that folder.
- I executed a Media Mover config that does nothing but move my file to my CloudFront distribution and then generate the CF file name. Media Mover is set to ACL Private
- I then put the file I am selling in my Ubercart store download directory.
- I created a product for sale that linked to that one file.
- As an admin, I added the file download to a testing user through the admin interface.
- Logged in as test user and went to my Files tab. click on the file link
- Same result, just an XML "dump" that is a directory listing of my distribution including dirs, sub dirs, and file names.
- Check ACL. Indeed only the owner has permissions
- Change ACL to include CloudFront having read permissions.
- Tried to download the file again.
- Same result. XML dir dump
So, the problem doesn't seem to even be ACL or we'd get an Access Denied error. The File links generated for the customer's "Files" tab simply do not link directly to the file sold.
If you want, I can send you the XML file generated.
Thanks very much for your help. You guys rock.
#4
Dave,
Can you ellaborate on how the CloudFront module interacts with the Media Mover module? Does CloudFront rely on the Media Mover file tables? It doesn't seem like it would but... what do I know ;-)
*IF* it does rely on the MM file table would it be possible to write the CloudFront module to not rely on that table?
Again... thank you for all your efforts.
#5
Customer pinged me on Skype so I checked a couple logs.
I did find these couple errors (there are duplicate errors but here I am only cutting & pasting one of each) in the user's PHP error_log:
[24-May-2010 XX:39:49] PHP Fatal error: Call to undefined method S3::checkKeys() in /XXXXXXXXXXXXXXXXX/sites/all/modules/media_mover/contrib/mm_s3/mm_s3.module on line 141
[01-Jun-2010 XX:00:18] PHP Fatal error: Cannot use object of type stdClass as array in /XXXXXXXXXXXXXXXXX/sites/all/modules/mediafront/mediafront.module on line 710
Steve
#6
JUST got this working. This module relies HEAVILY on Media Mover. Media Mover *must* be the interface to move your files. Anything else will not work. I wish that had been explained in much more detail. But alas...
The module, uc_cloudfront, *still*does not work with files that have spaces in the file name even though Media Mover handles moving the files. This isn't that big o' deal for us. I have a larger bug report that I posted in a different thread that is much more of a concern.
The ACL problem is still there too. But that is easily remedied, although manually, via CloudBerry for us.
#7
I ran into this issue today with filenames containing spaces. Note that Media Mover is not the culprit! (We don't even use it - we upload everything to S3 and then add it to our uc_file_s3 files table with a custom hook).
The Cloudfront integration on our site was made possible by borrowing some of the signature-generating code from this module and putting it in a standalone .include file for serving our large files through Cloudfront. I did this mainly to avoid adding another module to the site, when I really only needed a couple functions - and everything on the AWS websites seemed to fail. In this case, everything worked fine except when dealing with filenames containing spaces.
The issue mainly appears to be with not removing whitespace, which should be done, but isn't by default, and replacing that character with its urlencoded equivalent, %20. Here's my hacked version of the uc_cloudfront_sign function:
<?phpfunction uc_cloudfront_sign($policy, $private_key, $remove_whitespace=false) {
if($remove_whitespace) {
$policy = preg_replace('/\s/', '%20', $policy); // Instead of replacing with '', replace with '%20' otherwise
// spaces will cause this signature to fail!
}
$policy = utf8_encode($policy);
$binary_signature = '';
$pkeyid = openssl_get_privatekey($private_key);
openssl_sign($policy, $binary_signature, $pkeyid, OPENSSL_ALGO_SHA1);
openssl_free_key($pkeyid);
return uc_cloudfront_urlsafe($binary_signature);
}
?>
After making this small modification (and setting $remove_whitespace to TRUE when I call it in my module), files who have spaces in their names work without any issue. Hope this helps someone!