Upgrading the Media module from beta4 to beta5 (specifically, immediately after performing the database updates) breaks private downloads. The file download is skipped by file_file_download, but image_file_download no longer recognised the files as downloadable by image.module and passed the download back to file_file_download, which skips the download again, leading to a "Page not found error".

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

mesr01’s picture

This may be related: I have a site under development with download method set to private. When I install Media 7.x-1.0-beta5, the images I upload won't show up at all (either in media browser or as node). If I switch to public download, everything works well. Media library is empty, so it's not about broken links.

gboubou’s picture

I can confirm this issue. I have a similar development setup with download method set to private and the images do not show up anymore since I upgraded to Media 7.x-1.0-beta5. I'd be happy to help debug more this issue, but I'm not sure where to start.

mpotter’s picture

Priority: Normal » Major

I am having this same issue. This is a serious site-breaking issue for me. Anybody have any workaround before I start diving into the code myself?

It's not just images. Applies to any media that I upload to the site (videos, MP3, etc). Files are still getting uploaded to the sites/default/files_private directory. But trying to play or download the files does not work.

I seem to recall in the previous version that I had to edit the File field to set it to Private. Now that we have the File entity instead, there doesn't seem to be any setting for this in the Media settings. So something must have gotten overlooked. So many changes between beta4 and beta5 that it's been tough to debug.

fkargl’s picture

Just to keep the topic alive: same problem here, Drupal 7.4 and Media Module 7.x-1.0-beta5.

And I can also confirm that the interrelations between components makes it hard to debug ;-)

wizian’s picture

Title: Upgrade to beta5 breaks private downloads » Media declared private no longer visible: Site broken

As #1 all private stored images are not being rendered in media or in nodes, public images are ok. In the db table "file_managed", the files URI are declared as public://.. and private://...
Has anybody got an idea where I can tweak to get the stuff visible again.
Using D7.7, media 1.x-beta5, entity 1.x-beta10,

ropaolle’s picture

Same issue with D7.7, media-7.x-2.0 and file_entity-7.x-1.x-dev.

Vladimir Pasashnikov’s picture

It looks like that module doesn't provide file_download hook or doesn't update references to a image that File module could use.
In the last case file_file_download is looking for a references to an image in the field_data_field_image table, but Media module doesn't use it, as beta-5 is using file entity and it uses file_managed table instead.

I'm not sure what is a proper way to fix this issue, either add a hook or add a proper reference into field_data_field_image.

waynedpj’s picture

Version: 7.x-1.0-beta5 » 7.x-1.0-rc1

ahoy all,

here with Media 7.x-1.0-rc1 + Drupal 7.8 and same deal: files uploaded with private access via Media are not accessible, whether via thumbnail/original/etc. the files are uploaded to the private directory correctly, but an "access denied" is created in the Drupal log whenever access is tried.

note that public access works fine, as well as private access with non-Media uploads (e.g. Image field in an Article).

i would be happy to help with testing/debugging, though i am unfortunately not a PHP hacker.

thanks for the great module, looking forward to using it once this is fixed.

peace

Dave Reid’s picture

We shouldn't need to add a media_file_download() - there shouldn't be anything restricting the files. I think the problem is higher up and needs to be properly investigated by someone who normally uses private files and has the time. Possibly related to file usage not being properly assigned?

discipolo’s picture

the only thing mentioning references and file_usage is http://drupal.org/node/1222576 and i cannot figure out whether its related or what it is i am looking for. trying again with a clean install to avoid media_update_7201 which mentions above issue. why dont we need file_download anyway? and what else can cause access denied errors?

looking for infos on file_references i stumbled across http://drupal.org/node/993082#comment-5101440

robby.griffin’s picture

I don't understand why we wouldn't need to use hook_file_download. It seems to be the preferred, and perhaps only, way to deal with private files. It isn't required for public files, but it seems like we should use it here. This issue with media and private files is a real headache for me since I can't use a public scheme and I don't have experience patching drupal modules to know how to diagnose this issue.

fkargl’s picture

Version: 7.x-1.0-rc1 » 7.x-1.0-rc2

As we're nearing 1.0, I wonder if there is any progress on this. Setting file up/download to public is a nasty workaround and it would be nice if one of the Media module developers could look into this.

waynedpj’s picture

ditto fkargl's comment: though i am not a programmer and unfortunately cannot help in that way, this module is so useful and important for Drupal. this bug is a real show stopper. thanks again.

peace

CamroS’s picture

Hi folks!

I am facing the same problems when using private files. And I agree that it is sad, this module is not working the desired way, as it is such a fundamental one! But what is even worse, there seems to be a lack of documentation on how it is supposed to work. After playing with this module for some time, I still can't figure out the dependencies to other modules involved.

First of all, here are the modules I am using at the moment:

  • Core 7.7
  • media 7.x-1.0-rc2
  • File Entity 7.x-1.x-dev
  • Content Access 7.x-1.2-beta1
  • Media Gallery 7.x-1.0-beta7
  • Colorbox 7.x-1.2

As mentioned in former posts, everything works fine, when using public files.

Now for my questions:

  • Under Home » Administration » Configuration » Media » File System I may define two directories, one for the public, the other for the private files. Why do I have to select a default download method?

    As public and private files are kept in separate directories, why can't public files be delivered by the server and private files by drupal simultaneously?
  • When uploading media, I would have expected being able to choose the public folder or the private folder as destination. Actually the destination is controlled by the default download method! Uploading media, while the option "Private local files served by Drupal." is set, stores the files in the private folder! Otherwise media will be stored in a public directory. Is this the way it should be?
  • Uploaded public files are stored in drupal-home/sites/default/files...
    When editing media, public images are shown and can be accessed by http://drupal-home/sites/default/files/image.jpg.
    Private files are stored in drupal-home/sites/default/files/private... in my system.
    Editing private media, the image is not displayed. The linked URL is drupal-home/system/files/image.jpg and results in "Page not found". Accessing private files using http://drupal-home/sites/default/files/private/image.jpg is blocked by .htaccess. So far, so good.
    But...
    Using media gallery module, the images are transfered to specific directories containing formated images. Those can be accessed by URL http://drupal-home/system/files/styles/media_gallery_large/private/image.jpg, bypassing any access control :-(
  • Furthermore, when editing media (public or private) drupal is telling me:

    Notice: Use of undefined constant FILE_TYPE_NONE - assumed 'FILE_TYPE_NONE' in media_page_edit() (line 31 of public_html/drupal-7.7/sites/all/modules/media/includes/media.pages.inc).

    Is this related to the problems with private media?

I have no experience in developing drupal modules so far. But I am quite familiar with programming (C, Java, Perl, PL/SQL). So... If someone could give me a hint, I might be able to help.

CamroS’s picture

Hello all!

I have done some debugging and found out, that in my case (see #14) ckeditor-module denied access to the files! As ckeditor is not mentioned in the issue summary, it is not for sure that this comment is really related to the actual problem.

But for those who would like to get some more informations:

When switching to private default file scheme, uploaded images might not be displayed because ckeditors global settings (admin/config/content/ckeditor/editg) restrict access!
You have to check "Enable access to files located in the private folder " in order to make images visible.

Unfortunatelly, regarding the remark for this option, this seems to be inconsistent with the original intent of restricting file acces via direct URL:

Use this option with care. If checked, CKEditor will allow anyone knowing the URL to view a file, if it located inside of the private path (sites/default/files/private) and if there is no information about the file in the Drupal database.

So this might not be the final solution :-(

To be continued...

fkargl’s picture

Version: 7.x-1.0-rc2 » 7.x-1.0-rc3

I can reproduce this bug without using CKEditor at all (uploading images via Content -> Media -> Add Image.

It would really be nice if anyone from the Media module developers would feel responsible for this severe bug.

Or has anyone found a solution already?

7.x-1.0-rc3 does not fix it.

bsuttis’s picture

Dealing with the same issue. I wanted to note that this error is logged when attempting to view a private file.

Warning: Cannot modify header information - headers already sent by (output started at ..\includes\common.inc:2607) in drupal_send_headers() (line 1243 of ..\includes\bootstrap.inc).

stijndm’s picture

One of the issues I found so far:

Files uploaded with the media field are stored in the file_usage table with the module set to media.

When you try to load/view the file the callback file_file_download is called. This function takes to parameters: $uri and $field_type (default is file)

The first (second actually, this first is a check if the file exists) thing this callback does is check for references, if no references are found everything stops and access is denied. If references are found the necessary hooks are invoked (hook_file_download and hook_file_download_access).

The issue is in the reference check: our record is stored with the module set to 'media', but file_file_download is looking for a record with the module set to 'file'. No reference is found, everything stops, access denied.

I didn't have time to investigate further, but I think it's obvious one of the two following problems occur:
- the wrong module is passed to file_file_download (should be media, if I hardcode media everything works)
- no module is passed to file_file_download (fallback is the default 'file')

On a side note: it is not up to media to implement hook_file_download or hook_file_download_access, unless media starts provided files access controls. These hooks are preserved for modules wanting to implement access rules on files. By default, if the user has access to the parent node/entity, the user has access to the file. If you implement node access (through modules like Node Acces, Taxonomy Access or even Organic Groups), files liked from a node will inherit the same access settings as the node. (Sadly this does not count for files linked through textareas, WYSIWYG)

stijndm’s picture

Minor correction to the above. file_file_download is called in file_download. It is the latter that invokes hook_file_download. hook_file_download_access is called in file_file_download.

I also noticed the following;
- file_download invoke hook_file_download (and thus file_file_download) without passing a type, which makes sense because it only knows the uri
- file_download is looking for an explicit deny (return -1)
- since file_file_download returns nothing after it didn't find a reference it continues on to the next implementation of hook_file_download

This is just thinking out load, but could it be possible this is (partially) solved by doing the following:


function media_file_download($uri) {
  return file_file_download($uri, 'media');
}

stijndm’s picture

Ok, so the above let's us fall through to the real access check, but this time the check stops when the complete reference entity is loaded. The file access extracts all file items from the entity and then looks for the provided uri. Media widget only loads the fid, so no uri is found, so access is denied.

Adding the following to the media widget solved the problem, making my files accessible as expected.

function media_field_load($entity_type, $entities, $field, $instances, $langcode, &$items, $age) {
  file_field_load($entity_type, $entities, $field, $instances, $langcode, $items, $age);
}

These two additions seem to be a valid solution for having private files work with the media widget.
Image module seem to do something similar. It first checks access to the image styles directory, if all is ok file_file_download($uri, 'image') is called.

I suppose the media module might want to add some of its own logic, or maybe invoke a custom hook in media_file_download. Allthough I don't see a real benefit in that, other than maybe some permission check for viewing media. But this can easliy be done in hook_download_access as well. (I believe this might be more something for the media browser plus module which can have access control on folders etc.)

I'll see if I can provide a patch on the latest dev.

crifi’s picture

Version: 7.x-1.0-rc3 » 7.x-1.x-dev

I can confirm, that this fixes the issue. Thanks to stijndm! I can create the patch if you need help. But I think it's your privilege to do that. ;-)

stijndm’s picture

Patch for 1.x and 2.x. I noticed that the mediafield in 2.x has been deprecated in favor of the file field + media browser, so I'm not sure how relevant the patch is though.

stijndm’s picture

A side note: the patches above only fix the visibility of media attach through a media field to an entity. New files, without usage in the site, still don't show up in the media browser, and most likely won't show up when used through the wysiwyg. This is the next step to investigate.

crifi’s picture

Status: Active » Needs review

This needs review now.

Status: Needs review » Needs work

The last submitted patch, media-2.x-private-files.patch, failed testing.

Dave Reid’s picture

ursula’s picture

I ran in to a similar issue with the media_gallery module, and posted about my findings in the media_gallery issue queue (http://drupal.org/node/1274464#comment-5878638) and drupal core (image project) (http://drupal.org/node/1438940#comment-5878636).

crifi’s picture

@stijndm (#23) The second problem is caused by a bug in Drupal core #1414990: Orphaned private files can not be accessed.

Taxoman’s picture

More practical links to the issues mentioned in #27:
Media_Gallery issue: #1274464: Thumbnails are not created and displayed when using private file system
Drupal core issue: #1438940: Private stored default images give access denied

Matt Lewis’s picture

Status: Needs work » Needs review
Issue tags: -D7 stable release blocker

#22: media-2.x-private-files.patch queued for re-testing.

Status: Needs review » Needs work
Issue tags: +D7 stable release blocker

The last submitted patch, media-2.x-private-files.patch, failed testing.

Devin Carlson’s picture

Status: Needs work » Fixed

I believe that this should have been fixed by #1420812: Files from private stream cannot be downloaded.

Automatically closed -- issue fixed for 2 weeks with no activity.

yannick.C’s picture

22: media-2.x-private-files.patch queued for re-testing.

The last submitted patch, 22: media-2.x-private-files.patch, failed testing.

nerdoc’s picture

Issue summary: View changes

Hm, this thread is now 2 years closed - but I'm using Media 7.x-2..0-beta1, and it does not seem to be fixed. I have the usual config: pivate file system, images uploaded via Mmedia module into private directory, the links are like this:
http://example.local/system/files/screenshot.png

Additionally: the "view private files" permissions are set like this: anonymous users cannot see private files, authenticated users can.

This seems a bit weird:

  • If I enable anonymous users to see private files, they see the pictures. But this seems weird as Drupal warns me at the permission settings page about not letting non-trusted users have this permission. And then, with the files/ link, anyone has access to this.
  • OTOH, if I restrict to authenticated users, what I thought is that the view permissions of an attached file entity (even embedded via CKEditor etc.) should be inherited from the node. And then, if the user can see the node with the embedded image, he should see the image as well.

Am I completely wrong, or is this bug unfixed in Media 2.0?

davemurphy’s picture

subscribe

CKIDOW’s picture

Subscribing

BiigNiick’s picture

i have the same issue as well. it seemed to be working up until last week... files stored in the private file system return 404 or 500. the media browser in ckeditor just shows permission denied instead of browsing available media.

media 7.x-2.3
drupal 7.54