Hi.

_

I submit an URL (for a YouTube video) on the "Web" tab.
That URL is already in the library.

_

I then get this error: See attachment

_

I have to questions about that:
- Shouldn't it just take the one from the library and not bother me?
- If it should indeed notify me about that: Can the API be changed there?

_

We have an issue over at Feeds: #1032792: Mapper for Media

_

If a call to $provider->validate() fails, I can't detect if it was an error or if it was because of the existing file in the library. It would be cool if you could
a) say, that the validate() method should not fail, if dupes occur. Instead just reuse the file.
b) create a dedicated exception type for that case. (Might be the easiest one)
c) split validate() to validate() and validateDupes() or something.

_

Currently this only is an issue with media_youtube, but this should be cleared up in general.

_

The relevant code:

class MediaInternetYouTubeHandler extends MediaInternetBaseHandler {

  // [...]

  public function validate() {
    // @todo Media module currently fails when two files try to have the same
    //   URI, so catch that in the validation step. Some day, it would be nice
    //   to allow it, however. #952422.
    $uri = media_youtube_media_parse($this->embedCode);
    $existing_files = file_load_multiple(array(), array('uri' => $uri));
    if (count($existing_files)) {
      throw new MediaInternetValidationException(t('You have entered a URL for a video that is already in your library.'));
    }
  }

  // [...]
}

_

-- Kind regards, Niklas

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

JacobSingh’s picture

Title: Shall MediaInternetBaseHandler::validate() look for duplicates? » When pasting the URL of a file which already exists in the media library, media_internet throws a validation error
Category: support » bug

Yes, this is a bug, thank you for reporting it so well.

I'd be in favor of just re-using the existing one. can you think of any reasons why this would be a bad idea? One issue which will arise is when you have access control issues. Right now, we have no access control, but if we institute a "use my media only" or something and someone else has already added it to the files managed table... we will have a nasty issue on our hands. This is a core bug in a way because the uri field is a unique key :(

Thoughts on this?

JacobSingh’s picture

Priority: Normal » Major
Niklas Fiekas’s picture

Well, URI is unique ... that is defined by Drupal core. That is the meaning of uniform resource identifier. It could be changed from (uri) to (uid, uri) or (rid, uri) or (new group id, uri). Or it could stay (uri).

solution altering or touching core required other thoughts
(new group id, uri) The unique index on (uri) must be altered to a lookup index. A new table to join or a new schema field would have to be created. This solutions looks random and it wouldn't reuse anything existing. A completely new UI and so on would have to be created. On the first glance I don't think it's good. I didn't give it a secound glance though.
(rid, uri) A user can have multiple roles, so the schema must be altered. A new table to join or a new schema field would have to be created. (uri) can't stay unique. Also the user would have to decide: I am creating a file. To what role will I attach it?
(uid, uri) Alter unique index on (uri) to (uid, uri). This looks relatively easy to do. But media that is available to multiple users would have to be duplicated. Something like groups would be hard to do.
(uri) - Add - whatever will be necessary - to the local URI. schema://v/any-group-token/file-id or so. This looks most flexible to me. Random Ideas: If the file already exists (and the user has access to it) ask the user which file to reuse or if he want's to create a new file. Organize the files in folders (looks like a job for taxonomy?). Permission per folder / tag (managed by plugin module).

All in all permissions for media seems to be a difficult problem.

What other solutions might there be to this issue, that is powerful enough to enable fine grained permissions on the long run?

Update: Either way this also depends on the decision in #685818: Provide an API function to convert a URI to a file object.

iandickson’s picture

In my case the Library is essentially unusable as a "look through and reuse" - it will soon comprise thousands of YouTube urls, 99% of which will only ever be in ONE Node. *

So what I need is EITHER that the system doesn't care about accumulating dupes (acceptable), or automatically recognises them and handles accordingly (better).

I am not a programmer, so I will not use programming terms, but to me this looks like a taxonomy issue where people get confused between Terms and Semantics. You can have duplicate Terms in Different Vocabs because the combination has different Semantics - Animal Sounds / Bark and Tree Biology / Bark. Bark is not a duplicate.

So, can you have it so that when the user ads an item, it goes into the Library as a Package, where the Package comprises URI + Data. In a basic system the Data is User, and added automatically.

Then, even if ten users add the same video, the Packages are unique. (Note, this allows for a simple "users see own media only in the Library" option, one that will be in demand by a lot of Admin running multi contributor sites!)

To be a dupe, the PACKAGE must be a dupe, and if a user wants to create what the system sees as a dupe is flagged as such with "create Duplicate or use existing?" option.

Duplicate Packages are then useful if permissions can be applied to the Package. Example Thor Trailer (1) was posted in a "What's On", by me, but it's in my user library, not available to other people posting Thor Reviews for example. Later I was posting inside my Marvel Group and this time I said YES to Duplicate because that would make it available for all the members of the Marvel Group to use in their posts too.

[Note - I suppose in an ideal world you'd never need to create a Dupe as what you really want to is "apply a new permission to Package" - "make available to Group Library" in the example above. But that's one for programmers]

Summary - To allow Permissions, Media Library stores Packages rather than links. Permisisons act on Packages. Quick fix to make YouTube Media usable NOW, automatically pull out the existing link if user suggests a dupe.

Cheers

*Likal supports local community media, each of which is run by an Editor. They video and run their own YouTube channels, dropping video into nodes as needed.

Meanwhile those filmed can lso use the content via YT. This does several things - gives us free video hosting, avoid bandwidth issues, enables us to support the people we film, and we own the views counter.

My trial Likal for Gloucester built up more than 1000 videos and 870,000 views. A Likal run full time would generate 20-50 clips a week, most of which are only ever added to ONE node, because tagging and views pretty much ensures the the audience sees relevent nodes, thus avoiding people creating near duplicates.

kscott22’s picture

sub

webankit’s picture

+1

mstrelan’s picture

+1

oregonwebsiteservices’s picture

I have this issue as well - updated to the latest yesterday:
Drupal core 7.4
Media 7.x-1.0-beta5
Media: YouTube 7.x-1.0-alpha5

Your work on this is much appreciated!
Here is my site: http://humanhighlights.com

Dave Reid’s picture

I am really quite frustrated at this. The proper solution would have been to default the file_uri_to_object() to always re-use an existing file's record based on its URI - since {file_managed}.uri is unique. Instead, we created a default validation (that doesn't come from the parent class, but rather is copy/pasted into each MediaInternetPROVIDERHandler class, so we need to patch every single media provider module to fix this.

Dave Reid’s picture

Status: Active » Needs review
FileSize
1.17 KB

Patch to switch file_uri_to_object to re-use files by default. Once this is committed we'll have to file follow-up patches to remove MediaInternetYouTubeHandler::validate(), MediaInternetVimeoHandler::validate(), etc.

Dave Reid’s picture

Title: When pasting the URL of a file which already exists in the media library, media_internet throws a validation error » Change file_uri_to_object to re-use files by default and remove duplicate validation checks in handlers

Issue filed with Media: Youtube at #1213184: Allow YouTube videos to be re-used once this lands.

Niklas Fiekas’s picture

I am using a hacky workaround in media_feeds, which I want to remove as soon as possible. (It involves hook_file_presave().)
Is there a chance you will bump the version number of media, after this is fixed? Then plugins could depend on this update easily.

Dave Reid’s picture

Marked #1166568: Media_internet.module, existing files. as a duplicate of this issue.

Dave Reid’s picture

Committed to file_entity in 7.x-1.x (media 7.x-2.x).
http://drupalcode.org/project/file_entity.git/commit/330a186

aaron’s picture

dave, can we mark this issue as fixed then?

aaron’s picture

Status: Needs review » Fixed

looks like that's a yes, from testing... thanks!

Dave Reid’s picture

Status: Fixed » Needs review

I didn't commit this to 7.x-1.x (only 7.x-2.x) as it should probably be discussed if we should change that behavior or not.

Dave Reid’s picture

.

Dave Reid’s picture

Version: 7.x-1.x-dev » 7.x-2.x-dev
Status: Needs review » Closed (fixed)