Use of copy() and simplexml_load_file() rather than drupal_http_request() causes Media:YouTube thumbnails to be empty

Yeah, I have also seen this, and the strange thing is I have one mirrored installation where my local copy is able to create the thumbnails and the other not. Makes me wonder if this is related to the server env.?

What PHP version are you using?

same here

I'm thinking out loud here, could this be a bug in the media module? I have not looked enough at the code to figure out who of the module that is responsible for storing the thumb. I also think this issue is similar #717704: On some web hosts, preview thumbnail image isn't generated I posted some debugging info. at #11

Tested PHP 5.3 and 5.2 in my local setup, without being able to recreate the problem. It only materialize on the live server :-/

Still willing to help out squash this bug, any hints where in media I should start digging?

We moved this down to the media module, bumping this up to major in hop for it to get some more eyeballs. Anyone got some idea where we could start digging? This is related or the same as #717704: On some web hosts, preview thumbnail image isn't generated.

Thanks for a detailed feedback, time to drush en devel :)

Checked these variables in getLocalThumbnailPath(), and they seems OK.

$local_path: public://media-youtube/Uz1U2dRGVrA.jpg
$dirname is : public://media-youtube

3) Not sure about this when we also see this on Vimeo, and I do not think it is related to what API we use.

4) I think file_prepare_directory($dirname, FILE_CREATE_DIRECTORY | FILE_MODIFY_PERMISSIONS); check, create dir, change permissions if it have/able to. (http://api.drupal.org/api/drupal/includes--file.inc/function/file_prepar...) and works OK. I also tried to remove 'files/media-youtube' and the code is able to recreate the it.

5) That is correct, I have also seen this.

In my situation

I see this on a live production site running on greengeeks.com.

Todo

Make sure, when we get the thumbs created, that is also is able to clean them up. Right now they are just felt there after the stream is deleted.

Would not surprise me that there bugs related to running default in private mode is is the least tested/used one. As the server I see the bug on though are running with public as default storage area so at least for me do I think it is unrelated.

I'm still is trying to get 1) in #9, I'll prob. need to read up on PHP objects ;) btw. found time to check
@copy($this->getOriginalThumbnailPath(), $local_path);
by running the output drupal log dd(($this->getOriginalThumbnailPath()), 'Image URL '); and the complete URL is correct, example http://img.youtube.com/vi/399hFkQYnV4/0.jpg.

I'm trying to get where Styles fit in, if copy() does it thing should it copy the file that the URL points to and to where ever $local_path points to, right? Styles picks up these files, and run them through image module?

I have the same issue, it doesn't save the youtube thumbnail.
Btw, I used media gallery on another site and youtube thumbnails work there. They are stored in sites/default/files/media-youtube
rather than sites/default/files/styles/thumbnail/public/media-youtube ...

Subscribe

Hello
Same issue : thumbnails ok on one hoster, not on another. I checked the logs, and first one is :

"Warning: simplexml_load_file() [function.simplexml-load-file]: http:// wrapper is disabled in the server configuration by allow_url_fopen=0 in MediaInternetYouTubeHandler->getMRSS() (line 72 of /home/thewcul9/public_html/ng8/sites/all/modules/media_youtube/includes/MediaInternetYouTubeHandler.inc)."
If it can help...

This is the response of AN hosting, the great, after sending them the message above:

"Dear a.
Looking at the custom php.ini in the /home/tttttt/public_html directory I found that the allow_url_fopen was set to off. I turned this to on and now see the thumbnail of the video. Please validate that this is now functioning properly for you now."

And the guy is right : my thumbnail are now created!
Check your php.ini maybe it'll help.

@artatum that did change the behavior I'm seeing as Drupal's now displaying an error message. Hmm.

Warning: filesize(): stat failed for youtube://v/xxxxxxxxxx in file_save() (line 573 of /var/www/html/xxxxx/includes/file.inc).

@artatum, Thanks, it fixed my problems as well!

@artatum

me too. #16 solves it - after having a look to sys-log of drupal i found the message
you are my hero today :)

It's good to be the king... (of the day ;-)

BTW I spoke to a guy at my webhost the other day about allow_url_fopen, and he said that to avoid to have to set it to on, you could do this:

function curlGet($url) {
        $ch = curl_init();
        curl_setopt($ch, CURLOPT_URL, $url);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
        return curl_exec($ch);
}
$file = curlGet('http://www.urltofile.com');

I don't know if this is helpful, perhaps there's drawbacks in this method that I'm not aware of, but since many people are having this problem it might be a better solution to have this in the module.

subscribing

I would very much favor a solution were allow_url_fopen may stay off for media_youtube to work. Isn't there a drupal-core-function for loading URLs (something like curlGet above)?

I have allow_url_fopen enabled but still not getting any preview images. Also no error seems to be getting generated in my PHP error log.

EDIT: Moron alert, file permissions issue here :3. Maybe the module could provide an alert in status report if it can't write to public://media-youtube ?

I am also keen on getting this working without having to turn on allow_url_fopen

What about using 'drupal_http_request()' instead?

I installed the latest dev and it is working a bit better. Not all formats are working but looking not too bad.

Hello,

I have the same problem, thumbails are empy. This is the error log:

Warning: filesize() [function.filesize]: stat failed for youtube://v/xy_nKpZLo-0 en file_save() (línea 573 de /homepages/29/d292176751/htdocs/des/includes/file.inc).

#18 Have the same problem.

Active allow_url_fopen not solve the problem.

Any ideas?

If you look at the documentation for allow_url_fopen, it says.
"This option enables the URL-aware fopen wrappers that enable accessing URL object like files."

In includes/MediaYouTubeStreamWrapper.inc, line 34 uses the function copy($source, $dest). In the documentation for copy(), it states "Both source and dest may now be URLs if the "fopen wrappers" have been enabled."

That is, the use of copy() is what necessitates allow_url_fopen.

Here is a patch that uses drupal_http_request, which should be compatible with all installs, since it's part of the Drupal API.

If it turns out that some installations need to use copy() for some reason, we could always use copy() as a fallback. I've attached a patch that has code for that as well, but I don't think it'll be necessary.

So it'd be great if both people who have this issue and people who don't could try the media_youtube-empty-thumbnails-1180386.patch.

If it happens that it was working for you originally, but doesn't with the patch, try media_youtube-empty-thumbnails-1180386-alt.patch. (If it works, the "alt" patch is not needed.)

I just saw from comment #16, there's another place that needs corrected too. Here's a patch which addresses the error in #16. I haven't seen this error in what I'm doing, so I can't verify, but it should eliminate it without needing to turn on allow_url_fopen.

The changes for the patch in #28 aren't included in this patch. It might be easier for troubleshooting to keep them separate.

So please install both the patch in #28 (media_youtube-empty-thumbnails-1180386.patch) and this patch (media_youtube-empty-thumbnails-1180386-16.patch).

(Again, only install media_youtube-empty-thumbnails-1180386-alt.patch from #28 if the first patch breaks the thumbnails when they used to be working.)

I tried using #28 media_youtube-empty-thumbnails-1180386-alt.patch and it works.

I also have the same problem and #28 media_youtube-empty-thumbnails-1180386-alt.patch works for me too, nice work, thanks

Patch #28 worked for me.
I have allow_url_fopen disabled, is that why youtube content names are generated from youtube url not from the video title
I found this patch Set $file->filename as video title but it only effected the video content.
When a new video content is submited a log is created when the thumbnail preview shows up;
Warning: filesize() [function.filesize]: stat failed for youtube://v/x-xxxxxxxxx in file_save() (line 573 of includes/file.inc).

Have these patches been incorporated into the latest beta (7.x-1.0-beta2) and dev (7.x-1.x-dev) releases, updated in the last week?

Thank you,
Joe

@pkchoo Looking at latest dev, the #28, #30 patches haven't been included. And the latest version adds more dependencies on allow_url_fopen being on by using get_headers().

@DevilEye - The name issue seems like discussion for a separate issue since this one is only about empty thumbnails. You might want to ask about it in #1368714: Set $file->filename as video title.

That warning message is from the save() method of MediaInternetYouTubeHandler when it calls file_save in core's file.inc.

$file->filesize = filesize($file->uri);

I'm not sure what the affects of changing $file->uri to the local path would be. Since there's been no attention to this issue and additional dependencies on allow_url_fopen have been added, I'm not going to do any more work on this. It seems like the direction of the module is to require allow_url_fopen.

If the maintainers want to remove that dependency, please let me know and I'll look into it. If not, I would suggest adding the information that that configuration is required to the readme and/or project page.

http_request alternative is very welcome
Appreciated your support Cristinawithout

Hey,

with the current script + patches, the thumbnails are being stored on the server,
but the youtube script doesnt seem to pick the right directory - I get this (within the gallery elements):

http://homepage.com/sites/default/files/styles/media_gallery_thumbnail/public/media-youtube/thevideothumbnail.jpg

When I try to manipulate the local_path / dirname string and just hardcode the url it still adds up this:

http://styles/media_gallery_thumbnail/http/homepage.com/content/sites/default/files/media-youtube/thevideothumbnail.jpg

Any ideas? Thanks in advance!

I'm somewhat encouraged by the fact that Dave Reid took out the get_headers() function and replaced it with drupal_http_request() in #1480556: Use drupal_http_request() instead of get_headers() in MeidaInternetYouTubeHandler::valid_id(). So these patches once again work to correct the issue. With some attention being paid to something similar, I'm hopeful that the efforts in this issue won't be wasted.

@DevilEye - Regarding the filesize() warning you were getting in #33, the file_save() function is no longer in MediaInternetYouTubeHandler.inc. Would you be able to test against the current dev and see if you still get the warning?

@megatron777 - That seems to me to be the behavior of the module with or without the patches (enabling allow_url_fopen without the patches) . I don't quite understand the problem, because that seems like the correct path to a thumbnail to me. You may want to open a separate issue for that.

Rerolling #30 against latest dev.

Crossposting against media_vimeo #1504050: Use drupal_http_request() instead of of copy() and simplexml_load_file(), or else Media:Vimeo thumbnails may be empty that I created to make sure we also fix it there.

Coming to this from the vimeo cross post. I feel like it might be better to throw exceptions instead of returning an empty element... Fail loudly rather than silently.

cristinawithout, it is still the same with the dev. release; I believe that I will have to enable fopen to use media module smoothly.

@DevilEye you need to test with patch #40 applied. Does not get committed until it have been tested and reviewed.

steinmb, still filesize() warning using the #40 patch and latest dev

@steinmb, I tested patch #40 with latest dev and also get that filesize() [function.filesize]: stat failed warning. Fopen function is disabled and I am using Media module version 7.x-1.0-rc3.

My guess is that your code is not failing on getMRSS() but in getOEmbed(). Agreeing with @drewish (#42), that we should try and return more gruesome error messages to flush out these kind of errors, instead of silently failing and returning empty objects. This throws errors and dump out the http return code, what error are you getting on your servers?

doh! Just fired up Sublime text 2 to test it out, forgot to check the tab setting.

Let's see if I got the settings in Sublime Text right this time. Rerolling with spaces instead of tabs (I hope).

And, let's test and debug.

Thanx cristinawithout
==> patch #28 good

thanx god AllaH

______________

la ilaha illa allah mohamed rassol allah = There is no god but God, and Muhammad is the messenger of God

Are we sure we're patching the right thing? I figured out that many hosts disable fopen for remote urls as a security precaution. And so its a problem that the media and youtube modules use the copy() function with a remote source. If we used curl instead that would work for most hosts.
Something like the following or if you can do the same thing with drupal_http_request() then use that.

  $ch = curl_init ($source);
    curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_BINARYTRANSFER,1);
    $rawdata=curl_exec($ch);
    curl_close ($ch);
    if(file_exists($destination)){
        unlink($destination);
    }
    $fp = fopen($destination,'x');
    fwrite($fp, $rawdata);
    fclose($fp);

ok I see we already have a good patch it appears with drupal_http_request: http://drupal.org/files/media_youtube-empty-thumbnails-1180386-alt.patch

That I think will work well.

Reroll #50 for latest dev.

Hi guys

Everything was ok but after upgrading to latest drupal version, my last 20 video thumbnails went blank but not all the previous ones.
But when i click on a thumbnail a error appears.
I've checked the php.ini file but the allow_url_fopen was already on.

I've installed all patches but the error maintains and it's the following:

Fatal error: Call to a member function getLocalThumbnailPath() on a non-object in /home/topgoals/public_html/sites/all/modules/media_youtube/includes/media_youtube.formatters.inc on line 150

I don't know any php by the way :)

What can it be?

Cheers

Reroll the complete patch based on #28 and #50 for dev.

Patch in #55 resolved for us.

Patches #28 and #30 resolved problem. Thanks!

Whist the path in #55 has resolved the problem in most cases, in some cases, I get the following warning and no thumbnail:

Warning: simplexml_load_file(): https:// wrapper is disabled in the server configuration by allow_url_fopen=0 in _media_retrieve_xml() (line 38 of profiles/concern/modules/contrib/media/includes/media.xml.inc).

Patch at #57 works for me. Not seeing the issue noted in #60 (yet).

@mrfelton
try #57

#57 worked for me. Thank you!

If I enter the name of the YouTube user in the Media browser search field, the error goes away.

#57 worked for me too. Thank you!

Also had success with #57. Would like to see this patch added soon, with or without addressing #63.

#57 worked for us.

Going to give this issue some attention and testing soon. Looks like #57 can be applied to 1.x -- for those who spent some time in this issue already, is this something that can be fixed in the same way on 2.x as well?

#57 committed to 2.x-dev, with the return removed as suggested in #63. A big thanks to everyone who worked on this.

http://drupalcode.org/project/media_youtube.git/commit/6ded54e.