MediaFlickrStreamWrapper causes huge view render time

Enabling the Media:Flickr module, or more specificly the MediaFlickrStreamWrapper the Views view render time is suddenly huge when a page contains a lot of Flickr images.

I'm creating a view for the media browser (Media module) and this requires me to sort the files on the newest first (desc). Using the performance statistics I found out this view had a huge view render time, around 5 seconds.
To exclude any possible problems caused by the media module I made a new view that simply shows 100 files and just their title. Displaying a page with no Flickr images the view render takes 59.62 ms, a page with just Flickr images view render takes 41384.54 ms.

When I disable the Media:Flickr module there is no real difference. When I enable the module, but I comment out the media_flickr_stream_wrappers function there is no real difference.
Commenting out the functions in MediaFlickrStreamWrapper doesn't help.

Is there anything I can do to speed up the rendering of these Flickr photo's?