Trying to create a stream wrapper class, at Media: YouTube. Making progress, except that when file_save() is called for the newly created file object, we receive a warning on $file->filesize = filesize($file->uri);

Warning: filesize(): stat failed for youtube://v/-jubiv7QUco in file_save() (line 482 of /var/www/d7-media/includes/file.inc).

It looks like filesize() doesn't work with streams. This is obviously inconsistent with other stream wrapper file functions, as it apparently can't be overridden, and doesn't simply return FALSE (as does file_exists() or is_readable()).

I believe we should move the $file->filesize assignment outside this function, perhaps even to be handled by the stream wrappers somehow. Otherwise, the File API will fail for handling remote streams.

Comments

pwolanin’s picture

Do we implement http://www.php.net/manual/en/streamwrapper.stream-stat.php? It seems so - maybe make sure you are returning some size in the array returned by stream_stat() from your implementation

pwolanin’s picture

If that doesn't work, perhaps we should call stat() instead of filesize()

aaron’s picture

No, I've implemented stream_stat(), but it still fails. Let me try using stat() instead of filesize().

aaron’s picture

Hmm.. That fails too. Perhaps I'm implementing stream_stat incorrectly?

  /**
   * Support for fstat().
   *
   * @return
   *   An array with file status, or FALSE in case of an error - see fstat()
   *   for a description of this array.
   */
  public function stream_stat() {
    return array(
      'dev' => 0,
      'ino' => 0,
      'mode' => 0,
      'nlink' => 0,
      'uid' => 0,
      'gid' => 0,
      'rdev' => 0,
      'size' => 1000,
      'atime' => 0,
      'mtime' => 0,
      'ctime' => 0,
      'blksize' => 0,
      'blocks' => 0,
    );
  }

(I just have a fake value for size currently, obviously.)

aaron’s picture

And my stat() implementation, for the record:

/**
 * Save a file object to the database.
 *
 * If the $file->fid is not set a new record will be added. Re-saving an
 * existing file will not change its status.
 *
 * @param $file
 *   A file object returned by file_load().
 * @return
 *   The updated file object.
 *
 * @see hook_file_insert()
 * @see hook_file_update()
 */
function file_save(stdClass $file) {
  $file->timestamp = REQUEST_TIME;
  $stat = stat($file->uri);
  $file->filesize = $stat['size'];

  if (empty($file->fid)) {
    drupal_write_record('file', $file);
    // Inform modules about the newly added file.
    module_invoke_all('file_insert', $file);
  }
  else {
    drupal_write_record('file', $file, 'fid');
    // Inform modules that the file has been updated.
    module_invoke_all('file_update', $file);
  }

  return $file;
}
pwolanin’s picture

from php.net re: stream_stat()

This method is called in response to fstat().

NOT stat().

see: http://www.php.net/manual/en/function.fstat.php

aaron’s picture

@pwolanin: Unfortunately, that page also reads

Note: This function will not work on remote files as the file to be examined must be accessible via the server's filesystem.

I tried it anyway, and now get:

Warning: fstat(): supplied argument is not a valid stream resource in file_save() (line 482 of /var/www/d7-media/includes/file.inc).

What if we call the stream wrapper's ->stream_stat() instead?

aaron’s picture

oh wait, just saw that's expecting a handler. not sure if i can wrangle that, but i'll work on it this afternoon...

aaron’s picture

Assigned: Unassigned » aaron
Status: Active » Needs review
StatusFileSize
new906 bytes

OK, this takes care of the issue. We grab the stream and call ->stream_stat directly. Not sure why PHP isn't doing that natively; probably a PHP bug. Oh well.

Status: Needs review » Needs work

The last submitted patch failed testing.

pwolanin’s picture

Status: Needs work » Needs review

Odd indeed - how was this working even for local files like public://?

Quick look at the patch - looks reasonable.

aaron’s picture

Status: Needs review » Needs work

local files worked because it was calling filesave() directly on the filename, rather than the stream.

this patch fails now for normal uploads:

Warning: fstat(): supplied argument is not a valid stream resource in DrupalLocalStreamWrapper->stream_stat() (line 392 of /var/www/d7-media/includes/stream_wrappers.inc).

. unfortunately, i guess we'll need to switch on the stream type (drupallocal vs others).

aaron’s picture

Status: Needs work » Needs review
StatusFileSize
new1.02 KB

This one works fine with local and remote streams now.

aaron’s picture

this patch works by opening the stream before reading stream_stat, then closing it. i believe that filesave works the same way for local files, so this shouldn't add anything substantially different functionally. i'm still baffled by stat() not working on the remote stream; maybe dopry can clue us someday into why that's not working as i expected.

c960657’s picture

I have been using filesize() together with stream wrappers without problems, e.g. the stream wrapper included in the Services_Amazon_S3 PEAR package. I even have tests for it. The stream_stat() implementation is here.

Your stream_stat() should return an array similar to that returned by stat(). On my system, that array has both numerical and string keys:

array(26) {
  [0]=>
  int(769)
  [1]=>
  int(2606155)
  [2]=>
  int(33188)
  [3]=>
  int(1)
  [4]=>
  int(500)
  [5]=>
  int(1001)
  [6]=>
  int(0)
  [7]=>
  int(605)
  [8]=>
  int(1257696466)
  [9]=>
  int(1257696466)
  [10]=>
  int(1257696466)
  [11]=>
  int(4096)
  [12]=>
  int(8)
  ["dev"]=>
  int(769)
  ["ino"]=>
  int(2606155)
  ["mode"]=>
  int(33188)
  ["nlink"]=>
  int(1)
  ["uid"]=>
  int(500)
  ["gid"]=>
  int(1001)
  ["rdev"]=>
  int(0)
  ["size"]=>
  int(605)
  ["atime"]=>
  int(1257696466)
  ["mtime"]=>
  int(1257696466)
  ["ctime"]=>
  int(1257696466)
  ["blksize"]=>
  int(4096)
  ["blocks"]=>
  int(8)
}

Perhaps this is what's causing you trouble?

aaron’s picture

huh. well, i used that nearly verbatim, and it still didn't work. note that http://us3.php.net/manual/en/function.filesize.php writes:

As of PHP 5.0.0, this function can also be used with some URL wrappers. Refer to List of Supported Protocols/Wrappers for a listing of which wrappers support stat() family of functionality.

in fact, i'm still unable to get stat() working with my stream... :(

aaron’s picture

Status: Needs review » Fixed

OK, figured it out. I had to implement ->url_stat(). Really confusing. Not sure where ->stream_stat() is even used; i'd earlier interpreted its documentation to mean it was called for stat() functions.

c960657’s picture

stream_stat() is used by fstat() (get info based on a filehandle) and url_stat() is used by stat() (get info based on a filename).

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

giupenni’s picture

Witct Drupal 7.22 problem persists:
https://drupal.org/node/1586260#comment-7898653

micnap’s picture

Issue summary: View changes
StatusFileSize
new753 bytes
micnap’s picture

StatusFileSize
new721 bytes

Let's try this again without "docroot" in the path.

damienmckenna’s picture

These patches need to be moved to a new issue, or we need to have a maintainer reopen the issue.

szeidler’s picture

I don't think those patches are required. If the custom StreamWrapper is implementing the PHP stat() family properly, the retrieval of filesize will work out of the box and there is no need for the patch.