Running on the latest dev this happens. This is unexpected and is possibly a regression, and we need to introduce tests to ensure this works.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Devin Carlson’s picture

I see the same behaviour with unstable7.

How should determining the file type of existing managed files be handled?

A batch process that does something like:

$query = new EntityFieldQuery();

$query->entityCondition('entity_type', 'file')
      ->propertyCondition('type', FILE_TYPE_NONE);

$result = $query->execute();

if (isset($result['file'])) {
  $fids = array_keys($result['file']);
  $files = file_load_multiple($fids);

  foreach ($files as $file) {
    $type = file_get_type($file);

    if (isset($type)) {
      $file->type = $type;
      drupal_write_record('file_managed', $file, 'fid');
    }
  }
}

There has been some talk on IRC of exposing a button, i.e. "reclassify files", on a settings page which does something similar.

tchopshop’s picture

I uninstalled and reinstalled File Entity and Media (to solve a persistent update issue: http://drupal.org/node/1901886) and now under admin/content/file the existing files are not showing any "type".

Also -- and possibly related, but most importantly -- all thumbnails are broken on admin/content/file/thumbnails and in the Media Library and on the node/edit page with a media field.

tchopshop’s picture

Should I post my problem (see #2 above) in a different issue? Not sure it is related to this one. My main worry is that I have broken the link to all those images and won't be able to generate thumbnails for them. I can't figure out what to do.

Dave Reid’s picture

@tchopshop: If you edit the raw records in file_managed to change type column values to image, does the problem resolve itself? If so, then yes, your issue is this one here.

eidoscom’s picture

Just a note: If you edit a file and save, the file type is get ok.
I can remember other releases where once the module was installed, It has a need to update all the files to get the change from file to file entity ... I'm pretty sure of this but don't know exactly.

So it seems that the code to accomplish the function must be inside the module ... I revised the file_entity.install and found this:

/**
 * Update all files with empty types to use the first part of filemime.
 *
 * For example, an png image with filemime 'image/png' will be assigned a file
 * type of 'image'.
 */
function file_entity_update_7101() {
  db_update('file_managed')
    ->expression('type', "SUBSTRING_INDEX(filemime, '/', 1)")
    ->condition('type', '')
    ->execute();
}

It seems related to what we are getting here isn't? Perhaps this is not doing after install??

eidoscom’s picture

I runed UPDATE `file_managed` SET `type`=SUBSTRING_INDEX(filemime, '/', 1) WHERE 1 on Mysql and all image files are now ok, but not all the other files.

Used this to convert all the pdf files of my site UPDATE `file_managed` SET `type` = 'document' WHERE `filemime` = "application/pdf".

eidoscom’s picture

I can't understand exactly what code in #1, but I think that must get all the "type" values and the mimetypes associated in order to establish the correct "type" value for the file. I added a comment where this I think that must be.

An example is a pdf file where the mimetype is 'application/pdf' but the correct type string is 'document'.

$query = new EntityFieldQuery();

  $query->entityCondition('entity_type', 'file');
  $result = $query->execute();

  if (isset($result['file'])) {
    $fids = array_keys($result['file']);
    $files = file_load_multiple($fids);

    foreach ($files as $file) {
      $filemime = file_get_type($file);

      if (isset($filemime)) {
        // Here is where we must convert 'filemime' from the file to the correct 'type' string
        foreach (file_type_get_enabled_types() as $type) {
          if (file_entity_match_mimetypes($type->mimetypes, $filemime)) {
            $temp_type = $type->type;
          }
        }
        $file->type = $temp_type;
        drupal_write_record('file_managed', $file, 'fid');
      }
    }
  }
Dave Reid’s picture

I think what we'll probably need is a solution that:

  • On install, enqueues all existing file IDs into a queue (this should be a fairly fast operation - we only need to fetch file IDs).
  • On cron, add a queue processor using hook_cron_queue_info() that dequeues up to X items or X amount of time to do the processing for updating an unknown file type to a known file type like using the code in #1.
Devin Carlson’s picture

Status: Active » Needs review
FileSize
1.79 KB

A patch to, during installation, find all managed files and queue them up for processing during cron runs. When cron runs, batches of files will be analyzed, have their file types determines (if possible) and saved.

Status: Needs review » Needs work
Issue tags: -Needs tests, -7.x-2.0 alpha blocker

The last submitted patch, determine-file-file-types-during-install-1977662-9.patch, failed testing.

Devin Carlson’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, determine-file-file-types-during-install-1977662-9.patch, failed testing.

aaron’s picture

Status: Needs work » Needs review
Issue tags: +Needs tests, +7.x-2.0 alpha blocker
Devin Carlson’s picture

An updated version of #9 which includes tests and drops drupal_write_record() in favour of a simply doing a file_save().

Status: Needs review » Needs work
Devin Carlson’s picture

Status: Needs work » Needs review
aaron’s picture

Status: Needs review » Needs work

does not work if you uninstall/re-install the module.

Devin Carlson’s picture

@aaron, did you make sure to fully uninstall (not just disable) File entity in between your tests and run cron after each install of the module?

I'm not able to duplicate any problems after re-installing File entity.

jenlampton’s picture

I can confirm that uninstalling and re-installing file_entity and running cron does *not* update the 'undefined' file types.
(I'm using 7.x-2.0-unstable7 with this patch)

jenlampton’s picture

Status: Needs work » Reviewed & tested by the community

Okay, so things are different running 7.x-2.x-dev plus this patch. After uninstalling, reinstalling and running cron the files now have types.

* For those that follow, Please note that this patch will not work on the 7.x-2.0-unstable7 version.

Devin Carlson’s picture

Status: Reviewed & tested by the community » Fixed

Committed #14 to 7.x-2.x.

Automatically closed -- issue fixed for 2 weeks with no activity.

jenlampton’s picture

Issue summary: View changes

Thanks @Devin Carlson!
I love it when I can delete lines from my PATCHES.txt files! :)

maxplus’s picture

Thanks,

I needed to uninstall and re-install file entity but then on every cron run, several files changed from undefined to image.

Great

potassiumchloride’s picture

If you're like me and ran into undefined file types after installing Media module, wait for cron to run or go run cron manually (admin/config/system/cron) and the files will be categorized for you.