After installing and doing database update, migrate does not work. I get this error:

An AJAX HTTP error occurred.
HTTP Result Code: 200
Debugging information follows.
Path: /update.php/start?id=483&op=do_nojs&op=do
StatusText: OK
ResponseText: Recoverable fatal error: Argument 1 passed to Drupal\file\FileUsage\DatabaseFileUsageBackend::listUsage() must implement interface Drupal\file\FileInterface, null given, called in /var/www/vhosts/thejdngroup.com/domains/test8.portrevolt.com/sites/all/modules/focal_point/focal_point.install on line 91 and defined in Drupal\file\FileUsage\DatabaseFileUsageBackend->listUsage() (line 98 of /var/www/vhosts/thejdngroup.com/domains/test8.portrevolt.com/core/modules/file/src/FileUsage/DatabaseFileUsageBackend.php).

Comments

jnimchuk created an issue. See original summary.

bleen’s picture

I thought about throwing an exception if the file object doesnt load, but at the end of the day it doesn't really matter. All that means is that there is some extraneous file data in the focal point table in the DB.

This does lead me to think that if a file entity is deleted that focal point is not reacting accordingly, but that should be handled separately

bleen’s picture

Status: Active » Needs review
jnimchuk’s picture

Sorry, this one doesn't work either. As before: unrecoverable error — which means that I had to delete and replace the database AND fall back to a version that worked (probably in April).

Your mod makes two database updates:

==========
8001 - Install default config.
8002 - Migrates legacy values to crop entities.
==========

It's 8002 that causes the problem. Here's the error:

==========
An unrecoverable error has occurred. You can find the error message below. It is advised to copy it to the clipboard for reference.
Please continue to the error page

An AJAX HTTP error occurred.
HTTP Result Code: 200
Debugging information follows.
Path: /update.php/start?id=530&op=do_nojs&op=do
StatusText: OK
ResponseText: Recoverable fatal error: Argument 1 passed to Drupal\file\FileUsage\DatabaseFileUsageBackend::listUsage() must implement interface Drupal\file\FileInterface, null given, called in /var/www/vhosts/thejdngroup.com/domains/test8.portrevolt.com/sites/all/modules/focal_point/focal_point.install on line 91 and defined in Drupal\file\FileUsage\DatabaseFileUsageBackend->listUsage() (line 98 of /var/www/vhosts/thejdngroup.com/domains/test8.portrevolt.com/core/modules/file/src/FileUsage/DatabaseFileUsageBackend.php).
===============

(1) Love your lightweight module. For Drupal 6, I used something else. But yours is elegant and works.
(2) So please work to get this right (in 6 years of using Drupal, I've never before had a module that required a restore of my database.

jdn

bleen’s picture

I dont think you applied the patch in #2 correctly. The error you are seeing is complaining about the $file object being NULL (as opposed to being an object that implements FileInterface) on line 91, however when this patch is applied properly line 91 is a comment and therefore cannot cause the error you are seeing.

(1) Love your lightweight module. For Drupal 6, I used something else. But yours is elegant and works.
Thanks!!

(2) So please work to get this right (in 6 years of using Drupal, I've never before had a module that required a restore of my database.
I completely understand being frustrated but this is precisely the reason I have not really felt comfortable creating a release for D8 yet. In any event, I'm still pretty convinced that the patch in #2works. If you could try again that would be terrific.

jnimchuk’s picture

You're right: I did not apply the patch.

I assumed you integrated the patch into the main module. If the patch is a workaround for me: then I'll try this tomorrow ...

JDN

bleen’s picture

Great! once you can confirm that the patch works I will commit it

jnimchuk’s picture

Sorry, this is not working. Integrated the patch and when I try to run update.php, I get a blank page. When I revert to old version, everthing is fine.

Here is the install file with changes:

==============
<?php

/**
* @file
* Install hooks for focal_point.
*/

/**
* Checks if required version of Crop API is installed.
*
* @return bool
* TRUE if dependency is met and FALSE if not.
*/
function _focal_point_check_crop_version() {
if (\Drupal::moduleHandler()->moduleExists('crop')) {
$info = system_get_info('module', 'crop');
if (version_compare($info['version'], '8.x-1.0-alpha2') >= 0) {
return TRUE;
}
}

return FALSE;
}

/**
* Implements hook_requirements().
*/
function focal_point_requirements($phase) {
$requirements = [];
if ($phase == 'update' && !_focal_point_check_crop_version()) {
$requirements = [
'crop' => [
'title' => t('Focal point'),
'value' => t('Crop API missing'),
'description' => t(
'Crop API module is now a dependency and needs to be installed before running updates.',
[':url' => 'https://www.drupal.org/project/crop']
),
'severity' => REQUIREMENT_ERROR,
],
];
}
return $requirements;
}

/**
* Install default config.
*/
function focal_point_update_8001() {
if (!_focal_point_check_crop_version()) {
throw new \Drupal\Core\Utility\UpdateException('Crop API (drupal.org/project/crop) module is now a dependency and needs to be installed before running updates.');
}

\Drupal::service('config.installer')
->installDefaultConfig('module', 'focal_point');
}

/**
* Migrates legacy values to crop entities.
*/
function focal_point_update_8002(&$sandbox) {
/** @var \Drupal\file\FileUsage\FileUsageInterface $file_usage */
$file_usage = \Drupal::service('file.usage');
$file_storage = \Drupal::entityTypeManager()->getStorage('file');
$crop_storage = \Drupal::entityTypeManager()->getStorage('crop');
$crop_type = \Drupal::config('focal_point.settings')->get('crop_type');
if (!isset($sandbox['num_processed'])) {
$sandbox['last_fid'] = 0;
$sandbox['num_processed'] = 0;
$sandbox['total_items'] = \Drupal::database()
->select('focal_point', 'fp')
->countQuery()
->execute()
->fetchField();
}

$focal_points = \Drupal::database()
->select('focal_point', 'fp')
->fields('fp')
->condition('fp.fid', $sandbox['last_fid'], '>')
->range(0, 100)
->orderBy('fp.fid')
->execute();

foreach ($focal_points as $focal_point) {
/** @var \Drupal\file\FileInterface $file */
$file = $file_storage->load($focal_point->fid);

if (!isNull($file)) {
// Try to load width and height from the image fields and fall back to
// reading image file if that was not successful.
$usage = $file_usage->listUsage($file);
$usage = current($usage);

/** @var \Drupal\Core\Entity\FieldableEntityInterface $entity */
$entity = \Drupal::entityTypeManager()
->getStorage(key($usage))
->load(current(current($usage)));
$size = NULL;
foreach ($entity->getFieldDefinitions() as $field_definition) {
if ($field_definition->getType() == 'image') {
foreach ($entity->{$field_definition->getName()} as $item) {
if ($item->target_id == $file->id()) {
$size = [$item->width, $item->height];
}
}
}
}
}

if (empty($size)) {
$size = getimagesize($file->getFileUri());
}

// Now we have all information we need. Let's create crop entity.
$focal_point = explode(',', $focal_point->focal_point);
$crop_storage
->create([
'type' => $crop_type,
'entity_id' => $file->id(),
'entity_type' => 'file',
'uri' => $file->getFileUri(),
'x' => (int) round((intval($focal_point[0]) / 100.) * $size[0]),
'y' => (int) round((intval($focal_point[1]) / 100.) * $size[1]),
])
->save();
$sandbox['num_processed']++;
$sandbox['last_fid'] = $file->id();
}
}

$sandbox['#finished'] = $sandbox['total_items'] ? $sandbox['num_processed'] / $sandbox['total_items'] : 1;

// Intentionally leaving legacy table. You never know...
}

bleen’s picture

Now *THAT* was a dumb error

This patch should do it now

jnimchuk’s picture

Hey Bleen —

I tried making the change from the interdiff.txt file — but still have a problem.

Will you send me the install file that you think will work?

Thanks.

bleen’s picture

This patch should fix the case where a node was deleted (thus orphaning the file entity) such that there is focal point data for a deleted file. Its a bit of an edge case, but at the same time its very much not an edge case. There should be a followup issue for deleting focal point data when a file is deleted...

To help you out, here is the full focal_point.install file with this patch applied:

<?php

/**
 * @file
 * Install hooks for focal_point.
 */

/**
 * Checks if required version of Crop API is installed.
 *
 * @return bool
 *   TRUE if dependency is met and FALSE if not.
 */
function _focal_point_check_crop_version() {
  if (\Drupal::moduleHandler()->moduleExists('crop')) {
    $info = system_get_info('module', 'crop');
    if (version_compare($info['version'], '8.x-1.0-alpha2') >= 0) {
      return TRUE;
    }
  }

  return FALSE;
}

/**
 * Implements hook_requirements().
 */
function focal_point_requirements($phase) {
  $requirements = [];
  if ($phase == 'update' && !_focal_point_check_crop_version()) {
    $requirements = [
      'crop' => [
        'title' => t('Focal point'),
        'value' => t('Crop API missing'),
        'description' => t(
        '<a href=":url">Crop API</a> module is now a dependency and needs to be installed before running updates.',
          [':url' => 'https://www.drupal.org/project/crop']
        ),
        'severity' => REQUIREMENT_ERROR,
      ],
    ];
  }
  return $requirements;
}

/**
 * Install default config.
 */
function focal_point_update_8001() {
  if (!_focal_point_check_crop_version()) {
    throw new \Drupal\Core\Utility\UpdateException('Crop API (drupal.org/project/crop) module is now a dependency and needs to be installed before running updates.');
  }

  \Drupal::service('config.installer')
    ->installDefaultConfig('module', 'focal_point');
}

/**
 * Migrates legacy values to crop entities.
 */
function focal_point_update_8002(&$sandbox) {
  /** @var \Drupal\file\FileUsage\FileUsageInterface $file_usage */
  $file_usage = \Drupal::service('file.usage');
  $file_storage = \Drupal::entityTypeManager()->getStorage('file');
  $crop_storage = \Drupal::entityTypeManager()->getStorage('crop');
  $crop_type = \Drupal::config('focal_point.settings')->get('crop_type');
  if (!isset($sandbox['num_processed'])) {
    $sandbox['last_fid'] = 0;
    $sandbox['num_processed'] = 0;
    $sandbox['num_skipped'] = 0;
    $sandbox['total_items'] = \Drupal::database()
      ->select('focal_point', 'fp')
      ->countQuery()
      ->execute()
      ->fetchField();
  }

  $focal_points = \Drupal::database()
    ->select('focal_point', 'fp')
    ->fields('fp')
    ->condition('fp.fid', $sandbox['last_fid'], '>')
    ->range(0, 100)
    ->orderBy('fp.fid')
    ->execute();

  foreach ($focal_points as $focal_point) {
    /** @var \Drupal\file\FileInterface $file */
    $file = $file_storage->load($focal_point->fid);

    if (!is_null($file)) {
      // Try to load width and height from the image fields and fall back to
      // reading image file if that was not successful.
      $usage = $file_usage->listUsage($file);
      $usage = current($usage);

      /** @var \Drupal\Core\Entity\FieldableEntityInterface $entity */
      $entity = \Drupal::entityTypeManager()
        ->getStorage(key($usage))
        ->load(current(current($usage)));
      $size = NULL;
      foreach ($entity->getFieldDefinitions() as $field_definition) {
        if ($field_definition->getType() == 'image') {
          foreach ($entity->{$field_definition->getName()} as $item) {
            if ($item->target_id == $file->id()) {
              $size = [$item->width, $item->height];
            }
          }
        }
      }

      if (empty($size)) {
        $size = getimagesize($file->getFileUri());
      }

      // Now we have all information we need. Let's create crop entity.
      $focal_point = explode(',', $focal_point->focal_point);
      $crop_storage
        ->create([
          'type' => $crop_type,
          'entity_id' => $file->id(),
          'entity_type' => 'file',
          'uri' => $file->getFileUri(),
          'x' => (int) round((intval($focal_point[0]) / 100.) * $size[0]),
          'y' => (int) round((intval($focal_point[1]) / 100.) * $size[1]),
        ])
        ->save();
      $sandbox['num_processed']++;
    }
    else {
      $sandbox['num_skipped']++;
    }
    $sandbox['last_fid'] = $focal_point->fid;
  }

  $sandbox['#finished'] = $sandbox['total_items'] ? ($sandbox['num_processed'] + $sandbox['num_skipped']) / $sandbox['total_items'] : 1;

  // Intentionally leaving legacy table. You never know...
}
jnimchuk’s picture

OK, still a problem ...

1. I'm running Drupal 8.1.2.

2. I used your install file and update.php ran fine until the end. Again, a problem with migration, it seems. Here's the error:

============
focal_point module
Update #8002

Failed: Drupal\Component\Plugin\Exception\PluginNotFoundException: The "" entity type does not exist. in Drupal\Core\Entity\EntityTypeManager->getDefinition() (line 125 of /var/www/vhosts/thejdngroup.com/domains/test8.portrevolt.com/core/lib/Drupal/Core/Entity/EntityTypeManager.php).
=============

3. I'm hoping I don't have to rollback the DB and that you can resolve this quickly, Bleen ...

Thanks.

bleen’s picture

1. I'm running Drupal 8.1.2.

I've tested this on teh 8.1 branch with no problems

2. I used your install file and update.php ran fine until the end. Again, a problem with migration, it seems. Here's the error:

I cannot reproduce this error but it looks like you may have some wackado file entity defined on your site (or that you used to?)

3. I'm hoping I don't have to rollback the DB and that you can resolve this quickly, Bleen

This update is non-destructive so there is no reason you should have to rollback the DB. (Sidenote: I really hope you have been working on a local (or other non-prod env) to test your updates).

I'm not sure what else to do here ... I can say that the patch in #11 is a big improvement and should avoid errors that other people might hit but I'm not seeing any reason why it isnt working for you other that the possibility that you have erroneous data in your file_managed table.

I suspect that if you wrote your own function somewhere that did nothing but load each file in that table that you would have a similar error ...

bleen’s picture

... I recognize that what I am about to say is a hack, but if you take a look in your database at the crop_field_data table and it has all the same data that you have in your focal_point table than the update was successful (despite seeing that error).

If that is the case, you can run this query:
update key_value set value="i:8002;" where collection="system.schema" and name="focal_point";
And this will tell drupal that your update was successful and you're ready to just move on.

This is definitely a hack, but it may not be worth it for you to keep up the good fight here.

jnimchuk’s picture

Thanks Bleen —

This worked ...

Yes, I'm sure the issue is specific to my upgrade from Drupal 6 to 8. I've had a couple of other issues previously; appreciate your help here.

Didn't want to start all over with your module. It is now managing 500 + images, and will eventually manage over 1200.

Thanks for your help!

jdn

bleen’s picture

Status: Needs review » Closed (won't fix)
bleen’s picture

Status: Closed (won't fix) » Closed (fixed)

oops