Use expirable key/value for entity autocomplete

The piece of code we are talking about here is:

    // Store the selection settings in the key/value store and pass a hashed key
    // in the route parameters.
    $selection_settings = isset($element['#selection_settings']) ? $element['#selection_settings'] : [];
    $data = serialize($selection_settings) . $element['#target_type'] . $element['#selection_handler'];
    $selection_settings_key = Crypt::hmacBase64($data, Settings::getHashSalt());

    $key_value_storage = \Drupal::keyValue('entity_autocomplete');
    if (!$key_value_storage->has($selection_settings_key)) {
      $key_value_storage->set($selection_settings_key, $selection_settings);
    }

While talking to @larowlan he suggested we can just use \Drupal::keyValueExpirable() instead of \Drupal::keyValue() here.

\Drupal::keyValue('entity_autocomplete')->getAll() shows that it stores one value per EntityAutocomplete element.

According to #2 the change should be pretty straightforward and this might go in as a quick fix.

Two quick thoughts:

a) I suspect you are relying on some patches that allow to consider the current entity in the autocomplete, e.g. excluding it. To end up with that many rows, you need to have a dynamic component in there because by default, it's just the static field settings. So I don't think you can get millions of records in there without some special setup.

b) The problem wit hsetWithExpireIfNotExists() is that it doesn't renew the expire. 24 might sound fine at first, but it's perfectly possible that you call it 23h59m after it had been set and by the time you try to use the autocomplete, the records have been purged.

I think you misunderstood my second point. Even if you set the expiration to 1 year, it's still possible that you open a form with these settings just seconds before expiration. Unlike the form state storage, which is always written and always valid for 6h from when you start.

Also, in Drupal 8, the form state storage is actually only initialized on the first ajax request. The user/register form could for example have an autocomplete field, or a webform and that could be cached for days. And nothing will renew the expirable storage then, resulting in a broken autocomplete in both cases.

That's why we went with the non-expirable key value store. You should probably find the discussion about that in #2490420: EntityAutocomplete element settings allows sql injection and for arbitrary user-supplied data to be passed into unserialize() when we implemented this.

Well, that results in constant writes to the storage, which the current solution doesn't.

And note that the internal page cache does not currently respect max-age.

I'm not trying to just complain, and I agree there is an issue when using a patch like you do. But as I said, there's a reason we didn't do it like this initially.

The problem with any patch that passes the whole entity through this is that it doesn't just result in a lot of unique keys but the configuration is also massively bigger than what core deals with. Which means that an alternative solution like avoid persistent storage completely by passing a signed json structure for example doesn't work either.

Not sure. I guess the other issue is basically blocked on solving this, or finding a different approach that doesn't result in this problem.

I'm fine with keeping this open, I think there's another similar issue somewhere too. Maybe @amateescu has some thoughts on this and the related feature rquests.

Just ran into this on a project that had a mammoth 1.1gig db dump, tracked down to the key_value table with over 110,000 rows in the entity_autocomplete collection. I'm yet to track down what configuration is causing this dynamic nature.

Rerolled for D9.4. Just in case if someone require it. Without tests and hook_install, because it can affect on next core updates.
Need to clean database manually after patch with drush command:
drush ev "\Drupal::keyValue('entity_autocomplete')->deleteAll();"