Problem/Motivation

The link manager service allows to set a specific link domain by calling the method 'ConfigurableLinkManagerInterface::setLinkDomain($domain)'.

On the other hand, the type link manager manages a cache of type links present in the system. Each type link is identified by a URI that is based on the link domain set at the moment that the cache is written.

If the link domain name changes by calling setLinkDomain, the link type cache remains as it was.

Steps to reproduce

  1. fresh install of 8.5-dev Drupal with standard profile
  2. enable the HAL module. Will ask to enable serialization as well by dependency. Also enable the 'rest' module needed as a new dependency, see [#1155816]
  3. download and enable the devel module
  4. try at /devel/php this working-well PHP snippet, that serializes and un-serializes a node:
    $node = \Drupal\node\Entity\Node::create([
      'type' => 'page',
      'title' => 'Test node',
    ]);
    
    $serialized_node = \Drupal::service('serializer')->serialize($node, 'hal_json');
    
    $new_node = \Drupal::service('serializer')->deserialize($serialized_node, 'Drupal\node\Entity\Node', 'hal_json');
    echo $new_node->get('title')->getString();
    
  5. at the same place, replace previous test code by this one that does the same but changing the link domain, it causes a "Type http://example.com/rest/type/node/page does not correspond to an entity on this site." exception:
    \Drupal::service('hal.link_manager')->setLinkDomain('http://example.com');
    
    $node = \Drupal\node\Entity\Node::create([
      'type' => 'page',
      'title' => 'Test node',
    ]);
    
    $serialized_node = \Drupal::service('serializer')->serialize($node, 'hal_json');
    
    $new_node = \Drupal::service('serializer')->deserialize($serialized_node, 'Drupal\node\Entity\Node', 'hal_json');
    echo $new_node->get('title')->getString();
    
    \Drupal::service('hal.link_manager')->setLinkDomain('');
    

Rebuilding the cache and retrying the last step will work, because the type links cache is write again with the set link domain.

Note that it only affects the de-serialization process, in serialization the last set link domain is always used.

Proposed resolution

  1. Clear the type links cache when a new link domain is set.
  2. Write the type link by absolute URI (with no domain) and add it in getters.
  3. Cache types per link domain

Remaining tasks

Notify about it in docs (until solved) to module developers that makes usage of serialization. Currently, any module that changes the link domain should clear the "hal:links:types" cache to avoid this issue and also prevent malfunctioning of any other module that works with HAL, since type links cache remains with unexpected URI.

API changes

Depending on how it is solved, the API could be affected.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

manuel.adan created an issue. See original summary.

manuel.adan’s picture

Assigned: manuel.adan » Unassigned
Status: Active » Needs review
FileSize
805 bytes
borisson_’s picture

Status: Needs review » Needs work
Issue tags: -cache +API-First Initiative, +D8 cacheability, +Needs tests

Adding related tags. It would be great if we add test for this as well.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

neclimdul’s picture

Status: Needs work » Needs review
Issue tags: +Contributed project blocker
FileSize
2.53 KB
neclimdul’s picture

Issue tags: -Needs tests

forgot to remove test tag.

Wim Leers’s picture

Status: Needs review » Reviewed & tested by the community
alexpott’s picture

Status: Reviewed & tested by the community » Needs review

This looks super vulnerable to cache thrashing ie. multiple clears of the cache on the same request. Should the cid have the link domain as part of it instead?

neclimdul’s picture

Status: Needs review » Needs work

I don't know about "super" because I think it generally just get called settings up services but I guess is could basically nullify the cache making it more or less just overhead if it gets called on every run. So much for the trivial solution I guess...

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

manuel.adan’s picture

Status: Needs work » Needs review
FileSize
2.84 KB
1.19 KB

It now invalidates the "hal:links:types" cache only if not empty and it was built with a different link domain. Patch also updated to match with the 8.8 branch.

neclimdul’s picture

Couldn't apply the patch but the idea is interesting. Changed it up a bit and fixed the tests. Seems to mostly address Alex's concerns with out a seismic shift in the way we cache hal information.

To add a little extra thrash protection I compare the domain to the previous one as well. So if some code is just continuously setting the same domain we're protected.

yogeshmpawar’s picture

Assigned: Unassigned » yogeshmpawar
yogeshmpawar’s picture

Assigned: yogeshmpawar » Unassigned
FileSize
4 KB
588 bytes

Resolved one coding standard issue with an interdiff.

mikeryan’s picture

I don't have a good enough understanding of HAL type links and caching to intelligently review the code - but, it fixed a problem for me where after a drush site-install with our custom entity and *slightly* customized REST endpoints, a PUT of our entity would fail with "Type https://example.ddev.site/rest/type/our_entity/our_entity does not correspond to an entity on this site" unless we did an extra drush cr first. One thumb up.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Berdir’s picture

Status: Needs review » Needs work
+++ b/core/modules/hal/src/LinkManager/TypeLinkManager.php
@@ -99,6 +99,29 @@ public function getTypeUri($entity_type, $bundle, $context = []) {
+    // If we're changing domains we may need to invalidate the cache. Look and
+    // see if there are any old domains in the cache and if so invalidate.
+    if ($old_domain != $domain) {
+      $cache_values = $this->cache->get('hal:links:types');
+      if ($cache_values) {
+        foreach (array_keys($cache_values->data) as $bundle_uri) {
+          if (strpos($bundle_uri, $old_domain) === 0) {
+            $this->cache->invalidate('hal:links:types');
+            break;
+          }
+        }
+      }
+    }
+    return $this;

wouldn't it be easier to just make the link domain part of the cache key, aka making it a poormans-cache context? would just work then.

Also, The cacheability of the resulting page also doesn't correctly include the cache tag yet if the setting is currently empty.

ericdsd’s picture

#15 fixed my issue of import with default_content_deploy but but only using latest release (1.0-beta2 from april 15th) default_content_deploy 8.x-1.0-beta2
and including --uri parameter to my drush command to avoid this

vagrant@devd8:/var/www/drupalvm/d8vm/rd8.vbox.local/public_html$ drush dcdi --force-override
In ContentEntityNormalizer.php line 268:
 Type http://default/rest/type/block_content/basic does not correspond to an entity on this site.  using 

using

drush dcdi --uri=http://mysiteurl.com --force-override

instead works.

Tested on core 8.8.5

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

mkalkbrenner’s picture

One Drupal installation can serve multiple domains, protocols or ports. The current implementation in core regarding the cache is simply wrong. Also the approach in the proposed patch is wrong and is not robust against race conditions.

Here's a failing patch first.

mkalkbrenner’s picture

Title: Old link domain remains in link type cache after setting another » Type links are broken if diffferent domains, protocols or ports are used in multisite or multi-domain setup
Priority: Normal » Critical
Issue summary: View changes

I think berdir's proposal from #18 is the correct way to fix the issue without any race conditions or performance loss.

But as mentioned before, the current implementation must be considered to be broken. Multisite or multi-domain setups will fail.
Contrib modules like Default Content, Default Content Deploy, Domain Access and other are currently affected.
In core language negotiation by domain is incompatible for example.

mkalkbrenner’s picture

Relations are also affected, not just types.

mkalkbrenner’s picture

Title: Type links are broken if diffferent domains, protocols or ports are used in multisite or multi-domain setup » HAL links are broken if diffferent domains, protocols or ports are used in multisite or multi-domain setup
mkalkbrenner’s picture

Another try of a failing test.

mkalkbrenner’s picture

I missed to fetch the cached values in the previous tries of a failing patch.

Status: Needs review » Needs work

The last submitted patch, 28: 2928882_failing_test_3.patch, failed testing. View results

mkalkbrenner’s picture

Now the patch to solve the issue.

Berdir’s picture

+++ b/core/modules/hal/src/LinkManager/RelationLinkManager.php
@@ -155,7 +158,7 @@ protected function getRelations($context = []) {
    */
-  protected function writeCache($context = []) {
+  protected function writeCache($cid, $context = []) {
     $data = [];
 

changing the method just so we don't have to call getLinkDomain() a second time doesn't really seem worth it?

mkalkbrenner’s picture

changing the method just so we don't have to call getLinkDomain() a second time doesn't really seem worth it?

I checked the sources and didn't find any other calls of this protected method. But I don't care. If you'll set RTBC I'll change it to be be 100% backwards compatible.

BTW I run into this issue because of default_content_deploy.

mkalkbrenner’s picture

Berdir’s picture

Pretty sure it's much more likely to be committed without that change. We try to avoid changing protected methods, keep deprecated protected methods around and so on. There are known subclasses of this, it's theoretically possible that a custom project overrides or calls it in a subclass?

Patch looks good to me, we will wait for testbot to confirm.

mkalkbrenner’s picture

@berdir RTBC?

Berdir’s picture

Status: Needs review » Reviewed & tested by the community

Yeah, I think this is RTBC. fixes the problem with minimal changes and has test coverage.

catch’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/core/modules/hal/src/LinkManager/TypeLinkManager.php
@@ -152,7 +152,7 @@ protected function writeCache($context = []) {
+    $this->cache->set('hal:links:types:' . $this->getLinkDomain($context), $data, Cache::PERMANENT, ['entity_types']);

Should this get a cache tag for hal.settings?

https://api.drupal.org/api/drupal/core%21modules%21hal%21src%21LinkManag...

Also doesn't look like there's test coverage for the link domain being changed in config.

mkalkbrenner’s picture

Status: Needs work » Needs review

Should this get a cache tag for hal.settings?

I don't think so. The cache must not be invalidated when link_domain in hal.settings changes. This was an issue before this patch!
Now the cache entry can remain regardless that the link domain changed and regardless how it changed, via settings or via API calls at runtime.

mkalkbrenner’s picture

If you're argument is to save cache space when the setting changes, then we can add the tag.

Berdir’s picture

Status: Needs review » Reviewed & tested by the community

Cache tags don't clear space. And yes, the configured url is basically a cache context now with this patch.

  • catch committed 7dabb04 on 9.3.x
    Issue #2928882 by mkalkbrenner, neclimdul, manuel.adan, yogeshmpawar,...
catch’s picture

Status: Reviewed & tested by the community » Fixed

Oh good point didn't think it all the way through, so yeah when the settings are changed the cache key changes and everything will work fine.

Committed 7dabb04 and pushed to 9.3.x. Also cherry-picked to 9.2.x. Thanks!

  • catch committed e3dbbbd on 9.2.x
    Issue #2928882 by mkalkbrenner, neclimdul, manuel.adan, yogeshmpawar,...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.