Stop clearing the persistent entity cache when building the sitemap

Realised that related entities such as paragraphs might end up getting stuck in the static cache if they are loaded when we load the entity.

@gbyte just found #3170261: Entity static cache can cause memory exhaustion during sitemap regeneration and saw that the original solution there was pretty much what I'd recommend. Going to upload something very similar to #3170261-2: Entity static cache can cause memory exhaustion during sitemap regeneration. Please give @mstef issue credit too.

Hopefully at some point in the future one of:

• #1199866: Add an in-memory LRU cache
• #3190992: Add a WeakReference memory cache implementation

will land and then \Drupal\simple_sitemap\Plugin\simple_sitemap\UrlGenerator\EntityUrlGenerator::generate() can be removed because core will handle it for you.

Created a MR to clear the entity memory cache after generating the url instead of resetting the entity cache. This will result in sitemap generation doing less queries to the db cache backend because it's stop clearing it :D

I've generated 5000 nodes with paragraph fields and set the content type to be indexed see - https://gist.github.com/alexpott/874a0480b7097d7d97322c053dec6634

I'm generating the sitemap doing vendor/bin/drush ssg -v

The results show that subsequent runs are quicker because sitemap generation is no longer obliterating the entity cache.

With patch

Run 1

>  [notice] 3000 out of 5001 total items have been processed. memory: 54.5 MB, non-peak mem: 52.5 MB
>  [notice] 5001 out of 5001 total items have been processed. memory: 56.5 MB, non-peak mem: 54.5 MB

________________________________________________________
Executed in   16.84 secs   fish           external
   usr time    7.89 secs  106.00 micros    7.89 secs
   sys time    0.96 secs  690.00 micros    0.96 secs

Run 2

>  [notice] 5001 out of 5001 total items have been processed. memory: 54.5 MB, non-peak mem: 54.5 MB

________________________________________________________
Executed in    9.55 secs   fish           external
   usr time    5.16 secs  113.00 micros    5.16 secs
   sys time    0.62 secs  703.00 micros    0.62 secs

Without the patch

Run 1

>  [notice] 3311 out of 5001 total items have been processed. memory: 54.5 MB, non-peak mem: 52.5 MB
>  [notice] 5001 out of 5001 total items have been processed. memory: 56.5 MB, non-peak mem: 54.5 MB

________________________________________________________
Executed in   15.79 secs   fish           external
   usr time    7.12 secs  111.00 micros    7.12 secs
   sys time    0.96 secs  677.00 micros    0.96 secs

Run 2

>  [notice] 4487 out of 5001 total items have been processed. memory: 54.5 MB, non-peak mem: 54.5 MB
>  [notice] 5001 out of 5001 total items have been processed. memory: 54.5 MB, non-peak mem: 54.5 MB

________________________________________________________
Executed in   11.48 secs   fish           external
   usr time    5.36 secs  116.00 micros    5.36 secs
   sys time    0.72 secs  782.00 micros    0.71 secs

The reason for the speed up in subsequent runs is because it does way way less queries. With the patch subsequent runs is do around 25000 queries and without the patch we're doing around 50000 queries!

I've created #3202290: Add performance test script to make performance testing simpler in the future. It feels worth it because this module often has to deal with huge amounts of data and small changes can have a very big impact.

I've answered the review question. I think there's a misunderstanding of what deleteAll() is clearing. It is only clearing the entities from the memory used by the current process.