New #post_render_cache property for Render API: personalization without breaking render cache

The rationale explained in this change record still applies, but #post_render_cache was since replaced with an easier-to-use, more powerful concept: placeholders. See https://www.drupal.org/node/2498803.

If rendering something (e.g. a node) is expensive, then render caching (by using the #cache property) is a good way to avoid doing repeating that expensive rendering. Render caching was introduced in Drupal 7.

This is what enabled render caching could look like for entities:

$element['#cache'] = array(
  'keys' => array('entity_view', $entity_type, $entity_id, $view_mode),
  'granularity' => DRUPAL_CACHE_PER_ROLE,
);

And this is the data for a render cache entry:

$render_cached_value = array(
  '#markup' => 'The entire rendered mark-up, i.e. no more render arrays to be processed!',
  '#attached' => array(
    'css' => array(
      'sites/all/modules/mymodule/foo.css' => array(),
    ),
    'library' => array(
      array('system', 'drupal'),
    ),
  ),
);

(As you can see, a render cache entry itself is a render array, but a "flattened" one: it only contains the entire final markup in #markup and any attachments that are associated with/attached to that mark-up.)

However, one caveat when using render caching is that everything contained in the render array with render caching enabled (i.e. #cache set) must match the granularity of the containing item. In the provided example, that means that e.g. a comment entity's links may only be role-specific. However, that essential requirement actually prevents us from applying render caching in many cases. For example: a comment's "edit" link may only appear if the user has the "edit own comments" permission. Another example: the "post a new comment form" is usually rendered as part of the node, but the comment form must be tailored to the current authenticated user. And a final example: some #attached JavaScript setting may need to contain user-specific data … but since it's in the render array, it would get render cached and all users would get the same setting. Plenty more examples can be found.

All of these problems point to one important need: the need to render cache the majority of some markup, but to still be able to personalize either parts of the markup, or an attached asset (CSS, JS or JS setting).

That's the need #post_render_cache addresses.

Using #post_render_cache

For example, you may want to provide a JS setting (a drupalSettings entry) to let some JS on the front-end know what the last time was that the current user has read a certain node (this is necessary to make the "X new comments" link work). That's clearly user-specific, so you should use #post_render_cache:

+      // Embed the metadata for the "X new comments" link (if any) on this node.
+      $entity->content['links']['#post_render_cache']['history_attach_timestamp'] = array(
+        array('node_id' => $entity->id()),
+      );

So #post_render_cache accepts a callback (function name or static class method) as a key, with an array of contexts as corresponding value. The callback will then be called once for each context and will receive the render-cached element and must return that element, with whatever updates are needed. In the above example, there's just one: $context = array('node_id' => 'some node ID'). So there will be only one invocation of history_attach_timestamp().
Repeated for clarity: history_attach_timestamp() will be called only after the render cache entry has been retrieved, so the personalized data won't end up in the render cache.

+/**
+ * #post_render_cache callback; attaches the last read timestamp for a node.
+ *
+ * @param array $element
+ *  A render array with the following keys:
+ *    - #markup
+ *    - #attached
+ * @param array $context
+ *  An array with the following keys:
+ *    - node_id: the node ID for which to attach the last read timestamp.
+ *
+ * @return array $element
+ *   The updated $element.
+ */
+function history_attach_timestamp(array $element, array $context) {
+  $element['#attached']['js'][] = array(
+    'type' => 'setting',
+    'data' => array(
+      'history' => array(
+        'lastReadTimestamps' => array(
+          $context['node_id'] => (int) history_read($context['node_id']),
+        )
+      ),
+    ),
+  );
+
+  return $element;
+}

Using drupal_render_cache_generate_token() and drupal_render_cache_generate_placeholder()

These are helper functions for the most common use case of #post_render_cache: a small subset of a larger piece of markup that needs to be personalized, even though the majority of the markup remains the same for everybody. Instead of having to generate your own placeholders (which must have a random/unique identifier) and then find them in your #post_render_cache callback implementation, Drupal can do all that work for you automatically! :)

Instead, you can use this much simpler syntax:

+ $callback = '\Drupal\comment\Plugin\Field\FieldFormatter\CommentDefaultFormatter::renderForm';
+ $context = array(
+   'entity_type' => $entity->getEntityTypeId(),
+   'entity_id' => $entity->id(),
+   'field_name' => $field_name,
+   'token' => drupal_render_cache_generate_token(),
+ );
+ $output['comment_form'] = array(
+   '#post_render_cache' => array(
+     $callback => array(
+       $context,
+     ),
+   ),
+   '#markup' => drupal_render_cache_generate_placeholder($callback, $context, $context['token']),
+ );

The callback is analogous to the one above, except that this time we will find and replace the placeholder in $element['#markup'], again using the drupal_render_cache_generate_placeholder() helper function:

+ /**
+  * #post_render_cache callback; replaces placeholder with comment form.
+  *
+  * @param array $element
+  *   The renderable array that contains the to be replaced placeholder.
+  * @param array $context
+  *   An array with the following keys:
+  *   - entity_type: an entity type
+  *   - entity_id: an entity ID
+  *   - field_name: a comment field name
+  *
+  * @return array
+  *   A renderable array containing the comment form.
+  */
+ public static function renderForm(array $element, array $context) {
+   $callback = '\Drupal\comment\Plugin\Field\FieldFormatter\CommentDefaultFormatter::renderForm';
+   $placeholder = drupal_render_cache_generate_placeholder($callback, $context, $context['token']);
+   $entity = entity_load($context['entity_type'], $context['entity_id']);
+   $form = comment_add($entity, $context['field_name']);
+   $markup = drupal_render($form, TRUE);
+   $element['#markup'] = str_replace($placeholder, $markup, $element['#markup']);
+ 
+   return $element;
+ }

There is one edge case, for very advanced use cases: if there are multiple/many (dozens or hundreds) occurrences of a certain piece of markup that is personalized and they all must be replaced with the same value, you should use the same token (drupal_render_cache_generate_token()) for every placeholder (drupal_render_cache_generate_placeholder()) you generate, so you can replace all of the placeholders in a single #post_render_cache callback invocation.

Important: also works when render caching is disabled!

It is crucial that #post_render_cache callbacks continue to work when e.g. render caching is disabled on a node because somebody decided to set '#cache' => FALSE. That's why it will also work when #cache is not set!

The consequence is that it's always okay to use #post_render_cache for personalized markup! And in fact, it is strongly encouraged, because if you do so, then your code will automatically be compatible with render caching!

Background info

The solution had to meet six requirements:

1. Conceptually easy to reason about. In other words: sufficiently simple DX. (Points 2, 3 and 4 aid in this.)
2. It should be possible to use the same mechanism to either replace a placeholder in the markup, or attach additional JavaScript settings/libraries, or both.
3. Must continue to work when multiple render cacheable things are merged into one (i.e. #cache is set on an element, but also on its child or grandchild or …). In other words: nested #post_render_cache callbacks must continue to work even when they're no longer nested, after having been stored in/retrieved from the render cache
4. Even when not using render caching (i.e. an element that does not have #cache set), #post_render_cache callbacks must be executed. This allows (contrib module) developers to inject/alter render arrays with personalized data without breaking the render cache, and more importantly, without having to implement the same functionality in two ways: one implementation for when render caching is disabled, another for when it is enabled.
5. Must have a unique/random identifier for each placeholder, to guarantee the #post_render_cache callback will never accidentally replace user-generated content.
6. The default solution should be optimized to work out-of-the-box on any hosting. Roughly, there are two broad types of hosting: shared hosting and enterprise hosting (with a reverse proxy such as Varnish in front of the web servers). There are more types, but these are the extremes. Drupal core should accommodate both cases out of the box, and if that is impossible, it should favor shared hosting.

See #2118703: Introduce #post_render_cache callback to allow for personalization without breaking the render cache for an extensive explanation and discussion.