Problem/Motivation

This is a blocker for #2352155: Remove HtmlFragment/HtmlPage.

Any remaining drupal_render() calls that were called with $is_recursive = FALSE (the default), where they really shouldn't have been, need to be fixed.

drupal_render()'s optional second argument ($is_recursive_call, defaulting to FALSE), which is a huge PITA for developers and themers alike

  • HEAD today requires (but probably nobody knows, since not even HEAD does it correctly everywhere): drupal_render($build, TRUE) in 99% of cases
  • but everybody expects to be able to do just drupal_render($build)

While working on #2352155: Remove HtmlFragment/HtmlPage, I noticed that #post_render_cache callbacks and #attached assets were no longer being applied for the main content. Turns out the root cause was once again "wrong" drupal_render() calls; these caused the stack (that is used internally, see #2273277: Figure out a solution for the problematic interaction between the render system and the theme system when using #pre_render) not to be empty at the end of a "root" (non-recursive) drupal_render() call, which effectively means that some of the bubbleable metadata is being left behind in the stack, and is missing from the final result for the "root" call.

Proposed resolution

I first embarked on a mission to fix all drupal_render() calls to have $is_recursive = FALSE… but since this is the majority of the calls, it really is:

  1. A fool's errand
  2. Bad DX, because it means 99% of the time a Drupal developer uses drupal_render() without specifying $is_recursive = FALSE, that that is wrong.

So instead of "fixing" all existing drupal_render() calls, I instead fixed drupal_render() itself: the signature changed from

function drupal_render(&$elements, $is_recursive_call = FALSE) {}

to

function drupal_render(&$elements, $is_root_call = FALSE) {}

i.e. I inverted the second argument's meaning. Now the default is once again drupal_render($build), and only when rendering the final markup (i.e. just before generating a Response). The dozen or so cases in Drupal core that need to do this were updated to do this.

To prevent this problem in the future, even though the above already diminished the chance of this happening again greatly, drupal_render() now throws an exception whenever a root call to it does not end with an empty stack.

Note: All the conversions in the patch are just a sign that Drupal has a lot of special cases where you actually need a root call

To make the root calls more understandable and more discoverable, this introduces drupal_render_root($e), which is an alias for drupal_render($e, TRUE)

(Files with $is_root_call = TRUE calls for drupal_render(): whenever rendering the "final" markup, which happens in authorize.php, update.php, install.php, exception handling, maintenance mode and when rendering render arrays for non-HTML responses, such as feeds.)

As part of this work, we'll actually fix #2238835: #post_render_cache + #attached as well!

Remaining tasks

  1. Get green.
  2. Review.

User interface changes

None.

API changes

  1. Change: drupal_render()'s optional second argument now has an inverted meaning, to hugely reduce the burden imposed on developers and themers using drupal_render(): they'll be able to do drupal_render($build) rather than what they have to do in HEAD today (but probably nobody knows, since not even HEAD does it correctly everywhere): drupal_render($build, TRUE).
  2. Addition: drupal_render_root($e), which is an alias for drupal_render($e, TRUE)

Comments

wim leers’s picture

Status: Active » Needs review
StatusFileSize
new34.33 KB

This is a quick port from the parent issue's patch, the comment form is broken, so it looks like #post_render_cache callbacks in general don't work. Should be an easy fix somewhere in HEAD's page rendering pipeline.

wim leers’s picture

StatusFileSize
new42.48 KB
new6.87 KB

The patch in #1 contains only the hunks that come unchanged from the parent issue's patch. This one adds the changes necessary for HEAD's page rendering pipeline. But there's a mistake in this interdiff, since the comment form is broken.

The last submitted patch, 1: drupal_render_dx_tx-2361681-1.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 2: drupal_render_dx_tx-2361681-2.patch, failed testing.

wim leers’s picture

Great, our test coverage is catching this!

fabianx’s picture

#2331393: drupal_render() recursive parameter usage is unclear, safe default should be drupal_render(x) was the original issue.

Lets close one as a duplicate, please.

Thanks for your work!

dawehner’s picture

Issue summary: View changes
  1. +++ b/core/authorize.php
    @@ -104,7 +104,7 @@ function authorize_access_allowed() {
    -    $output = drupal_render($authorize_report);
    +    $output = drupal_render($authorize_report, TRUE);
    

    I updated the IS to make clear that drupal_render() is the common case for contrib ...

  2. +++ b/core/includes/common.inc
    @@ -2708,18 +2708,27 @@ function drupal_prepare_page($page) {
    - * @param bool $is_recursive_call
    + * @param bool $is_root_call
      *   Whether this is a recursive call or not, for internal use.
      *
      * @return string
    

    Can you please update the documentation to clearly define when to pass TRUE and when not? You should not have to read the entire drupal_render() documentation for it.

  3. +++ b/core/includes/common.inc
    @@ -2708,18 +2708,27 @@ function drupal_prepare_page($page) {
      */
    -function drupal_render(&$elements, $is_recursive_call = FALSE) {
    +function drupal_render(&$elements, $is_root_call = FALSE) {
    

    ... just as a follow up, I think it would be nice to have a drupal_render_root() which just does drupal_render(..., TRUE) which would be a little bit easier to read in the code.

  4. +++ b/core/modules/ckeditor/ckeditor.module
    @@ -96,3 +96,75 @@ function _ckeditor_theme_css($theme = NULL) {
    + */
    +function ckeditor_rebuild() {
    +  /** @var \Drupal\filter\FilterFormatInterface[] $formats */
    +  $formats = filter_formats();
    

    Did you considered to have a wrapper around the cache entry instead and just invalidate it in hook_rebuild()?

  5. +++ b/core/modules/views/src/Plugin/views/area/HTTPStatusCode.php
    @@ -64,8 +64,8 @@ public function buildOptionsForm(&$form, FormStateInterface $form_state) {
         if (!$empty || !empty($this->options['empty'])) {
    -      $this->view->getResponse()->setStatusCode($this->options['status_code']);
    -      $this->view->getRequest()->attributes->set('_http_statuscode', $this->options['status_code']);
    +      $build['#attached']['http_header'][] = ['Status', $this->options['status_code']];
    +      return $build;
         }
    

    Let's remove that hunk for now.

wim leers’s picture

Status: Needs work » Needs review
StatusFileSize
new44.69 KB
new31.69 KB

#6: I knew there already was an issue, I just couldn't find it! Sorry :( Since this one has all the activity, let's continue here. I've re-read that issue though, to make sure we don't forget anything.

#7:

  1. dawehner++
  2. Done; please review.
  3. Agreed! I'm doing it right here, since that'll also make the patch review a bit easier.
  4. It's not entirely clear to me what you mean here by "wrapper around the cache entry". But invalidating in hook_rebuild() (which then should happen in hook_cache_flush(), actually, but that's an aside) is not what we need. What we need is to build the cache outside of a drupal_render() call tree, and just invalidating it outside of the call tree is not sufficient, because that would still cause the #pre_render callback that attaches CKEditor settings to the text_format selector to recalculate the settings in that #pre_render callback and hence execute check_markup(), which calls drupal_render_root(), which is precisely the problem this needs to avoid.
  5. Oops, that was accidental. Removed. I already suspected this due to the test failures.
wim leers’s picture

dawehner’s picture

Status: Needs review » Reviewed & tested by the community
+++ b/core/authorize.php
@@ -104,7 +104,7 @@ function authorize_access_allowed() {
-    $output = drupal_render($authorize_report, TRUE);
+    $output = drupal_render_root($authorize_report);

Nice, this is much more explicit

fabianx’s picture

Issue tags: +Needs change record

RTBC + 1, but needs a short change record - if I see this correctly - or rather update the existing one.

Leaving as RTBC for core committers to decide what needs to be documented here as change.

catch’s picture

Status: Reviewed & tested by the community » Needs review
-
+    $format_tags = $this->cache->get('ckeditor_internal_format_tags:' . $format->id())->data;
     return $format_tags;

This has no fallback for if the cache item goes missing at all - for example a memcache eviction.

Should it go into state instead?

wim leers’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: -Needs change record
StatusFileSize
new44.72 KB
new2.42 KB

#11: CR created: https://www.drupal.org/node/2362887.

#12: Wow, that completely escaped me. And also dawehner and Fabianx apparently. Awesome catch! It indeed belongs in State.
I'd put this as NR, but since the changes are trivial, and the test coverage for the CKEditor module is solid, I'm moving this back to RTBC.

fabianx’s picture

RTBC + 1 again

FWIW, I saw it and wondered why there was no rebuild called on cache miss, but figured there some something else I was missing and given there was a rebuild function added I figured it would be called in that case.

Next time will speak up. Thanks!

Speaking of that:

- What happens if state is not yet populated at that point?

catch’s picture

Status: Reviewed & tested by the community » Needs review

Yes that's a good question, needs a default to empty array in the case of no formats saved yet I think?

wim leers’s picture

If no Text Formats exist, then there's nothing to attach a Text Editor to in the first place, so this code would never run? :)

Am I missing something?

wim leers’s picture

StatusFileSize
new45.59 KB
new45.44 KB
new2.17 KB

Talked to catch in IRC, he said: "But what if the user manually deletes things in State?" — to which I can only answer: then CKEditor will fail to initialize (I tested it manually), because it receives an invalid setting.

So it should indeed provide a default value. Comes with updated test coverage, to verify that all continues to work fine when manually deleting things in State.

fabianx’s picture

Status: Needs review » Reviewed & tested by the community

Back to RTBC then :).

catch’s picture

Status: Reviewed & tested by the community » Fixed

I asked Wim why we still even call check_markup() at all, and he reminded me it's been left around only for the case where you only want the final rendered string (and to completely discard #attached etc.), for example in an e-mail.

This means that https://api.drupal.org/api/drupal/core%21modules%21views%21src%21Plugin%... is using it wrong.

But ckeditor, while it's doing strange things here, is using it right.

I didn't have any other feedback apart from #12, and that's been dealt with.

So... Committed/pushed to 8.0.x, thanks!

  • catch committed 2d3266e on 8.0.x
    Issue #2361681 by Wim Leers: drupal_render(): invert second argument ($...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.