This is part of #2015147: [meta] Improve the DX of drupal_render(), renderable arrays and the Render API in general.

As part of the discussion from #1830588: [META] remove drupal_set_title() and drupal_get_title(), @moshe sums this up nicely:

Well, the guideline since D7 has been that you don't call drupal_render(). You are supposed to return render arrays from blocks and page callbacks and then kick off all rendering at the end with drupal_render_page(). I notice that we have been adding calls to drupal_render() instead of removing them in issues like #2006152: [meta] Don't call theme() directly anywhere outside drupal_render(). Lets try not muddy the render pipeline with more calls like that. We have to figure out a way to handle nested theme() calls without using drupal_render(). Or disallow nested theme() calls.

The issue mentioned by @moshe is trying to remove theme() calls (which are "worse" than drupal_render() calls), as part of a general API cleanup but the sentiment expressed by @moshe is valid in a more general sense - the more we can avoid "early rendering" render arrays into strings, regardless of the exact function doing the rendering, the better.

"Early rendering" introduces a few issues:

  • Code is harder to read and write because you have to be mindful of and mentally track how arrays and render calls are interleaved
  • It's harder for alters and other functions that want to process structured data to do their work on already-rendered strings than organised associative arrays. This is expecially damaging for the efforts to make variables in Twig templates "drillable".
  • In certain situations it can be a waste of CPU, e.g. consider the case where we do the work to render a child element and a parent element has #access => FALSE
  • Rendering elements can have "side effects" like introducing new javascript or CSS files into the page and this may not be appropriate if the early-rendered element ends up not being displayed in the page
  • The more we scatter calls to functions like theme() and drupal_render() around core, the harder it is for core contributors to see what the effects of making a modification to one of these API functions will be in all situations
  • Early rendering arrays that are used in attributes of other arrays lets us "stick our heads in the sand" rather than fix underlying incompatibilities between element configurations that are commonly used together
  • Early rendering elements also "flattens" their children too, meaning that we lose the structure of this element's children that may have been useful/important inside Twig templates that are the parent of this element

Of course, it's not as simple as just blindly returning arrays instead of strings everywhere that drupal_render() currently appears. Often there are good reasons why an early render is happening, either due to limitations in the Render API itself or limitations in the implementation of a particular element #type/#theme.

A common enough example of how this could happen is something like the following:

// Render array to build up a string commonly used as a description for other elements.
$foo = array(
  '#type' => 'foo',
  ...
  ...
);

// Render array that has a #description attribute that will throw an exception for anything other than a string.
$bar = array(
  '#type' => 'bar',
  '#description' => drupal_render($foo).
  ...
  ...
);
return $bar;

Here we see that unless the $bar element with the #description attribute (that, for the sake of this example, is blindly concatenated onto a string inside a theme function) is provided with some way to render #description arrays into strings (hopefully by calling drupal_render() during the element's processing phases before rendering) we have to "early render" $foo.

Another solution to this problem is to simply use the existing concept of "child elements" in render arrays. Depending on what $bar is trying to do, the following may or may not be an appropriate refactor that requires less structural change to the way #type 'bar' is implemented:

// Render array to build up a string commonly used as a description for other elements.
$foo = array(
  '#type' => 'foo',
  ...
  ...
);

// Render array that has a #description attribute that will throw an exception for anything other than a string.
$bar = array(
  '#type' => 'bar',
  'description' => $foo.
  ...
  ...
);
return $bar;

There are many (literally hundreds in core) calls to drupal_render() so we have a lot of work to do, but every little bit we clean up will make the API more coherent overall so we should just start filing issues and working through them - https://api.drupal.org/api/drupal/core%21includes%21common.inc/function/...

Hit list

Duplicate/superfluous drupal_render() calls

Functions that concatenate markup strings without using the Render API

Blocking issues

Related issues

#2056837: Modules can no longer alter page_bottom
#2060481: theme hooks are documented as responsible for rendering child elements in render arrays, most don't do this, many are incapable

Comments

thedavidmeister’s picture

Issue summary: View changes

Updated issue summary.

thedavidmeister’s picture

Issue summary: View changes

Updated issue summary.

Crell’s picture

the more we can avoid "early rendering" render arrays into strings, regardless of the exact function doing the rendering, the better.

I have to say I emphatically disagree. We should be more consistent with when we render things, absolutely. But the idea of building up One Giant Array To Rule Them All that we render all at once of the end of the page is the design flaw that WSCCI/SCOTCH have been trying to move away from for 3 years. That knot is particularly tightly wound (as the theme folks have run into, too), but the goal has always been to get the point that there is no such thing as a page array. Every block gets rendered down to a string individually, and there is no way to modify the entire page at once.

Quite simply, as long as we support modifying the page as a whole, we cannot do partial page rendering or partial page caching in a good way. Those two are incompatible.

Now, a best practice of "block::build() and controllers always return a render array", sure, that's fine. But 1) We have to still allow strings for DX (since we apparently got rid of #markup) and 2) whatever comes back from the block/controller needs to be string-ified on the spot, period.

thedavidmeister’s picture

That's a great point, any work done here should be tempered with those wise words in mind :)

When I wrote this issue summary I was definitely thinking more about things like theme_file_upload_help() that do nothing but render a string that is intended to be used in another render array's description attribute, or issues where we see drupal_render() sticking rendered markup straight into #children for other render arrays - #2043649: Make all #type 'link' arrays 100% renderable, use #theme_wrappers if necessary.

There are literally hundreds of calls to drupal_render() in core and many (most?) can be consolidated without hurting the ability to do partial page rendering and we get the tangible benefits outlined in the issue summary for each one we clean up.

So, to be clear, despite the quote from @moshe at the top of the issue summary sort-of implying we only ever want to render everything as a "page" (he did mention blocks, actually), I'm totally ok with rendering render arrays for blocks and other forms of "partial page rendering" well before we get to the "page level" of granularity with one mega-array.

I've tagged this with "Needs issue summary update" because it should be made clearer that we're not trying to turn everything back into a "page" here.

Oh also, I think you may have misinterpreted what I meant by "regardless of the exact function doing the rendering" - I meant "regardless of whether theme() or drupal_render() is handling the rendering", not "regardless of the function within which we are constructing a renderable array to be rendered".

since we apparently got rid of #markup

Um, I don't think so. #markup still exists and is in wide circulation within core (I get 399 matches for #markup doing a quick search), there's just no #type being shivved artificially into those render arrays that define '#markup', trying to suggest they're all somehow functionally related simply by virtue of being a string.

#markup is even now explicitly mentioned in the docs for drupal_render() - https://api.drupal.org/api/drupal/core%21includes%21common.inc/function/...

thedavidmeister’s picture

Title: [meta] Avoid calling drupal_render() "early" wherever possible, build and return structured data instead » [meta] Avoid calling drupal_render() "early", wherever it is beneficial to do so build and return structured data instead
pplantinga’s picture

Would it be helpful to divide this up somewhat like #2006152: [meta] Don't call theme() directly anywhere outside drupal_render()?

Fabianx’s picture

If #bar does not support render arrays, what about using the RenderWrapper instead with the array and drupal_render as a third solution?

For #bar it is just an object with __toString method.

star-szr’s picture

@pplantinga - definitely, but I think we should allow for more discussion first.

thedavidmeister’s picture

Issue summary: View changes

Updated issue summary.

thedavidmeister’s picture

yeah, this will absolutely need to be split up. That's a lot of work though as each call to drupal_render() will need to be "audited" individually, which is the reason it hasn't happened yet. I was thinking #2025629: [meta] Ensure that all #theme/#theme_wrappers hooks provided by core are associated with a sensible, re-usable element #type could happen first (but only after the "don't call theme" issue) to bring some more consistency to what's already in core, it might make the audit easier if everything is based on a #type rather than having a few different render array structures floating around.

Eric_A’s picture

The strategy to do straight and "Novice" conversions from theme() to drupal_render() is leading to instances of '#markup' => drupal_render($foo) and of return drupal_render($foo) in some menu page callback functions. The first is to be replaced by a (child) element. The second should just return the render element. It would be good to clean these up before beta 1. We can add types later, but we'd better get the elements in place in time.

thedavidmeister’s picture

well I can't say anything for other people's priorities, all I know is that if we can wait on the two issues linked in #4 and #7 before we start working on this issue it will reduce the amount of work we have to do here.

Personally I think making the render API more consistent/easier to use for developers building sites by consolidating theme()/#theme/#type is a higher priority than changing the way it's used internally - even though the early renders are "abuse" of the system, it's no worse than what we've always done until now with theme().

Also, if you want this issue to start moving along, feel free to start auditing drupal_render() calls and documenting it in sub-issues. I put all the issues up for the other two linked meta issues and I have an idea of how long it will take to just do the pre-requisite research on "legit" and "abusive" drupal_render() calls, before we can even start looking at patches - I was planning to do the audit for this one as well, but I don't have enough time for it this weekend.

I was thinking of doing something like 1 issue per module with a list of all the drupal_render() calls within that module in the issue summary. We can explain why each call is appropriate or inappropriate there.

Once we've done a sweep of all the modules, we should go back over the sub issues and look for broad "categories" of abusive drupal_render() calls (like "renders into #markup", for example) then write-up instructions on the preferred way to convert each category in this issue summary.

Once we're at this point we can hopefully start tagging some/most/all of the issues as novice issues as we'll have a clear list of instructions for what needs to be done and none of the individual patches will touch many lines of code. As soon as we've turned this meta issue into novice issues, they'll get cleaned up really fast :)

thedavidmeister’s picture

Title: [meta] Avoid calling drupal_render() "early", wherever it is beneficial to do so build and return structured data instead » [meta] Avoid calling drupal_render() "early", wherever it is beneficial to do so, build and return structured data instead
thedavidmeister’s picture

Issue summary: View changes

Updated issue summary.

pwolanin’s picture

We have a goal of removing the use of system paths from core. So l() in it's current for should be marked deprecated and we should stop using it.

Should we change it to take route + parameters in some way? Otherwise, I think the original title might have been correct. We need a simple way to render a link from a route + parameters?

We should probably alter #type => link to handle #route and #parameters and use the appropriate link-generation function.

thedavidmeister’s picture

#11:

Yeah, I guess in an ideal world l() would just internally build a render array and wrap drupal_render() and then one day we could just delete it when we think nobody is still using it. The problem with that historically has been performance (l() is called literally hundreds of times on many pages), but the profiling that I'm aware of largely focussed on the overhead of calling theme(). If #2052253: [META] Add #render property to drupal_render() and convert #type "#pre_render -> #markup" calls to use it landed we may be able to re-open that discussion as the performance of #type 'link' might become much closer to the current performance of l().

I have a bunch of relevant issues, closed and open, that I can link to for context around the discussion of what to do about rendering links from the POV of avoiding "early render" while supporting routes but I think it should be a new, child issue of this meta and we can track progress there.

thedavidmeister’s picture

Issue summary: View changes

add related

thedavidmeister’s picture

Issue summary: View changes

Updated issue summary.

thedavidmeister’s picture

Issue summary: View changes

Updated issue summary.

thedavidmeister’s picture

Issue summary: View changes

Updated issue summary.

thedavidmeister’s picture

Title: [meta] Avoid calling drupal_render() "early", wherever it is beneficial to do so, build and return structured data instead » [meta] Avoid "early rendered" strings where beneficial to do so, build structured data to pass to drupal_render() once instead

updating title to reflect that "early rendering" is not always caused by drupal_render(). Functions that concatenates an un-alterable, un-themeable string of markup to just pass it off to drupal_render() as an attribute of some theme hook are just as bad as early calls to drupal_render().

Also, I started putting a few issues up if anyone wants to take a crack at some patches :)

thedavidmeister’s picture

#2060481: theme hooks are documented as responsible for rendering child elements in render arrays, most don't do this, many are incapable is related as we're currently forced to jump through some hoops to avoid needing to turn everything into a string immediately just to make our system of theme hooks work.

thedavidmeister’s picture

Issue summary: View changes

Updated issue summary.

thedavidmeister’s picture

Issue summary: View changes

Updated issue summary.

thedavidmeister’s picture

Issue summary: View changes

Updated issue summary.

thedavidmeister’s picture

Issue summary: View changes

Updated issue summary.

thedavidmeister’s picture

Issue summary: View changes

Updated issue summary.

thedavidmeister’s picture

Issue summary: View changes

Updated issue summary.

thedavidmeister’s picture

In #2078287: [meta] Fix the DX of the link generator, @msonnabaum asserted that:

That is not a performance hit worth considering.

With regard to converting links from calling the generator directly to render arrays.

If that's the case, as per #11 and #12, I'd love to put up some issues around converting links over to render arrays where they are currently l() (or equivalent)

thedavidmeister’s picture

Issue summary: View changes

Updated issue summary.

sun’s picture

andypost’s picture

I closed all issues from IS as "can't reproduce" because all of them are fixed already
Maybe it makes sense to close the issue as well because there's #2509534: [Meta] document or refactor calls to drupal_render() but the summary looks better

Crell’s picture

Crell’s picture

Status: Active » Fixed

Sounds good, closing.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.