This is part of #2015147: [meta] Improve the DX of drupal_render(), renderable arrays and the Render API in general.
As part of the discussion from #1830588: [META] remove drupal_set_title() and drupal_get_title(), @moshe sums this up nicely:
Well, the guideline since D7 has been that you don't call drupal_render(). You are supposed to return render arrays from blocks and page callbacks and then kick off all rendering at the end with drupal_render_page(). I notice that we have been adding calls to drupal_render() instead of removing them in issues like #2006152: [meta] Don't call theme() directly anywhere outside drupal_render(). Lets try not muddy the render pipeline with more calls like that. We have to figure out a way to handle nested theme() calls without using drupal_render(). Or disallow nested theme() calls.
The issue mentioned by @moshe is trying to remove theme() calls (which are "worse" than drupal_render() calls), as part of a general API cleanup but the sentiment expressed by @moshe is valid in a more general sense - the more we can avoid "early rendering" render arrays into strings, regardless of the exact function doing the rendering, the better.
"Early rendering" introduces a few issues:
- Code is harder to read and write because you have to be mindful of and mentally track how arrays and render calls are interleaved
- It's harder for alters and other functions that want to process structured data to do their work on already-rendered strings than organised associative arrays. This is expecially damaging for the efforts to make variables in Twig templates "drillable".
- In certain situations it can be a waste of CPU, e.g. consider the case where we do the work to render a child element and a parent element has #access => FALSE
- Rendering elements can have "side effects" like introducing new javascript or CSS files into the page and this may not be appropriate if the early-rendered element ends up not being displayed in the page
- The more we scatter calls to functions like theme() and drupal_render() around core, the harder it is for core contributors to see what the effects of making a modification to one of these API functions will be in all situations
- Early rendering arrays that are used in attributes of other arrays lets us "stick our heads in the sand" rather than fix underlying incompatibilities between element configurations that are commonly used together
- Early rendering elements also "flattens" their children too, meaning that we lose the structure of this element's children that may have been useful/important inside Twig templates that are the parent of this element
Of course, it's not as simple as just blindly returning arrays instead of strings everywhere that drupal_render() currently appears. Often there are good reasons why an early render is happening, either due to limitations in the Render API itself or limitations in the implementation of a particular element #type/#theme.
A common enough example of how this could happen is something like the following:
// Render array to build up a string commonly used as a description for other elements.
$foo = array(
'#type' => 'foo',
...
...
);
// Render array that has a #description attribute that will throw an exception for anything other than a string.
$bar = array(
'#type' => 'bar',
'#description' => drupal_render($foo).
...
...
);
return $bar;
Here we see that unless the $bar element with the #description attribute (that, for the sake of this example, is blindly concatenated onto a string inside a theme function) is provided with some way to render #description arrays into strings (hopefully by calling drupal_render() during the element's processing phases before rendering) we have to "early render" $foo.
Another solution to this problem is to simply use the existing concept of "child elements" in render arrays. Depending on what $bar is trying to do, the following may or may not be an appropriate refactor that requires less structural change to the way #type 'bar' is implemented:
// Render array to build up a string commonly used as a description for other elements.
$foo = array(
'#type' => 'foo',
...
...
);
// Render array that has a #description attribute that will throw an exception for anything other than a string.
$bar = array(
'#type' => 'bar',
'description' => $foo.
...
...
);
return $bar;
There are many (literally hundreds in core) calls to drupal_render() so we have a lot of work to do, but every little bit we clean up will make the API more coherent overall so we should just start filing issues and working through them - https://api.drupal.org/api/drupal/core%21includes%21common.inc/function/...
Hit list
Duplicate/superfluous drupal_render() calls
- #2060401: Remove useless "early render" in authorize.php
- #2060413: remove useless "early render" in update_selection_page()
- #2073957: Remove useless "early render" in Comment Module
Functions that concatenate markup strings without using the Render API
- #2060441: Convert update_results_page() to return a renderable array rather than early render
- #2072639: Convert update_info_page() from a string concatenation function to a render array builder
Blocking issues
Related issues
#2056837: Modules can no longer alter page_bottom
#2060481: theme hooks are documented as responsible for rendering child elements in render arrays, most don't do this, many are incapable
Comments
Comment #0.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #0.1
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #1
Crell CreditAttribution: Crell commentedI have to say I emphatically disagree. We should be more consistent with when we render things, absolutely. But the idea of building up One Giant Array To Rule Them All that we render all at once of the end of the page is the design flaw that WSCCI/SCOTCH have been trying to move away from for 3 years. That knot is particularly tightly wound (as the theme folks have run into, too), but the goal has always been to get the point that there is no such thing as a page array. Every block gets rendered down to a string individually, and there is no way to modify the entire page at once.
Quite simply, as long as we support modifying the page as a whole, we cannot do partial page rendering or partial page caching in a good way. Those two are incompatible.
Now, a best practice of "block::build() and controllers always return a render array", sure, that's fine. But 1) We have to still allow strings for DX (since we apparently got rid of #markup) and 2) whatever comes back from the block/controller needs to be string-ified on the spot, period.
Comment #2
thedavidmeister CreditAttribution: thedavidmeister commentedThat's a great point, any work done here should be tempered with those wise words in mind :)
When I wrote this issue summary I was definitely thinking more about things like theme_file_upload_help() that do nothing but render a string that is intended to be used in another render array's description attribute, or issues where we see drupal_render() sticking rendered markup straight into #children for other render arrays - #2043649: Make all #type 'link' arrays 100% renderable, use #theme_wrappers if necessary.
There are literally hundreds of calls to drupal_render() in core and many (most?) can be consolidated without hurting the ability to do partial page rendering and we get the tangible benefits outlined in the issue summary for each one we clean up.
So, to be clear, despite the quote from @moshe at the top of the issue summary sort-of implying we only ever want to render everything as a "page" (he did mention blocks, actually), I'm totally ok with rendering render arrays for blocks and other forms of "partial page rendering" well before we get to the "page level" of granularity with one mega-array.
I've tagged this with "Needs issue summary update" because it should be made clearer that we're not trying to turn everything back into a "page" here.
Oh also, I think you may have misinterpreted what I meant by "regardless of the exact function doing the rendering" - I meant "regardless of whether theme() or drupal_render() is handling the rendering", not "regardless of the function within which we are constructing a renderable array to be rendered".
Um, I don't think so. #markup still exists and is in wide circulation within core (I get 399 matches for #markup doing a quick search), there's just no #type being shivved artificially into those render arrays that define '#markup', trying to suggest they're all somehow functionally related simply by virtue of being a string.
#markup is even now explicitly mentioned in the docs for drupal_render() - https://api.drupal.org/api/drupal/core%21includes%21common.inc/function/...
Comment #3
thedavidmeister CreditAttribution: thedavidmeister commentedComment #4
pplantinga CreditAttribution: pplantinga commentedWould it be helpful to divide this up somewhat like #2006152: [meta] Don't call theme() directly anywhere outside drupal_render()?
Comment #5
Fabianx CreditAttribution: Fabianx commentedIf #bar does not support render arrays, what about using the RenderWrapper instead with the array and drupal_render as a third solution?
For #bar it is just an object with __toString method.
Comment #6
star-szr@pplantinga - definitely, but I think we should allow for more discussion first.
Comment #6.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #7
thedavidmeister CreditAttribution: thedavidmeister commentedyeah, this will absolutely need to be split up. That's a lot of work though as each call to drupal_render() will need to be "audited" individually, which is the reason it hasn't happened yet. I was thinking #2025629: [meta] Ensure that all #theme/#theme_wrappers hooks provided by core are associated with a sensible, re-usable element #type could happen first (but only after the "don't call theme" issue) to bring some more consistency to what's already in core, it might make the audit easier if everything is based on a #type rather than having a few different render array structures floating around.
Comment #8
Eric_A CreditAttribution: Eric_A commentedThe strategy to do straight and "Novice" conversions from theme() to drupal_render() is leading to instances of
'#markup' => drupal_render($foo)
and ofreturn drupal_render($foo)
in some menu page callback functions. The first is to be replaced by a (child) element. The second should just return the render element. It would be good to clean these up before beta 1. We can add types later, but we'd better get the elements in place in time.Comment #9
thedavidmeister CreditAttribution: thedavidmeister commentedwell I can't say anything for other people's priorities, all I know is that if we can wait on the two issues linked in #4 and #7 before we start working on this issue it will reduce the amount of work we have to do here.
Personally I think making the render API more consistent/easier to use for developers building sites by consolidating theme()/#theme/#type is a higher priority than changing the way it's used internally - even though the early renders are "abuse" of the system, it's no worse than what we've always done until now with theme().
Also, if you want this issue to start moving along, feel free to start auditing drupal_render() calls and documenting it in sub-issues. I put all the issues up for the other two linked meta issues and I have an idea of how long it will take to just do the pre-requisite research on "legit" and "abusive" drupal_render() calls, before we can even start looking at patches - I was planning to do the audit for this one as well, but I don't have enough time for it this weekend.
I was thinking of doing something like 1 issue per module with a list of all the drupal_render() calls within that module in the issue summary. We can explain why each call is appropriate or inappropriate there.
Once we've done a sweep of all the modules, we should go back over the sub issues and look for broad "categories" of abusive drupal_render() calls (like "renders into #markup", for example) then write-up instructions on the preferred way to convert each category in this issue summary.
Once we're at this point we can hopefully start tagging some/most/all of the issues as novice issues as we'll have a clear list of instructions for what needs to be done and none of the individual patches will touch many lines of code. As soon as we've turned this meta issue into novice issues, they'll get cleaned up really fast :)
Comment #10
thedavidmeister CreditAttribution: thedavidmeister commentedComment #10.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #11
pwolanin CreditAttribution: pwolanin commentedWe have a goal of removing the use of system paths from core. So l() in it's current for should be marked deprecated and we should stop using it.
Should we change it to take route + parameters in some way? Otherwise, I think the original title might have been correct. We need a simple way to render a link from a route + parameters?
We should probably alter #type => link to handle #route and #parameters and use the appropriate link-generation function.
Comment #12
thedavidmeister CreditAttribution: thedavidmeister commented#11:
Yeah, I guess in an ideal world l() would just internally build a render array and wrap drupal_render() and then one day we could just delete it when we think nobody is still using it. The problem with that historically has been performance (l() is called literally hundreds of times on many pages), but the profiling that I'm aware of largely focussed on the overhead of calling theme(). If #2052253: [META] Add #render property to drupal_render() and convert #type "#pre_render -> #markup" calls to use it landed we may be able to re-open that discussion as the performance of #type 'link' might become much closer to the current performance of l().
I have a bunch of relevant issues, closed and open, that I can link to for context around the discussion of what to do about rendering links from the POV of avoiding "early render" while supporting routes but I think it should be a new, child issue of this meta and we can track progress there.
Comment #12.0
thedavidmeister CreditAttribution: thedavidmeister commentedadd related
Comment #12.1
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #12.2
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #12.3
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #13
thedavidmeister CreditAttribution: thedavidmeister commentedupdating title to reflect that "early rendering" is not always caused by drupal_render(). Functions that concatenates an un-alterable, un-themeable string of markup to just pass it off to drupal_render() as an attribute of some theme hook are just as bad as early calls to drupal_render().
Also, I started putting a few issues up if anyone wants to take a crack at some patches :)
Comment #14
thedavidmeister CreditAttribution: thedavidmeister commented#2060481: theme hooks are documented as responsible for rendering child elements in render arrays, most don't do this, many are incapable is related as we're currently forced to jump through some hoops to avoid needing to turn everything into a string immediately just to make our system of theme hooks work.
Comment #14.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #14.1
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #14.2
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #14.3
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #14.4
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #14.5
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #16
thedavidmeister CreditAttribution: thedavidmeister commentedIn #2078287: [meta] Fix the DX of the link generator, @msonnabaum asserted that:
With regard to converting links from calling the generator directly to render arrays.
If that's the case, as per #11 and #12, I'd love to put up some issues around converting links over to render arrays where they are currently l() (or equivalent)
Comment #16.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #17
sunComment #18
andypostI closed all issues from IS as "can't reproduce" because all of them are fixed already
Maybe it makes sense to close the issue as well because there's #2509534: [Meta] document or refactor calls to drupal_render() but the summary looks better
Comment #19
Crell CreditAttribution: Crell as a volunteer commentedWhat's left now that #2511472: Refactor all usages of drupal_render()/Renderer::render() that break #2450993 is in?
Comment #20
andypostSuppose leftovers:
#2512382: Follow-up for #2407195: #attached['http_header'] being added to Response in two places
#2450993: Rendered Cache Metadata created during the main controller request gets lost
#2509534: [Meta] document or refactor calls to drupal_render() - meta with children
So could be closed
Comment #21
Crell CreditAttribution: Crell as a volunteer commentedSounds good, closing.