[Meta] Untangle Drupal 8 page rendering

Updating as per the long "Hard Problem" meeting we had in Amsterdam. (Attendees: msonnabaum, dawehner, Crell, catch, effulgentsia, Wim Leers.)

Directly created from my conclusion notes, which I triple-checked with all attendees. This is the plan.

On my way home I tried to apply the following strategy: use render arrays everywhere where html page/fragments are used. This seems to be a) quite hard and b) does not solve really the problems we discussed.
One alternative would be to try to go back to the code how it looked like before HtmlPage.

So the most important for the rendering chain is to have:

Begin of request:

$render = array(
'#pre_render' => 'process_page_etc',
);

drupal_render($render, TRUE);

To _ensure_ that everything that is rendered is part of the rendering stack, which

a) Allows #post_render_cache everywhere and that #post_render_cache is resolved at the last possible point
b) Allows e.g. different rendering strategies to work on the whole rendered page with a complete #post_render_cache set

It does not matter if the render chain is interrupted in between as long as the $stack is correct.

+1 overall!
I see twig templates as pieces of render cache too, so here's another layer of complexity.

It's not clear how attached assets will bubble up to page template and the reason to use "head_link"? To be alterable?

The only question what to do if controller returns string?

I actually think we could get rid of controllers returning strings and standardize on response vs. render array. Strings was a backwards compatibility layer for Drupal 6.

I agree.

I think controllers could return a RenderArrayResponse object for now to wrap the render array?

Also please note:

- We want to get away from calling the controller directly, but instead use a #pre_render callback to call its callable later, but have the route context available for all blocks (and the main controller): https://www.drupal.org/node/2274167#comment-9244301

It seems to be feasible with only around 630 test failures.

Where controller and block need to communicate they need to do so via contexts instead of global state like now (views_get_page_view() / views_set_page_view())

I actually think we could get rid of controllers returning strings and standardize on response vs. render array. Strings was a backwards compatibility layer for Drupal 6.

I probably don't know enough about this, but why do we want to get rid of returning strings from Controllers?
That sounds really handy for prototyping if something is going to work or not and getting up and running and may have some use-cases were the output has been already generated and just needs to be sent to the browser.

Again, maybe there is a way around this and I'm overreacting a bit.
This is a "sanity check" piece of code I'd hope would work:

class MyController extends ControllerBase {
  public function myAction {
    return 'Is this working?';
  }
}

Which would shortly be followed by a more elaborate bit of code..., but also, I could dump an early return with some test data to the output of this controller.

I'm likely "doing it wrong" or something, but not being able to do that seems like a DX loss.

Couldn't this just be a is_scalar($var) ? new Response($var); wrap on the "support this" side?

Trying to avoid these kind of questions a bit too: (Warning ASP.NET code, NSW)
http://stackoverflow.com/questions/553936/in-mvc-how-do-i-return-a-strin...

Opened a small additional child issue: #2357937: Remove {{ feed_icons }} from page template (page.html.twig). It helps pave the way for #2352155: Remove HtmlFragment/HtmlPage and also helps with Convert all page template (page.html.twig) variables to blocks.

I am adding a viewport tag from within my contrib theme and I came across this issue but I am not sure if I should open a new issue or if this one covers the problem I am encountering. Update, opened a new issue. #2359987: Overriding core metatag using hook_page_attachments_alter results in two tags

@Danny Englander
You should probably open a separate issue. You want to override a meta tag, and it makes perfect sense to do that in page_attachments_alter, but you can't. This is related, but a separate issue is better.

Another, bigger child issue to help pave the way: #2359901: Discourage $main_content['#title'] in favor of route titles and title callbacks.