Twig debug output does not show the base or derived hooks when a theme suggestion is called directly

Patch, then I will set to CNW and tag for tests.

We have some Twig debug tests already in \Drupal\system\Tests\Theme\TwigDebugMarkupTest, so we can add tests there for this case.

I also think adding tests for this should be a good novice automated testing task :)

Thanks @cilefen, looks good at a glance. Setting the issue to 'needs review' so that this gets tested.

If you can also upload a test-only patch that would be great - see the screenshot near the bottom of https://drupal.org/contributor-tasks/write-tests. Uploading a test-only patch shows very clearly on the issue that the test is doing its job and failing appropriately.

This is very good and is a perfect example of why we write tests :) the fix in #1 was not correct for the case of a theme hook with no additional suggestions. Node inserts additional template suggestions in its preprocess function (template_preprocess_node()) by adding to the 'theme_hook_suggestions' key in the $variables array. Sidenote: see #1751194: Introduce hook_theme_suggestions[_HOOK]() and hook_theme_suggestions[_HOOK]_alter() for a new plan to handle that functionality.

Here is an updated "fix" patch, I think you should be able to get a failing/passing test for this with some minor tweaks. It might be helpful to use debug() to check the $output when running tests if you aren't already.

Needed a quick reroll after #1843650: Remove the process layer (hook_process and hook_process_HOOK), the \Drupal\forum\Tests\ForumTest failure was random.

#12: 2029225-complete-12.patch queued for re-testing.

@Fabianx - Sure, we could do that. But I'm also realizing:

+++ b/core/modules/system/lib/Drupal/system/Tests/Theme/TwigDebugMarkupTest.php
@@ -67,6 +68,30 @@ function testTwigDebugMarkup() {
+    $output = theme('comment__node_article', comment_view($comment));
+    $this->assertTrue(strpos($output, "CALL: theme('comment__node_article')") !== FALSE, 'Theme call information found.');
+    $this->assertTrue(strpos($output, 'x comment' . $extension) !== FALSE, 'Comment template file shown as current template.');
+    $this->assertTrue(strpos($output, '* comment--node-article' . $extension) !== FALSE, 'Full name template file found.');
+    $template_filename = $templates['comment']['path'] . '/' . $templates['comment']['template'] . $extension;
+    $this->assertTrue(strpos($output, "BEGIN OUTPUT from '$template_filename'") !== FALSE, 'Full path to current template file found.');

This should probably use the standard suggestion format of comment__node__article - two underscores as delimiters.

And then the debug output should show:

<!-- THEME DEBUG -->
<!-- CALL: theme('comment__node__article') -->
<!-- FILE NAME SUGGESTIONS:
   x comment.html.twig
   * comment--node.html.twig
   * comment--node--article.html.twig
-->

And all of these possibilities should be shown, even if comment--node.html.twig or comment--node--article.html.twig templates are implemented. This is not the case with the current patch.

Currently if you call comment__node__article and have a comment--node--article.html.twig template you don't get any information about suggestions because theme_hook_called == theme_hook_derived, or in other words an implementation was found for the hook passed into theme().

To accomplish this, I think our next steps will be to update theme() and/or twig_render_template() and expand test coverage. There is some code in theme() used to determine the derivative suggestions, but for the debug output we can't stop once an implementation is found.

    // Iteratively strip everything after the last '__' delimiter, until an
    // implementation is found.
    while ($pos = strrpos($hook, '__')) {
      $hook = substr($hook, 0, $pos);
      if (isset($hooks[$hook])) {
        break;
      }
    }

Thanks @cilefen, that's really helpful. Go TDD :)

This patch doesn't add any code to theme() - the added code only fires when twig_debug is on. This also fixes the incorrect order of suggestions in the debug output - they should start with the most specific suggestions. That change could be moved to a separate issue but at this point I think it's a small enough change.

The fix code being added could really use some refactoring but I wanted to get it up here. It works for the test case provided in #21 as well as the case of node__foo__bar and displays the suggestions in the correct order of how the templates are matched, most specific to least specific.

I'd like to see more test coverage for the order that the suggestions are displayed in, particularly for the more complex node case below. Having consistent debug markup is going to be very important for good TX.

<!-- THEME DEBUG -->
<!-- CALL: theme('comment__node__article') -->
<!-- FILE NAME SUGGESTIONS:
   * comment--node--article.html.twig
   * comment--node.html.twig
   x comment.html.twig
-->

<!-- THEME DEBUG -->
<!-- CALL: theme('node__foo__bar') -->
<!-- FILE NAME SUGGESTIONS:
   * node--foo--bar.html.twig
   * node--foo.html.twig
   * node--3.html.twig
   * node--page.html.twig
   x node.html.twig
-->

<!-- THEME DEBUG -->
<!-- CALL: theme('node') -->
<!-- FILE NAME SUGGESTIONS:
   * node--2.html.twig
   * node--page.html.twig
   x node.html.twig
-->

We already display the full path to the template file being used, check your debug markup :)

Example can be seen in the Twig settings/debug change notice: https://drupal.org/node/1922666

Here's a reroll with some improved inline comments. Looking at this again I'm okay with the fix but will leave it up to someone else to review :)

Thanks again @cilefen for working on the tests, you rock!

Thanks @joelpittet! We could maybe simplify the test and not use the comment example :)

@dawehner pointed out on IRC that the debug output also doesn't match up when an array of suggestions is passed into theme().

Created a separate issue for that: #2118743: Twig debug output does not display all suggestions when an array of theme hooks is passed to #theme

Update to reflect more recent status

Add related issue for an array of suggestions

Small reroll for #2173655: Refactor theme() to _theme(); make it a private API to discourage module developers from circumventing the renderable build system.

Updating the added test case to not use theme() or _theme(). Going to cancel the last one.

Thanks @joelpittet!

We can't use assertRaw here because we aren't doing a drupalGet(). I agree the lines are a bit long/hard to read but not sure how to improve upon them, we really want to check the order that things are output. See #23/#24.

I'd rather the "CALL" stuff be changed in #2029225: Twig debug output does not show the base or derived hooks when a theme suggestion is called directly.