Coming from #2742585-102: Deprecate dangerous assertTrue/False() compatibility overrides in PHPUnit Kernel, Functional and FunctionalJavascript tests:

@mondrake

How about deprecating also the assertEquals override in KernelTestBase and BrowserTestBase, so forcing tests to call explicitly castSafeStrings when needed?

@alexpott

the gain with it is less obvious.One thing that is interesting is instead of the assertEquals hack we could register a Comparator to compare MarkupInterface objects...
CommentFileSizeAuthor
#60 interdiff_57-60.txt3.93 KBmondrake
#60 3082340-60.patch10.07 KBmondrake
#57 interdiff_53-57.txt2.79 KBmondrake
#57 3082340-57.patch10 KBmondrake
#53 interdiff_48-53.txt2.98 KBmondrake
#53 3082340-53.patch9.75 KBmondrake
#48 3082340-48.patch9.54 KBmondrake
#48 interdiff_45-48.txt978 bytesmondrake
#45 3082340-45.patch9.45 KBmondrake
#45 interdiff_33-45.txt1.23 KBmondrake
#40 interdiff_38-40.txt1.19 KBmondrake
#40 3082340-40.patch10.86 KBmondrake
#38 interdiff.3082340.33-38.txt2.29 KBlongwave
#38 3082340-38.patch9.67 KBlongwave
#33 interdiff.3082340.30-33.txt2.18 KBlongwave
#33 3082340-33.patch9.82 KBlongwave
#30 interdiff.3082340.29-30.txt1.11 KBlongwave
#30 3082340-30.patch9.64 KBlongwave
#29 interdiff.3082340.22-29.txt10.07 KBlongwave
#29 3082340-29.patch9.77 KBlongwave
#22 3082340-22.patch11.58 KBmondrake
#22 interdiff_19-22.txt946 bytesmondrake
#19 interdiff_17-19.txt3.37 KBmondrake
#19 3082340-19.patch11.59 KBmondrake
#17 3082340-17.patch12.06 KBmondrake
#17 interdiff_15-17.txt2.26 KBmondrake
#15 interdiff_14-15.txt3.53 KBmondrake
#15 3082340-15.patch11.64 KBmondrake
#14 3082340-14.patch10.81 KBmondrake
#14 interdiff_11-14.txt8.15 KBmondrake
#11 interdiff_9-11.txt2.21 KBmondrake
#11 3082340-11.patch4.53 KBmondrake
#9 3082340-9.patch5.18 KBmondrake
#9 interdiff_7-9.txt1013 bytesmondrake
#7 interdiff_6-7.txt1.13 KBmondrake
#7 3082340-7.patch5.15 KBmondrake
#6 3082340-6.patch4.61 KBmondrake
#4 3082340-4-test-only.patch3.28 KBmondrake
#3 3082340-3-test-only.patch3.21 KBmondrake
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

mondrake created an issue. See original summary.

mondrake’s picture

Out of curiosity, I run the tests with the assertEquals override removed from the base classes in #3065023-56: Ignore, testing issue, and we only land with 44 test fails.

mondrake’s picture

Status: Active » Needs review
FileSize
3.21 KB

Test only patch, conditionally triggering deprecation errors when MarkupInterface objects are cast into strings in assertEquals.

mondrake’s picture

FileSize
3.28 KB

Status: Needs review » Needs work

The last submitted patch, 4: 3082340-4-test-only.patch, failed testing. View results

mondrake’s picture

Title: Consider deprecation of assertEquals() overrides in PHPUnit Kernel, Functional and FunctionalJavascript tests » Replace assertEquals() overrides in PHPUnit Kernel, Functional and FunctionalJavascript tests with a MarkupInterfaceComparator implementation
Status: Needs work » Needs review
FileSize
4.61 KB

This is a first attempt at trying implementing a MarkupInterfaceComparator as suggested by @alexpott. No interdiff, it's a different approach.

mondrake’s picture

If both expected and actual are MarkupInterface objects, there's no need to cast them to string (also because that fails in unit tests where there's no container available to resolve them to strings), and we can let the standard ObjectComparator do its job.

The last submitted patch, 6: 3082340-6.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

mondrake’s picture

#7 does not work. We need to work on higher level and check if the container is initialised so that casting a MarkupInterface object to string is possible. If not, we are in a unit test and we need to fall back to ObjectComparator.

The last submitted patch, 7: 3082340-7.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

mondrake’s picture

Actually, we do not need to resolve the conversion to string of MarkupInterface array items in the Comparator like in the assertEquals override. PHPUnit will use its own ArrayComparator to walk through the comparison of two arrays and break down each item into its own needed Comparator - and ours will then fit with the comparison of single MarkupInterface objects.

Krzysztof Domański’s picture

Status: Needs review » Needs work

1. In the new class MarkupInterfaceComparator the assertEquals method contains an additional parameter ($processed) that should be described. Dock block for assertEquals cannot be inherited.

+  /**
+   * {@inheritdoc}
+   */
+  public function assertEquals($expected, $actual, $delta = 0.0, $canonicalize = false, $ignoreCase = false, array &$processed = []) {

2. Coding standards.

--- lib/Drupal/Core/Test/Comparator/MarkupInterfaceComparator.php
+++ PHP_CodeSniffer
@@ -39,7 +39,7 @@
   /**
    * {@inheritdoc}
    */
-  public function assertEquals($expected, $actual, $delta = 0.0, $canonicalize = false, $ignoreCase = false, array &$processed = []) {
+  public function assertEquals($expected, $actual, $delta = 0.0, $canonicalize = FALSE, $ignoreCase = FALSE, array &$processed = []) {
mondrake’s picture

Assigned: Unassigned » mondrake

Also, needs tests. I am on it.

mondrake’s picture

Assigned: mondrake » Unassigned
Status: Needs work » Needs review
FileSize
8.15 KB
10.81 KB

Thank you @Krzysztof Domański

This is fixing #12 and adding thorough testing of the new comparator.

mondrake’s picture

Code style fixes.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

mondrake’s picture

Status: Needs review » Needs work

The last submitted patch, 17: 3082340-17.patch, failed testing. View results

mondrake’s picture

The tests were not testing what I was expecting and the now stricter assertTrue and assertFalse helped find that out!

mondrake’s picture

Status: Needs work » Needs review
longwave’s picture

+++ b/core/lib/Drupal/Core/Test/Comparator/MarkupInterfaceComparator.php
@@ -0,0 +1,65 @@
+    // When either expected or actual are MarkupInterface objects, we take over
+    // and convert to strings before comparing.

I think it is slightly more accurate to say something like "If at least one argument is a MarkupInterface object" here, as we also handle the case when both are objects?

Otherwise this looks good to me and will help with #3097892: Remove all @deprecated code in test base classes

mondrake’s picture

longwave’s picture

+++ b/core/lib/Drupal/Core/Test/Comparator/MarkupInterfaceComparator.php
@@ -0,0 +1,65 @@
+use SebastianBergmann\Comparator\Comparator;

Do we need an explicit dev dependency on sebastian/comparator now?

mondrake’s picture

#23 I don't think so, it is a requirement of phpunit/phpunit which would not work without, and I doubt we would be using the comparator in a dev environment without phpunit :)

longwave’s picture

No, but do we need to pin a version/set of known working versions? Otherwise Comparator 4.0 may be released and it could be backward incompatible with what we are doing here?

mondrake’s picture

#25 I'd let phpunit/phpunit manage it's own dependencies:

PHPUnit 6 has "sebastian/comparator": "^2.1"
PHPUnit 7 has "sebastian/comparator": "^3.0"
PHPUnit 8 has "sebastian/comparator": "^3.0.2"

If we start pinning it down explicitly we may incur in PHPUnit not being able to update later

longwave’s picture

Status: Needs review » Reviewed & tested by the community

Thanks for the explanation, I think that is OK - in which case this looks good to go to me.

alexpott’s picture

Hmmm... the container thing is a bit nasty. I wonder if we can add the comparator in a different place. For example, can we add it in the base test classes. I dunno.

longwave’s picture

Status: Reviewed & tested by the community » Needs review
FileSize
9.77 KB
10.07 KB

This might work?

longwave’s picture

Fixed up the test data provider comment.

edit: @returns should be @return, that can be fixed on next patch or commit

alexpott’s picture

@longwave nice one! TIL about \PHPUnit\Framework\TestCase::registerComparator() :)

mondrake’s picture

So much nicer!

  1. +++ b/core/tests/Drupal/KernelTests/Core/Test/Comparator/MarkupInterfaceComparatorTest.php
    @@ -0,0 +1,135 @@
    +  /**
    +   * Prepares the comparator.
    +   */
    +  protected function setUp($mock = FALSE) {
    +    parent::setUp();
    +    $this->factory = new Factory();
    +    $this->comparator = new MarkupInterfaceComparator();
    +    $this->comparator->setFactory($this->factory);
    +  }
    

    this is now an override of ::setUp, so the docblock should be @inheritdoc and $mock argument is no longer needed

  2. +++ b/core/tests/Drupal/KernelTests/KernelTestBase.php
    @@ -231,6 +232,7 @@ public static function setUpBeforeClass() {
         parent::setUp();
    +    $this->registerComparator(new MarkupInterfaceComparator());
     
    
    +++ b/core/tests/Drupal/Tests/BrowserTestBase.php
    @@ -390,6 +391,7 @@ protected function registerSessions() {}
         parent::setUp();
    +    $this->registerComparator(new MarkupInterfaceComparator());
    

    I think these would benefit a short comment what they are about

longwave’s picture

Status: Needs work » Needs review
FileSize
9.82 KB
2.18 KB

Thanks for review, addressed #30 and #32.

mondrake’s picture

+1 to RTBC, but I cannot set it myself

longwave’s picture

As a followup I wonder if we will be able to refactor away Drupal\Tests\AssertHelperTrait::castSafeStrings() as the code is suspiciously similar (although it also deals with arrays of MarkupInterfaces)

mondrake’s picture

#35 interesting...

alexpott’s picture

What's also super interesting is that I think we only need this comparator when both sides are MarkupInterface objects - otherwise the regular scalar comparator is going to do the work for us. And yes I think a follow-up to see if this allows us to deprecate Drupal\Tests\AssertHelperTrait::castSafeStrings() is a good idea. But that might mean that we need to do something for Unit tests.

longwave’s picture

Let's try your hypothesis from #37.

Status: Needs review » Needs work

The last submitted patch, 38: 3082340-38.patch, failed testing. View results

mondrake’s picture

Status: Needs work » Needs review
FileSize
10.86 KB
1.19 KB

Let's see if it's as easy as this.

edit - however hmmm... this can be a change in behaviour impacting contrib tests

alexpott’s picture

Yeah I think we should go back to #33 unfortunately.

+++ b/core/lib/Drupal/Core/Test/Comparator/MarkupInterfaceComparator.php
@@ -0,0 +1,47 @@
+  /**
+   * Casts MarkupInterface objects into strings.
+   *
+   * @param mixed $value
+   *   The value to act on.
+   *
+   * @return mixed
+   *   The input value, with MarkupInterface objects cast to string.
+   */
+  private function safeCastString($value) {
+    return $value instanceof MarkupInterface ? (string) $value : $value;
+  }

Let's drop this in favour of doing

$expected = (string) $expected;
$actual = (string) $actual;

Less methods and simpler.

longwave’s picture

I think #33 is better in order to deal with the int vs MarkupInterface case, as otherwise this would trigger NumericComparator which doesn't support objects, but I think this is a valid thing to try and do in a test.

Status: Needs review » Needs work

The last submitted patch, 40: 3082340-40.patch, failed testing. View results

mondrake’s picture

Assigned: Unassigned » mondrake

Working on #41. @longwave re. #42 I'm not sure I understand what you mean, can you elaborate?

mondrake’s picture

mondrake’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 45: 3082340-45.patch, failed testing. View results

mondrake’s picture

Status: Needs work » Needs review
FileSize
978 bytes
9.54 KB

This should do.

longwave’s picture

In #42 I was just trying to say that I think the #33 approach is both valid and correct, and we need to handle more than just the case where both sides are MarkupInterfaces, but I did not explain that very well :)

mondrake’s picture

@longwave :) so nothing else to do here then

alexpott’s picture

Status: Needs review » Needs work
+++ b/core/lib/Drupal/Core/Test/Comparator/MarkupInterfaceComparator.php
@@ -0,0 +1,34 @@
+    $expected_safe = $expected instanceof MarkupInterface ? (string) $expected : $expected;
+    $actual_safe = $actual instanceof MarkupInterface ? (string) $actual : $actual;

I'm pretty sure that we can cast to string here without the $expected instanceof MarkupInterface

What is fixing #45 is now we stringify when both are MarkupInterface objects.

longwave’s picture

@alexpott That blew up the test when I tried it as neither array nor stdClass can be cast to string, but I guess that part of the test is invalid anyway, as it will never be called with these arguments. It does however mean int is cast to string but I suppose that doesn't ultimately matter?

mondrake’s picture

Status: Needs work » Needs review
FileSize
9.75 KB
2.98 KB

Like this?

longwave’s picture

Status: Needs review » Reviewed & tested by the community

I think that looks OK, a clever way of testing the three possible states that I hadn't thought of. Tentatively marking RTBC even though I worked on some of this, hope that's OK.

longwave’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/core/tests/Drupal/KernelTests/Core/Test/Comparator/MarkupInterfaceComparatorTest.php
@@ -0,0 +1,148 @@
+      $this->assertNull($this->comparator->assertEquals($expected, $actual));

On second thoughts I think we need to assert that $equals_result is TRUE after this, if no exception was thrown?

mondrake’s picture

Assigned: Unassigned » mondrake

Fair, I am on it

mondrake’s picture

Assigned: mondrake » Unassigned
Status: Needs work » Needs review
FileSize
10 KB
2.79 KB

Done #55, and improved tests a bit.

longwave’s picture

Status: Needs review » Reviewed & tested by the community

Nice work, the changes are all good improvements.

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/core/lib/Drupal/Core/Test/Comparator/MarkupInterfaceComparator.php
@@ -0,0 +1,34 @@
+namespace Drupal\Core\Test\Comparator;

Just realised this in the main core namespace. Let's put this in the test tools. There's no need for this to be autoloadable by the non test core autoloader.

Also this will make our life simpler if as probable Comparator changes signature in different PHPUnit versions.

mondrake’s picture

longwave’s picture

Status: Needs review » Reviewed & tested by the community

Back to RTBC.

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

So we should favour composition over inheritance :)

Committed and pushed 398a61b7e0 to 9.0.x and 8d8cff8854 to 8.9.x. Thanks!

  • alexpott committed 398a61b on 9.0.x
    Issue #3082340 by mondrake, longwave, alexpott, Krzysztof Domański:...

  • alexpott committed 8d8cff8 on 8.9.x
    Issue #3082340 by mondrake, longwave, alexpott, Krzysztof Domański:...
longwave’s picture

mondrake’s picture

How about backporting this to 8.8.x? To have consistency across.

alexpott’s picture

@mondrake I don't think it is worth it. We're not going to make major changes to 8.8.x now. And there's enough to think about with 9.0.x and 8.9.x

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.