We noticed that attachment merging is suboptimal. Something like the attached patch should be faster. Needs work to pass tests and profiling to prove that it's faster.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Wim Leers’s picture

Status: Needs review » Needs work

The last submitted patch, ddd-profiling-attachment-merging.patch, failed testing.

mr.baileys’s picture

Assigned: Unassigned » mr.baileys

Assigning to myself to work on this during DDD Montpellier.

mr.baileys’s picture

Assigned: mr.baileys » Unassigned

Unassigning, the specifics of the array merging going on here are way above my head.

Wim Leers’s picture

Issue tags: -Novice

Hm, sorry to hear that! :( That means this is definitely not a Novice issue, if even mr.baileys is bailing out.

Hjarnmastara’s picture

Assigned: Unassigned » Hjarnmastara
Hjarnmastara’s picture

Assigned: Hjarnmastara » Unassigned
Status: Needs work » Needs review
FileSize
1.09 KB

Tests should pass, profiled a bit at the website 3v4l.org, just saw a very small improvement but I don't have any more inspiration for now.

Wim Leers’s picture

Assigned: Unassigned » Fabianx

Can you provide the 3v4l links? Thanks :)

Assigning to Fabianx for review.

Hjarnmastara’s picture

I used this: http://3v4l.org/ukJiQ
Please take into consider that this was my first time with profiling.
Cheers

Hjarnmastara’s picture

I will work on a bit better 3v4l.org link (test case) after food time!

Hjarnmastara’s picture

Without patch: http://3v4l.org/2PgFh/perf#tabs
With patch: http://3v4l.org/Q4qEn/perf#tabs

I believe my patch will only have a small impact when the 'drupalSettings' key is present in both arrays that we wish to merge. Anyway, it was fun trying to improve this function.

Cheers

Fabianx’s picture

Assigned: Fabianx » Unassigned

I would like to see some XHProf profiling for this.

The current change makes me uncomfortable as we now use, use mergeDeepArray() all the time?

jcnventura’s picture

FileSize
506 bytes
1.06 KB

I've modified Hjarnmastara's patch a bit, so that we get the desired performance boost.

According to 3v4l, it uses only about 50% of the previous user time (http://3v4l.org/4ht5i/perf#tabs). I did a few simpletest tests on my rig, but I need to see the results from the official bot.

jcnventura’s picture

Duplicate

Fabianx’s picture

Indeed, we have only library and html_head left, which are both not deep nested ...

Lets see if tests pass ...

Status: Needs review » Needs work

The last submitted patch, 13: 2471228-mergeAttachments-12.patch, failed testing.

jcnventura’s picture

FileSize
493 bytes
1.07 KB

Yes, those fails made sense.. I was able to get a few of those test failures to pass with this improved patch that uses array_merge_recursive instead of array_merge. Still only about 50% of before in 3v4l (http://3v4l.org/bRiOX/perf#tabs)

@Fabianx are there any instructions on how to provide the xhprof data?

jcnventura’s picture

Status: Needs work » Needs review
Fabianx’s picture

I am okay with 3v4l with latest patch now, its obvious native code is faster.

However:

https://api.drupal.org/api/drupal/core!lib!Drupal!Component!Utility!Nest...

Can we justify why we no longer need to use mergeDeep() instead of array_merge_recursive?

jcnventura’s picture

Fabianx. we can justify it according to https://api.drupal.org/api/drupal/core!includes!common.inc/function/drup... by the fact that all the allowed values for the #attached property must be arrays, so array_recursive_merge will result in a correct merge in this case.

Wim Leers’s picture

If we'd make \Drupal\system\Tests\Common\MergeAttachmentsTest comprehensive, we could justify that.

jcnventura’s picture

FileSize
13.47 KB
13.6 KB
14.68 KB

That was a lot more work than I expected, as \Drupal\system\Tests\Common\MergeAttachmentsTest was testing for stuff that doesn't exist anymore.. It should be very comprehensive now. And I made sure to add some kittens.

Wim Leers’s picture

It should be very comprehensive now.

Awesome! :)

And I made sure to add some kittens.

:D :D

Fabianx’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: -needs profiling

And I made sure to add some kittens.

Nooooooooo ... OK :-D

Fortunately I just added some more llama's :-p.

--

The patch looks great to me, the test coverage was extended extensively and it is absolutely clear that a native array_merge_recursive is always faster than our own ...

Therefore => RTBC

catch’s picture

Status: Reviewed & tested by the community » Fixed
+++ b/core/modules/system/src/Tests/Common/MergeAttachmentsTest.php
@@ -195,4 +167,246 @@ function testJsSettingMerging() {
+//    $this->assertIdentical($expected['#attached'], $renderer->mergeAttachments($b['#attached'], $a['#attached']), 'Attachments merged correctly; opposite merging yields opposite order.');

Fixed on commit.

diff --git a/core/modules/system/src/Tests/Common/MergeAttachmentsTest.php b/core/modules/system/src/Tests/Common/MergeAttachmentsTest.php
index b430ee6..466ed3a 100644
--- a/core/modules/system/src/Tests/Common/MergeAttachmentsTest.php
+++ b/core/modules/system/src/Tests/Common/MergeAttachmentsTest.php
@@ -295,8 +295,7 @@ function testHtmlHeadMerging() {
         'system_meta_content_type',
       ),
     );
-//    $this->assertIdentical($expected['#attached'], $renderer->mergeAttachments($b['#attached'], $a['#attached']), 'Attachments merged correctly; opposite merging yields opposite order.');
-    $this->assertIdentical($expected['#attached'], $renderer->mergeAttachments($b['#attached'], $a['#attached']));
+    $this->assertIdentical($expected['#attached'], $renderer->mergeAttachments($b['#attached'], $a['#attached']), 'Attachments merged correctly; opposite merging yields opposite order.');
   }
 
   /**

Committed/pushed to 8.0.x, thanks!

  • catch committed 9b4dd55 on 8.0.x
    Issue #2471228 by jcnventura, Wim Leers, Hjarnmastara: Optimize merging...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.