Problem/Motivation

Order's Adjustment can't be normalized, So the field adjustments can't been serialized
there is a core bug cause this, #2915705: TypedData 'Any' can't be normalized to array if it stores an object
the core will not recursive normalzie object,

But the resolution require that object itself saved to a 'any' type data must been normalizable or traversable

Proposed resolution

Make Adjustment been traversable

Remaining tasks

make Adjustment implement IteratorAggregate

User interface changes

API changes

Data model changes

CommentFileSizeAuthor
#89 2916252-3.0.x-89.patch2.33 KBalexpott
#88 2916252-3.0.x-88.patch2.3 KBalexpott
#77 2916252-77.patch2.3 KBjsacksick
#75 2916252-75.patch1.68 KBjsacksick
#73 interdiff-2916252-66-73.txt388 bytesalex.bukach
#73 2916252-73.patch9.56 KBalex.bukach
#72 2916252-72.patch9.56 KBakhil babu
#66 interdiff-2916252-58-66.txt6.97 KBlawxen
#66 2916252-66.patch9.55 KBlawxen
#58 interdiff-2916252-57-58.txt450 byteslawxen
#58 2916252-58.patch1.98 KBlawxen
#57 interdiff-2916252-56-57.txt549 byteslawxen
#57 2916252-57.patch1.67 KBlawxen
#56 interdiff-2916252-54-56.txt1.25 KBlawxen
#56 2916252-56.patch1.39 KBlawxen
#54 interdiff-2916252-53-54.txt2.02 KBlawxen
#54 2916252-54.patch2.13 KBlawxen
#53 2916252-53.patch1.03 KBwim leers
#51 2916252-51.patch12.69 KBlawxen
#36 2916252-36-a.patch7.97 KBlawxen
#36 2916252-36-b.patch10.25 KBlawxen
#36 normalze-denormalize-flow.png1.54 MBlawxen
#30 2916252-30.patch5.3 KBlawxen
#30 interdiff-2916252-25-30.txt553 byteslawxen
#27 2916252-27.patch5.3 KBlawxen
#27 interdiff-2916252-25-27.txt553 byteslawxen
#25 2916252-25.patch5.39 KBlawxen
#25 interdiff-2916252-21-25.txt4.34 KBlawxen
#21 2916252-21.patch2.5 KBlawxen
#21 interdiff-2916252-19-21.txt630 byteslawxen
#19 2916252-19.patch2.49 KBlawxen
#19 interdiff-2916252-18-19.txt5.25 KBlawxen
#18 2916252-18.patch3.38 KBlawxen
#18 interdiff-2916252-7-18.txt2.17 KBlawxen
#7 interdiff-2916252-4-7.txt863 byteslawxen
#7 2916252-7.patch1.04 KBlawxen
#4 interdiff-2916252-1-4.txt4.33 KBlawxen
#4 2916252-4.patch1.6 KBlawxen
#2 2916252-1.patch3.82 KBlawxen

Issue fork commerce-2916252

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

caseylau created an issue. See original summary.

lawxen’s picture

StatusFileSize
new3.82 KB

Adjustment's amount stores object Price, but Price can't be normalized either, so I make Drupal\commerce_price\ implements TypedDataInterface too.

bojanz’s picture

That patch looks very wrong. Adjustment and Price are dumb value objects, we should not pollute them with a dozen Drupal-specific methods. The conversion needed for normalization/serialization should ideally happen one level above, at the field type level.

lawxen’s picture

Issue summary: View changes
StatusFileSize
new1.6 KB
new4.33 KB

@bojanz

That patch looks very wrong. Adjustment and Price are dumb value objects, we should not pollute them with a dozen Drupal-specific methods

I agree this
meanwhile the normalization/serialization shouldn't need function change the basic logic

Combine https://www.drupal.org/node/2915705‘s patch
The code will run to here
https://github.com/symfony/serializer/blob/3.2/Serializer.php#L149

        if (is_array($data) || $data instanceof \Traversable) {
            $normalized = array();
            foreach ($data as $key => $val) {
                $normalized[$key] = $this->normalize($val, $format, $context);
            }

So we can just make the object Adjustment and Price to be traversable.

skyredwang’s picture

#4 Looks worse. Why do we need to change Price at all? It has been working with serialization on Product Variant and Order.

skyredwang’s picture

#4 Looks worse. Why do we need to change Price at all? It has been working with serialization on Product Variant and Order.

lawxen’s picture

StatusFileSize
new1.04 KB
new863 bytes

@skyredwang yes, you are right
I make some change
1. don't change Price
2.on Adjustment's toArray(), return the Price's toArray() value

lawxen’s picture

Product Variant and Order don't storage Price object ,It just storage Price's value and currency, But “don't change Price” still is right,
It has some benefit:

  1. If Adjustment add another object tomorrow, It make an example of just call new object's toArray() function, no change for the new object is needed always
  2. Faster
skyredwang’s picture

#7 looks cleaner, but #3 question still remains: Can we do serialization at the field level? Specifically speaking, why can't we normalize $Order->adjustments? One reason I can think of is that Adjustment uses 'Any' for storage, therefore on the field level, there isn't metadata we can utilize to normalize $Order->adjustments, therefore, we need to go to one level deeper.

@caseylau, can you prove or disapprove that $Order->adjustments can be serialized at field level?

wim leers’s picture

Issue tags: +API-First Initiative
wim leers’s picture

wim leers’s picture

AFAICT the real problem here is that Adjustment doesn't have a proper @DataType. It's not clear to me why it doesn't at least use \Drupal\Core\TypedData\Plugin\DataType\Map in its base field definition at in Order.

skyredwang’s picture

It's not clear to me why it doesn't at least use \Drupal\Core\TypedData\Plugin\DataType\Map in its base field definition at in Order

@bojanz has told me: this is because when they started doing Adjustment, they couldn't decide on a clear data structure for Adjustment. Core has 'inferior' support on field type change (or something like that), therefore, there isn't a plan to change it for now.

bojanz’s picture

Correct. Adding a column to an existing field type is hell (which we've had to go through with Address, but wanted to avoid here). Adjustments changed multiple times. Since we only needed a serialized, non-queriable solution, the current approach made sense.

lawxen’s picture

#9's question:

can you prove or disapprove that $Order->adjustments can be serialized at field level?

Because all Adjustment's properties are protected, and Drupal can't konw it's get*** function,

So there are ony two possible plan as far as know:

  1. Write Custom normalizer in commerce itself, to make Adjustment can be normalzed on field level
  2. Custom normalizer use Adjustment's several get*** function to get value;
    If Adjustment change, the normalizer need change too. And every situation like this need a standalone normalizer.
    So many garbage codes!

  3. Take full advantage of symfony Serializer->normalize() design, Make Adjustment traversable(#7's solution 2916252-7.patch).

    1. https://www.drupal.org/node/2915705
    Core change: TypedDataNormalizer.php->normalzie():

    -    return $object->getValue();
    +    $value = $object->getValue();
    +    // If the value is object or array, return the normalized result, object
    +    // must be typed data or traversable to be normalizable.
    +    if (isset($value) && (is_object($value) || is_array($value))) {
    +      $value = $this->serializer->normalize($value, $format, $context);
    +    }
    +    return $value;
    

    Then the code will run to symfony's Serializer->normalize()

       public function normalize($data, $format = null, array $context = array())
        {
            // If a normalizer supports the given data, use it
            if ($normalizer = $this->getNormalizer($data, $format)) {
                return $normalizer->normalize($data, $format, $context);
            }
    
            if (null === $data || is_scalar($data)) {
                return $data;
            }
    
            if (is_array($data) || $data instanceof \Traversable) {
                $normalized = array();
                foreach ($data as $key => $val) {
                    $normalized[$key] = $this->normalize($val, $format, $context);
                }
    
                return $normalized;
            }
    
            if (is_object($data)) {
                if (!$this->normalizers) {
                    throw new LogicException('You must register at least one normalizer to be able to normalize objects.');
                }
    
                throw new UnexpectedValueException(sprintf('Could not normalize object of type %s, no supporting normalizer found.', get_class($data)));
            }
    
            throw new UnexpectedValueException(sprintf('An unexpected value could not be normalized: %s', var_export($data, true)));
        }
    

    Let's focused on

    instanceof \Traversable
    

    If Adjustment is Traversable, it wll loop Adjustment to get it value.
    So just make Adjustment implements \IteratorAggregate to make it Traversable
    #7's 2916252-7.patch

  1. We can see that solution one:Write Custom normalizer is boring and chaos, result in a lot of similar codes in Drupal. So normalize Adjustment through custom normalizer on field level is not recommend.
  2. And the solution two use symfony's design to make things easier and clear.(recomend)
lawxen’s picture

There is a new solution,
change TypedDataNormalizer->normalize() to use (array) to translate object;

  public function normalize($object, $format = NULL, array $context = []) {
    $value =  $object->getValue();
    if (isset($value) && is_object($value))
    $value = (array) $value;
    //then write code to remove * on protected value's key
    return $value;
  }

But It can't handle two situations:

  1. Situation here on Adjustment , (array) can't return object's object value, like object Price stored on Adjustment->amount
  2. 'any' typed data stored an array with child object value
    If we want handle this situation, we must loop the array on every level's child element, it's inefficient.

conclusion:
(array) is not recommend

skyredwang’s picture

Status: Active » Needs work

@bojanz has told me on IRC:

<bojanz> skyred: lets just write a custom serializer then
<bojanz> that way we can avoid patching core
<skyred> bojanz: this custom normalizer, do you want it to be in Commerce project, or you want it to be in a contrib?
<bojanz> skyred: in commerce itself
lawxen’s picture

StatusFileSize
new2.17 KB
new3.38 KB
<bojanz> skyred: lets just write a custom serializer then
<bojanz> that way we can avoid patching core

on the basis of your suggestion. before https://www.drupal.org/node/2915705 be committed in core
I add a custom normalizer, naming it a more generic name : ObjectAnyNormalizer, to make it can support more similar situations through this one normalizer in commerce.

lawxen’s picture

StatusFileSize
new5.25 KB
new2.49 KB

#17 suggest add custom normalizer on field level with no changing for Adjustment
This is the patch

skyredwang’s picture

Status: Needs work » Needs review

I tested #19, it seems working fine, below is a sample output, you can see the adjustments are serialized:

{
    "data": {
        "type": "commerce_order--default",
        "id": "66c5f00b-530e-4410-8a18-4c49e7deb3c3",
        "attributes": {
            "order_id": 20,
            "uuid": "66c5f00b-530e-4410-8a18-4c49e7deb3c3",
            "order_number": "20",
            "mail": "test@test.com",
            "ip_address": "172.17.0.1",
            "adjustments": [
                {
                    "type": "custom",
                    "label": "adjustment",
                    "amount": {
                        "number": "4.32",
                        "currency_code": "CNY"
                    },
                    "percentage": null,
                    "sourceId": "custom",
                    "included": false,
                    "locked": true
                },
                {
                    "type": "promotion",
                    "label": "Custom promotion",
                    "amount": {
                        "number": "-5.00",
                        "currency_code": "CNY"
                    },
                    "percentage": null,
                    "sourceId": "custom",
                    "included": false,
                    "locked": false
                },
                {
                    "type": "shipping",
                    "label": "Deliver fee",
                    "amount": {
                        "number": "12.0",
                        "currency_code": "CNY"
                    },
                    "percentage": null,
                    "sourceId": "custom",
                    "included": false,
                    "locked": false
                }
            ],
            "total_price": {
                "number": "6097.160000",
                "currency_code": "CNY"
            },
            "state": "completed",
            "data": null,
            "locked": null,
            "created": 1480914436,
            "changed": 1508399562,
            "placed": 1480916496,
            "completed": 1480916496,
........
lawxen’s picture

StatusFileSize
new630 bytes
new2.5 KB

fix the comment
@see \Drupal\commerce\Normalizer\ObjectAnyNormalizer::supportsNormalization
to
@see \Drupal\commerce_order\Normalizer\ObjectAnyNormalizer::supportsNormalization

mglaman’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests
  1. +++ b/modules/order/commerce_order.services.yml
    @@ -62,3 +62,10 @@ services:
    +  serializer.normalizer.any_adjustment:
    

    Nit: Can't this be `commerce_order.normalizer`. It's a normalizer and not provided by the Serializer module.

  2. +++ b/modules/order/src/Normalizer/AnyAdjustmentNormalizer.php
    @@ -0,0 +1,51 @@
    +      if (isset($value) && is_object($value) && $value instanceof Adjustment) {
    

    This can be simplified to just $value instanceof Adjustment

Can we just add a kernel test which normalizes a field and asserts the array output? Then it looks good to me. But we cannot commit this without tests.

bojanz’s picture

commerce_order.normalizer doesn't make sense since it doesn't normalize orders, it normalizes adjustments.
So, commerce_order.adjustment_normalizer, unless a service ID pattern is enforced somehow?

mglaman’s picture

Sorry I meant `commerce_order.normalizer.adjustment` not just `commerce_order.normalizer`. Was regarding to the namespace of the service.

lawxen’s picture

StatusFileSize
new4.34 KB
new5.39 KB
  1. changeisset($value) && is_object($value) && $value instanceof Adjustment to $value instanceof Adjustment
  2. rename serializer.normalizer.any_adjustment: to commerce_order.normalizer.adjustment
  3. add AdjustmentItemNormalizeTest
mglaman’s picture

+++ b/modules/order/commerce_order.services.yml
@@ -62,3 +62,10 @@ services:
+    # @see \Drupal\commerce_order\Normalizer\ObjectAnyNormalizer::supportsNormalization

I think this comment is outdated and should be removed, correct?

Test failures look to be due to the random coupon redemption test failures.

lawxen’s picture

StatusFileSize
new553 bytes
new5.3 KB

Remove outdated comment:
# @see \Drupal\commerce_order\Normalizer\ObjectAnyNormalizer::supportsNormalization

@mglaman #25:2916252-25.patch tests succeed at the second time when choosing the same conditons.
PHP 7.1 & MySQL 5.5, D8.4

lawxen’s picture

Status: Needs work » Needs review
mglaman’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: -Needs tests

This looks good to me! Just want to check with bojanz

lawxen’s picture

StatusFileSize
new553 bytes
new5.3 KB

If wget https://www.drupal.org/files/issues/2916252-27.patch to local, It display

+    # This normalizer must just be higher than serializer.normalizer.typed_data,
+    # So setting priority as 0 here.

But review on the web, it display
+ # This normalizer must just be higher than serializer.normalizer.typed_data, So setting priority as 0 here.

Something wrong with drupal'org. Maybe caused by updaloading the 2916252-27.patch twice when edit the comment.

I reupload the patch no mater whatever happens.

mglaman’s picture

Assigned: lawxen » mglaman

caseylau thanks! I'll commit this today.

wim leers’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs tests
  1. +++ b/modules/order/commerce_order.services.yml
    @@ -62,3 +62,9 @@ services:
    +    # This normalizer must just be higher than serializer.normalizer.typed_data,
    +    # So setting priority as 0 here.
    +      - { name: normalizer, priority: 0}
    

    If we're just using priority 0, then we can omit both the comment and the service priority.

  2. +++ b/modules/order/src/Normalizer/AnyAdjustmentNormalizer.php
    @@ -0,0 +1,51 @@
    +class AnyAdjustmentNormalizer extends NormalizerBase {
    ...
    +  protected $supportedInterfaceOrClass = 'Drupal\Core\TypedData\Plugin\DataType\Any';
    

    This would affect not just the Commerce Order module, but any entity type that uses a \Drupal\Core\TypedData\Plugin\DataType\Any.

    IOW: this introduces unwanted side effects.

  3. +++ b/modules/order/src/Normalizer/AnyAdjustmentNormalizer.php
    @@ -0,0 +1,51 @@
    +  public function supportsNormalization($data, $format = NULL) {
    +    /* @var \Drupal\Core\TypedData\Plugin\DataType\Any $data */
    +    if (parent::supportsNormalization($data, $format)) {
    +      $value = $data->getValue();
    +      if ($value instanceof Adjustment) {
    +        return TRUE;
    +      }
    +    }
    +    return FALSE;
    +  }
    

    Oh but this then does the necessary extra checking. Why not just declare Drupal\commerce_order\Adjustment the supported class?

  4. +++ b/modules/order/src/Normalizer/AnyAdjustmentNormalizer.php
    @@ -0,0 +1,51 @@
    +  public function normalize($object, $format = NULL, array $context = []) {
    

    Looks good, but I don't see an equivalent denormalize() method?

    This means only GET will work, PATCH and POST will still fail.

  5. +++ b/modules/order/tests/src/Kernel/AdjustmentItemNormalizeTest.php
    @@ -0,0 +1,109 @@
    +class AdjustmentItemNormalizeTest extends CommerceKernelTestBase {
    

    Please, please, please create a functional test for this.

    We have \Drupal\Tests\rest\Functional\EntityResource\EntityResourceTestBase for that in Drupal core. (See its many subclasses.)

    See \Drupal\Tests\datetime\Functional\EntityResource\EntityTest\EntityTestDatetimeTest for an example of a functional test specifically for a field type's normalization.

  • mglaman committed fbe8dd2 on 8.x-2.x authored by caseylau
    Issue #2916252 by caseylau, skyredwang: Order's Adjustment can't be...
mglaman’s picture

Assigned: mglaman » Unassigned
Status: Needs work » Fixed

Committed, thanks!

mglaman’s picture

Status: Fixed » Needs work

Putting to needs work. Totally missed #32. I am not reverting. Let's just make a new patch which

  • Removes priority definition and comment
  • Is able to denormalize data
  • Create a matching Functional test beyond the kernel test
lawxen’s picture

StatusFileSize
new1.54 MB
new10.25 KB
new7.97 KB

First reply #32 @Wim Leers:

  1. If we're just using priority 0, then we can omit both the comment and the service priority.

    This is right

  2. This would affect not just the Commerce Order module, but any entity type that uses a \Drupal\Core\TypedData\Plugin\DataType\Any.
    IOW: this introduces unwanted side effects.

    Because commerce suggest don't change object Adjustment and Hack core or waiting core's issue be fixed, So abandon #7's patch, Just write custom normalizer, More checking will appear in supportsNormalization();

  3. Why not just declare Drupal\commerce_order\Adjustment the supported class?
    This can not be done,What this doing is replacing core's and jsonapi's severial FieldItemNormalzer
    1. If priority just higher than Drupal\serialization\Normalizer\FieldItemNormalizer, This normalzer won't replace hal or jsonapi's FieldItemNormalzer
    2. If priority higher than any other FieldItemNormalizer, You can only follow one FieldItemNormalizer's design, Other will totally crash
    3. So this is wrong

  4. but I don't see an equivalent denormalize() method?

    This is right, denormalize is needed.

  5. please create a functional test for this.
    This is right, functional test is needed too.

Patch:

  1. 2916252-36-a.patch(suggest):
    1. add some code to AdjustmentItem's setValue() to make FieldItemNormalze's denormalize() work well
    2. remove priority and comment
    3. add denormalize test
    4. add functional test
  2. 2916252-36-b.patch(disapprove):
    1. Modify AdjustmentNormalizer to make it replacing FieldItemNormalizer when denormalizing adjustment filed
    2. set priority to 22
    3. add denormalize test
    4. add functional test

Two patch's Pros and cons:

  1. 2916252-36-a.patch(suggest):
    pros:
    1. conform to origin flow of execution
    2. After core https://www.drupal.org/node/2915705 fixed, The AdjustmentItem has no need to change

    cons:

    1. Hack AdjustmentItem
  2. 2916252-36-b.patch(disapprove):
    pros:
    1. No affection on AdjustmentItem and Adjustment itself

    cons:

    1. Maybe has some affection when other module like jsonapi use it's own FieldItemNormalzer, I haven't find definite evidence
    2. Speed a little slow than 2916252-36-a.patch

Besides:
Upload a normalize and denormalize flow debug image:

mglaman’s picture

Status: Needs work » Needs review

This is above my head, now, so I'll see what Wim says :) Patch A looks to be my preference as well.

+++ b/modules/order/src/Plugin/Field/FieldType/AdjustmentItem.php
@@ -48,6 +49,11 @@ class AdjustmentItem extends FieldItemBase {
+    // Used for denormalizing.
+    if (is_array($values)) {
+      $values['amount'] = new Price($values['amount']['number'], $values['amount']['currency_code']);
+      $values = new Adjustment($values);
+    }

I think hacking into AdjustmentItem here is not the end of the world. Just because we are supporting non-TypedData value objects.

The last submitted patch, 36: 2916252-36-b.patch, failed testing. View results

Status: Needs review » Needs work

The last submitted patch, 36: 2916252-36-a.patch, failed testing. View results

drugan’s picture

Seems something went wrong with the latest commit.

drush cr

PHP Fatal error: Class 'Drupal\serialization\Normalizer\NormalizerBase' not found in /home/drugan/public_html/drugan.dev/web/modules/commerce_core_modules/commerce/modules/order/src/Normalizer/AdjustmentNormalizer.php on line 11

Fatal error: Class 'Drupal\serialization\Normalizer\NormalizerBase' not found in /home/drugan/public_html/drugan.dev/web/modules/commerce_core_modules/commerce/modules/order/src/Normalizer/AdjustmentNormalizer.php on line 11
Drush command terminated abnormally due to an unrecoverable error. [error]
Error: Class 'Drupal\serialization\Normalizer\NormalizerBase' not found in
/home/drugan/public_html/drugan.dev/web/modules/commerce_core_modules/commerce/modules/order/src/Normalizer/AdjustmentNormalizer.php,
line 11

mglaman’s picture

Argh, okay. I need to revert that commit. We need to add commerce_order.normalizer.adjustment through a ServiceProvider when the Serialization module is available.

EDIT: I'm not able to reproduce this error when rebuilding cache through Drush or the UI (Performance page, core/rebuild.php)

drugan’s picture

My DC is updated to this latest commit:

commit a31816a4f0c3ca17d7fd0b9105b049a00e90cc87
Author: TommyChris
Date: Tue Oct 24 08:53:54 2017 -0500

Issue #2918392 by TommyChris: ParseError: syntax error, unexpected '$this' (T_VARIABLE) in Composer\Autoload\includeFile() (line 5

drugan’s picture

OK, the reason is the Serialization module was not enabled. Now drush cr runs without errors.

Should not be the Serialization put in the commerce_order dependencies now? Or at least to emit somehow a warning to users? In any case commerce_base install profile need to be updated to enable the Serialization on install. I'll make a PR.

mglaman’s picture

drugan, in #41 I said we need to revert the commit if it causes a soft-dependency. However, I have not been able to reproduce and our tests did not fail. Also, it should not cause an issue unless the service is invoked, which it only should be when Serializer is enabled.

I'll be reverting later today.

  • mglaman committed 4710d6f on 8.x-2.x
    Revert "Issue #2916252 by caseylau, skyredwang: Order's Adjustment can't...
mglaman’s picture

Okay, I reverted.

We'll need to merge #30 and a patch from #36. We'll also need to only register this service when the Serialization module is installed using a ServiceProvider.

See https://github.com/drupalcommerce/commerce/blob/8.x-2.x/modules/log/src/... for an example.

lawxen’s picture

@mglaman
I test #30 patch on drupal8.5.x drupal8.4.x drupal8.4.0, drupal-composer/drupal-project, even drupal8.3.7(force enable commerce2.x), cant't reproduce #40 error. I'm wondering whether we should

register this service when the Serialization module is installed using a ServiceProvider.

@drugan Can you direct me reproduce the bug?

lawxen’s picture

Oh, Even if I can't reproduce the bug, we should do

register this service when the Serialization module is installed using a ServiceProvider.

Because it use serialization's code

drugan’s picture

It looks like a bit weird but after uninstalling the Serialization module I can't reproduce the #40 error too. Magic!

My DC story:

I've installed site using my own forks of the commerce_base, project_base and DC itself. Basically, there are no any radical changes except I install some additional modules. Also, DC core modules mapped to be downloaded into web/modules/commerce_core_modules folder. All the rest modules download as usual into web/modules/contrib folder. That's is the entire difference.

So, first I've run this and then installed site (before the serialization commit):

composer create-project --repository=https://raw.githubusercontent.com/drugan/project-base/8.x/packages.json drugan/project-base some-dir --stability dev

Everything worked great. Sometimes I merge changes on the base DC project and again, before the serialization commit there was not any issues.

Note that I had the error not only with drush cr but also with drush updb --entity-updates command. Also, when trying to visit the site page I had the same fatal error reported in red as I have all errors enabled on the site. Then I've just removed the extends control from the AdjustmentNormalizer class:

/**
 * Custom normalizer for 'any' typed data with Adjustment stored.
 */
class AdjustmentNormalizer {
// ...
}

Successfully run drush cr and tried to restore the class file. Again the same error. At this stage I've decided to check if I have this module enabled at all. And no, it was not. After enabling everything worked as expected.

bojanz’s picture

I think #41 is the right approach. We always have occasional problems if a service isn't defined in a service provider but depends on an optional module.

lawxen’s picture

StatusFileSize
new12.69 KB
  1. combine 2916252-36-a.patch and 2916252-30.patch
  2. register this service when the Serialization module is installed using a ServiceProvider
skyredwang’s picture

Status: Needs work » Needs review

This issue actually do not have a dependency on #2915705: TypedData 'Any' can't be normalized to array if it stores an object anymore, as we are normalizing $order->adjustments on the field level instead of data type level.

wim leers’s picture

Title: Order's Adjustment can't be normalized and serialized » [PP-1] Order's Adjustment can't be normalized and serialized
Status: Needs review » Postponed
StatusFileSize
new1.03 KB

So let's postpone this on that issue then.

I believe that once #2915705: TypedData 'Any' can't be normalized to array if it stores an object is in, all this issue would need to do is:

-final class Adjustment {
+final class Adjustment implements \IteratorAggregate {

Sample implementation attached. We'd still want the tests that the previous patches have (to ensure it actually works, and to ensure we don't regress), but all of the complexity would live in that other issue :)

lawxen’s picture

StatusFileSize
new2.13 KB
new2.02 KB

Combined #7 #51 #53

like #53 described.
prepare for #2915705: TypedData 'Any' can't be normalized to array if it stores an object

Add denormalize

    // Used for denormalization.
    if (is_array($values)) {
      $values['amount'] = new Price($values['amount']['number'], $values['amount']['currency_code']);
      $values = new Adjustment($values);
    }
bojanz’s picture

I am fine with the #53 / #54 approach, since the additions to the Adjustment class are minimal, and consist of wrapping toArray(), which is useful on its own.

lawxen’s picture

StatusFileSize
new1.39 KB
new1.25 KB

Just a re-roll, Nothing change.

lawxen’s picture

StatusFileSize
new1.67 KB
new549 bytes
lawxen’s picture

StatusFileSize
new1.98 KB
new450 bytes
-      'amount' => $this->amount,
+      'amount' => $this->amount->toArray(),
josephdpurcell’s picture

Status: Postponed » Needs review

I encountered this issue using Commerce 8.x-2.7 and Drupal 8.5. I have confirmed that patch #58 applies correctly and is sufficient in terms of being able to serialize adjustments.

Given that this patch applies and is working fine, is there a need to postpone this issue? Setting to "Needs Review" to ensure this doesn't go missed. The patch is quite small. I apologize in advance if this patch does in fact still need to be postponed.

The last submitted patch, 53: 2916252-53.patch, failed testing. View results

Status: Needs review » Needs work

The last submitted patch, 58: 2916252-58.patch, failed testing. View results

mglaman’s picture

Status: Needs work » Postponed

It's postponed because core should fix the problem itself :/ The patches are just here as a workaround.

mglaman’s picture

Title: [PP-1] Order's Adjustment can't be normalized and serialized » Order's Adjustment can't be normalized and serialized
Status: Postponed » Needs work

#2895532: @DataType=map cannot be normalized, affects @FieldType=link, @FieldType=map landed in 8.6.x!

No longer postponed.

Needs work and review against the 8.6.x branch.

Will required is to force ^8.6.0 in order to land. Which means this is blocked on 8.6.0 (for official commit)

wim leers’s picture

Yep, and this still needs test coverage :)

See the many subclasses of EntityResourceTestBase for examples!

lawxen’s picture

Assigned: Unassigned » lawxen
Status: Needs work » Postponed

#2895532: @DataType=map cannot be normalized, affects @FieldType=link, @FieldType=map landed in 8.6.x!

No longer postponed.

#2895532: @DataType=map cannot be normalized, affects @FieldType=link, @FieldType=map has nothing relationship with this issue, And it can't fix Order's Adjustment's normalization problem with patch 2916252-58.patch .

#59 Maybe @josephdpurcell used both TypedData 'Any' can't be normalized to array if it stores an object and 2916252-58.patch.

Yep, and this still needs test coverage :)

See the many subclasses of EntityResourceTestBase for examples!

I have tested that 2916252-58.patch can't be fix this issue alone, I will write a test to prove it.

lawxen’s picture

StatusFileSize
new9.55 KB
new6.97 KB

Add resource test for adjustment field.

lawxen’s picture

Assigned: lawxen » Unassigned
wim leers’s picture

Thanks for working on test coverage! Now let's get it to pass :)

mglaman’s picture

Rerunning the tests to see where this is at :)

mglaman’s picture

Title: Order's Adjustment can't be normalized and serialized » [PP-1] Order's Adjustment can't be normalized and serialized
yonailo’s picture

I can confirm that 2916252-66.patch + 2915705-59.patch work nicely for me. I had found this issue when creating an Order Item + using the module 'content_sync'. Those 2 patches solved the issue for me.

akhil babu’s picture

StatusFileSize
new9.56 KB

Thanks for the patch. Adjustment values are now getting displayed in API response after applying the latest patch from https://www.drupal.org/project/drupal/issues/2915705 and then 2916252-66.patch.
But there was a waring after applying 2916252-66.patch.
Deprecated function: Return type of Drupal\commerce_order\Adjustment::getIterator() should either be compatible with IteratorAggregate::getIterator(): Traversable, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in include()
Updating 2916252-66.patch to remove the warning.
Change

  public function getIterator() {
    return new \ArrayIterator($this->toArray());
  }

to

  public function getIterator(): \ArrayIterator {
    return new \ArrayIterator($this->toArray());
  }
alex.bukach’s picture

StatusFileSize
new9.56 KB
new388 bytes

It should be \Traversable, not \ArrayIterator.

Agence Web CoherActio’s picture

The last patch caused side-effects (e.g. price substract() method with adjustment).
I ended up extending Drupal TypedDataNormalizer class to support Ajdustment normalizer

namespace Drupal\my_module\Normalizer;

use Drupal\commerce_order\Adjustment;
use Drupal\Core\TypedData\TypedDataInterface;
use Drupal\serialization\Normalizer\TypedDataNormalizer;

class MyModuleAdjustmentNormalizer extends TypedDataNormalizer {

  /**
   * {@inheritdoc}
   */
  public function normalize($object, $format = NULL, array $context = []): array|string|int|float|bool|\ArrayObject|NULL {
    $this->addCacheableDependency($context, $object);
    $value = $object->getValue();
    // Support for stringable value objects: avoid numerous custom normalizers.
    if (is_object($value) && method_exists($value, '__toString')) {
      $value = (string) $value;
    }
    if ($value instanceof Adjustment) {
      $value = $value->toArray();
    }
    return $value;
  }
}
jsacksick’s picture

StatusFileSize
new1.68 KB

The attached patch works... However I don't think we can commit it as this means Commerce just provides a normalizer for the "Any" data type, which is used by other field types... which is a problem... But attaching the patch anyway..

jsacksick’s picture

Perhaps we could provide our normalizer, only if the core one introduced in #2915705: TypedData 'Any' can't be normalized to array if it stores an object doesn't exist?

jsacksick’s picture

Status: Postponed » Needs review
StatusFileSize
new2.3 KB

Actually, let's go with this patch... Which provides the "Any" normalizer conditionally in case it is not defined by the serialization module.

tBKoT made their first commit to this issue’s fork.

jsacksick’s picture

We were missing the test group (added that). But also suddenly not sure about the fix, whenever I don't skip the test methods defined by the base class, there seems to be an issue with the denormalization of the adjustments field... Also... now extending getSupportedTypes() from the normalizer as the previous approach :

  /**
   * {@inheritdoc}
   */
  protected $supportedInterfaceOrClass = Any::class;

is deprecated in D10.2. I'd need to double check when this change was introduced to see if we need ot raise the required core version... Since I'm not sure the issue is actually fixed... Putting this issue on hold for now.

mrweiner’s picture

@jsacksick re: #80, I can confirm that the patch in #77 fixes the issue for me on D10.3.0.

trickfun’s picture

Patch #77 works fine with Drupal 10.3
Thank you

jsacksick’s picture

Last time I worked on the tests, they were failing, and can't remember why, so looks like the patch doesn't address all the problems...

trickfun’s picture

jsacksick’s picture

@tBKoT: Interesting approach, just wondering if this isn't a breaking change?
Also, we might as well copy what was done in Commerce API. The Commerce API module defines an Adjustment data type as well as an AdjustmentDataDefinition definition class.

The data definition class defines all the adjustment properties. This should help getting rid of the "value".

tbkot’s picture

@jsacksick I've tried to get rid of the "value" key but it seems that it is not possible without rewriting a lot of staff or removing the denormalization process in the test (it would fail in this case)

The main thing is that JSON:API uses the field property definition as a foundation to generate the structure if we change it as a result all other things related to this field would be broken such as getAdjustments() etc. as those methods use "value" and I guess many other contrib/custom solutions may use this field like that.

Another thing that can help is to define more properties with their own data types to set values for Label, Type, Amount, etc. I do not think it is the best way to resolve this.

One more solution is to disable access to the main "adjustment" field for JSON:API as it is done in the "commerce_api" module and define a new computed field for it to show adjustment without the "value" key.

ivnish’s picture

The patch from MR fix the WSOD. Looks like RTBC

alexpott’s picture

StatusFileSize
new2.3 KB

Need the patch for a D11 site.

alexpott’s picture

StatusFileSize
new2.33 KB

Ah we need return typehints...

goz’s picture

#89 works great on Commerce 3.0

goz’s picture

But we should also support denormalizer implementing DenormalizerInterface and adding denormalize method, so we can create order item with adjustment from jsonapi.


  /**
   * {@inheritdoc}
   */
  public function denormalize($data, $type, $format = NULL, array $context = []) {
    if (isset($context['field_type']) && 'commerce_adjustment' === $context['field_type']) {
      if (isset($data['amount'], $data['amount']['number'], $data['amount']['currency_code'])) {
        $data['amount'] = new Price($data['amount']['number'], $data['amount']['currency_code']);
      }
      return new Adjustment($data);
    }

    return $data;
  }
goz’s picture

Status: Needs review » Needs work
goz’s picture

#89 works with 3.0 but use previous fix based on "Any".
MR add the adjustmentType which should be kept for 3.0

goz changed the visibility of the branch 3.0.x to hidden.

jsacksick’s picture

Do we actually need to declare each individual property like the following? The normalizer is on the value property, isn't that sufficient?
The normalization still works without those... Or is that needed for setting adjustments though the API?

$properties['type'] = DataDefinition::create('adjustment_property')
      ->setLabel(t('Type'))
      ->setComputed(TRUE);

    $properties['label'] = DataDefinition::create('adjustment_property')
      ->setLabel(t('Label'))
      ->setComputed(TRUE);

    $properties['amount'] = DataDefinition::create('adjustment_property')
      ->setLabel(t('Amount'))
      ->setComputed(TRUE);

    $properties['source_id'] = DataDefinition::create('adjustment_property')
      ->setLabel(t('Source ID'))
      ->setComputed(TRUE);

    $properties['percentage'] = DataDefinition::create('adjustment_property')
      ->setLabel(t('Amount'))
      ->setComputed(TRUE);

    $properties['included'] = DataDefinition::create('adjustment_property')
      ->setLabel(t('Included'))
      ->setComputed(TRUE);

    $properties['locked'] = DataDefinition::create('adjustment_property')
      ->setLabel(t('Locked'))
      ->setComputed(TRUE);
jsacksick’s picture

Version: 8.x-2.x-dev » 3.0.x-dev
Status: Needs work » Needs review

Ok removing the extra properties from the AdjustmentItem failed... But removing the AdjustmentPropertyDefinition worked... Wondering why it was added in the first place?

rhovland’s picture

So on our site we're writing new adjustments through the API, not just updating existing ones so we need all of those extra properties to be able to set the characteristics of the adjustment.

jsacksick’s picture

Issue tags: -Needs tests

@rhovland: Does the current patch allow for that? (from the MR)? I'm considering merging it as it looks ready.

rhovland’s picture

I will update our patch on our test site and make sure the removal of AdjustmentPropertyDefinition didn't mess anything up.

goz’s picture

With last patch, no error is thrown, i can list adjustments for orders and order items, but i cannot create them.

GET /jsonapi/commerce_order/default?filter[drupal_internal__order_id]=3807'

{
    "jsonapi": {
        "version": "1.0",
        "meta": {
            "links": {
                "self": {
                    "href": "http://jsonapi.org/format/1.0/"
                }
            }
        }
    },
    "data": [
        {
            "type": "commerce_order--default",
            "id": "ad15f584-05f7-43a8-bd55-ab18a4eaba2c",
            "links": {
...
            },
            "attributes": {
                "drupal_internal__order_id": 3807,
                "order_number": "3807",
...
                "adjustments": [
                    {
                        "type": "shipping",
                        "label": "Expédition",
                        "amount": {
                            "number": "4.95",
                            "currency_code": "EUR",
                            "formatted": "4,95 €"
                        },
                        "percentage": null,
                        "source_id": "1058",
                        "included": false,
                        "locked": false
                    }
                ],
                "total_price": {
                    "number": "28.950000",
                    "currency_code": "EUR",
                    "formatted": "28,95 €"
                },
                "total_paid": null,
                "balance": {
                    "number": "28.950000",
                    "currency_code": "EUR",
                    "formatted": "28,95 €"
                },
                "state": "completed",
                "locked": false,
                "created": "2025-03-14T08:13:38+00:00",
                "changed": "2025-03-14T13:53:41+00:00",
                "placed": "2025-03-14T13:53:41+00:00",
                "completed": "2025-03-14T13:53:41+00:00",
                "customer_comments": null,
                "cart": false,
                "checkout_step": "complete",
...
            },
            "relationships": {
...
            }
        }
    ],
...
}

But creating new order doesn't :

{
    "data": {
        "type": "commerce_order_item--abonnement_single",
        "attributes": {
                            "title": "Abonnement individuel - 1 an",
                            "quantity": "1.00",
                            "unit_price": {
                                "number": "15.000000",
                                "currency_code": "EUR",
                                "formatted": "15,00 €"
                            },
                            "total_price": {
                                "number": "15.000000",
                                "currency_code": "EUR",
                                "formatted": "15,00 €"
                            },
                            "adjustments": [
                                {
                                    "type": "tax",
                                    "label": "TVA",
                                    "amount": {
                                        "number": "0.31",
                                        "currency_code": "EUR"
                                    },
                                    "percentage": "0.021",
                                    "source_id": "france|0|0",
                                    "included": true
                                }
                            ]
        },
        "relationships": {
            "uid": {
                "data": {
                    "type": "user--user",
                    "id": "565a095e-d398-4ebb-9030-f877d9301601"
                }
            },
            "order_id": {
                "data": {
                    "type": "commerce_order--default",
                    "id": "f1fe4660-dba4-4c90-8c0e-4090310a1e78"
                }
            },
            "store_id": {
                "data": {
                    "type": "commerce_store--online",
                    "id": "1adc9d7c-c976-4951-9cd4-38413b150a51"
                }
            },
            "purchased_entity": {
                "data": {
                    "type": "commerce_product_variation--abonnement_single",
                    "id": "e12569c7-24a4-4a5c-844f-d67ea1e01a21"
                }
            }

        }
    }
}

Result :

{
    "jsonapi": {
        "version": "1.0",
        "meta": {
            "links": {
                "self": {
                    "href": "http://jsonapi.org/format/1.0/"
                }
            }
        }
    },
    "data": {
        "type": "commerce_order_item--abonnement_single",
        "id": "cea7ce7c-84c2-4154-85b6-e65c87bf233c",
        "links": {
...
        },
        "attributes": {
            "drupal_internal__order_item_id": 5932,
            "title": "Abonnement individuel - 1 an",
            "overridden_title": false,
            "quantity": "1",
            "unit_price": {
                "number": "15.000000",
                "currency_code": "EUR",
                "formatted": "15,00 €"
            },
            "overridden_unit_price": false,
            "total_price": {
                "number": "15.00",
                "currency_code": "EUR",
                "formatted": "15,00 €"
            },
            "adjustments": [],
            "uses_legacy_adjustments": false,
            "data": null,
            "locked": false,
            "created": "2025-05-16T10:34:03+00:00",
            "changed": "2025-05-16T10:34:03+00:00"
        },
..
}

Same thing creating order_item.

jsacksick’s picture

@goz: What we don't see in your response is whether the order is a draft order? IF it is a draft order then on order save, the adjustment will be cleared by the OrderRefresh service right? So that is probably expected? You'd need to try on a non draft order.

goz’s picture

My bad, you are right, adjustments are cleared and recalculated. I forgot that. Checked again and everything is fine

jsacksick’s picture

So RTBC then? Would be great to finally close this... I think this is ready! Would appreciate an RTBC.

goz’s picture

Status: Needs review » Reviewed & tested by the community

Let's do it !
Thanks

  • jsacksick committed e1ca95d8 on 3.x authored by goz
    Issue #2916252 by lawxen, tbkot, jsacksick, goz, alexpott, alex.bukach,...
jsacksick’s picture

Status: Reviewed & tested by the community » Fixed

Merged! 🎉! Thanks everyone!!

rhovland’s picture

So far we have not been able to use PATCH to replace adjustments. They go poof with no error returned. This is on non-draft orders.

rhovland’s picture

If I try to send a patch request to https://example.com/jsonapi/commerce_order/default/ab42ab1c-b21f-4da2-9e...

With JSON

{
    "data":{
        "type": "commerce_order--default",
        "id": "ab42ab1c-b21f-4da2-9e9b-82213e622e59",
        "attributes": {
            "adjustments": [
                {
                    "type": "shipping",
                    "label": "Shipping",
                    "amount": {
                        "number": "126.940000",
                        "currency_code": "USD"
                    },
                    "percentage": null,
                    "source_id": "209676",
                    "included": false,
                    "locked": false
                }
            ]
        }
    }
}

I get the response

{
    "jsonapi": {
        "version": "1.0",
        "meta": {
            "links": {
                "self": {
                    "href": "http://jsonapi.org/format/1.0/"
                }
            }
        }
    },
    "data": {
        "type": "commerce_order--default",
        "id": "ab42ab1c-b21f-4da2-9e9b-82213e622e59",
        "links": {
            "self": {
                "href": "https://example.com/jsonapi/commerce_order/default/ab42ab1c-b21f-4da2-9e9b-82213e622e59"
            }
        },
        "attributes": {
            "drupal_internal__order_id": 289391,
            "order_number": "289391",
            "version": 15,
            "mail": "redacted@example.com",
            "ip_address": "208.56.233.218",
            "adjustments": [],
            "total_price": {
                "number": "226.22",
                "currency_code": "USD",
                "formatted": "$226.22"
            },
            "total_paid": {
                "number": "253.160000",
                "currency_code": "USD",
                "formatted": "$253.16"
            },
            "balance": {
                "number": "-26.94",
                "currency_code": "USD",
                "formatted": "-$26.94"
            },
            "state": "shipped",
            "data": {
                "paid_event_dispatched": true
            },
            "locked": false,
            "created": "2024-12-16T21:19:32+00:00",
            "changed": "2025-05-16T20:19:03+00:00",
            "placed": "2024-12-16T21:28:24+00:00",
            "completed": null,
            "customer_comments": null,
            "cart": false,
            "checkout_step": null,
            "field_customer_comments": null,
            "field_customer_motorcycles": [],
            "field_newsletter": false
        },
        "relationships": {
               removed for brevity
        }
    },
    "links": {
        "self": {
            "href": "https://example.com/jsonapi/commerce_order/default/ab42ab1c-b21f-4da2-9e9b-82213e622e59"
        }
    }
}

The adjustments are just empty.

rhovland’s picture

Ok so after combing through things, we were using #72 plus the core Any normalizer patch and it was working.

The issue seems to be the final commit never implemented a denormalizer so that PATCH operations can work. The patch we were using had one. Kinda.

diff --git a/modules/order/src/Plugin/Field/FieldType/AdjustmentItem.php b/modules/order/src/Plugin/Field/FieldType/AdjustmentItem.php
index 81ae6c90..cbc96972 100644
--- a/modules/order/src/Plugin/Field/FieldType/AdjustmentItem.php
+++ b/modules/order/src/Plugin/Field/FieldType/AdjustmentItem.php
@@ -3,6 +3,7 @@
 namespace Drupal\commerce_order\Plugin\Field\FieldType;

 use Drupal\commerce_order\Adjustment;
+use Drupal\commerce_price\Price;
 use Drupal\Core\Field\FieldItemBase;
 use Drupal\Core\Field\FieldStorageDefinitionInterface;
 use Drupal\Core\TypedData\DataDefinition;
@@ -48,6 +49,11 @@ class AdjustmentItem extends FieldItemBase {
       // The property definition causes the adjustment to be in 'value' key.
       $values = reset($values);
     }
+    // Used for denormalization.
+    if (is_array($values)) {
+      $values['amount'] = new Price($values['amount']['number'], $values['amount']['currency_code']);
+      $values = new Adjustment($values);
+    }
     if (!$values instanceof Adjustment) {
       $values = NULL;
     }

So as committed this lets you read adjustments via API but not write them.

rhovland’s picture

@goz You got this to successfully write order adjustments? Do you have commerce_api installed?

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.