Download & Extend

Improve the TypedData API usage of EntityNG

Project:Drupal core
Version:8.x-dev
Component:base system
Category:task
Priority:major
Assigned:Unassigned
Status:needs review
Issue tags:Entity Field API, sprint, typed data

Issue Summary

Right now, entity types are not exposed as proper typed data API types. There is just a type 'entity', which gets you a typed data object *wrapping* the actual entity object. But furthermore, $entity is an instance of the ComplexDataInterface of the typed data API and with #1778178: Convert comments to the new Entity Field API also an instanced of the ContextAwareInterface.

First, I'd consider this a DX is wtf, it's implementing the typed data just the half way, while all the derived fields are implementing it the full way, i.e. you can call $field->getType() or $field->getDefinition() what you cannot do for an entity.

Second, this special separation makes things more complex, e.g. it will cause problems when implementing #1696648: [META] Untie entity validation from form validation based upon TypedData validation (as the entity itself does not implement the TypedDataInterface - ouch).

Third, the parent relationship of typed data APIs is broken for entities. E.g. when you have a referenced entity and you use the TypedData API to derive a property of the entity from the TypedData object (EntityWrapper), it's parent will be another object (the entity object) - ouch. Also, right now $typed_data->getParent() cannot assume the parent is typed data, it can just assume it's complex data or a list. We should fix that.

Once we've done, we can let the TypedData ListInterface and ComplexDataInterface extend the TypedDataInterface also - we do not need to support implementing the API half-way any more ;-)

I'm thinking about options to implement this best such that it results in no WTFs, will come back with my thoughts.

Comments

#1

ok, I think the following should work out:

  • Make EntityInterface extend TypedDataInterface also. An entity is always the root of the data tree.
  • Have registered typed data types like "node", "comment" and return that from the getType() method.
  • If you have an entity reference the typed data object wrapping the entity must actually be a different object, as it's not the root of tree but a property in another tree, so it will have a different 'type', e.g. "entity_reference".
  • To point to the referenced entity/tree I think we should introduce a new interface DataReferenceInterface or so. (Delegating interface methods would not result in cleanly decoupled trees. We cannot rely on the plain value being the object either as this does not make sense for other reference cases where the object do not implement the typed data API natively, e.g. language.)
  • For implementing the TypedDataInterface with entities we need to have a) getValue() setValue() functions that can set the entity object "value" and b) support (re-)constructing an object via typed_data()->createInstance($type, $value). For that to make sense I think $value should be the complete array of property values, including translations.
  • For creating entity objects via typed_data()->createInstance($type, $value) we would have to adapt the construct or make creating typed data objects more flexible, e.g. allow specifying a factory class.

#2

Could you please post some concrete examples or code snippets in the issue summary so that we can understand the problem better? What are the implications for a node article for example?

#3

What it fixes is that the following works:

<?php
$entity
instanceof TypedDataInterface;
$entity->getType() == 'node';
$entity->getDefinition(); $entity->validate(); // etc. works as for any other typed data.
$entity->user_id->getParent() instanceof TypedDataInterface;
$node = typed_data()->create('node', $values);
?>

whereas the following would stop to work:

<?php
// The entity reference would not be complex data any more,
// so you cannot do:
$entity->get('user_id')->get('entity')->get('field');
// while the following continues to work (=delegation is removed)
$entity->get('user_id')->get('entity')->getValue()->get('field');

// And you cannot run in the issue that the getParent() is inconsistent:
$entity_reference = $entity->get('user_id')->get('entity');
$entity_reference->get('field')->getParent() !== $entity_reference // The parent is the entity object as of now
?>

Instead, you could do something like the following (while using TypedData API only):

<?php
  $entity
->get('user_id')->get('entity')->resolveReference()->get('field');
 
// What in the case of an entity reference equals to
 
$entity->user_id->entity->field;
?>

We need resolveReference() as we cannot rely on the value being a TypedData object of the referenced value, e.g. for language references it wouldn't.

#4

Adding tags.

#5

#6

This patch allowed the EntityWrapper to support all entity types with an array('type' => 'entity:node') style invokation. type 'entity' is still supported, though internally it is known as entity:entity, and there's a little bit of a bc layer for that. I assume we want to remove that in the long term, but even if we don't it's only a couple small if statements.

The definition constraints didn't appear to be getting passed to the plugins appropriately, and since the derivatives were adding constraints for EntityType, I sort of needed it to work. It still respects whatever the user passed so array('type' => 'entity:node', 'constraints' => array('EntityType' => 'user')) should actually give you a user entity, but I didn't try it. The patch is very straight forward otherwise.

Eclipse

AttachmentSizeStatusTest resultOperations
typed-data-deriver.patch4.22 KBIdleFAILED: [[SimpleTest]]: [MySQL] Unable to apply patch typed-data-deriver.patch. Unable to apply patch. See the log in the details link for more information.View details | Re-test

#7

+++ b/core/lib/Drupal/Core/TypedData/TypedDataFactory.php
@@ -57,6 +65,11 @@ public function createInstance($plugin_id, array $configuration, $name = NULL, $
+    // If constraints exist in the definition, pass them to the configuration.
+    if (isset($type_definition['constraints']) && !empty($type_definition['constraints'])) {
+      $configuration['constraints'] += $type_definition['constraints'];

No need to do that, that's taken care of for you when you get the constraints. So you always can differentiate between type constraints and data definition constraints. Check the e-mail example - it works and has type constraints.

#8

ok, as discussed on IRC we also need away to deal with data types that cannot be instantiated properly. E.g if we do this what would we instantiate for type = entity ? It works for a user, that we could instantiate, but we cannot even instantiate node without knowing the bundle (as to the current impl.).

  • So the idea we came up with is the following:
  • We introduce the notion of abstract types, i.e. a usual typed data plugin with a special flag, e.g. a key in the typed definition.
  • Different to others is that abstract types are for dealing with their metadata only, so you cannot set/get a value on those object, i.e. we throw an exception in that case.
  • Examples for abstract types would be "entity" and every entity types that has bundles (node, comment, but not user).
  • If you instantiate an abstract type, you get an usual typed data object based upon new helper class. Those help you to deal with the metadata and can even work from situations where you specify a bunch of possible bundles (a node reference pointing to articles and pages only).
  • For dealing with the metadata of non-abstracts type we'll continue to instantiate the usual classes without data.

#9

ok, started working on this. Attached patch starts with implementing #1 and partially also #8. It's not finished yet but should show where it goes.

Todo:
* Finish abstract type implementation
* Finally make $entity implement the typed data interface
* Convert tests dealing with type => 'entity'
* Make sure all entity types and bundle combinations are registered
* Write tests for the new API

On what the value of an entity should be? I think it should be plain array representation of all the values of an entity (including translations?) That way you can create a valid entity object by using the typed data factory + passing in all necessary values and such this also serves as correct entity factory.

#10

Status:active» needs work
AttachmentSizeStatusTest resultOperations
d8_expose.patch.txt25.36 KBIgnoredNoneNone

#11

#10 looks good
any progress here?

#12

I've worked a bit more on that, so we are getting closer. Attached patch is functionally mostly complete, but has test-fails as reference-updates are not reflected due to the stale $target property in the reference.

TODO:
* Complete the abstract entity implementation (metadata with multiple-bundles) and add tests for it.
* Finally make $entity implement the typed data interface
* Write tests for the new API

For the stale data in reference to get updated, we could do an interim hack for comparing the target-id with the source id. But for doing that properly + for #1869562: Avoid instantiating EntityNG field value objects by default + for being able to track changes to an entity we should best add support for notifying the parent data item when a property changes. Thus, we could introduce this as part of #1869562: Avoid instantiating EntityNG field value objects by default and build upon it here then.

AttachmentSizeStatusTest resultOperations
d8_expose.patch39.85 KBIdleFAILED: [[SimpleTest]]: [MySQL] Unable to apply patch d8_expose.patch. Unable to apply patch. See the log in the details link for more information.View details | Re-test

#13

Issue tags:+sprint

As we've got a patch ready in #1869562: Avoid instantiating EntityNG field value objects by default, it's time to move on with this next.

#14

Priority:normal» major

This is a blocker for validation of REST write requests - #1696648: [META] Untie entity validation from form validation.

#15

Once #1869562: Avoid instantiating EntityNG field value objects by default gets in entities already implement the typed data interfaces but only with stubs, so we've to re-roll this and implement those stubs + add some tests for using them.

#16

Status:needs work» needs review

Have no clue yet what I'm doing, just a stupid re-roll and took care of the obvious fatal errors due to interface mismatches. Not yet sure what to do in some cases with setValue() and the $notify and some other changes.

AttachmentSizeStatusTest resultOperations
entity-typed-data-1868004-16.patch41.05 KBIdleFAILED: [[SimpleTest]]: [MySQL] 52,726 pass(es), 871 fail(s), and 939 exception(s).View details | Re-test

#17

Status:needs review» needs work

The last submitted patch, entity-typed-data-1868004-16.patch, failed testing.

#18

Status:needs work» needs review

This should at least fix most of those exceptions.

$node->langcode was sometimes array(0 = NULL), which seems to be related to #1957888: Exception when a field is empty on programmatic creation.

AttachmentSizeStatusTest resultOperations
entity-typed-data-1868004-18.patch41.78 KBIdleFAILED: [[SimpleTest]]: [MySQL] 53,102 pass(es), 708 fail(s), and 479 exception(s).View details | Re-test
entity-typed-data-1868004-18-interdiff.txt589 bytesIgnoredNoneNone

#19

Status:needs review» needs work

The last submitted patch, entity-typed-data-1868004-18.patch, failed testing.

#20

+++ b/core/lib/Drupal/Core/Entity/Field/Type/Entity.phpundefined
@@ -0,0 +1,158 @@
+ * example 'entity:user' or 'entity:node:article'. Entity types that make use of
+ * bundles cannot be instantiated without bundles either, i.e. entity:node is
+ * an abstract type as well as it requires a bundle for instantiation.

#1983548: Convert contact message entities to the new Entity Field API is an example were bundles are currently optional, would be great to get your feedback there.

+++ b/core/lib/Drupal/Core/Entity/Field/Type/EntityReference.phpundefined
@@ -0,0 +1,113 @@
+ * Contains \Drupal\Core\Entity\Field\Type\EntityWrapper.

Still says EntityWrapper

+++ b/core/lib/Drupal/Core/Entity/Field/Type/EntityReference.phpundefined
@@ -0,0 +1,113 @@
+   * Implements \Drupal\Core\TypedData\DataReferenceInterface::getTargetDefinition().

All the implements/overrides should be replaced with {@inheritdoc}

+++ b/core/lib/Drupal/Core/Entity/Field/Type/EntityReference.phpundefined
@@ -0,0 +1,113 @@
+
+  /**
+   * Overrides \Drupal\Core\TypedData\TypedData::getValue().
+   */
+  public function getValue() {
+    // If we have a valid reference, return the entity object, otherwise NULL.
+    $target = $this->getTarget();
+    return !$target->isempty() ? $target : NULL;

+++ b/core/lib/Drupal/Core/TypedData/DataReferenceBase.phpundefined
@@ -0,0 +1,80 @@
+  /**
+   * Overrides TypedData::getValue().
+   */
+  public function getValue() {
+    return $this->getTarget()->getValue();

Why is the isEmpty() check only required in EntityReference? e in isEmpty should be uppercase.

+++ b/core/lib/Drupal/Core/TypedData/DataReferenceBase.phpundefined
@@ -0,0 +1,80 @@
+    if (!isset($this->target)) {
+      $this->target = typed_data()->create($this->getTargetDefinition(), $this->getSource()->getValue());

Do we need to check if we have a source here? what happens if not?

+++ b/core/lib/Drupal/Core/TypedData/DataReferenceBase.phpundefined
@@ -0,0 +1,80 @@
+    return (string) $this->getTargetIdentifier();

Hm, what is getString() used for example? Would something like getType() . ':' . $this->getTargetIdentifier() be helpful?

+++ b/core/lib/Drupal/Core/TypedData/DataReferenceInterface.phpundefined
@@ -0,0 +1,38 @@
+   * @return mixed
+   *   The identifier of the referenced data.
+   */
+  public function getTargetIdentifier();

Wondering about the return value here. Any reason to not limit it to int|string or something like that?

+++ b/core/lib/Drupal/Core/TypedData/Type/Language.phpundefined
@@ -17,13 +17,6 @@
class Language extends TypedData {

+++ b/core/lib/Drupal/Core/TypedData/Type/LanguageReference.phpundefined
@@ -0,0 +1,41 @@
+class LanguageReference extends DataReferenceBase {

What's the difference between those two?

+++ b/core/modules/system/lib/Drupal/system/Tests/Entity/EntityFieldTest.phpundefined
@@ -469,21 +469,15 @@ public function testDataStructureInterfaces() {
+    // @todo: Make the Entity class implement the TypedDataInterface.
+    return;

Huh? Why not just let the tests fail until this is done? Also, isn't that what we are doing here?

#21

Huh? Why not just let the tests fail until this is done? Also, isn't that what we are doing here?

Ops, that should not have been part of the patch.

Wondering about the return value here. Any reason to not limit it to int|string or something like that?

Yeah, I think that's reasonable.

Do we need to check if we have a source here? what happens if not?

I think we should fix the code to throw a typed data MissingContextException in that case.

Hm, what is getString() used for example? Would something like getType() . ':' . $this->getTargetIdentifier() be helpful?

I'm not aware of any usage of it yet, but I'd use it whenever a string representation of the object is needed, e.g. when referring to it in an exception message. Given that use-case, I think the suggestion is good.

Why is the isEmpty() check only required in EntityReference? e in isEmpty should be uppercase.

It depends on the target whether it supports isEmpty, e.g. language does not as we are not implementing the ComplexDataInterface for language objects yet.

#1983548: Convert contact message entities to the new Entity Field API is an example were bundles are currently optional, would be great to get your feedback there.

Replied there!

#22

Status:needs work» needs review

ok, I worked more on this and cleaning references up. So attached patch does the following:

  • It remove the "source" reference of the reference object. Instead of letting the reference object read from the source, I've changed the approach to let the parent item take care of keeping ID and reference synched. We've to do that anyway in order to update the stale target object in the reference when the ID property changes, so I think it makes the overall flow much simpler and easier to follow that way.
  • The language reference is now quite simple and clean, it consists of three parts: the typed data object of the referenced language, the language reference object and the language reference item. As the typed data language object holds the language object or ID for us.
  • Entity references work a bit differently as there the typed data entity object is the same as the entity object and cannot handle instantiation without IDs. So for entity references the "entity_reference" object takes care of the ID vs object logic.
    Then, while $reference->getTarget() should always give you a valid typed data object, the value should be NULL when there is no data. So that's the cause of the current isEmpty() check in there, alternatively we could check for the abstract entity typed data class there. Let's revisit once we figured that part out also.
  • In order to allow the parent item to handle synching of the reference property values (ID + reference object) we need to move the notify call to the parent to be POST-change instead of PRE-change. That way we allow the parent to access the changed value.
  • I found some problems with the notify() code, i.e. set() did not correctly trigger the onChange() method of the changed complex-data. Fix was necessary for this and is included.

Attached patch also addresses the review.
Please review, in particular how the references work!

todo:
* Update onChange() docs.
* Make entity instantiation via typed data work and complete abstract typed data plugins.
* Fix tests and add tests.

AttachmentSizeStatusTest resultOperations
d8_expose.interdiff.txt30.91 KBIgnoredNoneNone
d8_expose.patch56.7 KBIdleFAILED: [[SimpleTest]]: [MySQL] 54,692 pass(es), 219 fail(s), and 55 exception(s).View details | Re-test

#23

Status:needs review» needs work

The last submitted patch, d8_expose.patch, failed testing.

#24

Status:needs work» needs review

Tried to fix some of the failures.

I'm a little bit concerned about following change I had to made:

<?php
function comment_prepare_author(Comment $comment) {
 
// The account has been pre-loaded by CommentRenderController::buildContent().
 
$account = $comment->uid->entity;
  if (empty(
$account->uid)) {
  ...
?>

After the change $comment->uid->entity always returns an entity object - even though an empty one. Thus the if clause has to be changed to check for a proper uid.
Is this the intended behaviour or should $comment->uid->entity return FALSE / NULL if no proper entity was found.

AttachmentSizeStatusTest resultOperations
d8-expose-entity-typeddata-1868004-24.patch62.38 KBIdleFAILED: [[SimpleTest]]: [MySQL] 57,120 pass(es), 29 fail(s), and 8 exception(s).View details | Re-test
interdiff-1868004-22-24-do-not-test.diff5.68 KBIgnoredNoneNone

#25

Status:needs review» needs work

The last submitted patch, d8-expose-entity-typeddata-1868004-24.patch, failed testing.

#26

Status:needs work» needs review

Next bunch of fixes:

  • Converted ImageItem
  • Fixed TypedDataManager - ProcessDecorator was lost (thanks to Berdir for pointing this out)
  • core/modules/field_sql_storage/lib/Drupal/field_sql_storage/Entity/Tables.php fixed the check if it's a valid entity reference - needs review, I've no clue if this is the right way to go.
  • LocaleContentTest added static cache reset for the language list to ensure the newly registered languages are known.
AttachmentSizeStatusTest resultOperations
d8-expose-entity-typeddata-1868004-26.patch67.4 KBIdleFAILED: [[SimpleTest]]: [MySQL] 55,810 pass(es), 2 fail(s), and 4 exception(s).View details | Re-test
interdiff-1868004-24-26-do-not-test.diff6.03 KBIgnoredNoneNone

#27

Status:needs review» needs work

The last submitted patch, d8-expose-entity-typeddata-1868004-26.patch, failed testing.

#28

Status:needs work» needs review

I've really trouble to run the affected test locally (timeout). But I guess that's another case where resetting the static cache of language_list() before using WebTestBase::drupalCreateNode() helps after adding new languages using the UI.

AttachmentSizeStatusTest resultOperations
d8-expose-entity-typeddata-1868004-28.patch67.78 KBIdlePASSED: [[SimpleTest]]: [MySQL] 55,835 pass(es).View details | Re-test
interdiff-1868004-26-28-do-not-test.diff701 bytesIgnoredNoneNone

#29

Status:needs review» needs work

The last submitted patch, d8-expose-entity-typeddata-1868004-28.patch, failed testing.

#30

Status:needs work» needs review

#28: d8-expose-entity-typeddata-1868004-28.patch queued for re-testing.

#31

Title:Expose entity types as proper TypedData API types» Improve the TypedData API usage of EntityNG

So we've got #2002134: Move TypedData metadata introspection out of the way now, which solves our metadata problem for entities without bundles (nodes without a type). So we embrace that instead of the Abstract Type notion.

Together with #2002138: Remove TypedDataInterface I guess we can re-title the issue to just improve TypedData API usage, i.e. clean entity references up and just skp the AbstractType stuff for now as #2002134: Move TypedData metadata introspection out of the way will take care of that.