Config Inspector crashes on 10.1.x + 10.2.x for `type: mapping` with `nullable: true` due to core bug

I had problem with custom migration config

e.g.

The website encountered an unexpected error. Try again later.

Symfony\Component\Validator\Exception\UnexpectedTypeException: Expected argument of type "array", "null" given in Drupal\Core\Validation\Plugin\Validation\Constraint\ValidKeysConstraintValidator->validate() (line 23 of core/lib/Drupal/Core/Validation/Plugin/Validation/Constraint/ValidKeysConstraintValidator.php).
Drupal\Core\TypedData\Validation\RecursiveContextualValidator->validateConstraints(NULL, '000000000000079a0000000000000000', Array) (Line: 154)
Drupal\Core\TypedData\Validation\RecursiveContextualValidator->validateNode(Object) (Line: 164)
Drupal\Core\TypedData\Validation\RecursiveContextualValidator->validateNode(Object, Array, 1) (Line: 106)
Drupal\Core\TypedData\Validation\RecursiveContextualValidator->validate(Object, NULL, NULL) (Line: 93)
Drupal\Core\TypedData\Validation\RecursiveValidator->validate(Object) (Line: 132)
Drupal\Core\TypedData\TypedData->validate() (Line: 352)
Drupal\config_inspector\ConfigInspectorManager->validateValues('migrate_plus.migration.rss_flickr_picture') (Line: 176

also search api solr

The website encountered an unexpected error. Try again later.

Symfony\Component\Validator\Exception\UnexpectedTypeException: Expected argument of type "array", "null" given in Drupal\Core\Validation\Plugin\Validation\Constraint\ValidKeysConstraintValidator->validate() (line 23 of core/lib/Drupal/Core/Validation/Plugin/Validation/Constraint/ValidKeysConstraintValidator.php).
Drupal\Core\TypedData\Validation\RecursiveContextualValidator->validateConstraints(NULL, '0000000000001d2a0000000000000000', Array) (Line: 154)
Drupal\Core\TypedData\Validation\RecursiveContextualValidator->validateNode(Object) (Line: 164)
Drupal\Core\TypedData\Validation\RecursiveContextualValidator->validateNode(Object, Array, 1) (Line: 106)
Drupal\Core\TypedData\Validation\RecursiveContextualValidator->validate(Object, NULL, NULL) (Line: 93)
Drupal\Core\TypedData\Validation\RecursiveValidator->validate(Object) (Line: 132)
Drupal\Core\TypedData\TypedData->validate() (Line: 352)
Drupal\config_inspector\ConfigInspectorManager->validateValues('search_api_solr.solr_cache.cache_document_default_7_0_0')

i was able to uninstall search_api_solr and config is removed. Some of these are quite big. For migrations, these need to be manually deleted set as a forced dependency to remove.

Maybe there is an issue with search_api_solr config?

Well the it seems there is an expectation here that the schema is of a certain type/format/structure.

I am thinking either that assumption is either wrong here, or, search_api has implemented their config schema incorrectly.

Looks like the error is in core's code path. Not sure how we could pre-validate the schema so core does not fail this way. It is definitely a mismatch between core's understanding of how schema should be and the schema provided by your contrib module. But this does not look like the fault of config schema, rather the schema as applied when validating a config item.

If the exception is thrown all the way forward to the config inspector level, the inspector may be able to catch it. Is that the case?

So looks like it is failing here

    $violations = $typed_config->validate();

ok, so we also have this issue which appears to have been closed now.

https://www.drupal.org/node/3360991

So looks like that could be updated now. Not sure if that will make a difference here?

uploaded pic that shows that it seems to be validating all values. Seems to be tripping up here

Difficult to see what is breaking. Here are two configs that appear to break with the same or similar error

langcode: en
status: true
dependencies:
  module:
    - search_api_solr
id: cache_document_default_7_0_0
label: 'Document Cache'
minimum_solr_version: 7.0.0
environments: {}
cache:
  name: document
  class: solr.LRUCache
  size: 512
  initialSize: 512
  autowarmCount: 0

and

id: rss_flickr_picture
label: 'StreathamLife - Flickr picture feed'
migration_group: rss
status: true
migration_tags:
  - rss feed media

source:
  plugin: url
  data_fetcher_plugin: http
  data_parser_plugin: simple_xml
  urls: 'https://api.flickr.com/services/feeds/photos_public.gne?tags=streatham&format=rss_200'

  item_selector: /rss/channel/item
  fields:
    -
      name: guid
      label: GUID
      selector: guid
    -
      name: author
      label: Author
      selector: author
    -
      name: title
      label: Title
      selector: title
    -
      name: pub_date
      label: 'Publication date'
      selector: pubDate
    -
      name: summary
      label: Summary
      selector: description
    -
      name: image
      label: Image
      selector: 'media:content/@url'
    - 
      name: height
      label: Height
      selector: 'media:content/@height'
    - 
      name: width
      label: Width
      selector: 'media:content/@width'
    -
      name: thumbnail
      label: Thumbnail
      selector: 'media:thumbnail/@url'

  ids:
    guid:
      type: string

process:
  field_title: title
  field_media_image:
    -
      plugin: skip_on_empty
      method: row
      source: image
      message: 'Field image is missing'
    -
      plugin: file_remote_image
      width: width
      height: height
      alt: title
      title: title
  created:
    plugin: format_date
    from_format: 'D, d M Y H:i:s O'
    to_format: 'U'
    source: pub_date
  uid:
    plugin: default_value
    default_value: 1
  status:
    plugin: default_value
    default_value: 1

destination:
  plugin: 'entity:media'
  default_bundle: remote_image

stacktrace for search_api_solr

Symfony\Component\Validator\Exception\UnexpectedTypeException: Expected argument of type "array", "null" given in Drupal\Core\Validation\Plugin\Validation\Constraint\ValidKeysConstraintValidator->validate() (line 23 of core/lib/Drupal/Core/Validation/Plugin/Validation/Constraint/ValidKeysConstraintValidator.php).
Drupal\Core\TypedData\Validation\RecursiveContextualValidator->validateConstraints(NULL, '00000000000018080000000000000000', Array) (Line: 154)
Drupal\Core\TypedData\Validation\RecursiveContextualValidator->validateNode(Object) (Line: 164)
Drupal\Core\TypedData\Validation\RecursiveContextualValidator->validateNode(Object, Array, 1) (Line: 106)
Drupal\Core\TypedData\Validation\RecursiveContextualValidator->validate(Object, NULL, NULL) (Line: 93)
Drupal\Core\TypedData\Validation\RecursiveValidator->validate(Object) (Line: 132)
Drupal\Core\TypedData\TypedData->validate() (Line: 352)
Drupal\config_inspector\ConfigInspectorManager->validateValues('search_api_solr.solr_cache.cache_document_default_7_0_0') (Line: 176)
Drupal\config_inspector\Controller\ConfigInspectorController->overview()
call_user_func_array(Array, Array) (Line: 123)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 627)
Drupal\Core\Render\Renderer->executeInRenderContext(Object, Object) (Line: 124)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array) (Line: 97)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 181)
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1) (Line: 76)
Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1) (Line: 58)
Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1) (Line: 48)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1) (Line: 28)
Drupal\Core\StackMiddleware\ContentLength->handle(Object, 1, 1) (Line: 32)
Drupal\big_pipe\StackMiddleware\ContentLength->handle(Object, 1, 1) (Line: 106)
Drupal\page_cache\StackMiddleware\PageCache->pass(Object, 1, 1) (Line: 85)
Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1) (Line: 48)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1) (Line: 51)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1) (Line: 36)
Drupal\Core\StackMiddleware\AjaxPageState->handle(Object, 1, 1) (Line: 51)
Drupal\Core\StackMiddleware\StackedHttpKernel->handle(Object, 1, 1) (Line: 704)
Drupal\Core\DrupalKernel->handle(Object) (Line: 19)

I run out of memory trying to step through.

https://www.drupal.org/project/search_api_solr/issues/3415861

Ok reading up on this it seems that one solution is to disable this change that exposes the validation errors. https://www.drupal.org/project/config_inspector/issues/3359418

class ValidKeysConstraintValidator extends ConstraintValidator {

  /**
   * {@inheritdoc}
   */
  public function validate(mixed $value, Constraint $constraint) {
    assert($constraint instanceof ValidKeysConstraint);

    if (!is_array($value)) {
      throw new UnexpectedTypeException($value, 'array');
    }

Perhaps there also an issue in core with ValidKeysConstraintValidator?

Certain values are expected are set as null, but ValidKeysConstraintValidator is expecting it to be an array?

I am not sure why UnexpectedTypeException is thrown here as well, as it seems reasonable that a key value can be set to null without throwing an exception?

I'm getting this for a null value of a mapping which is nullable. Maybe we need #3364109: Configuration schema & required values: add test coverage for `nullable: true` validation support in Drupal 10.2?

Applying the patch for issue #3364109 linked in the previous comment resolves this error for me.

@megan_m which patch and which version of Drupal?

Thank you @megan_m. Rerolled patch, uploaded and seems to work. i.e. no longer getting fatal error at /admin/reports/config-inspector/search_api_solr.solr_cache.cache_document_default_7_0_0/list

OK @alexpott states on slack.

I think config inspector should not have made changes that assume this code is present.

Config inspector has also been updated see

https://git.drupalcode.org/project/config_inspector/-/commit/6799e12839f...

It seems plausible that this change has broken config inspector when running in 10.2.x or later

Updating as it appears latest update to config_inspector breaks compatibility with 10.x

see

https://www.drupal.org/project/drupal/issues/3364109#comment-15504971

My guess is this can be merged into 11.x release and we can probably revert for 10.x compatibility

I made Config Inspector compatible with Drupal 10.2 in #3413217: Make tests pass on Drupal 10.2.

Investigating… 🕵️

Linking the issues that @2dareis2do has mentioned.

#16: Yes, https://git.drupalcode.org/project/config_inspector/-/commit/6799e12839f... landed for Config Inspector 2.1.8 — that's the commit for the issue I linked in my previous comment.

Well … the issue title is very broad, but AFAICT this has only ever been reported or reproduced for search_api_solr config, and even then only for migrated-from-Drupal 7 config for that module? 😅

1. Installed Drupal core 10.2.4
2. Installed latest Config Inspector, all is well.
3. Installed search_api 1.32, all is still well.
4. Installed search_api_solr tip of the 4.x branch, currently b9608ac764efa3df72a90e8102712ce3c36c3a17. BTW, this includes the following scary message: There are 154 configuration objects updated. after installation 🫣
5. Crash reproduced in Config Inspector UI at /admin/reports/config-inspector👍:
   

   The website encountered an unexpected error. Try again later.
   
   Symfony\Component\Validator\Exception\UnexpectedTypeException: Expected argument of type "array", "null" given in Drupal\Core\Validation\Plugin\Validation\Constraint\ValidKeysConstraintValidator->validate() (line 23 of core/lib/Drupal/Core/Validation/Plugin/Validation/Constraint/ValidKeysConstraintValidator.php).
   Drupal\Core\TypedData\Validation\RecursiveContextualValidator->validateConstraints(NULL, '000000000000208e0000000000000000', Array) (Line: 154)
   Drupal\Core\TypedData\Validation\RecursiveContextualValidator->validateNode(Object) (Line: 164)
   Drupal\Core\TypedData\Validation\RecursiveContextualValidator->validateNode(Object, Array, 1) (Line: 106)
   Drupal\Core\TypedData\Validation\RecursiveContextualValidator->validate(Object, NULL, NULL) (Line: 93)
   Drupal\Core\TypedData\Validation\RecursiveValidator->validate(Object) (Line: 132)
   Drupal\Core\TypedData\TypedData->validate() (Line: 352)
   Drupal\config_inspector\ConfigInspectorManager->validateValues('search_api_solr.solr_cache.cache_document_default_7_0_0') (Line: 176)
   Drupal\config_inspector\Controller\ConfigInspectorController->overview()
   call_user_func_array(Array, Array) (Line: 123)
   Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 627)
   Drupal\Core\Render\Renderer->executeInRenderContext(Object, Object) (Line: 124)
   Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array) (Line: 97)
   Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 181)
   Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1) (Line: 76)
   Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1) (Line: 58)
   Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1) (Line: 48)
   Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1) (Line: 28)
   Drupal\Core\StackMiddleware\ContentLength->handle(Object, 1, 1) (Line: 32)
   Drupal\big_pipe\StackMiddleware\ContentLength->handle(Object, 1, 1) (Line: 106)
   Drupal\page_cache\StackMiddleware\PageCache->pass(Object, 1, 1) (Line: 85)
   Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1) (Line: 48)
   Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1) (Line: 51)
   Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1) (Line: 36)
   Drupal\Core\StackMiddleware\AjaxPageState->handle(Object, 1, 1) (Line: 51)
   Drupal\Core\StackMiddleware\StackedHttpKernel->handle(Object, 1, 1) (Line: 704)
   Drupal\Core\DrupalKernel->handle(Object) (Line: 19)
   

Repeating the same steps for Drupal 11 does not trigger a crash.

Root cause analysis

Part one: the "trigger"

The "trigger" is

search_api_solr.solr_cache.*:
  type: config_entity
  label: 'Solr Cache Config'
  mapping:
…
    solr_configs:
      type: mapping
      nullable: true

in config/schema/search_api_solr.cache.schema.yml

Drupal core config schema infrastructure

Drupal core introduced the ability to mark type: sequence as nullable in #1978714: Entity reference doesn't update its field settings when referenced entity bundles are deleted. But … it implemented it at the ArrayElement level, which is the parent class of both Sequence (type: sequence) and Mapping (type: mapping).

Not a single config schema in Drupal core specifies nullable: true for a type: mapping. So when </code> was introduced in #3324150: Add validation constraints to config_entity.dependencies, that was overlooked. <code>10.1.0 was the first release in which it shipped.

Subsequently, #3364109: Configuration schema & required values: add test coverage for `nullable: true` validation support landed after 10.3.x and 11.x and it contains this:

      // If the value is NULL, then the `NotNull` constraint validator will
      // set the appropriate validation error message.
      // @see \Drupal\Core\Validation\Plugin\Validation\Constraint\NotNullConstraintValidator
      if ($value === NULL) {
        return;
      }

… which is why @2dareis2do reported at #3415861-15: ValidKeysConstraintValidator thrown by Config Inspector on Drupal 10.1.x and 10.2.x due to core bug triggered only by Config Inspector that that "fixed" the problem.

Conclusion

• This is not a bug in Config Inspector; it's just executing Drupal core's validation logic.
• Drupal core evolved its validation infrastructure in 10.1 and subsequent releases, but does not apply it to any non-core config! Precisely because we knew that not all contrib config would be compliant/correctly use config schema … and because we knew that Drupal core's config validation infrastructure was not yet in a good enough place, we'll only be able to consider it ready for prime time once at a minimum all core config is validatable.
• But in this particular case … the contrib module's "trigger" actually is valid config schema and is just triggering a bug/oversight in 10.1.x and 10.2.x!

The only possible way forward is … to make Config Inspector "simulate" the part of the fix that #3364109: Configuration schema & required values: add test coverage for `nullable: true` validation support landed, and which I quoted above. So I'll do that next 🤓 This will only be necessary for as long as this module supports Drupal <10.3.

Hah, looks like @claudiu.cristea caught the root cause 2 weeks ago in #12. I wish you had renamed the issue title, @claudiu.cristea, then your crucial single line would've been lost in this ocean of long comments that only talked about symptoms 😄 Either way, fixed now — will get a release out shortly.

thanks @Wim Leers.

fyi iirc I also had similar with migrate yaml config and not specifying enforced dependencies.

fyi iirc I also had similar with migrate yaml config and not specifying enforced dependencies.

That must've been with the migrate_plus module then, because Drupal core's migration system is completely independent of the configuration system.

Did the commit above fix that too? I'd appreciate it if you could test + confirm that :) (A quick scan of https://git.drupalcode.org/project/migrate_plus/-/tree/6.0.x/config/sche... reveals no occurrences of nullable: true, so it seems that's an unrelated problem. If you can still reproduce that problem, could you please create a new/separate issue for that? 🙏)

@Wim Leers this issue was marked fixed but the merge request is still open. Could you merge it and tag a new release?

Paging @Wim Leers - the issue branch has still not been merged (there is no MR).