Problem/Motivation
Working on #2531564: Fix leaky and brittle container serialization solution it was found that some classes using settings as injected service which leads to issues when this objects are serialized
Steps to reproduce
\Drupal\Core\Cache\CacheFactory::$settings, \Drupal\Core\EventSubscriber\ExcludedModulesEventSubscriber::$settings and more
Proposed resolution
- stop using Settings object as service and declare public API to use it
- ensure runtime still fail when settings (like database) are serialized
Remaining tasks
- review patch with BC deprecations
- commit clean-up to 9.5 and 10.*
- deprecate in 10.1.x branch via #3307190: [PP-1] Deprecate settings service
User interface changes
no
API changes
Core's classes no longer accept settings service
Data model changes
no
Release notes snippet
no
Comments
Comment #2
longwaveI do wonder many how much of settings.php should really be container parameters, if we are to follow Symfony more closely. We have too many places to store configuration :)
e.g.
This is only used in \Drupal\Core\Http\ClientFactory so could just be injected from the container.
Similarly the reverse proxy settings could be injected into the relevant services as container parameters,
omit_vary_cookieis only used in one event subscriber,cache_ttl_4xxcould even be config for the page_cache module,file_public_base_urlcould be injected into the PublicStream wrapper, etc.Comment #3
longwaveDuplicate of (or closely related to) #2443351: Ensure that settings can't be serialized by not injecting them / replace stuff with container parameters.
See also #3061535: Add settings from settings.php to the container as parameters
Comment #4
andypostI think it should be a plan, I bet it will need for child issues to fix cache backend factories and other places, and not sure it needs more tests to catch it
Comment #5
bradjones1IMO this is a case of older-school Drupalism that we've hung on to, far too long. In my custom (and contrib, when I'm writing greenfield) code, it's all container parameters all the time. Paired with a module like
services_env_parameter, this works really well. The challenge is that Drupal has its own version of the Yaml parser (which I've filed some issues relating to, elsewhere) so setting parameters in a 12-factor way akin to what Symfony core provides is a blocker to sunsetting$settings. But it's for sure a noble cause.Comment #6
andypostwondered if env parameters can take over serialized into container defaults
I think #3061535: Add settings from settings.php to the container as parameters may land in 10.1.x only as a feature
Comment #7
andypostComment #8
andypostComment #9
andypostThat's all usage I found
Comment #10
andypostbit more clean-up
Comment #11
catchWe'd normally ask for a 'best effort' bc layer for changes like this, but this is a tricky one.
Would have to:
- remove the type hint
- check if it's a Settings object
- get $default_bin_backends using func_get_args() if not.
Maybe we can grep contrib for CacheFactory to see if anything's inheriting from it, and if not just leave it?
Some more similar ones like this where it's not the last argument.
Ones like this on the other hand we can just ignore entirely since it's the last argument.
Comment #13
andypostRe #11
1. http://codcontrib.hank.vps-private.net/search?text=CacheFactory&filename= there's some wrappers in contrib (devel webprofiler for ex)
2. I gonna make patch green, then add BC
Comment #14
andypostFix few tests, mostly all failed test are because of FileSystem service failed to be created
Comment #15
andypostAdded BC for FileSystem as I can't find why it's instantiated with wrong argument in installer
Comment #18
andypostfixed remaining tests
now needs work for BC and fix for
FileSystemserviceComment #19
andypostfixed last place, if get green I will work on BC
Comment #20
andypostfix cs
Comment #21
andypostThis changes does not need BC so could be splitted off from that
Comment #22
andypostFiled CR to deprecate
settingsservice, hope it possible for 9.5 https://www.drupal.org/node/3306646Comment #23
andypostbtw in 9.5 slightly different because of #1538118: Update status does not verify the identity or authenticity of the release history URL
Comment #24
andypostpatch for 9.5
needs
- BC for constructors
- deprecation test or separate issue for deprecation settings service
- split of changes without BC (OTOH it may need deprecation from settings service)
Comment #25
andypostOne more fix found
Comment #28
andyposta couple of more usages found
Comment #30
andypostfixed patch for 10.0
Comment #31
andypostHere's BC added for 9.5
Now it needs postponed follow-up to deprecate according to
Comment #32
andypostComment #33
andypostfixed version string, now need to remove deprecation (left to make sure tests passed
Comment #34
andypostFiled follow-up #3307190: [PP-1] Deprecate settings service
Here's interdiff and patch for 10.0.x
Comment #35
andypostFix CS
Comment #36
andypostUpdate title as IS
Comment #37
catch(reviewing the 10.x patch).
The bc layers all look good to me.
Main question is this mismatch in the deprecation versions:
This says deprecated in drupal:9.5.0 and removed in drupal:10.0.0
But this says deprecated in 9.5.0 and removed in 11.0.0.
We broadly have three options I think:
1. Deprecate in 9.5.0 and removed in 10.0.0 - disadvantage is it will potentially break modules that are already marked compatible with 10.0.x if they inject the service themselves or modify one of the services if we remove the constructor bc in 10.x
2. Deprecate in 10.1.x and remove in 11.0.0 (but with all the other service/constructor changes committed to 9.5.x) - less likely to break modules but also means that modules could end up with issues on PHP 8.2 and without any useful notice from us what the problem is.
3. Deprecate in 9.5.x for removal in 11.x - IMO this is the least worst option. Modules that are already compatible with 10.0.x won't break, they'll just get a new deprecation, everyone gets as much warning as possible about the change to help with PHP 8.2 compatibility. Also means least churn between 9.5 and 10.x just before beta.
Will ping other committers to get additional opinions too.
Comment #38
andypostI think the best is deprecate in 9.5 for 11.0 because it will give more time to clean-up and BC will work
Comment #39
andypostfix typo
Comment #40
andypostmaybe message could be improved, we have no standard about constructor messages
Comment #41
andypostbetter wording
Comment #42
catchLatest wording looks good to me for the constructor deprecations.
Comment #43
andypostdelComment #44
andypostwhat's left here for RTBC?
Comment #45
kristen polTagging
Comment #46
longwave@andypost what are the actual issues with having this as a service? The IS doesn't explain, and the parent issue just says "after PM discussion with chx" but doesn't give an explanation either that I can see.
How does the BC promise work if we are removing arguments? If someone has subclassed one of these services and swapped it in, and then relies on
@settingsbeing passed in, or$this->settingsbeing set correctly, what happens?Also if we are going to convert settings to container parameters, I wonder if we should be removing the constructor arguments here, as maybe we could switch
@settingsfor%some_container_parameter%? In that case maybe we should pass NULL in instead for now?Comment #47
longwaveThis variable should be snake_case (or just inline it into the if statement).
This shouldn't be in the patch.
Comment #48
andypost@longwave I only recall test of
ConfigImportFormwhich said serialisation of settings (in ignore issue)Otoh maybe it's because of incomplete serviceId issue fix
At lest 3 tests been fixed
Comment #49
andypostfixed #47
Still not sure passing NULL is good idea, anyway primary focus is on hard blocker #2531564: Fix leaky and brittle container serialization solution
Comment #50
andypost@longwave one more test to catch it is
\Drupal\FunctionalTests\Bootstrap\UncaughtExceptionTest::testMissingDependencyCustomErrorHandlerComment #51
catchThis looks like it also needs the func_get_args() treatment.
Apart from that, all looks great.
Comment #52
alexpottIf we did #3061535: Add settings from settings.php to the container as parameters first then we'd have way less constructor change. Because we could then use this issue to properly inject the container parameters.
Comment #53
andypostFixed #51
Comment #54
catchI'm not sure about #3061535: Add settings from settings.php to the container as parameters at least in its current state:
- has to special case hash_salt to avoid it getting written to the database, but there can be similar custom/contrib settings you might also not want to get into the db.
- it's not impossible that people put downright strange things into $settings that would break.
Comment #55
longwaveI think we have two possible approaches here:
Not sure which is the best route.
Comment #56
longwaveGiven that Symfony has explicit support for injecting environment vars as container parameters, and also when dumping a container it can dump the original reference to the env var name (instead of the env var value), we could leverage or duplicate this technique for injecting settings variables?
See https://github.com/symfony/dependency-injection/blob/c1831682d0a02ee2ce0...
Comment #57
andypost+1 to convert partially settings to parameter because some contrib (redis for ex) still require additional settings
I think we should get rid of settings service anyway
Re #56 env vars should never contain secrets that's why we use settings.php (hash salt, file/config path)
Also db-connection settings could be tricky because of BC for prefixes
Comment #58
andypostThere's existing child issue for partial conversion #2443351: Ensure that settings can't be serialized by not injecting them / replace stuff with container parameters.
Comment #59
wim leersI just learned that the Drupal 10 beta will likely be held up over this. So I'm trying to find my bearings.
I think it'd be helpful if somebody who's familiar with this area could suggest concrete next steps. I'm also fine with @catch and @alexpott as core committers simply deciding a direction given the very narrow timeframe before Drupal 10 beta.
Comment #60
andypostI'm coming to DrupalConEur let's get consensus here)
Comment #61
longwaveDiscussed this with @alexpott at Drupalcon. We talked about a number of issues:
@andypost we will be in the contribution room for most of today if you want to discuss this!
Comment #62
longwave#61.1 has some related issues already:
Comment #63
andypostI bet it 10.1 material as task
Comment #64
andypostI don't think we should block the issue on environment variables as this approach considered as insecure in cloud based infra because it exposes secrets to all processes in containers.
This issue is just a clean-up and complimentary to #2443351: Ensure that settings can't be serialized by not injecting them / replace stuff with container parameters.
Moreover
Settingscontain secrets and errors in code may expose its properties in backtraces, so security++Comment #65
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".
Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.