Problem/Motivation
Follow-up of #2284111: Exceptions caught and printed from index.php should not return a 200 status code.
basic_auth loads a full user entity and sets that as current user. That results in a circular recursion due to LocaleLookup trying to get the current user roles and t() calls while fetching the field definitions to load them.
This is what happens:
-> t()
-> LocaleTranslation::getStringTranslation
-> new LocaleLookup ...
-> getRoles()
-> UserRole::baseFieldDefinitions
-> t()
-> new LocaleLookup
->getRoles()
->...
Proposed resolution
Three layers of defense:
1. @dawehner came up with a way to lazy-load the cache ID within lazyLoadCache(). Because that sets the already-loaded flag before actually trying to load, any following calls will stop there. They will then result in cache miss lookups and resulting cache writes, but at least we can prevent a fatal error/loop.
2. Then, we add TranslationWrapper's to the few known current t() calls during early bootstrap. That prevents those t() calls for core/current HEAD.
3. To prevent the cache writes, we reset the persist list after building the cache ID. Then the worst thing that can happen if someone adds more super-early t() calls is a few uncached translation lookup queries.
With that, I feel comfortable with this as a solution to this problem.
Remaining tasks
Can we test this somehow? (the cache misses)
There is also an issue that wants to look into loading the user entity for the session backend too, to get rid of the hardcoded queries there, now that we have the entity cache. This is a hard blocker for trying that. We might encounter more t() calls there, we can fix them there.
Follow-up task in #1751274: Do not query user_roles directly, not to be done here.
User interface changes
API changes
Comment | File | Size | Author |
---|---|---|---|
#35 | 2288123-35-interdiff.txt | 1.45 KB | Berdir |
#35 | 2288123-35.patch | 12.34 KB | Berdir |
#26 | 2288123-26-interdiff.txt | 785 bytes | Berdir |
#26 | 2288123-26.patch | 12.01 KB | Berdir |
#20 | 2288123-20-interdiff.txt | 6.44 KB | Berdir |
Comments
Comment #1
BerdirComment #3
bbinkovitz CreditAttribution: bbinkovitz commentedI'm trying to reproduce this bug so I can test the patch but I'm not getting it. I've installed from the latest Drupal 8 dev with the language set to German and then I followed the instructions on https://www.drupal.org/documentation/modules/basic_auth which culminate with "try various routes like router_test/test11". When I try that route, I don't get any indication of a circular recursion. I just get an "Access Denied" message, which I believe is what one would expect?
Comment #5
BerdirI can certainly still reproduce this when installing basic auth, making de the default language and then going to a rest route.
Adding random new TranslationWrapper()'s instead of t()'s seems to be a start in fixing this, but ugly, as mentioned.
Comment #9
BerdirMaking sure that DataDefinition and EntityType return strings for their labels and descriptions.
The test-only test did fail as expected (out of memory due to endless loop)
Comment #10
dawehner@berdir
It seems not so nice if you have to create TranslationWrappers for your own all over the place. One simple way could be to introduce \Drupal::t() which creates these objects?
Do we write code in german now ;) Let's lowercase "New".
Comment #11
Berdir"seems not so nice". I'd say it's ugly as **** :)
I was just trying to check if this is enough or if we need something more. I don't know what a good API would look like.
If we wouldn't have the ever-recurring problem of having to be able to identify translatable strings, we could just say that DataDefinition will take care of translating the title/description and you can simply pass in the string.
Comment #12
dawehnerAnother idea I had was to pass along some kind of object to
baseFieldDefinition
Comment #13
Gábor HojtsyWas this happening due to the nonsensical logic in #2240459: Applying default language fallback rules to interface translation has impressively bad results that was fixed since then? Or still happening? Will send the test only patch for retest.
Comment #15
Gábor HojtsyBerdir says still a problem.
Needs to be updated in the summary.
Comment #16
dawehnerOkay, so as always, it is a bad idea to execute any code beside
$this->foo = $foo;
in the constructor.What about deferring the call a bit. I haven't actually tried whether the approach works, but could be interesting.
Comment #18
BerdirThat will not change anything. I thought about this before and I think there is a cache collector where we do this, although I would have to check which one it is. However, this isn't something that we happen to just initialize early... We actually *use* it early, and while I agree that it would be cleaner to not do this in __construct(), it will not change anything for this issue.
Combining your patch with my test should prove that.
Comment #19
dawehnerSo that approach is at least green.
So the thing is that now the t() call goes like this:
So as you see the deferring helps for the next call.
In HEAD it works like the following:
Comment #20
BerdirWe discussed this, and the reason the patch works is that lazyLoadCache() sets the flag about being loaded before loading before we actually load it. Then the next call assumes that the cache was loaded, looks if it is in the cache, and because it is (obviously) not. That calls resolveCacheMiss(), which loads from the database.
That's not too bad, but the real problem is that also marks that string to be written into the cache. So on every request, we'd write the cache again, which also involves a lock. That's not pretty. But at least this prevents the fatal error/loop.
However, we debugged this a bit and we only found two/three calls that happen that early. That's those default languages in LanguageManager that currently trigger the initial t() call and that's StringItem::propertyDefinitions(), which is called from FieldItemBase::__construct() for the roles field item. So we can add TranslationWrappers for those, so we prevent the cache write problem on HEAD with basic authentication.
Finally, to prevent the cache write problem, we can clear the persist list after getting the cache ID. Then the worst thing that can happen if someone adds more super-early t() calls is a few uncached translation lookup queries.
Comment #21
BerdirComment #22
dawehnerGreat!
Should we explicit document these cases and why we do this? PS: Does potx has support for this already?
Comment #24
BerdirOuch, that looks like we're somehow writing those language types out into config and fail to deal with TranslationWrapper there?
@dawehner: Honestly, I just updated those to be consistent with everything else in entity annotations. Right now, they would end up being cached translated as removed the language from the cache key. They're not strictly related to this issue, should work without that.
Comment #25
Gábor Hojtsy@dawehner: yes, potx supports TranslationWrapper.
@berdir: we have language.entity.und and language.entity.zxx already pre-shipped with the language module, we don't write them from the code modified here. Or you referred to something else that is written out into config?
Comment #26
BerdirWeird, it's just that test that's trying to write those languages, so I used installConfig() there. I haven't been able to figure out why exactly we are doing it, all tests pass fine without that logic. But I don't want to change something there, so switching to the installConfig() call.
Comment #27
dawehnerSo this would also include language.(negotation, mappings and types).yml
do we really want those as well? I guess it is not problematic at all.
Comment #28
BerdirI don't think that is a problem. In a real installation, that default config exists too.
Comment #29
dawehnerMe neither now :)
Comment #30
Gábor HojtsyThe fix looks as good as it can get I guess :) This is a tricky circular dependency.
Is there some hard rule to where should we use TranslationWrapper? I mean people may be puzzled that one field type uses it while others don't. IMHO we should explain it with comments or do some other kind of long term knowledge preservation.
////
Also do we want to pursue any of the remaining tasks in this issue? I think not, but then would be good to say it there. That section is to be used for things still outstanding for commit on an issue.
Comment #31
BerdirThanks, yes adding comments makes sense, I will add them to Language Manager and StringItem. The group label changes are IMHO just a logical thing to do as all other annotation labels are tranlation wrappers too. But I guess I could add that as a comment.
There is not really a hard rule, that was what I was worried about it my patches. That adding another call would make it fail again. We're protected against that now, as explained in the issue summary, so the rule is basically "We use TranslationWrapper when we know that something is called early".
The first task I'm not sure if possible, I don't know how. We would have to do very dark things (like manually messing with the cached data).
Comment #32
andypostIt would be great to mention in some update docs/CRs that
TranslationWrapper
is replacement forst()
;)Comment #33
Gábor Hojtsy@andypost: No, not at all a replacement. st() did translation right away using an array in memory reading from a specific file in the filesystem, while TranslationWrapper() does not do translation until cast to a string sometime later and will use all the regular services that t() would use at the time, same backend. So the behaviours have nothing to do with each other.
Comment #34
Gábor Hojtsy@Berdir: looks like this is down to the comment cleanups discussed.
@all: special thanks for maintaining a stellar issue summary.
Comment #35
BerdirThose comments OK?
Comment #36
Gábor HojtsyLooks fine by me. While there is some black magic to which ones to apply the TranslationWrapper treatment, there does not seem to be a possibility to do this cleaner.
Comment #37
catchLooks good to me as well. TranslationWrapper is the only way we have at the moment to resolve these circular dependencies, so as long as we explain why we're using it I'm happy.
Comment #39
Gábor HojtsyYay, thanks.
Comment #41
BerdirSee you in #2345611: Load user entity in Cookie AuthenticationProvider instead of using manual queries :)