Closed (fixed)
Project:
Drupal core
Version:
8.0.x-dev
Component:
routing system
Priority:
Critical
Category:
Task
Assigned:
Unassigned
Issue tags:
Reporter:
Created:
21 Mar 2014 at 03:55 UTC
Updated:
8 Nov 2015 at 21:58 UTC
Jump to comment: Most recent, Most recent file
Comments
Comment #1
tim.plunkettThis patch originally from #2124749-26: [meta] Stop using $request->attributes->get(MAGIC_KEY) as a public API
Comment #2
tim.plunkettNote that this is still just a proposal and only makes a few conversions.
Comment #3
Crell commentedWhy the extra method? Why not just map the arguments directly to the access method? That would mean the interface is less useful on it, but that would be true of arguments() as well so it's a wash. But splitting the methods makes it inconsistent with other places we use the callable resolver style.
I'm open to this change if we can make it cleanly.
Comment #4
effulgentsia commentedOne issue with using the access() method is that it also receives $route and $account, which aren't on the request attributes. I started #2223473: Simplify AccessManager methods to act on either route or request, but not both in order to remove the $route at least, since that actually is on the request attributes (just not named 'route').
Comment #5
dawehnerIt is the old disussion. Either you have an interface, in which case you need default values, or you have a separate setter or whatever.
To be honest I think the DX would be worse than before if you would have to write an extra method, which by some magic is executed before the main access method.
We could leverage the request info to get some of the parameters or just accept that the request object is part of the api. If you look at the proposed patch it will be one parameter so an extra method really feels like an overhead.
Comment #6
Crell commentedThe attributes() method is also setting stateful data, which therefore disqualifies the request object as a service. So I'm -1 to that.
So we either pass the request (as now) or declare access() to be an attributes-derived method and figure out how to document that in the interface/API spec.
Comment #7
effulgentsia commentedThat's the option I'd prefer: change it from being interface-defined to a non-interface-defined callable that receives arguments via reflection only. But we still need to figure out how to get $route and $account in there for implementations that need it. I think the use case for $route is minimal, and the few implementations that need it can use RequestInfo::getRouteObject($request) instead. Not sure how to best handle $account though. I kind of think we should go back to an _account attribute on $request, but I don't think I'm ready to reopen that battle up quite yet. In lieu of that, perhaps this means we need a separate AccessCallableResolver service rather than reusing ControllerResolver? Or are there any other ideas?
Comment #8
Crell commentedI've felt for a while that ControllerResolver needs to be split up; one service that does just the callable resolution from an arbitrary string, which ControllerResolver then leverages. Then ControllerResolver can be responsible for the argument mapping, while TitleResolver, AccessResolver, or whatever can do their own argument mapping with a different set of magic values.
That should be done at least partially upstream in Symfony; that doesn't block this issue though, as we could have an AccessResolver already. Actually we should probably continue to insist that it's an "access" method on objects of that interface and just do the attribute resolution with $account as the magic case.
Comment #9
dawehnerMoving to non-interface method would be interesting, especially if a module provides multiple access checkers. This though
would call issues with the applies method.
Just to be clear, there is an issue: #2165475: Provide a generic class resolver
If we would move that upstream we would have to override it anyway 100%, but sure let's upon up on, and don't rely on it being there.
Comment #10
effulgentsia commentedWork in progress. Provide feedback if you like, or you can wait for a more complete patch.
Comment #11
dawehnerjust a quick note
I would argue that you want to have the account required on there, as it is an edge case not take the current user into account, if you are honest.
Comment #12
catchThis will need profiling where there's lots of access checks on the same page (i..e menu links/tabs). We're caching menu link blocks by default now, so can take a bit of hit there, although primary/secondary menu variables, breadcrumbs, tabs are still not at all.
Comment #13
effulgentsia commentedFar enough along for bot to test, but does not yet implement #11.
Comment #15
effulgentsia commentedSimplified the access() signatures for the easy checkers. There are still some tricky ones left.
Comment #16
tim.plunkettWas taking a look locally and decided to help out with the unit test failures.
There are still two broken, AccessManagerTest::testCheckException() and CustomAccessCheckTest::testAccess
Comment #19
dawehnerFixed the other bits of the tests.
I really hate that you have to use both the interface as well as the interface ... Having the interface here does not help as a lot. What about making the interface optional and just used in the really edge-cases like the custom one?
I wonder whether we can let the DefaultAccessCallbackTrait implement the AccessInterface. Having to implement both is REALLY odd
Comment #20
effulgentsia commentedThis converts all the remaining access checks to get route parameters via their signature rather than via $request->attributes. $request->attributes is still being used for internal attributes that aren't route parameters, because removing those is outside the scope of this issue (see #2124749: [meta] Stop using $request->attributes->get(MAGIC_KEY) as a public API for that). And Edit module has access checks that still do funky things with $request, but I updated the @todo to link to #1837388: Provide a ParamConverter than can upcast any entity. for fixing that.
Comment #22
effulgentsia commentedWe have 7 access checks that don't depend on $account: CsrfAccessCheck, DefaultAccessCheck, ThemeAccessCheck, BookNodeIsRemovableAccessCheck, FormatDisableCheck, CronAccessCheck, UpdateManagerAccessCheck, and those all seem fairly legit to me, so I wouldn't call that an edge case.
Yeah, that would be good. The current patch does the reflection for every invocation, but we could add static caching (in the AccessArgumentsResolver service) of the reflection results per callable, so a menu with the same access checker running lots of times (e.g., a menu with a lot of node links) wouldn't need to do all that reflection for every link.
Sadly, PHP traits can't implement interfaces; they can only implement the methods, but the using class must be the one to specify whether it implements the interface or not.
I thought about doing that by making AccessManager::performCheck() check to see if the interface is implemented, and if not, then fall back to the default of
array($this->checks[$service_id], 'access'). Then we wouldn't need DefaultAccessCallbackTrait either. However, that felt wrong because it creates an implied interface (that the access() method exists and means what it means) rather than an explicit one. Also, the interface is what contains the constants (ALLOW, DENY, KILL), and it's nice for the access checkers to be able to refer to them viastatic::.Comment #23
effulgentsia commentedComment #25
Crell commentedHere's a thought: If it's too wonky to do for the checker services due to the interface, what about focusing on the custom access callbacks? They're intended for the one-off cases. They may be more common in practice so more relevant for the typical developer anyway. (Note: I haven't actually checked to see if that's the case, I'm just wondering if it is.)
Comment #26
dawehnerI agree that having the constants available is a huge win.
Couldn't we also write down the used methods into the service definition, so the interface would be just the constants?
Comment #27
effulgentsia commentedI like #26. Will work on that after getting this green.
Comment #29
effulgentsia commentedComment #30
effulgentsia commentedFor simplicity, just forcing the method name to access(). That's no more constricting than what's in HEAD. The flexibility of #26 might be a good follow up to work on, but I'd rather not hold this up on it.
I think all that's left now is profiling and optimizing the arguments resolver, unless there's additional feedback.
Comment #31
effulgentsia commentedHere's a scenario I tested for profiling: standard install profile, give anon users permission to "Administer site configuration", "Use the administration pages and help", and "Use the administration toolbar". Then, as anon, visit the home page, and see the URL for the toolbar JSONP request (
/toolbar/subtrees/INSERT_HASH_HERE/en). Thenab -c1 -n100that URL. This results in a few dozen menu links being rendered and not much else happening. On my machine with xdebug disabled:HEAD: 105ms
Patch: 108ms
With patch, AccessArgumentsResolver::getArguments() is called 48 times, and I'm guessing that that's where the bulk of the time difference is, in which case that's ~60us per invocation, and I suspect can be optimized per #22.2.
Comment #32
effulgentsia commentedBased on discussion with webchick and xjm, raising this to beta blocker, since it's a child of #2124749: [meta] Stop using $request->attributes->get(MAGIC_KEY) as a public API. We'll need to file more children of that issue, and not every one necessarily needs to block beta, but access checkers is one of the more common module-developer-facing ones.
Comment #33
dawehnerIt is great to get rid of the trait madness, though I don't really get why we don't do it right in the beginning and put the callable onto the service
Comment #34
neclimduldoes not apply after #2235347: Field instance operations are out of order
Noticed this though.
we get rid of the usage of subrequest but not its creation?
Comment #35
effulgentsia commentedJust a reroll. Doesn't yet address #33 or #34.
Comment #36
effulgentsia commentedFixes #34.
Re #33: I think doing so would require some nontrivial changes to AccessManager that would be better reviewed independently in a separate issue. If I'm wrong about that, and it's actually more straightforward than I assume, can you post the patch for it?
Comment #37
tim.plunkettI really don't feel comfortable with calling that access method without any checks.
If it's named anything else (or missing), this is the resulting error:
Recoverable fatal error: Argument 1 passed to Drupal\Core\Access\AccessArgumentsResolver::getArguments() must be callable, array given, called in /Users/tim/www/d8/core/lib/Drupal/Core/Access/AccessManager.php on line 350 and defined in Drupal\Core\Access\AccessArgumentsResolver->getArguments() (line 22 of core/lib/Drupal/Core/Access/AccessArgumentsResolver.php).That's not very helpful.
After trying to add something to the service definition to specify the method name, it completely didn't work, but I have another idea after discussing with @msonnabaum.
I have to think it through a bit more, and I also have to leave the sprint soon. I will post my thoughts and code examples next time I'm at my laptop.
Comment #38
tim.plunkettI think that in all but a very few cases, access checks as tagged services is overkill. And in all cases except maybe
_permission, it is immensely confusing. Moving theapplies_toto the service definition helped only minimally.Almost every other checker is used 4 or less times, most 1-2 times.
When using _custom_access, the DX is improved because you know *exactly* where the class is, you know and can control the method used, and it matches the pattern used for controllers, title checkers, etc.
@msonnabaum wanted me to split the value of _custom_access into an array, with a callable and whatever other values needed.
I did not do that here, I just added them to the defaults for now. This can be changed later if we want...
I've converted the two node checkers for now. My brain is a little fuzzy, so if I'm not being clear, please inform me of that :)
Comment #40
tim.plunkettMarking needs review to keep discussion moving here, I filed #2239307: Remove obsolete _node_revision_access to help fix that fail.
Comment #41
tim.plunkettFixes for node revision access.
Comment #42
dawehnerHAHA, yeah i really think it is a bad idea to switch all stuff to custom access checkers, obviously. It is good that we got get rid of that.
Even if it is hard to accept, people will write modules outside of /core which will depend on our existing stuff. We should not make their lives increadible hard for the sake of purity.
Comment #43
Crell commentedI don't follow the implications of #38. Are you suggesting we move more things to custom checkers and just make custom checkers do the reflection magic, or something else...?
Comment #44
tim.plunkettCustom checkers already support reflection magic, but they cannot get the route object in HEAD, that's the only functional change here for them.
I think that switching to them everywhere is just as reusable as a magic string, you're still going to have to copy/paste it, and at least you can more readily figure out what you're copy/pasting.
As far as improving the non-_custom_access checks, I see three options:
= NULLhack and work around the interface (matching the pain from FormInterface)@dawehner seems opposed to prefer custom access checks, but we haven't heard from @effulgentsia since I posted that, assigning to him for some feedback.
Comment #45
effulgentsia commentedStraight reroll. Will answer #44 in following comment.
Comment #46
effulgentsia commentedI support converting most of the module-specific access check services currently tagged
access_checkinto custom access checkers. For example, I see absolutely no benefit in BookNodeIsRemovableAccessCheck or ContactPageAccess being tagged services. To me, those are examples of ones that should clearly fall into the "custom" category.For the two that Tim chose to convert in #38 though: NodeAddAccessCheck and NodeRevisionAccessCheck, I'm a little less sure. I actually wonder if we want to abstract those into generic entity access checkers that other entity types could leverage for per-bundle or cross-bundle creation checking and operation-on-revision checking.
Because I think each specific access check conversion to either "make custom" or "make more generic" or "leave alone" should get its own review, I'd prefer to defer all such conversions to a separate issue from this one (not even a follow up, as that work can be done independently of this issue).
The way those conversions got lumped into this issue is the comment in #37:
Would it be sufficient for resolving this issue to make that error more helpful? For example, in AccessManager::performCheck(), throw a friendly AccessException if
!is_callable($callable)?Per #36, I still think we can have a follow up for making it possible to specify the method name as another attribute on the access_check tag, to help with the DX of whatever checks we decide to keep as tagged services. I'd just like that to be a follow up, since I think we'll need to do a little refactoring of AccessManager's addCheckService() and friends, which I don't think will be that big a deal, but just not something I want cluttering this patch. If you insist on it being done in this issue though, I'll go along with it.
Comment #48
effulgentsia commentedHappily rerolled for shortcut's conversion to PSR-4.
Comment #49
tim.plunkett#46, NodeAddAccessCheck is just a special case of EntityCreateAccessCheck, but includes the fallback for for /node/add in addition to node/add/{node_type}. That can and should be refactored anyway...
I tried doing this, I couldn't get it to work, since only ContainerBuilder has the ability to check the tags, and it is *really* messy to do this on compile-time, and within the AccessManager we only have a Container.
If addCheckService() was the only entry point, we'd be fine, but AccessManager::setChecks() runs on a completely different code path, and cannot introspect the tags of the service.
Not sure about the !is_callable, but I guess that would be a way forward.
Comment #50
effulgentsia commentedLike this? What's messy about it?
Comment #51
effulgentsia commentedComment #53
tim.plunkettMessy, because as I said in #49:
That's why your patch failed, and that's what I don't know how to fix.
Comment #55
tim.plunkettOr maybe I have no idea what is going on anymore. But at NYCCamp I wrote a patch very similar to #51, and it failed miserably.
Comment #56
effulgentsia commentedComment #57
effulgentsia commentedGiven #42 and #46, reverting the conversions of Node module access checkers to custom ones. Let's have a separate issue to refactor them if desired.
Comment #58
effulgentsia commentedReroll.
Am working on optimization per #31 next.
Comment #59
effulgentsia commentedThis is the (rather ugly) optimization I mentioned in #22.2. It doesn't seem to improve the
abnumbers of the #31 scenario by much though. So both this and #58 need to be profiled for real, to see what's causing the difference.Comment #60
effulgentsia commentedSo I tried xhprof'ing on my machine, and it appeared that #59's approach of trying to statically cache and reuse reflection results didn't improve anything because reflection isn't what's slow. Actually, it appeared that parameter bag's get(), has(), and array_key_exists() were some of the slowest contributors, so here's a different micro-optimization. It discards #59, so interdiff is relative to #58.
However, I'm not posting xhprof results because I'm having issues with my machine setup, where I'm using PHP 5.5, and I can't get xhprof to work without segfaulting unless I also enable the xdebug.so extension, but doing that makes wall time numbers meaningless.
Also, when I test locally, I get as much noise between individual runs on the same codebase as the signal between HEAD and patch.
Hoping someone else can profile and post cleaner results. The goal is to profile a request that checks access on many menu links (#31 is one way to do that, and doesn't need to be done as anonymous, that was just for the benefit of running
ab). The biggest culprit should be the inclusive time of AccessArgumentsResolver::getArguments(). Other differences are likely to be noise.Comment #61
star-szrI did some quick profiling here using xhprof-kit and the scenario outlined in #31. This is roughly an average result. I had wt results as low as +1.3% and as high as +2.3%, but the function calls were always consistent.
http://www.lionsad.de/xhprof-kit/xhprof/xhprof_html/?run1=5361b94b74ee9&...
http://www.lionsad.de/xhprof-kit/xhprof/xhprof_html/?run1=5361b94b74ee9&...
(edited to add missing link for the actual comparison).
Comment #62
effulgentsia commentedThanks, Cottser! That does look like a clean result.
I think the best way to analyze this is to sort by function calls diff: http://www.lionsad.de/xhprof-kit/xhprof/xhprof_html/?sort=ct&run1=5361b9....
Adding up the exclusive wall diff of the top 13 lines (calls diff >= 47) is 3.3ms of the 3.7ms total. Remaining differences include things like the load of AccessArgumentsResolver.php (0.145ms), plus noise (note that #61's comparison of 2 different runs of HEAD showed 0.2ms of noise).
So, if we ignore noise and the loading of a PHP file, and just look at what would scale linearly with number of access checks performed, 3.3ms is 1.6% of the total time of a request whose only job is to render 40ish menu links (the toolbar JSONP request). @catch: is that acceptable?
This patch hasn't been RTBC'd yet, but assigning to catch for feedback on performance cost.
Comment #63
effulgentsia commentedOne possibility, if we're concerned about the performance impact, is that we can create a new interface (e.g.,
UnreflectedAccessInterface) with an access() method that's hardcoded to the same signature as in HEAD ($route, $request, $account), and we make AccessManager::performCheck() check if the service implements that interface first, and only if it doesn't, do the whole AccessArgumentsResolver lookup.In core, we could then at a minimum make PermissionAccessCheck and EntityAccessCheck use that interface, and I think that would take care of the vast majority of menu links, so that we'd only have a relatively small number of AccessArgumentsResolver lookups happening for any given request.
But then, each contrib access checker would need to make a decision as to whether to implement UnreflectedAccessInterface or allow reflection to be used. Is the optimization worth adding the extra decision and code branch?
Alternatively, we could do what tim.plunkett suggested, and only implement reflection for custom access checks, and move many of our core ones to that. That would remove a decision point for contrib: they either implement a 'access_check' service or a custom callback, just as in HEAD, and based on that one decision, they know whether to implement an interface-defined access() signature or a reflection-resolved one. However, then we'd be coupling the DX of the signature to whatever other factors are involved in deciding between tagged service and custom callback, and is that making things easier or harder on contrib?
Note that the entire purpose of this issue's attempt to support reflection-based signature is to eliminate
$request->attributes->get('foo')as a common thing for contrib to deal with. If something like$route_match->getArgument('foo')(from #2238217: Introduce a RouteMatch class) within access() methods would be seen as acceptable DX, we could alternatively "won't fix" this.Comment #64
catchThis sounds like something we could do as a follow-up including after release?
Assuming the performance regression here is correct, I think that's acceptable given we have critical issues to cache all menu output by default - including the access checks.
That won't apply to local tasks etc. but there's less of them .
Comment #65
effulgentsia commentedYay! Ok, anything left to do on the patch then before RTBC?
Comment #66
effulgentsia commentedRemoved even the commented out access() method from the interface, and inlined access() docs into each using class. RTBC, anyone?
Comment #67
dawehnerThis opens up the problem as we basically solved in #1906810: Require type hints for automatic entity upcasting already.
Comment #68
pwolanin commentedAgreed, this seems wrong - if you don't type hint, you should only get the raw value:
Also, an updated summary would help to review.
Comment #69
effulgentsia commentedFixes #67/#68. Working on updating the summary.
Comment #70
effulgentsia commentedComment #71
effulgentsia commentedUpdated the summary. And here's a small improvement I discovered while writing it.
Comment #72
effulgentsia commentedComment #75
effulgentsia commentedSigh. Found a landmine. Sidestepping it here, but will open a separate issue if there isn't one already.
Comment #76
dawehnerDo we have a proper way to do that? Everybody uses their own way to document it.
Comment #77
xjmAssigning to catch to review the profiling.
Comment #78
sunNo need to invent something. @return documents the data type, not the origin. The description can point to the origin.
Comment #79
catchI don't have much to add on the profiling beyond #64 - it's a noticeable regression but we have caching issues in progress, and the interface idea if the uncached performance turns out to be a problem.
Let's fix those docs. I haven't done a final review of the patch yet, but that doesn't have to be me either.
Comment #80
xjmI'll clean up the docs; I think @sun's suggestion is spot on.
Comment #81
xjmDid a quick read for any other docs gate issues or other docs cleanups while I'm at it; few notes...
blank line
Issue for this?
Missing descriptions
Needs a one-line summary about what it actually does. Also missing parameter documentation.
s/id/ID/g
This is documented differently from the other one.
(and elsewhere) This is sort of a vague description.
These are the ones to change. Wow but we repeat that a lot... feels like a code smell... I'll ask effulgentsia to clarify why this is.
So we're now using the interface as a container for three constants. Is there an issue to explore this?
Defines. :)
Comment #82
effulgentsia commentedRe #81.2: #2265939: [sechole] AccessManager::checkNamedRoute() leaks attributes from current request. I still need to file issues for the other two @todos that are missing issue links.
Comment #83
xjmHere we go. I addressed everything in #81. Couple unrelated things I noted during the process.
FormModeAccessCheckandViewModeAccessCheckhave incorrect class docblocks. Will file a followup issue for that.\Drupal\quickedit\Access\EditEntityAccessCheckandEditEntityFieldAccessCheckshould probably be renamed to indicate they're specifically for quickedit now that the module has been renamed (as maybe should their relevant methods possibly other classes in the namespace); I was confused at first since there's a different checker for entity CRUD.I also noticed that there are some places where we're not properly documenting optional parameters (comparing the @param docs to the signatures). The docblock should indicate when the parameter is optional and what happens in the default case when it is not provided. E.g.:
Should $node really be optional here, and is the revision really a string? Also, the parameters should be marked as optional in the docblock if they are.
And the rest:
Comment #84
effulgentsia commentedFiled the remaining todos that were missing issue links: #2266817: Deprecate empty AccessInterface and remove usages and #2266809: Make QuickEditEntityFieldAccessCheck::access() use the $account that's passed in, and fixed #83, but did not file the two issues mentioned at the top of #83.
Comment #85
xjmThe order of the arguments switched around here; does that not matter/impact calling code because of upcasting? (I can see how this is a more logical optional order based on the docs.)
Comment #86
xjmFiled:
Comment #87
xjmCircular issue references are circular.
Comment #88
xjmTalked to @effulgentsia about #85 and the answer is "yes that's the point". :) So AFAIK all the followups and docs improvements are taken care of now.
Comment #89
tim.plunkettThis has long diverged from my work, so I feel okay RTBCing.
My only comment is that I wish we had unit tests for AccessArgumentsResolver, but I think that can be a follow-up.
Thanks for persisting on this @effulgentsia, I like where we ended up.
Comment #90
xjmYay, thanks @tim.plunkett!
Filed #2268537: Add unit tests for AccessArgumentsResolver since as @tim.plunkett points out there's already integration test coverage, but unit tests for the new class would be good.
Comment #91
tim.plunkettActually, now that I work on these tests, I think I found something odd. Give me a bit to work through this. Also, I felt bad about trying to circumvent the testing gate.
Comment #92
tim.plunkettOkay, here is a patch with the test and my fix, a patch without the fix, and the interdiff of the test and fix.
If these changes are fine with @effulgentsia, I think he can re-RTBC.
Comment #94
effulgentsia commentedThanks for the tests. The 2 tiny non-test changes look great. For the tests:
s/default/provided value/?
I don't think it's clear what the purpose of these two tests are, especially the last one. Are we trying to test a generic parameter name and the ability for it to be in ->attributes as NULL, even if the callable typehints? Or are we specifically testing the functionality of 'route' as a named attribute overriding $this->route? Related, do we also want a test for in that latter case, to still be able to receive $this->route into the callable via a
Route $pick_any_name_you_likeparameter?Comment #95
tim.plunketttestGetArgumentRouteNoTypehintAndValue() is to prove that just naming something
$routewill not cause any conflict with the Route object.testGetArgumentBypassUpcasting() is to cover this case:
I added some comments.
Also, I added testGetArgumentRouteCustomName() to test
Route $custom_name, as suggested.Comment #96
effulgentsia commentedThanks.
Comment #97
dries commentedThis look like a good DX win and it helps us make progress on #2124749: [meta] Stop using $request->attributes->get(MAGIC_KEY) as a public API. Given that catch feels comfortable with the performance issue, I decided to commit it to 8.x. Thanks everyone!
Comment #100
dawehnerWe certainly should improve our horrible documentation now, see #2611272: Add documentation to \Drupal\Core\Routing\Access\AccessInterface how to implement it