In a shared hosting environment, it can be useful to run cron from a centralised location, through drush. Another example is Aegir, which runs repeated tasks on sites, as a common, privileged "aegir" user (privileged in a sense that it can reload apache, read any settings.php, have a root mysql user, etc).
Now the way drush works is that it happily loads any .drush.inc files in the context it's running in. It's going to go through a great deal of effort to find and load those files. In a shared hosting environment, this will create very serious security issues as giving developpers access to code will be equivalent to giving them access to the aegir user (another way to see this is detailed in #762138: Design security issue with developer access to sites' modules and themes).
The other attack vector is through hook_cron() and hook_init() in modules. The provision_plesk module works around this issue by patching core (ouch!) which is to say the least not ideal. A cleaner fix here would be to modify the drupal bootstrap in drush to avoid loading certain modules, although that could break some functionality.
So in short, we have three ways of hacking a Aegir install through drush, if we can upload code in sites/example.com/modules or /themes:
* create a hook_init() (that gets run during any drupal bootstrap, which happens all over the place in aegir) or;
* create a hook_cron() (which happens during cron jobs) or;
* create a .drush.inc file which gets executed during the drush bootstrap
I'm going to work on a patch to at least implement protection against the third attack vector. The first and second vector are arguably a "by design" error, and would probably be handled better by aegir switching user IDs before running cron or bootstrapping Drupal, although the latter is much harder than just running cronjobs through curl/wget... Of course, all code would be contained in an option switched off by default.
I am aware of #446736: Have drush update be told to ignore a module/theme -- simple solution, but it's blacklist-based: here, I want a whitelist of modules I can load (e.g. all modules in modules/ or sites/all/modules) and nothing else. Furthermore, that issue is about updatecode and not the general bootstrap procedure.
| Comment | File | Size | Author |
|---|---|---|---|
| #12 | 769264_drush_inc.patch | 2.28 KB | anarcat |
| #12 | evil.tgz | 1.43 KB | anarcat |
| #1 | drush_include.patch | 1.51 KB | anarcat |
Comments
Comment #1
anarcat commentedA first attempt at hitting the #3 vector.
Comment #2
anarcat commentedThere's another attack vector: .install files are loaded by drupal_get_schema(), and that function is called all over the place.
The hook_init() gets called only if we're not in MAINTENANCE_MODE
... so maybe it's something we could use to remove that attack vector... However, some module *do* need that to work properly, so a lot of stuff would break, and it doesn't resolve the hook_update() stuff.
Still digging.
Comment #3
greg.1.anderson commentedI'm in favor of giving control over drupal configuration file loading for the purpose of convenience, but I don't see how this is actually improving security. If I understand correctly, you want to prevent someone who can write to sites/somesite/drushrc.php from being able to have code executed during your call to
drush cron; however, as you point out in #0, anyone who could write to the sites folder could also define somehook_cronmethod that would also take control at the same time. In short, if you are callingdrush cronfrom user X, and user Y can write to the php files in the target Drupal site, I don't see any way to prevent them from being able to hijack user X, sincehook_cronfunctions can be inserted just about anywhere, and if drush bootstraps it it will run.Comment #4
anarcat commentedYou are confusing two attack vectors: hook_cron() is not what I'm trying to resolve with the above patch. I'm only trying to resolve the vector #3, which is the .drush.inc files that get loaded when drush itself bootstraps. Right now, I assume I will *not* be able to resolve the hook_cron() problem and I have a workaround: don't run cron through aegir, but run it through the web interface.
Which leaves us with hook_init(), hook_update(), hook_install(), hook_schema(), etc.. Those are real bitches...If we solve those, we also solve the hook_cron() problem, but it's so hard to figure out that i'm not sure how to actually do it. Maybe it's something that needs to be done outside of drush, in aegir, that would disable all non-sites/all modules before bootstrapping...
Comment #5
greg.1.anderson commentedI understand what you're saying in #4. However, there is no benefit to closing vector #3, because any user who might be able to exploit vector #3 could always exploit the hook_cron vector. It's not just that it's "hard" to fix the hook_cron vector; the simple fact of it is that any user who can write to a file that Drupal bootstraps when you run
drush croncan get code to execute as your special user. settings.php and *.module are places where you can define *_cron() and have code run. If you disable all of your modules, then you'll remove legitimate cron behavior. The only fix here is to make cron run as the individual user who owns the site in question. Legitimate functions of aegir should then be made available via php system() to a command invoked by sudo; you then allow your users to call the script by naming it explicitly in the sudoers file. Ideally, you don't pass any parameters to this script (e.g. write one script per site and hardcode the parameters inside it) to avoid extra attack vectors. Writing the commands in "C" rather than an interpreted language might add a little value too.Comment #6
anarcat commentedThere is a way to force Drupal to a limited set of modules by calling module_list() before the full bootstrap is done. The way to do this would be to do something like this:
The trick here is that module_list() can be passed a fixed_list of modules to load that persists through a static cache... Not sure it would work, but it's certainly something to investigate...
This would address attack vectors 1, 2 and 4 (init, cron and schema hooks).
Comment #7
greg.1.anderson commentedIf you whitelist some module, say 'foo' that does not implement hook_cron, then the user could implement foo_cron() in settings.php, and gain access to the system. Of course you also have to make sure that the user cannot write to any file that is whitelisted as well. Even if you took special care to make sure that foo_cron was implemented for all whitelisted modules, you would also have to make sure that none of the modules that implemented cron called any Drupal functions that invoked any hooks that the user could exploit. This is "little dutch boy" security, because the problem space can potentially change every time you add or upgrade a module. Soon you run out of fingers.
The only way to be secure is to insure that the user cannot write any php files in their Drupal folder (including settings.php), or run cron with the permissions of the user who owns the Drupal account as in #5. You might be able to make #6 work, but the amount of effort would be too large to be practical. It would be better to have your aegir-user cron task call some script that called drush cron as a less-privileged user.
That said, let me finally review your patch, which I think is fine in concept, though probably ineffective from a security standpoint.
1. You shouldn't duplicate the same block of code; call a function that is passed the candidate list as a reference and the filter list that goes with it.
2. If you're going to the trouble of creating an explicit load path setting, it would be helpful if you could add additional places to load drush config files from (that is, relocate, not just remove setting files). It would require a slightly more robust (or at least more verbose) descriptor, though, to specify both when and where to run each file (two lists rather than one?).
Comment #8
greg.1.anderson commentedI had another thought. Rather than make the code more complex as I suggested above, make it simpler. Two choices on how to do that.
Choose one:
1. Restrict by location. Use a binary flag. Default value means 'load config files from anywhere'. Restricted setting means 'do not load config files in Drupal root or Drupal site folders'.
2. Restrict by permissions. Again, use a binary flag. Default means 'load anything', restricted setting means 'only load config file if it is (a) owned by the current user, (b) not group-writable, and (c) not world-writable'.
Comment #9
anarcat commentedJust regarding the .drush.inc vector of attack here..
So I like your approach: the load path is the first thing that crossed my mind, and I implemented it as such, but when I looked at the code, I realized there was already some kind of a search path there and I was basically over-complexifying this thing.
Therefore, I think having some "security" binary flag makes a lot of sense for .drush.inc. As much as I hate the notion of "safe mode" function, I think we could have some kind of "restricted shell" notion (a bit like rbash works) implemented. It's a far-ranging thing that potentially touches a lot of areas, but I think it would be good to start *somewhere*. I would prefer option 1 from #8 (do not load Drupal modules' config files): it would make the code much simpler, much cleaner. I am not sure how we should name the option then however, as --restricted seems a bit too wide (although it could be reused in other places).
Comment #10
anarcat commentedNow regarding the Drupal bootstrap attack vector (we really have two attack vectors here: the drush and the drupal bootstrap)...
First, let me say that I assume the user doesn't have write access to settings.php and can only modify its own modules, files and themes. I know it's a bad idea to allow that to people, but "my boss wants it" (I don't really have a boss, but clients do want this functionality ;).
You make good points in #5, which I considered: running the tasks as the right user makes perfect sense, and in fact it's probably the best solution all around. However, it requires a *major* rearchitecturing of the aegir backend (which is already being re-written) in order to have that kind of privilege separation. Basically, it means splitting a lot of provision commands in two and call the second command through "sudo -u anyuser" or with the first command running as root, both of which mean giving the aegir user root, something we want to avoid. So I have put that solution aside for the moment. Suid scripts will have the same problem.
Again in #7, you seem to assume the problem only stands with hook_cron(): it doesn't. Any drush command that's going to hit DRUPAL_BOOTSTRAP_FULL (provision-verify does that, for example) will fire hook_init(). Anything that calls drupal_write_record() will call hook_schema() hooks, and the list goes on. So the problem is not only with cron, it's with any command.
I admit the solution of calling module_list() early is hackish and ugly as hell, but it *should* work if modules are not loaded at all as a result of us hacking the list. We could implement a solution similar to what you're suggesting for the drush bootstrap problem: just a flag that would tell drupal/drush to bootstrap only modules in sites/all/modules or modules/ and ignore the site-specific modules. The whitelist could be constructed out of the output of modules_list and we could refeed it a filtered list...
Besides, short of patching core, modules_list() is the only thing we got to restrict that bootstrapping process... We could try to get that patch (see this for an example) in core, but then it would hit only Drupal 8, *may* be backported to 7, and never hit 5 or 6... it's not what we want. The idea here is to restrict the loaded modules, and have a more fine-grained control over the drupal bootstrap process. I think that would only be a good thing over all.
I agree it's not a complete solution to the security problem. It's only fixing one bit. But it's the only bit I got right now, short of rewriting the Aegir dispatch, which will happen, but not in the next 6 months (by my wild guess), while a Drush patch would be much simpler to implement.
Anyways, all this is moot if hacking at modules_list() doesn't work in the first place... I have to dig into this deeper, but my bet is that the list is reset in a few places anyways, so initializing it would fail in the first place... :(
Comment #11
anarcat commentedForget that last part: even if module_list() is called with
$refresh = TRUEthe$fixed_liststill applies:Also... Just defining myevilmodule_cron() is not sufficient for the hook to be called: module_invoke_all() explicitely requests the module_list() and will only call hooks on those modules. So unless some retarded module explicitely calls module_invoke("myevilmodule", "cron"), just changing the module_list() should be sufficient.
I am therefore considering writing that module_list() hack. However, I'll probably start by writing an attack scenario so that we actually verify it works. :)
Oh, and thanks for the generous feedback and the patch review. I know this is insane stuff and I'm insane to try and fix this, so I really appreciate your openness here.
Comment #12
anarcat commentedAlright, a new patch here. This creates a "--restricted" mode (also -R) that keeps drush from loading .drush.inc files from site-specific directories. The code is slightly cleaner, but it may need improvements as it's 1AM here and I mostly need to brain dump this stuff so I don't forget and we can forward the discussion. :)
Note that I think that -R can do more stuff than just that: the next step is to look at the module_list() hijack, in a similar way.
One thing that may be missing is the passing of -R around backend_invoke() calls... not sure that would get propagated now, but it would probably need to.
I also include an "evil" module to demonstrate how to test for this attack. With that module, I can confirm that the attached patch fixes the drush bootstrap attack vector successfully.
Comment #13
greg.1.anderson commentedRegarding #10, yes I agree. If you somehow manage to get the module_list thing to work, and you insure that the user cannot write to any of the modules in the whitelist and cannot write to settings.php, then you'll be secure. However, you've removed the option to let the user install modules that need a cron hook called, so I think if it were me, I would still try to get cron to run as a more restricted user.
I'll review your latest patch later in the day.
Comment #14
anarcat commentedThe problem with the module_list() approach is that it's a all-or-nothing approach: we can't make it work for hook_cron() and not for other uses... So yes, that will have the unfortunate implication of breaking drush cron for regular user. However, cron.php will still work, with all installed modules! So that problem is really not that bad...
Thanks for the review again.
Comment #15
anarcat commentedSo I misread module_list(): the $fixed_list parameter doesn't survive $refresh = TRUE; which gets called during bootstrap_all() and module_load_all().
So in short: without patching Drupal or fiddling with the system table to disable those modules, it's not possible to only bootstrap some whitelisted modules. It's all or nothing.
I have investigated the bootstrap code and it seems like DRUSH_BOOTSTRAP_DRUPAL_DATABASE is safe, while DRUSH_BOOTSTRAP_DRUPAL_FULL and DRUSH_BOOTSTRAP_DRUPAL_LOGIN are not (ie. they load all modules). So in short, to be on the safe side, we could make the --restricted mode simply abort when trying to bootstrap to full.. Other ideas?
Comment #16
greg.1.anderson commentedWell, I would guess that you would have to bootstrap to the level where modules are loaded in order to run
drush cron, so it looks to me like you're stuck, pending Drupal changes. We can always turn a quick drush-3.1 to fix this issue if you don't come up with a solution in time for drush-3.0-stable.Comment #17
anarcat commentedRight. Basically, I consider the Drupal attack vector to be "by design". Note that the current patch doesn't try to address that issue at all, so let's please focus on getting that in (see #12 for the patch), which still covers some part of the vulnerability..
Comment #18
greg.1.anderson commentedSorry, I thought that your comments in #15 meant that you were abandoning your patch in #12. I'll go back and review it in a few minutes.
Comment #19
greg.1.anderson commentedMaybe I'm just tired, but it seems that your patch does not apply to HEAD, and I could not find the revision of command.inc that you branched from. Anyway...
I think that "--restricted" is okay, but you should not use -R.
The rest of the patch is okay; it looks like it does what it is supposed to. I don't know that what it does is valuable unless you can also solve the bootstrap issues and other security issues that you say this patch does not address.
Please make sure that your patch applies against head; making it would with patch -Np0 rather than -Np1 would also be more standard. After that, I think this patch could be committed, but I defer to Moshe on the issue of whether or not it should be committed. Wouldn't hurt, anyway.
Comment #20
anarcat commentedI may have gotten lost in my git mirror.. I'll reroll the patch based on the CVS code later tomorrow.
I think it's still valuable if commands are careful not to bootstrap drupal completely or bootstrap it only in controlled (ie. suid/sudo) environments. Without the patch, *any* drush command that runs is vulnerable to user-supplied code because of the too enthusiastic drush bootstrap.
The other thing I was saying was that the patch could also be improved to keep drush from bootstrapping drupal's module altogether...
Comment #21
greg.1.anderson commentedYeah, I think something like that is necessary before this patch becomes useful. "More secure" is like a spacesuit that is "more airtight", but still has some holes. "Secure" is what we need. I do agree that this patch is part of the solution, though.
Comment #22
greg.1.anderson commented-1 for this patch. After sleeping on it, I decided that having this in drush might give someone a false sense of security, or encourage someone to pursue an inadvisable security model. Drupal wasn't designed to partially bootstrap modules, so you'd always be in danger of your module-list hack breaking with new versions. I also don't feel good about committing this issue as a 'partial' security solution.
You should run cron on your client sites with appropriately-privileged accounts (similar privs to the web server). This will allow your clients to use modules that need cron.
For your privileged periodic actions, you should run cron as your privileged user, but you should never bootstrap a client site as this user. You can put your privileged tasks either in a module or in a drush command associated with an admin drupal site. This code should query the client sites using backend invoke.
For example:
In your alias records for the client site, set 'remote-host' to the URL of your local server (by name, not by 'localhost' or '127.0.0.1'), and 'remote-user' to your less-privileged user. Use ssh keys so that the privileged user can call the less-privileged account without needing a password. You could also request a patch to backend_invoke to make it use 'sudo' if there is a 'local-user' item in the alias record.
I won't complain if you convince Moshe to commit this patch, but I for one don't think it is a good idea and won't commit it. Please set to "won't fix" to agree, or "needs review" to appeal.
Comment #23
greg.1.anderson commentedSetting to 'postponed' pending resolution.
I do have another thought for you. Have you considered using PHP's open_basedir feature to protect your files? How this would work is you would call drush in your cron script as your privileged user, also insuring that you used a custom php.ini just for cron that set open_basedir so that only your core modules were protected. This might require that you re-organize your project layout, but that's probably doable. Presumably, Drupal would then throw a bunch of warning messages when it tried to bootstrap modules that were not in the open_basedir path, but your user code would be inaccessible in a way that was safe and un-hackable.
See related issue #509506: Drush users should turn of open_basedir in their drush-specific php.ini file.
Anyway, I'm sorry for harshing your mellow on this issue; I don't want to play security cop for your website configuration, but at the same time I think this solution is inadvisable for drush core, at least until things change with Drupal. I hope you understand & can eventually come up with a good workaround.
Comment #24
greg.1.anderson commentedOh, and in case it's not clear, if #23 does work for you, then I would very much support you fixing #509506 to get rid of the warnings (at least for drush). I think that solution would look fairly similar to what you originally proposed, and it would be secure.
Comment #25
anarcat commentedI would like to appeal this, if possible. I'm at the BOF to talk about this, but basically I think that the current patch is just the first step. As I mentionned in #15:
I think that the patch should be improved to abort the bootstrap with an error if a command attempts to go beyond BOOTSTRAP_DRUPAL_DATABASE. With that, the solution would be quite complete and drush would be effectively isolated from the users's modules.
Would that make it a sufficiently complete solution to be included in drush? Thanks for your consideration, and I'm available to discuss this at the BoF.. although there's a lot of people here. :)
Comment #26
anarcat commentedI failed to communicate at the BoF (sorry!)... Can we meetup at the codesprint tomorrow? :)
Comment #27
greg.1.anderson commentedI won't be at the code sprint tomorrow, and this issue is basically up to Moshe now, but I wanted to clarify what I made reference to at the BOF.
In any module, you may define a file MODULE.drush.load.inc that contains a function
MODULE_drush_load()will be called to determine if that module should be loaded. If it returns TRUE, then it will never be called again, but if it returns FALSE it will be called on every bootstrap phase, and asked again if now, with new information does it want to be loaded. If you made yourself a nice little commandfile that had an associated .drush.load.inc file, then the later could return TRUE if the current user is not the restricted user, and FALSE if the current user is the restricted user, and it could instead just call exit() if the bootstrap level was DRUSH_BOOTSTRAP_DRUPAL_SITE or greater (and the current user was the restricted user).From
environment.inc:That almost does it. The only other thing that will get you here is that you need to make sure that your load test is called before any 'unsafe' module's load test.
The code in find commandfiles currently looks like this:
I for one would be fine with changing the above so that 'sites/all/modules' appeared first in this list, so that you could bail out before conf_path() . '/modules' had a chance to be called.
If this works for you, talk it over with Moshe and see what he thinks.
Comment #28
moshe weitzman commentedThanks for the detail. Its helpful.
I'm not too inclined to change the order of searchpaths right before a release. I don't know all the implications. Can you not make use of the $share_path feature that you added?
Comment #29
anarcat commentedI don't think we'd need to change the load path order: if this override sits in aegir (~/.drush or $sharepath), it loads before the the drupal-specific modules.
Greg, I think I like your idea! It keeps drush clean and allows us to hook it. I think we can stick with that, but let me think about it a bit more and get the code working if you will. :)
Thanks,
Comment #30
anarcat commentedI'm digging again in this marvelous issue here. :)
By the way, this is a key issue in shared hosting security in Aegir, so I will not let go of this. I will, however, take it to third party modules if Drush thinks it's too invasive. ;) See #762138: Design security issue with developer access to sites' modules and themes for the Aegir side of things.
I think the hook_drush_load() hack is probably the best option so far. It could allow dropping modules from the DRUSH_COMMAND_FILES context externally, as the context can be edited by third party modules (now that's sick, I know).
It's the approach I'm looking at right now - of course, this doesn't fix the issue for Drupal core itself or site files, for which nothing less of "running as the right user" will solve the problem. Maybe a mix of the two approaches could work.
Thanks very much for the feedback.
Comment #31
anarcat commentedassigning the right component and owner.
Comment #32
greg.1.anderson commentedI commented in the other issue in the provision queue on why I think this issue should not be fixed in drush. My vote is to set this to 'will not fix', and insure that user privs are correct in Aegir when calling cron on a user account.
Comment #33
anarcat commentedI think I agree. Let's close this.
By the way, the "evil module" I uploaded here is now a fully fledged Drupal project :P
https://drupal.org/project/evil