Optimize settings.php discovery

A little while back I was discussing conf_path() with effulgentsia and pwolanin, and they noted that a sites.php definition might allow us to skip some of the searching for settings.php during bootstrap.

If you watched Rasmus' keynote speech at Drupalcon Copenhagen, you may remember the section where he asked for "some kind of deploy mechanism" to save hunting around for settings.php on every request. Post deploy scripts are hard to do with Drupal since we let people add multisites at any time they like, but sites.php has the potential to be such a thing.

Looked at a strace (on my localhost) with straight core, just using sites.default.php as you'd get with a normal installation via the UI or drush. Note I left APC, xdebug and xhprof all enabled doing the straces, I think the mmap here are APC on a cache miss, but this doesn't really affect the rest of the output significantly:

6555  access("/home/catch/www/7/sites/sites.php", F_OK) = -1 ENOENT (No such file or directory)
6555  access("/home/catch/www/7/sites/9090.d7.7/settings.php", F_OK) = -1 ENOENT (No such file or directory)
6555  access("/home/catch/www/7/sites/d7.7/settings.php", F_OK) = -1 ENOENT (No such file or directory)
6555  access("/home/catch/www/7/sites/7/settings.php", F_OK) = -1 ENOENT (No such file or directory)
6555  access("/home/catch/www/7/sites/default/settings.php", F_OK) = 0
6555  lstat("/home/catch/www/7/sites/default/settings.php", {st_mode=S_IFREG|0555, st_size=15272, ...}) = 0
6555  lstat("/home/catch/www/7/sites/default", {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0
6555  lstat("/home/catch/www/7/sites", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
6555  open("/home/catch/www/7/sites/default/settings.php", O_RDONLY) = 9
6555  fstat(9, {st_mode=S_IFREG|0555, st_size=15272, ...}) = 0
6555  fstat(9, {st_mode=S_IFREG|0555, st_size=15272, ...}) = 0
6555  fstat(9, {st_mode=S_IFREG|0555, st_size=15272, ...}) = 0
6555  fstat(9, {st_mode=S_IFREG|0555, st_size=15272, ...}) = 0
6555  mmap(NULL, 15272, PROT_READ, MAP_SHARED, 9, 0) = 0x7fb1aa209000
6555  stat("/home/catch/www/7/sites/default/settings.php", {st_mode=S_IFREG|0555, st_size=15272, ...}) = 0
6555  munmap(0x7fb1aa209000, 15272)     = 0
6555  close(9)                          = 0

After this, I created a sites/sites.php file with the following line:

$sites['9090.d7.7'] = 'default';

Stracing again the output looks like this:

6897  access("/home/catch/www/7/sites/sites.php", F_OK) = 0
6897  stat("/home/catch/www/7/sites/sites.php", {st_mode=S_IFREG|0644, st_size=1782, ...}) = 0
6897  lstat("/home/catch/www/7/sites/sites.php", {st_mode=S_IFREG|0644, st_size=1782, ...}) = 0
6897  lstat("/home/catch/www/7/sites", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
6897  open("/home/catch/www/7/sites/sites.php", O_RDONLY) = 9
6897  fstat(9, {st_mode=S_IFREG|0644, st_size=1782, ...}) = 0
6897  fstat(9, {st_mode=S_IFREG|0644, st_size=1782, ...}) = 0
6897  fstat(9, {st_mode=S_IFREG|0644, st_size=1782, ...}) = 0
6897  fstat(9, {st_mode=S_IFREG|0644, st_size=1782, ...}) = 0
6897  mmap(NULL, 1782, PROT_READ, MAP_SHARED, 9, 0) = 0x7fa5a9c79000
6897  munmap(0x7fa5a9c79000, 1782)      = 0
6897  close(9)                          = 0
6897  access("/home/catch/www/7/sites/default", F_OK) = 0
6897  access("/home/catch/www/7/sites/default/settings.php", F_OK) = 0
6897  access("/home/catch/www/7/sites/default/settings.php", F_OK) = 0
6897  lstat("/home/catch/www/7/sites/default/settings.php", {st_mode=S_IFREG|0555, st_size=15272, ...}) = 0
6897  lstat("/home/catch/www/7/sites/default", {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0
6897  open("/home/catch/www/7/sites/default/settings.php", O_RDONLY) = 9
6897  fstat(9, {st_mode=S_IFREG|0555, st_size=15272, ...}) = 0
6897  fstat(9, {st_mode=S_IFREG|0555, st_size=15272, ...}) = 0
6897  fstat(9, {st_mode=S_IFREG|0555, st_size=15272, ...}) = 0
6897  fstat(9, {st_mode=S_IFREG|0555, st_size=15272, ...}) = 0
6897  mmap(NULL, 15272, PROT_READ, MAP_SHARED, 9, 0) = 0x7fa5a9c76000
6897  stat("/home/catch/www/7/sites/default/settings.php", {st_mode=S_IFREG|0555, st_size=15272, ...}) = 0
6897  munmap(0x7fa5a9c76000, 15272)     = 0
6897  close(9)  

Not as good as I'd hoped.

It removes the following:

6555  access("/home/catch/www/7/sites/9090.d7.7/settings.php", F_OK) = -1 ENOENT (No such file or directory)
6555  access("/home/catch/www/7/sites/d7.7/settings.php", F_OK) = -1 ENOENT (No such file or directory)
6555  access("/home/catch/www/7/sites/7/settings.php", F_OK) = -1 ENOENT (No such file or directory)

But, sadly, it adds some back because we're actually including sites.php now:

6897  stat("/home/catch/www/7/sites/sites.php", {st_mode=S_IFREG|0644, st_size=1782, ...}) = 0
6897  lstat("/home/catch/www/7/sites/sites.php", {st_mode=S_IFREG|0644, st_size=1782, ...}) = 0
6897  lstat("/home/catch/www/7/sites", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
6897  open("/home/catch/www/7/sites/sites.php", O_RDONLY) = 9

And there's this section which is two steps forward two steps back:

555  access("/home/catch/www/7/sites/default/settings.php", F_OK) = 0
6555  lstat("/home/catch/www/7/sites/default/settings.php", {st_mode=S_IFREG|0555, st_size=15272, ...}) = 0
6555  lstat("/home/catch/www/7/sites/default", {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0
6555  lstat("/home/catch/www/7/sites", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
6555  open("/home/catch/www/7/sites/default/settings.php", O_RDONLY) = 9

vs.

897  access("/home/catch/www/7/sites/default", F_OK) = 0
6897  access("/home/catch/www/7/sites/default/settings.php", F_OK) = 0
6897  access("/home/catch/www/7/sites/default/settings.php", F_OK) = 0
6897  lstat("/home/catch/www/7/sites/default/settings.php", {st_mode=S_IFREG|0555, st_size=15272, ...}) = 0
6897  lstat("/home/catch/www/7/sites/default", {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0
6897  open("/home/catch/www/7/sites/default/settings.php", O_RDONLY) = 9

So, still a bit messy.

So we could assume that sites/default/settings.php will always exist, if there's an entry for it in sites.php, i.e. this patch:

diff --git includes/bootstrap.inc includes/bootstrap.inc
index 0739332..c995bdc 100644
--- includes/bootstrap.inc
+++ includes/bootstrap.inc
@@ -395,7 +395,7 @@ function conf_path($require_settings = TRUE, $reset = FALSE) {
   for ($i = count($uri) - 1; $i > 0; $i--) {
     for ($j = count($server); $j > 0; $j--) {
       $dir = implode('.', array_slice($server, -$j)) . implode('.', array_slice($uri, 0, $i));
-      if (isset($sites[$dir]) && file_exists(DRUPAL_ROOT . '/' . $confdir . '/' . $sites[$dir])) {
+      if (isset($sites[$dir])) {
         $dir = $sites[$dir];
       }
       if (file_exists(DRUPAL_ROOT . '/' . $confdir . '/' . $dir . '/settings.php') || (!$require_settings && file_exists(DRUPAL_ROOT . '/' . $confdir . '/' . $dir))) {

Now it looks like:

8027  access("/home/catch/www/7/sites/default/settings.php", F_OK) = 0
8027  access("/home/catch/www/7/sites/default/settings.php", F_OK) = 0
8027  lstat("/home/catch/www/7/sites/default/settings.php", {st_mode=S_IFREG|0555, st_size=15272, ...}) = 0
8027  lstat("/home/catch/www/7/sites/default", {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0
8027  open("/home/catch/www/7/sites/default/settings.php", O_RDONLY) = 9

So that saves us one access.

Then you have to wonder while we're doing an access twice, to remove the double access, we can do this:

diff --git includes/bootstrap.inc includes/bootstrap.inc
index 0739332..3dfcc75 100644
--- includes/bootstrap.inc
+++ includes/bootstrap.inc
@@ -395,8 +395,9 @@ function conf_path($require_settings = TRUE, $reset = FALSE) {
   for ($i = count($uri) - 1; $i > 0; $i--) {
     for ($j = count($server); $j > 0; $j--) {
       $dir = implode('.', array_slice($server, -$j)) . implode('.', array_slice($uri, 0, $i));
-      if (isset($sites[$dir]) && file_exists(DRUPAL_ROOT . '/' . $confdir . '/' . $sites[$dir])) {
-        $dir = $sites[$dir];
+      if (isset($sites[$dir])) {
+        $conf = "$confdir[$sites][$dir]";
+        return $conf;
       }
       if (file_exists(DRUPAL_ROOT . '/' . $confdir . '/' . $dir . '/settings.php') || (!$require_settings && file_exists(DRUPAL_ROOT . '/' . $confdir . '/' . $dir))) {
         $conf = "$confdir/$dir";

Now we're down to one access:

8361  access("/home/catch/www/7/sites/default/settings.php", F_OK) = 0
8361  lstat("/home/catch/www/7/sites/default/settings.php", {st_mode=S_IFREG|0555, st_size=15272, ...}) = 0
8361  lstat("/home/catch/www/7/sites/default", {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0
8361  open("/home/catch/www/7/sites/default/settings.php", O_RDONLY) = 9

Now let's look at drupal_setting_initialize():

  if (file_exists(DRUPAL_ROOT . '/' . conf_path() . '/settings.php')) {
    include_once DRUPAL_ROOT . '/' . conf_path() . '/settings.php';
  }

.

So currently, we have duplicate file_exists() calls if you just do a normal install. If you use sites.php, there are three file_exists() for the same file!

Let's kill that file_exists() - if it fails, we're going to redirect you to the installer, where you should have been anyway. PHP will throw warnings in this case, but it doesn't actually break anything. It looks like install.php might need to add some extra checks here though so it doesn't try to include sites/default/settings.php for no reason.

694  lstat("/home/catch/www/7/sites/default/settings.php", {st_mode=S_IFREG|0555, st_size=15272, ...}) = 0
8694  lstat("/home/catch/www/7/sites/default", {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0
8694  open("/home/catch/www/7/sites/default/settings.php", O_RDONLY) = 9

And try include instead of include_once

8545  stat("/home/catch/www/7/sites/default/settings.php", {st_mode=S_IFREG|0555, st_size=15272, ...}) = 0
8545  lstat("/home/catch/www/7/sites/default/settings.php", {st_mode=S_IFREG|0555, st_size=15272, ...}) = 0
8545  lstat("/home/catch/www/7/sites/default", {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0
8545  open("/home/catch/www/7/sites/default/settings.php", O_RDONLY) = 9

This is actually worse, need to look at that...

In either case, after the first request, if you have the PHP realpath cache set up properly (may not be the case on many hosts), then the lstat calls are cached, then best case we're down to:

8545  access("/home/catch/www/7/sites/sites.php", F_OK) = 0
8545  stat("/home/catch/www/7/sites/sites.php", {st_mode=S_IFREG|0644, st_size=1782, ...}) = 0
8545  stat("/home/catch/www/7/sites/default/settings.php", {st_mode=S_IFREG|0555, st_size=15272, ...}) = 0

At this point, I think that's as much as we could possibly get away with for Drupal 7. Will open a separate issue for Drupal 8.

This definitely still needs work for the installer, but CNR to see what the bot thinks.