| Project: | Drupal |
| Version: | 7.x-dev |
| Component: | base system |
| Category: | feature request |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | closed |
| Issue tags: | multi-site |
The following patch to conf_path() adds support for aliasing the directory of a multi-site installation. That makes it possible to develop a multi-site on a dev server that does not have the same domain name as the production server without the site breaking when it moves to a new domain. It also means two people can have two different domain-dependent sandboxes that share the same database, and everything still works as long as they have different mapping files. (That's the actual use case I needed this patch for; it works great for us in Drupal 5, but the patch is against Drupal 7.) The performance impact should be miniscule, and can be skipped completely on a production site by simply naming your directories to assume the live server; then the mapping file doesn't even get included in production.
#1
Oh yes. Patch added during DrupalCon code sprint. :-)
#2
+1 from me, would be very handy. There's some weird character encoding noise in the patch file though.
#3
@catch: Where? Viewing the file in KWrite I don't see any "noise".
#4
Hmm, my browser auto-detects ISO-8859-1 when I view the patch (on two different machines), manually selecting UTF-8 the noise goes away again. I'll put it back to needs review since it may not be an issue. It's the conf_path() section of the patch only, first bit is fine either way.
#5
I can confirm the problem catch is talking about. Where spaces should be, there are … some other kind of symbols. I tried UTF-8 and ISO-8859-1 (Latin1 and Windows).
#6
Oh for the love of... I know the problem. Windows sucks. Even remotely. I'll fix it and reroll the patch shortly, but in the mean time it's fully testable so I'm leaving it as CNR.
#7
It turns out something did change in conf_path() between D5 and D6, so the previous patch would have been a regression anyway. Here's a new version that should (a) have no regressions and (b) not have any stupid character encoding. I hope. :-)
Please review.
#8
@Crell: I saw the encoding before as well (catch had already mentioned it). The latest patch does not show any oddities. Thanks.
#9
Thank me by testing it and setting it RTBC if appropriate. ;-) Thanks.
#10
Will this allow us to store credentials outside of web root?
#11
@moshe: No, this is unrelated to putting settings.php outside the web root. That would be a much larger change, since you'd need to move settings.php but not the module and theme directories (since those must be web-accessible for CSS, JS, and image files). Not that allowing settings.php to be moved is a bad idea, but it's a separate issue from this one. (I suppose the sites.php file could be used for that as well, but let's take it one step at a time.)
#12
This was on my list for several months. It is a pain to deploy a site that was at test1.example.com without symlinks.
No longer do we need to move files around and hack the database for file locations ...
Thanks for doing it sooo much. I owe you a (non-alcholic) beer ...
Works as advertised.
So, +1 from me.
#13
Here are the benchmarks.
Without the patch, install to sites/default as usual
Concurrency Level: 5
Time taken for tests: 6.286319 seconds
Complete requests: 500
Failed requests: 0
Write errors: 0
Total transferred: 2444000 bytes
HTML transferred: 2180500 bytes
Requests per second: 79.54 [#/sec] (mean)
Time per request: 62.863 [ms] (mean)
Time per request: 12.573 [ms] (mean, across all concurrent requests)
Transfer rate: 379.55 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 4.8 0 46
Processing: 23 61 47.5 51 628
Waiting: 0 59 48.1 50 628
Total: 23 62 47.6 53 628
Percentage of the requests served within a certain time (ms)
50% 53
66% 68
75% 76
80% 82
90% 96
95% 123
98% 192
99% 252
100% 628 (longest request)
With patch and a sites.php that has one entry:
<?php$sites = array(
'sub1.example.com' => 'khead-test',
);
?>
Concurrency Level: 5
Time taken for tests: 6.347545 seconds
Complete requests: 500
Failed requests: 0
Write errors: 0
Total transferred: 2444000 bytes
HTML transferred: 2180500 bytes
Requests per second: 78.77 [#/sec] (mean)
Time per request: 63.475 [ms] (mean)
Time per request: 12.695 [ms] (mean, across all concurrent requests)
Transfer rate: 375.89 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 2 6.7 0 48
Processing: 23 60 43.9 55 738
Waiting: 0 58 45.1 53 736
Total: 23 62 44.0 56 739
Percentage of the requests served within a certain time (ms)
50% 56
66% 70
75% 77
80% 82
90% 100
95% 123
98% 158
99% 188
100% 739 (longest request)
Caching is off in both cases. One node on the front page.
Conclusion: the difference is negligible, and this patch has no adverse performance impact.
Get it in Dries ...
#14
Subscribing, I'm having a similar problem I solved (for now) using symlinks.
#15
Good stuff - subscribing.
#16
Crell, just to be sure: would this allow
<?php$sites = array(
'example.com' => 'foo'
);
?>
where foo is not a domain, just a directory?
If so you would make me very happy.
#17
@gaele
Yes, that is how I tested it.
#18
Yes it would, although I think the better use of it is
<?php$sites = array(
'mydevserver.com' => 'liveserver.com',
);
?>
That way when you move to live you can simply omit the settings.php file, everything will continue to work, but you save the small extra cost of the file load.
#19
Update patch for new coding standards. No other changes. Still RTBC.
#20
No tests?
#21
Tell me how I can write unit tests for a function that depends on the file system having a certain state when I can't reliably modify the file system as part of the test and I will happily write tests for it. :-)
#22
Still applies with offset. I don't see how this can be tested either. It's annoying messing around with system paths etc. when moving sites around, so confirming the RTBC.
#23
We have been using this patch on a few D5 sites in production for 6 months now without incident.
#24
is this working on d6 as well?
#25
The latest patch will probably apply against D6, but I haven't tried. I will probably make backport patches available, but not until this patch lands in Drupal 7. Which means someone needs to commit it (hint hint).
#26
The Crell patch #19 still applied successfully on CVS HEAD.
#27
And of course, all of this time and I JUST NOW find the one bug in this thing. We should be using include, not include_once, as if conf_path() is called with a $reset a second time (as happens in the installer, and nowhere else, I think) then we don't get the alias file.
No other changes from last version.
#28
Is it relevant to use
DRUPAL_ROOThere?#29
I don't believe so, since it's used higher up when files get included. I wasn't tracking that issue too closely, though, so I'll defer to Dries/webchick.
#30
We did take a lot of steps to *not* ever include directly without DRUPAL_ROOT. So I think this is needed ask DamZ for a review to be sure.
#31
Patch adds DRUPAL_ROOT
#32
Hm. This seems a little bit hackish, and something symlinks were basically designed for. We handle this by checking in symlinks to our SVN repository and it works just fine.
Someone care to sell me on why this should be a Drupal problem and not a filesystem problem?
#33
The primary reason is so that what gets into the database is, in fact, the "production" domain.
In our dev setup, everyone working on a given site has their own domain for their own sandbox. That would map to a different directory on disk in the current Drupal setup. That means places where Drupal stores a file path into the database (files, system table, etc.) will always be wrong for someone, because the database is shared.
A symlink may get Drupal to the right settings.php file, but it won't get it to save the right, shared values to the database.
The underlying problem is that Drupal multisite is far too tightly coupled to the domain name it happens to be running on at any given time. As soon as you try to move the site to a new domain (company rebranding, administrative decision, multi-server development process), it breaks down and cries.
#34
Also Windows and symlinks don't go well together.
#35
In our shop we experience the same problems as outlined by Crell in #33. We handle the problem by adding symlinks, but it easily becomes a lot of symlinks.
<background-info>
Here is a brief description of our setup. Buzy developers may skip this section :-)
We have a number of developers (John, Alice, Peter) who each develop on a number of sites (site1.org, site2.com etc.). We have several shared development servers (dev1.acme.tld, dev2.acme.tld etc.). We use rule-based hostnames like these:
Developers on the same development server can share the same symlink, e.g. site1-org.dev1.acme.tld, so we need one symlink per development server and one for the production hostname (and one for the temporary production hostname used when testing the site on the production server prior to launch).
The product of developers, sites and development servers is big and rapidly changing, so with this patch we would probably not enumerate all combinations in sites.php but rather do something like this:
<?php// Translate e.g. "peter.site3-com.dev2.acme.tld" to site3.com
$production_hostname = getProductHostnameByDevHostname();
$sites = array(
$_SERVER['HTTP_HOST'] => $production_hostname,
);
?>
This may not be the intended use of this feature, but it solves the problem :-)
</background-info>
#36
Folks - please reread Crell's answer in #33. The sticky problem is about file paths in the database, not about identifying a sites directory.
#37
Well done. ;)
Ok, I can commit this. I want to make one change though. 'example.com' => 'otherexample.com' isn't a very clear example. I'd rather use something "real world" which *seems* like it would be:
example.com => localhost
or
example.com => dev.example.com
is that true? It'd be easier to tell if the docs were a bit more clear in that section, so let's update that as well. I'd like to see those capture some of the discussion on this thread: why this is useful, what the left vs. right side of the array should contain (a code comment like 'The normally publicly accessible domain => A directory within the sites directory), etc.
#38
*sigh* Well at least someone is finally paying attention to this patch. I just wish I'd gotten this sort of feedback in March.
I'll see about Doc revisions sometime today.
#39
Also, can we get a CHANGELOG.txt entry too?
#40
Does this make more sense?
#41
I'm not sure the first is the best possible example, because IIRC, the existing system will allow that mapping automatically. Maybe it should be:
or
#42
I thought webchick might not like me for not listening to one of her excellent suggestions so I quickly added some more comments and patched is attached.
@sprugman:I don't believe it happens now. Maybe the attached patch will give a better description.
#43
sprugman convinced me on IRC that dev.example.com to example.com already works. So changing the docs to devexample.com
#44
This probably doesn't work.
In
conf_path():<?php- $confdir = 'sites';
+ $confdir = DRUPAL_ROOT . '/sites';
+
+ $sites = array();
+ if (file_exists($confdir . '/sites.php')) {
+ // This will overwrite $sites with the desired mappings.
+ include($confdir . '/sites.php');
+ }
?>
But in
conf_init():<?phpif (file_exists(DRUPAL_ROOT . '/' . conf_path() . '/settings.php')) {
include_once DRUPAL_ROOT . '/' . conf_path() . '/settings.php';
}
?>
Conclusion:
conf_path()should return a path relative to DRUPAL_ROOT. The DRUPAL_ROOT must only be prepend infile_exists()andinclude()inconf_path().#45
Cleaned up DRUPAL_ROOT in conf_path().
#46
+ * named $sites, which maps domains to directories. It should be ini the formExtra "i" on the end of "in".
#47
Fixed the i
#48
Went to test this patch and multisite stopped working after applying. Still looking into the cause.
Steps to reproduce.
1) new checkout
2) copy sites/default/*.* to sites/site1/
3) Go to http://site1
4) Reach requirements page and it is looking for the settings.php in the sites/site1 directory. Good
5) Apply patch and do the same. It is looking for the settings.php in the default directory. Bad
#49
Found the issue now patching
#50
Ok the issue was that I used for example, file_exists("DRUPAL_ROOT/blahblah") it needed to be file_exists(DRUPAL_ROOT . '/blahblah')
I changed it throughout and tested on my local machine. Works great! Thanks Crell.
#51
Yeah, you can't put constants in strings and have the substituted like variables:
<?phpdefine('DRUPAL_ROOT', dirname(realpath(__FILE__)));
echo "DRUPAL_ROOT/ferzle";
?>
Outputs: DRUPAL_ROOT/ferzle
#52
Oops forgot to move to CNR
#53
D6 Patch, excludes DRUPAL_ROOT fixes. Does it exist in D6?
#54
I'm already using #27 in a Drupal 6 site, and a similar version on two Drupal 5 sites.
#55
Couple quick final nitpicks:
<?php+ * $sites = array(
+ * 'The normally publicly accessible domain' => 'A directory within the sites directory'
+ * );
?>
a) Line doesn't wrap at 80 chars
b) If it's publicly accessible, mymachine/example doesn't make sense.
<?php+ * $sites = array(
+ * 'devexample.com' => 'example.com',
+ * 'mymachine/example' => 'example.com'
+ * );
?>
a) The spacing is off between the ' and the =
b) Let's use dev.example.com
c) Let's use localhost/example
d) 2nd array line needs a comma.
Note that b) and c) need updating below as well.
#56
Fixed
#57
Committed. :) Thanks.
This needs to be documented... somewhere... hm. I guess whatever handbook page is talking about settings.php. Doesn't need to be extensive. Just the fact that this file exists and you can use it for what you can use it for. Read the file for more information. It might be a good to coordinate with the docs team on this.
#58
The documentation should probably end up on the d7 version of http://drupal.org/getting-started/6/install/multi-site
#59
Blah didn't switch the project
#60
Backport for D6. Should this go into an issue of its own?
I am not sure whether new features like this are still being added to D6 or even D5. The patch is low-risk and doesn't break BC (unless people happen to have a file named
sites.phpinside theirsitesdirectory).#61
Features are not back-ported to stable versions.
#62
We do however sometimes maintain backports in the issue queue for important features, and this definitely qualifies. I had planned to maintain a D6 port myself anyway as we use it frequently. What's the proper way to do that again?
#63
How about get this documented first, and THEN worry about whether we do or do not backport to 6.x. :P
#64
Problem with documenting this is that to my knowledge we don't have a 7.x section in the handbook and until we do the documentation will have to wait. Any plans of adding a d7 section to the handbook anytime soon or possibility of copying the d6 over to d7 and make changes as things go?
#65
suggestion - sub-page of the relevant D6 page, clearly flagged as D7-specific for now
I'm finding all kinds of nodes relevant to D5 that got moved to the D6 sections, which therefore are gone as far as D5 people are concerned, makes for an even more "interesting" doc situation for newbies trying to find stuff. IMO copying's not the way to go, as pages that apply to both versions then get edits in only one version.
OT (not really, but sticking my neck out farther than I'd like :)
I'd suggest finding a good systemic solution to this problem before repeating the same method for D6->D7. Hate to say it but IMO we're really pushing the limits of Book and it's pushing back. . .
Separate platform? Or if we really have to keep this very complex doc set in our own dog food - well-managed vocabularies + Views and/or better search?
#66
@ HansBKK, yep, versioning of docs is one of our biggest, hardest problems to solve. Feel free to join the redesign group documentation discussion.
#67
@add1sun
OK, but the way I'm feeling at the moment I'll probably regret it in the morning
http://groups.drupal.org/node/15965#comment-55186
Actually I'm regretting it already ;{
#68
The most important place to document this is in Drupal's '/sites' directory. Especially until we figure out where to document it on drupal.org.
People who know about this feature are likely going to look for instructions and an example in the very place where they need to implement it. This is also a likely place for people (who are unaware of it) to fortuitously stumble upon it.
Patch attached to add a 'sites/default.sites.php' file to core, analogous to 'sites/default/default.settings.php' (except that this one is never copied and renamed automatically). The code comments are borrowed and modified from conf_path() in includes/bootstrap.inc (i.e. from the original patch in this thread) and from settings.php.
#69
A core patch needs to go in the core queue. No one important will see it in the docs queue. ;-)
#70
#71
I have applied the Documentation patch, and following the instructions contained within, have setup the sites.php file.
Having it as an example file is helpful for shopwing the capabilities, but I also think that some d.o documentation will be good, when there are 7.x pages.
#72
The last submitted patch failed testing.
#73
I cannot reproduce the testbot failure. Requesting review again.
#74
Looks like this was a testbot glitch.
#75
Attached is a Drupal 5 backport of c960657's Drupal 6 backport. For all the people, like me, who need it.
Untested but should be fine.
#76
#77
The last submitted patch failed testing.
#78
Set -D5 extension on last patch to avoid bot.
RTBC is on the documentation D7 patch, for clarity
#79
I know it's probably a weird place to bring this up, but i was going through some _VERY_ old issues of mine, and i found this :
http://drupal.org/node/25720
Basically, it allows the sites to add additional paths to search for modules / themes. One of the issues you will run into with windows is that without symlinks you will be duplicating a lot of code on multi-site installs.
Say if you had 10 sites on one drupal install, half of them need access to one shared module, the other some other module, and each of these has their own set of custom modules.
Without symlinks, or a way to specify this in code, you would need to either copy the exact set of modules each site needs into it's sites directory, or put the shared modules into the sites/all, and just live with the fact that half of the sites will have access to modules they shouldn't.
#80
@adrian - that is the reason for using the sites/all/[modules/themes] directories. Admittedly it does clutter up other sites modules.
Have you looked into Junctions - they are the NTFS equivilent of sym- and hard-links.
#81
Cool. Committed Jaza's documentation patch at #68. Thanks a lot!
Marking this fixed, for lack of a better status.
#82
Can we get a comment on porting to D6? #60 has a D6 backport, and webchick did say we could discuss backporting it once the documentation was done...
#83
Here is a backport for D6.
#84
Setting title back for D6.
#85
Successfully applied and configured D6 sites with the patch in #83.
#86
New features are not added to stable versions. This is a won't fix in relation to Drupal 6.
#87
@Gabor,
This isn't really a new feature. This fixes a fundamental flaw from testing to production. The fact that at the moment you have to go through each entry in the files table and replace the path of the files when you move from testing to production. You also have to change places in the variables table where this occurs. In the variables table it's not as easy because the data is serialized and depends on how the module stores the data.
Hopefully you can reconsider this for D6. It doesn't change anything fundamentally in core, but it fixes some serious issues in a multisite configuration.
#88
FWIW, I agree that this is an inappropriate change for a stable branch.
#89
Will try this. Althought it will never be commited it would be nice to have a working patch for people who need this in 6.
#90
@Pedro Lozano
#83 is the D6 patch
#91
all for the grace of statistics ;)
#92
Automatically closed -- issue fixed for 2 weeks with no activity.