(Refer from http://drupal.org/node/103519) According to the discussion of http://drupal.org/node/371 since #72, maybe we should rollback this change since it is a bit tricky and buggy:
- The primary goal of prefixTables() is for table prefix (e.g.
table_prefix_) replacement only, but not fordatabase_name.table_prefix_syntax (since tables are not located in same database):
Queries sent to Drupal should wrap all table names in curly brackets. This function searches for this syntax and adds Drupal's table prefix to all tables, allowing Drupal to coexist with other systems in the same database if necessary.
- The comment from default.settings.php never mention dot as valid syntax clearly, but only use alphanumeric and underscore as example valid database characters:
Be sure to use valid database characters only, usually alphanumeric and underscore.
- Most online reference from drupal.org or even somewhere else never mention dot as valid syntax:
Here are a host of tutorials that explain how to setup multisites with shared tables:
http://drupal.org/node/43816
http://groups.drupal.org/multisite
http://www.practicalweb.co.uk/blog/08/08/07/drupal-multisite-shared-tables - By using
database_name.table_prefix_syntax we have a lot of limitation, e.g. both database must come with identical connection parameters, which result as similar effect of locate all different tables under same database instance:
This is because we only define ONE SET of connection parameter correctly. In this case the use of split DB is just equal effect with all tables located within single database instance. As MySQL come with nearly none of maximum number of tables limitation within single database instance, I can't find the usefulness of split DB handling.
- Even if we define different set of connection parameters in settings.php, as no db_set_active() is ever called there is no actual effect:
The $db_url will only load during connection establish, where db_set_active() will take action for. As we do a tricky cross database switching with $db_prefix => database_name.table_prefix, Drupal is being cheated as all tables belongs to same database. It is function just because we hit the backdoor of both Drupal and MySQL, but not means this syntax is expected as valid :S
- When we are considering for escape all database constraint name as reserved word safe (e.g. http://drupal.org/node/371),
[{tablename}]will not able to convert as`prefix_tablename`simply, since it may result as`dbname.prefix_tablename`(if dot exists in $db_prefix) which is invalid SQL syntax. Even we can revamp prefixTables() for a proper handling, I guess this may not be our expected approach, e.g.:
- $sql = strtr($sql, array('{' . $key . '}' => $val . $key)); + $sql = strtr($sql, array('{' . $key . '}' => strtr($val, '.', '].[') . $key));
This patch rollback the checking from _install_settings_form_validate(), which close the backdoor of both Drupal and MySQL/PostgreSQL as expected handling. For existing incorrect Drupal site installation, we can propose some upgrade procedure for correction.
| Comment | File | Size | Author |
|---|---|---|---|
| #78 | db-prefixing.patch | 18.93 KB | bcn |
| #69 | 302327-reformatted.patch | 18.84 KB | josh waihi |
| #67 | 302327-schema-support.patch | 19.26 KB | josh waihi |
| #65 | 302327-reroll.patch | 19.25 KB | josh waihi |
| #59 | 302327-try-again.patch | 19.74 KB | Crell |
Comments
Comment #1
damien tournoud commentedAccording to both the discussion in #371: resolve ANSI SQL-92/99/2003 reserved words conflict in query statements and #103519: Install doesn't allow "." in db prefix for psql schema (that you vandalized), this should not be. A dot in $db_prefix is supported, and works for both MySQL (allows sharing of specific tables between instances of Drupal that are installed in different databases, with different credentials) and PostgreSQL (schema).
Comment #2
hswong3i commented@Damien: I totally understand your point of view, but please don't close an issue without enough discussion. Well, that is another story if you are able to answer my idea point by point with enough support.
Comment #3
damien tournoud commentedGod, I already did. You simply *don't listen*.
You have no argument there. You are simply stating your point of view, which is totally unsubstantiated.
That's a documentation issue that will need to be fixed, again this is not an argument.
Idem, that's a documentation issue.
That's really not true. In MySQL you can define *per table* and *per database* permissions. That means that you really can have the following setup:
This don't mean anything know that I clarified that your previous argument is false.
Well. I'm pretty sure you can easily change that. I don't see any limitation here, you are only being *lazy*. The world will not bend because you think it should. We will not drop a supported feature because you can't even program a simple replacement logic.
Comment #4
hswong3i commentedSomething need to clarify:
According to CVS log message, db_prefix_table() was first introduced to Drupal since Jul 10 2003; around 1 year later (Jul 14 2004), it was being well documented and NEVER MENTION cross database setup with dot syntax as valid. Once install.php was introduced for D5 since Jul 13 2006 (including the good works from chx and steven), the installer check $db_prefix correctly as expected: without dot support. If all these development footprint is still not enough for proving my point of view about "db_prefix_table() is originally designed for table prefix only but not for database name switching hack": sorry, I can't say anything more than that.
Form this point I need to agree with you; but I still can't find its usefulness. If: 1. MySQL almost come with no limitation of tables within single DB, and 2. MySQL allow for *per table* permissions, so why don't locate all tables within single DB + setup permission per tables + using $db_prefix without tricks? If the world is now bended as incorrect, why don't we give some effort in order to fix it!?
I have *already* change that with an example implementation! Yes, it is not elegant enough as it is just a demonstration of idea, but you also have the right to amen it.
If you DO think this is a documentation issue, please don't be *lazy* and file your idea in order to correct it. I will suggest you to: 1. change the title of this issue as documentation related, 2. submit YOUR patch and let the others review it, and 3. let it become a real standard that no longer debatable. Simply close and mask this issue don't give any help for people understand its background concern and indeed discussion.
Comment #5
chx commentedYou vandalized one issue already, your points are heard there and in another and it's over. This won't happen sorry.
Comment #6
hswong3i commented@chx: My suggestion should be very clear (if this is really a documentation problem ONLY):
If you are talking about "won't fix" of the "patch for rollback dot support" itself, just let it be and I don't really care; but if you are talking about we "won't fix the document and keep the confusion" (again, assume this is a documentation problem ONLY), I can't understand your point of view :S
Comment #7
dalinIn my opinion blocking cross-db table sharing is unacceptable. While there may not be any theoretical limitation of having a thousand tables in a DB, there certainly are practical limitations. Trying to get novice developers to work with database dumps of all of that would be near impossible.
Comment #8
damien tournoud commentedComment #9
seanburlington commentedHi,
interesting that my blog is referenced in this thread as an argument for preventing database prefixes
http://www.practicalweb.co.uk/blog/08/08/07/drupal-multisite-shared-tables
It's only since writing that post that I realised database prefixes are possible.
The project I am currently working on originally used two databases - one for "drupal" content and another for business data.
The other database was connected to using http://api.drupal.org/api/function/db_set_active/5
But the problem with this approach is that if an error occurs while connected to the non-drupal database - error handling attempts to log to the watchdog table in the wrong database
As a result we merged the two databases - this has had it's own problems.
We currently intend to use the database prefix method to use two databases - with one connection.
and I need to update my blog.
I'm strongly in favour of updating the documentation to refer to the database prefix
Changing this behaviour would break the sites of those who rely on it
see also
http://www.drupaler.co.uk/blog/multi-site-drupal-6x/55
http://drupal.org/node/291373 (Multi-site with single codebase, different content databases, shared user database, shared sign-on)
Comment #10
Crell commentedFor Drupal 7, actually, the solution is for selected parts of core that *must* use the main database to bypass db_query() and access the default connection directly. Watchdog and the registry are both good candidates there.
Comment #11
seanburlington commentedIs it possible for a documentation bug to be fixed in current versions?
Comment #12
greg.harveyI'd like to +1 Damien's suggestion as well as Sean's proposal that the docs change is back-ported. Seems to me the documentation could stand to be clearer. Until Damien told me I could use the "dot" in settings.php, I didn't know - it's a great feature, but I've never seen any documentation about it. =)
Comment #13
ivansb@drupal.orgIt seems that "." is going to cause problems during updates since the way D5 and D6 check for table existence in postgresql doesn't take into account schemas.
Problem seems solved in D7 (does actually D7 support schemas?)
I still have to figure how to "backport" D7 solution to D5 and D6 in a clean way...
I've no idea about how/if D5-7 support schemas (including update path), so I'm not sure if backporting the solution of D7 is going to make any good to D5-6.
It looks there are still some problems to fix in INDEX creation in D7 if schema are going to be supported:
Comment #14
josh waihi commented#140860: Consistent table names and database handling of table names is currently debating implementing schema logic seperate from db_prefix logic. This makes sense since schema/db reference in db_prefix has been more accidental than intentional.
Comment #15
ivansb@drupal.orgI do understand that using schema is a hack. But somehow it is a working manegable hack. (eg.PostgreSQL let you selectively restore schemas over existing DB and moving tables across schema preserve constraint and move sequences etc..).
This issue was about documentation. At the current state . in prefix is permitted but it may cause "solvable" issues.
From my point of view unless D7 fully support schema, a "." in prefix is still going to be permitted, but it is going to cause problems.
It would be nice if that was documented. It is going to be a reminder of the issues people may have to solve for D7, a warning to user updating etc...
I'm building up a reasonable experience with drupal, PostgreSQL and schema. If and when and where you're planning a patch let me know. Thanks.
As a side note: quoting object names may cause other "side effects". Names becomes case sensitive. SQL standard says objects with no quotes should be downfolded to uppercase. Unfortunately this is one of the rare cases where PostgreSQL doesn't follow the standard... and we may incur in other issues.
Furthermore... quoting is a pain ;)
PostgreSQL doesn't support writable VIEWS "out of the box", they may be ready for 8.4 (?) or they may be implemented with RULES... but I wouldn't go that route (at least now).
Comment #16
josh waihi commentedSchema support for PostgreSQL is currently broken. I've got a patch here to fix that. It also documents more in depth that schema support is available in Drupal as well as instruction on who to go about that.
This will need a decent review, also I think SQLite maybe able to use what I've got here, not to mention, MySQL might be able to get cross database support out of it too.
Marking as critical because PostgreSQL has support for schemas already in the driver layer, the fact that it doesn't work makes this a critical bug.
Comment #18
josh waihi commentedI've this apply to SQLite and MySQL also.
Comment #20
josh waihi commentedgit diff's don't apply :(, here's a cvs one.
Comment #22
josh waihi commentedfixed up MySQL
Comment #23
Crell commentedThe $info array at the top of getPrefixInfo() is badly indented.
getPrefixInfo() needs a complete docblock. I didn't immediately understand why $table has a default of "default". :-)
Method names should not begin with an underscore, ever. If they're non-public, make them protected.
Er. $this->getPrefixInfo() returns an array, doesn't it? If it's returning a handle the prefixing of this thing" string, then getPrefixInfo() is the wrong name for it.
Comment #26
josh waihi commentedchanges as requested
Comment #27
josh waihi commentedupdated
buildTableNameConditionto re-factor the newer code I've written.Comment #29
josh waihi commentedah, had to update tests.
buildTableNameConditionno longer requires a prefixed table nameComment #30
josh waihi commentedupdating status
Comment #32
josh waihi commented--no-prefix for git patches
Comment #34
josh waihi commentedHad MySQL working with a '.' based prefix but not without, works now.
Comment #35
Crell commented1) I talked with Josh about this patch to see what was really going on.
2) Changing title to reflect what we're actually doing here. :-)
3) Setting RTBC.
Comment #36
damien tournoud commentedThe SQLite part is not quite there yet. SQLite uses:
like PostgreSQL does.
Comment #37
josh waihi commentedHave altered and tested SQLite works now.
FYI: PostgreSQL doesn't follow the SQLite Syntax is uses:
Comment #39
bcn commentedre-roll
Comment #40
bcn commentedand for the test bot...
Comment #41
josh waihi commentedchoice thanks!
Comment #42
sethcohn commented+1, about to start on a multi-site multi-db development project, where this will be extremely useful.
Comment #43
josh waihi commentedwhat is stopping this getting in?
Comment #44
dries commentedI found the documentation in this patch to be a little cryptic.
Should be 'Optional advanced'.
Should be 'Optional advanced'.
Example comment that is hard to grok.
Missing dot at the end of the sentence. We should also explain what the 'prefix for a table' is, and why it useful.
We should use lower case letters here.
This is unclear. We should explain why you'd want to "specify a schema as part of the prexif if the schema already exists".
Always write 'Drupal' and not 'drupal' in documentation.
Comment #45
k4ml commentedWhat about:-
which failed because DatabaseConnection::escapeTable() would strip out the 'dot'. I don't see this fixed in the patch. This is very important when working through multiple schema in PostgreSQL. I'm not sure whether overriding escapeTable() in PostgreSQL driver is all we need to do.
Comment #46
josh waihi commented@k4ml, identifying the schema in postgresql should done in settings.php and not in runtime. Thats the whole point of this patch - and not to introduce th abilty to identify the schema when using db_select
Comment #48
bcn commentedHere's a re-roll, along with some small changes to account for #570900: Destroy remnants of update_sql().
Comment #50
bcn commentedThis one should be fixed. The two previous patches were missing an opening comment in includes/database/schema.inc...
Comment #51
Crell commentedJust comment cleanup, should be good to go.
Comment #52
josh waihi commentedlooks great, thanks noahb :)
Comment #53
webchick-i from braces.
Indentation.
Uh. Could we expand these comments out? I don't understand why we have both tablePrefix() and prefixTables(). The docs should make it clear what does what and which one to use when.
Formatting bug.
Uh. Wow. I don't remember the last time I've seen ++$something.
Can we comment this function internally a bit to explain what's going on?
If we're going to document this in only one place, let's do it in pgsql/schema.inc (though preferably in database/schema.inc.
Indentation is off here.
Um. I'm seeing double. :)
And do we need a similar hunk for INSTALL.msyql.txt?
I'm on crack. Are you, too?
Comment #54
josh waihi commentedI've cleaned up the comments plus fixed up some code that was missed
Comment #56
josh waihi commentedusing
git diff --no-prefix origin/master. Should work.Comment #57
Crell commentedCycling the bot.
Comment #58
Crell commentedBot, pay attention!
Comment #59
Crell commentedOK, reposting the same patch. Bot, wake up!
Comment #61
Crell commentedI forget, was this blocked on #633678: Make sequence API work on non-MySQL databases? Because that just went in. :-)
Comment #62
josh waihi commentedyes it was, my next focus
Comment #63
ivansb@drupal.orgWould it be possible to support schema for functions too?
I think functions should have their own db_query call/method.
Advantages would be: auto uninstall as with tables, share function across schema even if they refer to tables in different schemas etc...
I'm doing it right now to install functions I use for fulltext search and well some support in core would come handy.
Is there a way to make drupal/db_prefix aware of shared "entities".
Now D7 should transparently support transaction and eg. modules that insert records at install time may make a mess if the module has some shared table.
Where would be the right place to talk about shared tables and search_path?
Once the module install phase is passed... stacking multi-sites with common tables is awesome with postgresql... and quite performant. Cloning sites is a matter of minutes.
BTW
I noticed CONCAT is still around but a) not tested, just || get tested and b) used just once in a place where 1) it could be changed to || 2) the query should be rewritten anyway.
Why you test existence
SELECT COUNT(*) FROM pg_proc WHERE proname =
just for concat and rand and not for other functions?
Finally:
select greatest(10::numeric, 15::numeric, 32::numeric); works perfectly under pg>=8.1
so there seems to be no need to define them with 3 arguments or with CASE (tested with null too and they are equivalent).
Comment #64
josh waihi commentedivan, this is way off topic for this particular issue and since Drupal 7 is now in a code freeze, features can not be added however improvements to existing functionality is welcome but they need to have there own issues created and brought to the attention of the maintainers such as myself, DamZ and Crell.
Unfortunately, Drupal doesn't really know what a schema is, it just thinks of it as a prefix to the table. Because it doesn't know of any schemas other that 'public' we simply cannot create functions that reference tables from other schema. Is there a function inparticular your talking about? From memory, Drupal's postgres functions don't reference any tables.
In regards to concat, it must be used to align MySQL and PostgreSQL syntax - this is so contrib don't break postgresql support so easily.
Please feel free to email me if you have any other questions.
Comment #65
josh waihi commentedhere is a re-roll. Have tested on PostgreSQL and it works.
Comment #67
josh waihi commentedlets try this.
Comment #68
sun.core commentedAt least a duplicate "is" here. Could be shortened a bit in general though.
Why multi-line?
We can remove the duplicate braces here.
Wrong indentation.
This commment is a bit lengthy and could be shortened to the actual problem/action that is being performed.
Trailing white-space here.
...and here (and possibly elsewhere)
Wrong indentation, possibly tabs.
Missing @code + @endcode delimiters.
Powered by Dreditor.
Comment #69
josh waihi commentedI didn't apply the @code + @endcode delimiters becase no other code in default.settings.php has them (maybe that should be a separate issue)
Also this stays:
because it is more explanatory than
($pos = strpos($info['prefix'], '.') !== FALSE).Comment #70
sunNothing of the following should hold back a commit. But of course, would be cool if those issues could be fixed either prior to commit or in a quick follow-up patch.
This issue is marked as critical, so this stuff really doesn't matter.
Trailing white-space here.
Exceeds 80 chars.
Still duplicated parenthesis here.
Unlike webchick, I don't think that ++$pos needs explanation. That's plain and totally valid PHP.
Typo in "contstraints".
144 critical left. Go review some!
Comment #71
aspilicious commentedUnlike webchick, I don't think that ++$pos needs explanation. That's plain and totally valid PHP.
I agree with sun, ++var statements are well-known in almost every programming language I know (that doesn't mean that there aren't languages that doesn't support it)!
Comment #72
mikeytown2 commentedif ++$pos is an issue; replace it with $pos++ above. Everyone knows what that does.
Actually looking at the code $pos is not used again so this is valid
Comment #73
dries commentedLet's do a quick re-roll here. There was enough feedback on the code that it is worthy incorporating it. I agree that ++$pos is perfectly valid, but we should write $pos++ if we can.
Comment #74
Crell commented@Dries: Why? Both appear to be identical functionality-wise in this instance. Technically ++$pos is infinitesimally faster. Neither one should require a comment as we can expect anyone writing PHP to understand it.
I don't really care either way, I just don't see why there's a preference.
Comment #75
dries commentedCrell, sorry I wasn't suggesting we document
++$pos. I was suggesting we incorporate (some of) the feedback from reviews #70 and #72.Comment #76
grendzy commented#69: 302327-reformatted.patch queued for re-testing.
Comment #78
bcn commentedre-roll to include comments from 70 & 72, including going with this:
+ $info['table'] = substr($info['prefix'], $pos+1) . $table;Comment #79
josh waihi commentedThanks noahb
Comment #80
dries commentedAsked Crell for his final thumbs up (or thumbs down).
Comment #81
Crell commentedLet's get this long and storied issue committed. :-) I just read over the patch and didn't see anything to hold it up. (There were one or two stylistic things already in the code that this patch doesn't change; I am not holding this patch up for that.) Let's get this committed.
Josh, awesome work!
Comment #82
dries commentedCommitted to CVS HEAD. Thanks.