I'm starting to write some modules for my own use, and while I use external DB files, I think it would be better for maintenance issues down the road to put the CREATE/DELETE TABLE commands into the module itself, instead of separate .mysql (or .postgres) files. Instead, put it under admin/settings/module to Create/Delete/Administer the tables.

I can also see this helping for those admins who can't get cmdline access to the Database, but still need to do DB maintenance activities

Does this sound reasonable to add to the Module Writer's guidelines? Specifically the tips page on http://drupal.org/node/1395

And, what would be a reasonable set of default actions to take beyond Create/Delete. Seems that PURGE option to cleanup a table would be good. And Upgrades too, which are now handled this way.

Anecdote: One overzealous admin I have likes to extract the module tarballs, mv the .module file to the /modules area, and delete the sub-dir! Often loses the README and .mysql files in the process, even after I have shown him the problem. Just his habit, but got to break this bad habit.

Comments

nsk’s picture

No I don't think this is a good idea. I can easily manage an sql file and see what it does to my database, but I would have difficulty to analyse a module. As a user I prefer separate sql and php files.

--
NSK, Admin of Drupal-based site http://www.wikinerds.org

cpill’s picture

The module looked for the file and automatically ran the SQL contained within?

degerrit’s picture

I think it's a good idea, but would think a specific API for it would be better.

Then you can still review what the module does to your database by checking out the relevant API function. Heck, it could even be included from a separate file so you retain the "best of both worlds".

cpill’s picture

I don't understand why Drupal isn't using something like PEAR's MDB. The item to note in the list is "Altering of a DB from a changed xml schema". If the table structure changes in newer versions of a module, simply changing the XML table description will alter the table without touching the data in it.

This also expands the number of databases that Drupal can support from two to seven.

It uses this crazy new coding idea. I think they call it "Object Oriented Programming". Crazy stuff :)

sepeck’s picture

Do search on Object Orientated in the forums. Look through the dev list archives. There are several good reason's why they don't use this [insert sarcastic descriptive comment here] 'new coding method' Object Oriented Programming.

As to pear... I believe that it is to reduce reliance on yet more exteranl requirements. Not all web hosting companies have PEAR and then what version? How will you maintain backward compatibility. Will this offer a performance hit? Perhaps it's just better to put the database abstraction layer in the php code itself instead.

-sp
---------
Test site...always start with a test site.

-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide

cpill’s picture

Well I just did a search for "Object Orientated" in these forums and the only post that seemed to come back was this one:
http://drupal.org/node/866
Where people seemed to agree with me and advocated a complete rewrite (which is the real reason Drupal is still in procedural, I would guess suffering from the decisions made at the very beginning). Drupal emulating OO with the hooks idea in the modules, in OO they call it polymorphism. In truth Drupal is a testament to how bad PHP4 is, where it doesn't matter weather you use OO or procedural methods much except it might be slightly easier to enforce coding standard in OO (i.e. requiring modules to all inherit from a module class interface).

Perhaps Drupals design is starting to come undone as the image inclusion problem is becoming a bit of a thorn:
http://drupal.org/node/10271
If there where a simple image object that implemented your basic image upload and management, one might be able to inherit/extend it and add complexity or extend its behaviour for more specific modules requiring less work for the module developer while remaining flexible for future needs.

As for "reduce reliance on yet more external requirements". Have you hear of the saying "Why reinventing the wheel"? Hell why use PHP at all when you can write a standalone app in C that will require nothing and be faster than any web app out there?

  • PEAR comes standard with PHP now. Since somewhere in 4 I think. Also you can download it and include it in your distribution if you want more specific objects (since your also open source I think).
  • The performance hit is almost nothing as its an interface to the PHP native functions but uses Encapsulation (another OO term there ;).
  • As for which version, I guess the latest since the interface to the PEAR objects are guarantied not to change once stable, just get better and faster as they have a huge community testing/debugging (see mailing list or RSS feeds for examples of this).

The trouble is there is no "abstraction" if there is no 'Encapsulation' and there is none in procedural function calls (unless of course your emulating polymorphism, in which case why not go for the real thing?).

But OO vs. procedural is as old as C vs. C++ and is academic now. For the question of: "Why use someone else’s code library?" My question is: Why not use code that is continually updated/upgraded, supported by a huge community, tested by this community, and has identical functionality to your own code except that it does more (and perhaps does it better, if you believe in the open source methodology). PHP is an out standing example of why use other peoples libraries. You just have to look at the user manual and note that most of the useful things in PHP are external libraries. Not looking for existing code is just being provincial. Hell if everyone thought the same way, there would be no Drupal community.

But this is such a low level issue. Drupal developers should be focusing on the higher level functionality which is what using PEAR is supposed to be about. In respect the original question using PEAR's MDB will mean:

  • That there won't need to be a separate 'Import .mysql' file into the DB, making it easier for non developers.
  • Amateur PHP developers will not need to know SQL to setup or access their tables.
  • Maintenance/upgrade of schemes between module versions will be transparent (see previous post).

-aw

killes@www.drop.org’s picture

Try searching for "object oriented". Also search the mailing list:
http://lists.drupal.org/archives/cgi-bin/namazu.cgi?query=object+oriente...
--
If you have troubles with a particular contrib project, please consider filing a support request. Thanks. And, by the way, Drupal 4.5 does not work with PHP 5.

cpill’s picture

...no cigar. It did return a post with the word 'object' and the word 'oriented' separately I might add (http://drupal.org/node/1007).

The results I did look at didn't seem to be against OO. Please direct me...

killes@www.drop.org’s picture

Drupal development isn't done in the drupal.org forums but happens on the mailing list. If somebody on drupal.org advocates OO he is most likely not a core developer. Requests to rewrite Drupal to use classes usually get replies like "I'd rather rewrite it in LISP" (or Haskell, AWK, ...).

In short: It won't happen.
--
If you have troubles with a particular contrib project, please consider filing a support request. Thanks. And, by the way, Drupal 4.5 does not work with PHP 5.

cpill’s picture

Even Perl went OO after long enough. Hell even VB! Look at PHP5, what where all the major changes related to.

Your not addressing the issue with "development isn't done in the drupal.org forums". It isn't done in the list ether. Its done at peoples homes and offices. Issues are discussed in forums and lists. Discussed.

As for rewriting in LISP, see my above comment about rewriting in C. The reason PHP is dominate on the web is because its on every server on the web (even IIS machines).

In short: weather it happens or not is open for discussion, as are all aspects of an open source system. Its only a matter of time and frustration :)

killes@www.drop.org’s picture

Discussions are a very important part of the development process. As to LISP: Look up »irony« in your dictionary. Also, I am very non-frustrated that Drupal doesn't use classes. I don't think any of the core developers is frustrated about this. You are free to use classes in contributed project, btw.
--
If you have troubles with a particular contrib project, please consider filing a support request. Thanks. And, by the way, Drupal 4.5 does not work with PHP 5.

cpill’s picture

LISP wasn't mentioned. Nor was an counter argument as to why my suggestion for using MDB in Drupal to solve the above problem.

Technically classes are used in Drupal aren’t they? The Node is an example of this and would be considered fairly ‘core’, yes? Guess this might explain your lack of frustration :)

So apart from you are there any others in this inner circle of the "core developers" klan that has objections to a OO approach (and can supply argument ether way i.e. a logical progression of reasons/facts that build to a conclusion)? Or are you the only “nugget core developer” to appear in the light of public discussion?

killes@www.drop.org’s picture

I have a very limited knowledge of OO stuff, but I think the $node thingie is an object, not a class.
Most core developers have expressed reservations on introducing more OO stuff into Drupal. The authoritative answer is however always Dries' anwer:

http://lists.drupal.org/archives/drupal-devel/2004-07/msg00542.html
--
If you have troubles with a particular contrib project, please consider filing a support request. Thanks. And, by the way, Drupal 4.5 does not work with PHP 5.

micha_1977’s picture

the only issue i would have for the drupal php-code, is a documentation of the "workflow", doin it with the debugger is, like working with IBM->TSO->Xpediter and trying to learn the secrets of CICS

but as i step deeper into drupal, id like to see if it is possible to completely re-design drupal for OO...in a personal project

cpill’s picture

And in PHP5...

cpill’s picture

PEAR is available on all systems that support PHP (its just PHP code) and its DB object does everything that the current 'abstraction layer' does, in fact it does it better (i.e. more supported DB's, more thoroughly tested, more documentation in more languages) and offers more functionality.

I could go on.

The real argument in that email is that the majority of coders on Drupal don't have the understanding to take on OOP (as your comment about the '$node thingy' demonstrates). This might reduce the number of contributors but might also improve the general quality of the modules submitted. It hasn't hurt the PEAR projects numbers or quality.

javanaut’s picture

The flexibility found in the current design differs from the OO paradigm. Similar results could be created using classes, interfaces and such, but *everything* about drupal would have to be rewritten. All node modules or modules that act upon nodes would have to be rewritten. All hooks, all themes, everything. This is why, as far as I can tell, the current direction has not changed. I haven't surveyed the developer community here (core or otherwise), but I would guess that if drupal core were changed over to an OO system, most of the existing modules would never be updated further. Furthermore, I would predict a fork such that OO drupal and its fans would do one thing and procedural drupal would do another.

If you were truly interested in refactoring drupal into an OO paradigm, I know you would have fans, but you would be on your own otherwise. Also, it wouldn't make sense to do this on a PHP4 platform, so you would most likely take on a PHP5 requirement, which probably won't be acceptible for some time to come.

As for PEAR, not all systems have it installed. One of the things that Drupal is trying to deal with now is making installation easier. Requiring users to install PEAR modules on their servers will not help with this. The benefit of expanded db vendor support is traded for simplicity of installation and development. I realize, however, that many people want Oracle and SQLServer support. They're nowhere near as popular as MySQL and Postgres on public shared hosting services, which is the primary target for drupal installations.

cpill’s picture

The second major advantage that PHP has over other server side scripting languages (that it on every web server is the first) is its massive community. If you not taking advantage of that your crazy. PEAR has a massive community with teams of people working on just one of the hundreds of little problems that make up web development slow and painful.

If your worried about distribution you could set-up your own channel server and deploy packages on across the web at set-up.

Steven’s picture

Just because we don't use classes doesn't mean that Drupal is a tangled mess of functions. In fact, Drupal is quite object-oriented. We just don't use classes because they are too rigid and don't allow what we do now without a huge amount of overhead, both in code and in performance. I've seen both bad OO and non-OO PHP code. And I've seen good OO and good non-OO code. Whether or not something uses classes is not a very good criterium to judge quailty.

Oh and many of us /do/ have a computer science background. Dries spends his day-time on his Ph.D. about Java Virtual Machines, so I think he understands programming concepts well.

Comments such as yours achieve nothing but set bad blood between you and the others because you're essentially saying everyone is too stupid to understand your wonderful idea. It doesn't help that this entire discussion has been had before.

Maybe PearDB does have more features... but duplicating the entire SQL language as an object-oriented structure adds another layer in between, which can only slow down performance.

--
If you have a problem, please search before posting a question.

sepeck’s picture

Several years ago I got to attend a presentation by Rasmus Lerdorf (php) and he made a comment about accessing databases. He said there was nothing wrong with accessing the database from within your code. Just write the functions you need and make sure that if necessary you can easily update the functions to access a different database platform in one spot.

I didn't fully understand what that meant at the time but have a better idea now. Drupal supports MySQL out of the box as well as PostGRE. It used to upport MS SQL but there has been no maintainer for a version or two.

I can't find that original presentation from then (I will keep looking) but I did find this brief item
http://www.oracle.com/technology/pub/articles/php_experts/rasmus_php.html
Look at item 9 in the PHP Tips and Tricks sidebar.

9. Database abstraction is mostly a myth. There is nothing wrong with direct database calls' making use of all the tricks and cheats your chosen database has to offer, to tweak as much performance as possible out of it.

-sp
---------
Test site...always start with a test site.
Drupal Best Practices Guide

-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide

cpill’s picture

have you

killes@www.drop.org’s picture

I like the way the installation is currently done.
--
If you have troubles with a particular contrib project, please consider filing a support request. Thanks. And, by the way, Drupal 4.5 does not work with PHP 5.

cpill’s picture

:))

Bèr Kessels’s picture

There are much better solutions that adding the SQL in the module. Keep in mind that on every (every!, yes that inlcudes that one too!) pageview that code would be loadedinto the PHP engine. In other words incredible amount of overhead, for what?

No, an install system is on its way, but not in this way!

And if this solved you problem, would you be so kind to report back that it helped? This will help others whom are looking for the same solution.

[Ber | Drupal Services webschuur.com]

faxman’s picture

I read through the full node and from my experience as a database designer and programmer let me state two things out.
Keep the sql-code retainded somwhere but not in a Object or module or anything like that.
It makes more sense to have the sql-code residing maybe even in a .doc or .txt file.
This gives you an advantage when you for instance want to port to different databases.
The overhead isn't all that much and it remindes me of the question we takled in our own projects several years ago. Is it better to use ODBC or are stored procedures better?
At that time speed was a matter of the processor, but looking at the power that is available for little money these days, this question is absolete.

Second, if you set up a database system, you are bound to run into learning sql in one way or the other. So what's the problem? Learning something hasn't hurt anybody so far as i know. Looking at what started this discussion (one guy not knowing what to copy and where and how) is also a issue of learning. And hey, if he messes it up, let HIM fix it!
Best way to learn and not forget that easy again.

cpill’s picture

DB always say store it in the DB. Putting code in stored procedures is ugly. What 'sense' does it make? If it does make sense then all the SQL should be in this .txt (or .doc file *shudder*). I don't see the 'porting' advantage? If porting to a different DB as long as its standard SQL it doesn't make a difference if its in the code or a separate file, except that you have to generate another file and update it with all the other separate DB file. If its in the code the DB abstraction layer takes care of that. Less work of developer. Less work for end user. Less that can go wrong.

As for the second argument: 'We should all know the details of who everything’s works because its a learning experience', well I'm glad I don't know how my toilet, camera, car, heater, mouse, screen, network card, etc work because life is too short. Drupal provides an API for developing web applications, the idea being that if you want to wack out a blog quickly you don't have to known how the authentication system works, or the DB abstraction layer, you can work at a high level and assume the you don't need to know assembler to get your job done. Like wise the modules are ultimately just another layer on top of many others. They have to be massaged in most cases to produce the target functionality. If you putting together a site that used, say 15 modules, you don't want to have to dig around in the code to figure out what the table structures are supposed to be to figure out that one (or more) modules are missing a column because you didn't know that developer had changed it (or forgot to) or you used the wrong file or you have a newer version of the code and and old schema when you moved servers etc etc.

'Encapsulation' is the word. It means that people don't have to know obscure information that is not directly relevant to what they are doing.

Also your assuming its a 'HIM'. It might be a 'HER'.

faxman’s picture

Hey CpILL,
storing everything in a DB is not what i had in mind.
Granted, you don't need to know how everthing works, i tend to like to know how a few things arround me do work. But that's an engineers additude, so forgive me.
The problem with porting database programms to different DB's was very nicely handeled by BORLAND with the use of their BDE. Unfortunately they went of track sort off. But forget about this.
Take into consideration that you might need some additional fields in your very own database for various reasons. How would you accomblish this without some SQL knowlege and soem PHP Knowhow?
Get the meaning?
Now lets think that this would have to run on various DB's with somewhat different SQL statements? Just about every DB diff's in some ways.
If Drupal was to run on various DB's there will have to be some kind of userfriendly update mechanism. How this will be accomplished i don't know as of yet.
If the problem was a HIM or a HER doesn't really matter or are you assuming all women are stupid when it comes to computers and software?
The whole issue is not really worth it, don't you think so?

Michael

cpill’s picture

...in storing the SQL out of the code and in a separate file. I mean if your going to do it for some, what ever your reasoning it would dictate that you do it for all. So all your SELECT, UPDATE, DELETE etc statements should be in this location. Immediately this seems cumbersome, especially since most of the SQL is generated dynamically by code. So my reasoning is that if some of its in the code why not all? The CREATE TABLE statements could then become more intelligent and see if the table exists and then just upgrade it. Auto patching seems pretty sexy to me.

How would you get around having additional fields without SQL knowledge?
There are two answers:
1. At present 'flexinode' module. This is such a useful concept that I think it should be core functionality.
2. I was proposing above to use PEAR's MDB package, (a hard core DB abstraction layer) that lets you describe your table in XML and then any changes to this XML will change the DB automatically. "RDBMS independent xml based schema definition management" Module developers wouldn't have to realise dodgy patches to update changes in the DB schema, just a new XML file with it… so I guess I am advocating using a separate file… but it should be XML…yeah.

Steven’s picture

Your XML suggestion sounds like a horribly bloated solution to something which can be solved much simpler.

--
If you have a problem, please search before posting a question.

cpill’s picture

And the solution is...

Steven’s picture

and try to find out what Adrian is doing with the install system. A simple installation system tied into the module system which runs a couple of SQL statements is all we need, really. The database side is really not a topic of discussion.

The real problem is that it needs to be built into core, not per module, otherwise it will be inconsistent both on the code side and on the UI side. And we need to consider all possible installation scenarios and figure out good rules about them (should a module clean out its variables when uninstalled? should we have a separate "deinstall" "disable" step? how do we explain this to the user? what about updates? etc).

--
If you have a problem, please search before posting a question.

faxman’s picture

Fully agree!

SQL is just plain old text, that's all.
Why hassle with XML?
Try reading a XML-file which does some things to the DB and then read the plain SQL-script.
By the way, to check if a table exists is just one case statement in SQL.
If exists! Thats all!

Michael

mousse-man’s picture

I think it's a very bad idea unless the module configuration could specify such amenities as which tablespace to use.

Remember that there are free RDBMS now that support tablespaces, and maybe there's an Oracle-based drupal instance if one is crazy enough.

For my part, I use PostgreSQL 8 now and it works quite well except for some very funky inserts into the cache with even more funky inserts for Unicode...

Steven’s picture

Please don't implement this in your module, as it will be inconsistent and hackish. Adrian is working on an install system which does what you describe, but offers a good UI for it and clean code.

--
If you have a problem, please search before posting a question.