By adrian on

Update: an installer finally got into Drupal 5

Over the last few weeks, I have been attempting to simplify the deeper intricacies of installing, and maintaining a drupal system, while allowing for a more centralized/controlled method of providing database scripts for contributed modules.

A good example of the intended result would be quotes and menus in the contrib modules. The modules contain the neccesary SQL to create, and update the tables required for operaton/installation of the module directly within the .module code. This makes them incredibly userfriendly, and simple for even the greenest newbie to install a contributed module without needing to know anything about MySQL, PostgreSQL an ssh connection and yes , even phpmyadmin.

I have managed to develop the framework to such a point, that I have successfully installed the core system (with screenshots), with the desired database prefix, based on the absolute minimum amount of information required to install the system (ie: a successfull database connection, table selection and the optional prefix). I have managed to upgrade Drupal 4.3.0 systems to cvs head using the install system, and I have managed to install and update small demonstration modules for the API.

This has been discussed on the devel lists several times over the few months that I have been subscribed, and also used to penalize drupal whenever there is a review. The general consensus for a long time has been that it is either unneccesary , or that the current setup of .sql dump, and update.php is sufficient for our purposes. The real reason that drupal doesn't have a cleaner install , is that noone has put forward a clean , concise and internally consistent proposal for the functionality.

For the more technically inclined, I have detailed large parts of the api I have developed here (base install/update), and here (configuration/wizard system).

My requirements for an install system are probably not indicative of the actual user/developer base of drupal however. When I started this little project of mine, I simply wanted to fulfill the following:

  1. install drupal core system , for drupal core initially, but available to contrib modules too.
  2. integrated support for different RDBMS systems. I maintain the postgres port, and I would prefer to have a consistent interface with which to add postgres support to the core and / or contrib modules.
  3. database prefix installations without needing to modify the sql dump.
  4. a method to configure settings from user input, with full error checking.
  5. allow contributed modules access to the api , so that all database creation/update happen consistently.
  6. not encumber the modules themselves by including, and parsing the
  7. update the system, based on the changes made since the system was installed. This is directly analogous to the current update.php process, however in a more rigidly defined structure, and capable of updating not only the main system. This requires rudimentary version checking, which at this stage is done using a 'date' , as in update.php.

After this point, I feel the actual requirements for what the default system install script should do, are not too clear. There are quite a few dissenting opinions regarding this on the devel list, but I feel the input of as many people as possible could only help. The framework for developing all this is pretty much done, but nailing down the actual configuration and setup requirements need to be done.

What I would like to know, is how you think the drupal install should work. Several of the users on this site will have used other open source CMSes with install scripts (such as Xoops, and Xaraya and PostNuke). What can Drupal learn from these , and what would you like to see from the install system?

Over the next few days I will be trying to nail down as much of the code as possible, as I am still aiming for a Drupal 4.4.0 inclusion (with any luck, the next drupal will be a lot easier for newbies to use, it is already shaping up as one of our better release imho).

The current workflow , as I have described elsewhere, is the bare mimimum .. 1) ask for a valid database username and password, do not allow the user to continue until the connection has been made successfully. 2) ask the user for the site name and URL .. this is currently ignored, as I feel this is part of a bigger step somewhere. 3) install the database schema.

Comments

irwin’s picture

irwin commented

I have a mini-comment.

Many people have problems configuring their htaccess files and PHP installations, so if the script writes htaccess or suggests solutions to fixing magic_quotes and so forth, that would be very beneficial.

magic_quotes is off: CHECK
database tables are created: CHECK
register_globals is off: CHECK

This is very similar to PostNuke's installation system.

Second, there should be a user prompt for the generation of conf.php, which currently doesn't exist, as far as I recall. A form would be similar to:

Select Database --> [ MySQL, PostgreSQL, etc. ]
Enter Database Host --> [ localhost ]
Enter Database User Name --> [ dbuser ]
Enter Database Password --> [ ***** ]

Database Name --> [ drupal ]
Database Table Prefixes --> [ drupal ]

TWiki and Postnuke both do this, in addition to phpBB.

Third, make sure you tell the user to remove the update.php and install.php scripts.

That's a start....

-- Irwin

Steven’s picture

Steven commented

Drupal CVS works regardless of what register_globals and magic_quotes_gpc is set to.

escoles’s picture

escoles commented

Actually, I had to configure magic_quotes_gpc in order to get Drupal to work properly.

In fact, Drupal generated an error message that told me I needed to do that....

Gábor Hojtsy’s picture

Gábor Hojtsy
he/him
Primary language Hungarian
Location Hungary
commented

But you are surely not using the CVS version :)

adrian’s picture

adrian commented

I have been tirelessly hacking away at the code over the last few days,
and I have uploaded some screenshots to my website and uploaded the required patches/files to test out the system.

Check out this task node for some more details.

irwin’s picture

irwin commented

Do you think it'll make it for the freeze? (If it doesn't, then certainly it'll make a big splash in the next release).

A few observations:

1) Move License agreement to the first thing you do.
2) There's some English wording that could be worked on, I think, to make it more usable. However, that can be talked about later in detail. :) For example, write out "PostgreSQL" in the DB selection. Another one is to make it clearer where the conf.php file is located if you customize its location.

Other than that - I think it looks quite good at this point.

-- Irwin

adrian’s picture

adrian commented

I am concentrating on actual working code , as opposed to 100% syntactically and grammatically correct english.

I am sure that our leet user interface genius' will think of a million ways to improve what I have done.

Also .. i have spoken to dries, and he thinks the license agreement is lame .. so it's gone. Luckily the configuration system is incredibly flexible , takes a couple of minutes to write/test a wizard screen.

this is precisely what I mean when I say i need requirements. noone replied to this post, so I just had to go with what I had seen done before.

Stefan Nagtegaal’s picture

Stefan Nagtegaal commented

First of all, I want to say that this is an excellent usability efford for drupal.
When I tested the patch, everything goes very smooth.. No problems at all, and also the db_prefix works the way it should.

From an administrators point of view, I suggest that you add an option to let people choose if they would like to set the language which is used in the locale.module, because this also configured in conf.php. (Or isn't this neccesary anymore after Gerhard's locale-improvements which are expected?)

adrian’s picture

adrian commented

might enable us to select the language as the first step in the installation, and continue the installation in the users' native language.

that would be the ultimate aim

irwin’s picture

irwin commented

I replied. :) But you're right - I don't think there's a lot of special Drupal-specific requirements that you wouldn't have seen from installations of other CMS's. Generally, it's a question of "Select your database", "Select your tables", up to "Create your password" and "Don't forget to remove this install file."

Good work so far.

-- Irwin

Larry Cannell’s picture

Larry Cannell commented

So this is looking pretty good. I have some input though as role-management is something I have been looking at (in other capacities) for some time.

When you look to roles or groups consider using them in other contexts. Basically, a group is a list of users (ok, maybe a group can include other groups, but in the end you have a list of users). Security is one thing that can use this list. However, there are other applications that need lists of users: email distribution lists, instant messaging buddy lists, and probably others. Wouldn't it be cool if the group that lists the members of my team is the basis for providing me access to a drupal node, is the distribution list I can send email to, and is a server-side buddy list I use with my instant messaging client?

Also, the source of this list can come from elsewhere. In the same sense that user logins can come from say an LDAP directory, so can groups. LDAP is one potential external source for groups. So consider external hooks for this information.

To summarize, consider the following for groups:
- Use for other capabilities besides security, like email
- Leave open the option of hooking into external sources

--
Larry Cannell
larry@cannell.org

bertboerland’s picture

bertboerland commented

Excellent work!!

On a sitenote, please dont make links with the text "click here". click here to see why :-)

Do we need some guidelines for posters? Like the oreilly how to say what. In case such a quideline is wanted, I volunteer to setup a "book".

--
groets

bertb

--
groets
bert boerland

adrian’s picture

adrian commented

major major code cleanup ...

and it took me several hours, but i finally managed to get php error handling under control enough to get the install process working and stopping itself from trying to overwrite the configured site.

still loads of exceptions to handle , but it is getting close now.

atleast the subdir install works, and checking if another system has been installed on the same prefix.

I will hopefully post some more of the code today , but i seem to be coming down with something (all this late night hacking isnt helping any)

Mike@frazierhome.net’s picture

Mike@frazierhome.net commented

I just wrote about some of this on my blog last night (hadn't been to the Drupal site in awhile so I didn't realize this topic was going on). You may want to seriously look at how Mambo is doing just what you propose. In the Mambo install packages, they use XML to provide information about all of the files (you can go over to MamboPortal and download a complex component such as the Events component, and check out the XML file in it to see how "simple" it is). The XML tells what directories every file goes into, what directories need to be created and chmod'd, what sql needs to be run, etc. So installing a module, component, and/or template is as simple as clicking a button on the admin screen to upload a zip file. Its unzipped and executed by Mambo and then the user is given back a success or failure. Also installing MOS itself is very simple as the configuration.php are written out by the application (and you can even maintain key php and css files right from the admin screens).

I have to say that I love the capabilities of Drupal, but from a install perspective where I'm going to turn it over to someone that isn't as technical, I'm opting for Mambo right now was Drupal tends to be too complex for many to maintain / upgrade.

One final note. As mentioned in my blog, I'd love to see (and truely wish to see) a better method for indicating when modules and templates in Drupal have been updated. The web page simply shows what version of the core the module is for. But I've found in the past that there have been changes (bug fixes or enhancements made) to modules and templates, and there is no indication (without downloading and looking at timestamps and inside at the code) that this is the case.

kbahey’s picture

kbahey commented

Hey Adrian

Excellent excellent work! A much needed improvement.

I agree with the other posters on the XML metadata for modules as a way to "instruct" the installation scripts and/or human on what is exactly needed.

I also second the idea of writing a functional conf.php without the need to edit it to specify things like a site's name, db parameters, ...etc. It has to be able to do multi sites as well, which is one of the strong points for Drupal (site1.com.php has different content than site2.com.php, but both use the same codebase, same database and different db prefixes).

Great work. Keep it up!

--
Drupal performance tuning and optimization, hosting, development, and consulting: 2bits.com, Inc. and Twitter at: @2bits
Personal blog: Ba

adrian’s picture

adrian commented

i am using standard php

and a 'registry' hook inside the .install file to specify meta data.

this includes which updates are available for the module.

it's the 'drupal way' , and I don't think it's any less suited for the purpose than xml.

Gábor Hojtsy’s picture

Gábor Hojtsy
he/him
Primary language Hungarian
Location Hungary
commented

There is no need to use XML just because it is cool :) If the install API has some standard functionality (ie. setting permissions, importing DB dumps), then those can be called from the PHP logic. The Drupal way is not to overcomplicate things just because it is so cool and trendy :)

adrian’s picture

adrian commented

Although the code freeze has landed, I feel compelled to finish my plans for the install api. One of it's merits at the moment is that it requires only small, somewhat trivial modifications to drupal core, and is otherwise a completely seperate entity.

With this release available on the task node I have taken to heart what has been said in this forum. I have seperated the wizards into

1) Test for valid php installation
currently it tests for php > 4.1.0 , checks the list of available php extensions for atleast one valid database module (and only allows you to select from valid databases in the next screen) and finally wether the session_handler is set to 'user' (although install.php reverts to file based sessions to avoid errors)
2) Choose a valid config file
I will go into this screen in a bit more detail in a moment.
3) Configure database connection and several important settings that need to go into the config file.
I am aware that this screen seems a lot more confusing than it needs to be, I am confident with the right interface guidance it could be clarified A lot.

Once your site has been configured , future calls to install.php will present you with an administrator login for your site. This login is "almost" garaunteed to work on whichever drupal version you might have running on the site. Along with the use of file based sessions for anything related to the install , It completely rids us of issues such as the 'sessions' table problem from 4.2.0 and the 'check_login' file hack. Only with a valid login will you be allowed to proceed to the update screens. Essentially , I have made the install.php be 'safe' to have lying around in your distribution directory. No need to remember to delete it.

The real black magic in this release is the select config file screen however. When you configure (for a first time, probably the default file , which has been renamed to default.php) a config file, you are given the option of specifying the 'configuration override requirements'. The install script approaches this in the following way. Upon loading the script, it attempts to find a valid and working configuration file. In the case of no configuration file, it continues to a fresh install. When a site has already been configured, it loads a new variable that I have stored in the config files the script creates.

This variable is 'allow_override_config' , and it has 3 values 'any', 'auth' or 'no' (which is also the default). In the case of any, it allows anybody to create a config that will override the current config. When the variable is set to 'auth' , it will ask for the administrator login of the site who's config is going to be overridden. When it is said to 'no' , or not set at all , creating new configs in the presence of an already configured system is not possible

I have spent a considerable effort in trying to catch as many possible errors / configuration traps as I thought practical. And I feel that this install process (while a lot more complex than the previous one) is built to leverage drupal's strengths , and delivers a much smoother user experience, while allowing great power to more advanced users.

I have a more complete set of screenshots at my personal site. Now to clean out my test database , which at one stage had 1 300 tables from all the different prefix installs i had loaded into it during testing.

Rainy Day’s picture

Rainy Day commented

What about the CivicSpace (a Drupal distribution) install Wizard? Works wonderfully well. Perhaps there's something which can be leveraged off of there?

kbahey’s picture

kbahey commented

Good idea. I wonder why there is no reuse of what CivicSpace has done.

Has it been a year (actually 13 months) already since this thread was started? Wow. Time flies.

--

Consulting: 2bits.com
Personal: Baheyeldin.com

--
Drupal performance tuning and optimization, hosting, development, and consulting: 2bits.com, Inc. and Twitter at: @2bits
Personal blog: Ba

axel@debian.linuxrulez.ru’s picture

axel@debian.linuxrulez.ru commented

I think, code for installation need to be separate from work code (of module or core). Or must be way for easy deleting install code after installation. Because installation requires once, after it code needed for installation will useless.

--
Axel

moshe weitzman’s picture

moshe weitzman commented

the installer discusdsed here will soon be used to install and update your core and contrib modules. you will not be deleting it.

adrian’s picture

adrian commented

install code is completely seperate from the rest of the drupal code, although the code itself leverages as much of drupal that is available without requiring to set up the database connection

Also .. while an engine like postnuke can easily have a deletable install script, drupal's multi site hosting nature requires an install script to be available for any site that needs to be hosted on said directory.

For instance .. if you have 200 sites hosted in one directory (in a hosting situation), you will only be able to run the install ONCE for the first site and not be able to install any additional new systems in that directory.

dmitrig01’s picture

dmitrig01 commented

I like how civicspace does their install. (Didn't you write that?!)
Maybe the user could be asked for their hosting tech support e-mail address, give their username and password, and require the hosting company to respond with the following format:

DBTYPE: database type
DBUSER: database username
DBPASS: database password
DBNAME: database name
DBHOST: database host

That would make it very easy for the end user.

dgtlmoon’s picture

dgtlmoon commented

Grouping of modules or requirement matching of modules would be a big help, so many times newbies have gone to setup say - somethign in ecommerce that requires a couple of other modules but they didnt go into the system in the right order and it broke their whole site

kbahey’s picture

kbahey commented

An installer will be in 4.8

See details here http://drupal.org/node/73537
--
Drupal development and customization: 2bits.com
Personal: Baheyeldin.com

--
Drupal performance tuning and optimization, hosting, development, and consulting: 2bits.com, Inc. and Twitter at: @2bits
Personal blog: Ba