I am trying to create an installation profile that includes basic site pages (a home page, some standard content pages, etc.) and settings (new roles, preset permission entries, etc.), but I cannot figure out how that would work.

I have successfully populated the variable table using variable_set(), but whenever I try to use db_query() to add roles, permissions, and nodes (extrapolated out from the examples presented in http://drupal.org/node/67921), I end up with a site that does not work. After putting in the database name, user and password, the setup routine runs, but I end up with a blank page. A subsequent load of the base site yields the "Welcome to your new Drupal site" page, but attempts to create the super account result in a page with the "Your super account has been created, and you can change your password below..." entry, but an "Access Denied" message in the space where the password reset should be. Subsequent attempts to log in using the temporary password provided on that page give the error that the account is blocked, and attempts to change the password in MySQL are unsuccessful.

Moreover, an examination of the underlying MySQL tables doesn't show any of the records I want created. The code I am using looks like this:

  /** ROLES AND PERMISSIONS **/
  // Site Admin
  db_query("INSERT INTO role (rid, name) VALUES (3, 'Site Admin')");

  // Add user 1 to authenticated and admin roles
  db_query("INSERT INTO {users_roles} VALUES (1, 2)");
  db_query("INSERT INTO {users_roles} VALUES (1, 3)");

I suspect that I should use some other drupal hook (along the lines of variable_set) to add the pages, users, roles, and permissions, but I haven't got a clue where to start. A brief examination of the API doesn't show me an obvious set of commands to use. Can anyone steer me?

TIA,
David

Comments

sunfish62’s picture

Hello? Does anyone have any suggestions? I can't believe I'm the only one who's thought of doing this...

sunfish62’s picture

My own solution has been to set up my test site the way I want it, dump the entire database to an sql file, and db_query the resulting information in the .profile. It's a little brute force, but it seems to work reasonably well (at least once I realized that the website directory structure and content can get changed when you change a theme setting, and you need those new files and folders...).

sunfish62’s picture

For anyone else interested, here is what I came up with to take a MySQL dump file and install it during the setup routine. This code would be put in the profile_final function. I have a few comments in here about mods that I hope to do.

  $lines = file('./profiles/myprofile/myoutput.sql'); // set to the .sql file
  $str = "";
  $ctr = $fail = 0;
  foreach ($lines as $line) {
    if(substr($line, 0, 2) == "--") { continue; } // skip comment lines
    $str .= trim($line);
    if(substr($str, -1, 1) == ";") { // MySQL end of command reached
      $str = substr($str, 0, strlen(trim($str)) - 1); // remove ";"
      if(strlen($str) > 0) {
        // Note: modify to omit table structure, or omit structure in dump
        // Note: $str should be tested for validity; some entries seem to be invalid in dump
        (mysql_query($str) ? $ctr++ : $fail++); // Could prob use db_query
        $str = "";
      }
    }
  }

I have noticed that for some reason this interrupts the final stage of the install process, and instead of the "Welcome to your New site" message, you end up with an empty page. But the cloned site is actually created; I would love to know what I could do differently to fix this.

Perhaps others will benefit from this...

David

konsumer’s picture

the return from your profile_final() is what creates the text, so you can return a parsed template or just some inline text)

You can also use drupal_set_message() to add messages (with no return from your profile_final()), and it will show them on whatever is your front page.

ac_vivi’s picture

Maybe it's too difficult or too much to ask, but do you know any other way of making a copy of the site and create the "myownprofile.profile" with general characteristics for installing it in other sites? is it the only way by programming?

I hope someone will answer
Vi

sunfish62’s picture

Vi--

Based on my own experience and the little I've seen on these forums on the topic, there aren't any easy ways for cloning a site. I have looked into creating a backup of my site using the Backup module, but that module relies on access to server processes that my host doesn't provide, and so the backup files from that module are always empty for me.

The code I listed earlier is as close as I've been able to get to clone a site, and it generally works.

EXCEPT FOR THE FACT THAT certain configuration settings (like the suppression of a login block) don't take (see http://drupal.org/node/154393). You have to re-select the settings (even though they are entered in the load).

EXCEPT FOR THE FACT THAT there's no clearcut outline of *which* tables actually contain a site's content, and which contain tracking or other behind the scenes administrative data. So, you have to experiment with the dumped tables from your site until you find the right combination. (Oh, and beware that some of those tables will cause the installation profile to barf altogether).

EXCEPT FOR THE FACT THAT the installation profiles FAIL to access your data tables the first time you ask to create your new site using the onscreen profile (see http://drupal.org/node/154388), making the installation routine an obscure and impractical solution for general distribution--since you have to tell everyone that in order for your installation to work, when the installation routine pops up with what appears to be the opening screen again, RE-SELECT the profile again and press OKAY, and the install will work on the second try.

Until someone actually steps up and fixes some of the problems with the installation routines (I am not skilled enough with the Drupal code to be able to this myself), you're stuck with whatever you can get. For what it's worth (and I'll admit it's not worth much, when you come right down to it), here's a complete "mysite" profile that would take your dumped database and use it to clone your site.

<?php

/*
  * MySite profile
  * To use this code to create a MySite profile, 
  * 1 - create /profiles/mysite
  * 2 – copy this text into mysite.profile
  * 3 - create an sql dump of the site you want to clone "myoutput.sql", omitting structure, but including full markup. 
  *      Use trial and error to determine the necessary table dumps to recreate your site.
*/

/**
 * Return an array of the modules to be enabled when this profile is installed.
 *
 * @return
 *  An array of modules to be enabled.
 */
function mysite_profile_modules() {
  // Add your own modules to this list
  return array('block', 'color', 'comment', 'filter', 'help', 'menu', 'node', 'system', 'taxonomy', 'user', 'watchdog');
}

/**
 * Return a description of the profile for the initial installation screen.
 *
 * @return
 *   An array with keys 'name' and 'description' describing this profile.
 */
function mysite_profile_details() {
  return array(
    'name' => 'My Site',
    'description' => 'Select this profile to install your site from a dumped copy.'
  );
}

/**
 * Perform any final installation tasks for this profile.
 * Dump the content of your site into myoutput.sql
 * @return
 *   An optional HTML string to display to the user on the final installation
 *   screen.
 */
function mysite_profile_final() {
  $lines = file('./profiles/mysite/myoutput.sql');
  $str = "";
  $ctr = $fail = 0;
  foreach ($lines as $line) {
    if(substr($line, 0, 2) == "--") { continue; } // skip comment lines
    $str .= trim($line);
    if(substr($str, -1, 1) == ";") { // MySQL end of command reached
      $str = substr($str, 0, strlen(trim($str)) - 1); // remove ";"
      if(strlen($str) > 0) {
        // Note: modify to omit table structure, or omit structure in dump
        // Note: $str should be tested for validity; some entries seem to be invalid in dump
        if(substr($line, 0, 6) == "CREATE") { continue; } // skip CREATE lines
        (mysql_query($str) ? $ctr++ : $fail++); // Should prob use db_query
        $str = "";
      }
    }
  }

}
Kuldip Gohil’s picture

i have tried above profile code into my installation profile but it.. is not work properly main 2 thing i find is there is no entry in user table and another thing is required module is not enabled into the new site.. in system table also status value is '0' instead of '1'

Hi Please Help Me into This...
Thanks..
kuldip

Kuldip Gohil’s picture

i am not geting user table value and module is not enabled into new site
so please help me when i use to import .sql file using profile

sunfish62’s picture

First issue: user values:

So, did you set up a demo version of your site (including your users), dump that MySQL database to a text file and use that for the new profile source? Are you sure you included all the necessary tables from the database? To be honest, I haven't seen a list of the tables that really hold your data, but it should be moderately clear which tables contain administrative/session data, and which don't. At the very least, did you include the variable, users, user_profiles, & permissions tables in your dump?

Second issue: module not active:
Did you include your modules in the yourprofile_profile_modules routine? My code only applies after the modules you want have been installed by the modules routines.

David

Kuldip Gohil’s picture

Hi David
First of all Thanks For Giving Me Reply.
Now I am Explaining You what i have done

1)First I have download drupal from drupal.org and install it
2)Then Done Some changes create 2-3 users and put downloded module into sites/all/modules
3)take the backup of the database using phpmyadmin into databasename.sql file
4)take new fresh drupal and then put it into one folder and create empty database with name "instprofile"
5)take copy of your profile and it put into profiles/mysite/mysite.profile
5)Only make changes in database file name with my sql file name into mysite.profile
6)THen go through url : localhost/instprofile
7)Then its asking for select installation profile Then i have selected "mysite" profile from it
8)Drupal is installed , my page content is comes , user is not created , user table has only one row, and module is not enabled

So I want to clear you one thing that i have not done any changes into your profile code except the sql file name.

another my query is : Does This Way of Profile Is Work With Multisite concept..

Waiting For Your Reply...

Thanks

Kuldip Gohil

sunfish62’s picture

First things first: you keep mentioning TWO problems.

The module thing sounds like trouble with your steps 5:

5)take copy of your profile and it put into profiles/mysite/mysite.profile
5)Only make changes in database file name with my sql file name into mysite.profile

You don't mention changing the list of modules to include your special module. You have to include a list of all the modules you want Drupal to install in the modules code. Did you do that?

The user thing *may* have to do with the bug I outlined in http://drupal.org/node/154388, where it looks like the installation resets, but actually doesn't completely. Take a look at the patch mentioned there, and see whether it fixes that problem...

Kuldip Gohil’s picture

i have not chek further but i have done mistake only gives same number (5) to the point

i have need to add my module in array of module in profile???

sunfish62’s picture

If you are using any modules in addition to the modules that are listed in mysite_profile_modules(), you MUST ADD THEM THERE. Otherwise, they won't be activated (which is what you described). So if you want to use Jan's node (janode), your list will look like:

function mysite_profile_modules() {
  // Add your own modules to this list
  return array('block', 'color', 'comment', 'filter', 'help', 'menu', 'node', 'system', 'taxonomy', 'user', 'watchdog', 'janode');
}

HTH,
David

Kuldip Gohil’s picture

hi David

i have put my modules in

function mysite_profile_modules() {
// Add your own modules to this list
return array('block', 'color', 'comment', 'filter', 'help', 'menu', 'node', 'system', 'taxonomy', 'user', 'watchdog', 'event','banner');
}

my module is "event" and "banner" .....

i have already done the chages and cofiguration of both of the modules in site and then take backup into .sql file and after that i have done installation of site using myprofile .. so settings of the event module and banner modules comes into my new site.... but again i have changed some settings in event module then again take backup of the database and put it into mysite profile and then installl then settings not comes... i have tried it many times

--- Finally I want to know that what is the way of changes into profile (means changes into configuration)
---If i want to change is some configuration then again take backup and then put .sql file in mysite profile is this right way or any other way

sunfish62’s picture

I want to note at this point that my authority in the drupal realm is essentially zero. I have offered my experience in trying to clone a basic drupal website to another location, and the code I presented is at best considered Beta quality. To be honest, I was hoping that by putting my code up here, I might stimulate a dialog with more experienced drupal developers on methods to achieve the goal of cloning a site, without success.

I haven't got a clue what might be wrong on your site, but you might want to compare the MySQL content of the various tables in both the original version of your site and the cloned copy. Ideally, these should be the same, and if they are not, then there is something else going wrong. If this IS the case, I'd start by seeing if the SQL is valid--use phpMyAdmin to load the SQL directly and see whether any errors crop up.

If the SQL is valid, perhaps the problem I mentioned before is cropping up. Does the installation routine run EXACTLY as it is supposed to, or does it seem to "re-start"?

HTH,
David

Kuldip Gohil’s picture

installation process working ok. but data of the sql file not comes properly sometimes.

I have purpose to use this profile same to you like cloning . i have to install same site on other places but some time i have also change to it's features then i have need to create the site then take backup and put it in place of old file am i right on this way of doing updation??

sunfish62’s picture

Honestly, if it works for you, then it's as right as this solution can be. As I pointed out earlier, there doesn't seem to be any preset clone operation in the Drupal world. This code was my best attempt to create this. What you describe is exactly how I have attempted to use this solution. I will grant you that it is not a pretty process, but it's what I could come up with.

If you have the need for flavors of the same cloned site, you might be able to make use of some sort of versioning software to allow you to manage the different sites. This suggestion is made based on an exceptionally rudimentary understanding of the capabilities of such software, so you're on your own there. Good luck!

konsumer’s picture

My import function for SQL won't run out of memory. It assumes that lines end with ";", so it won't work if they don't. This should be ok with phpmyadmin dumps.

/**
 * this will import a very large sql dump
 * @param string $filename the full path to the SQL file
 * like dirname(__FILE__).'/site.sql'
 * @return integer the number of queries executed
 */

function import_sql($filename){
  $buffer='';
  $count=0;
  $handle = @fopen($filename, "r");
  if ($handle) {
      while (!feof($handle)) {
          $line = fgets($handle, 4096);
          $buffer.=$line;
          if(preg_match('|;$|', $line)){
            $count++;
            _db_query($buffer);
            $buffer='';
          }
      }
      fclose($handle);
  }
  return $count;
}

If you are already going the mysql dump route, then you might as well just import all your settings, modules, and everything (it's a lot easier then sorting through things and it will allow you to just import everything)

I made a simple install profile using this principal:
http://drupal.org/project/sql

If you prefer to be lots more selective (and cross-DB) you can get modules, variables, and content-types for install profiles pretty easily by using a running site:


// put these in a PHP node somewhere
$variables = variable_init();

$t = node_get_types('types', NULL, TRUE);
foreach($t as $type_name=>$type){
  $node_types[$type_name]=(array)$type;
}

$m = system_modules();
foreach($m['validation_modules']['#value'] as $modulename=>$moduledata){
  if($moduledata->status){
    $modules[]=$modulename;
  }
}

print '<pre>';
print '$variables='.var_export($variables,TRUE).";\n";
print '$types='.var_export($node_types,TRUE).";\n";
print '$modules='.var_export($modules,TRUE).";\n";
print '</pre>';


You can use these outputs (cut and paste) in your own profile_modules(), and profile_final() functions to add them back.

You can use one of these to utilize this principal, if you need something more advanced:
http://drupal.org/project/install_profile_api
http://drupal.org/project/profile_generator

sunfish62’s picture

konsumer--

Thanks for offering the updated code. FWIW, my version 5 profile assumed that one had dumped the entire database file--including settings as well as module-specific tables--with the exception of housekeeping tables. My goal was to clone a site--a goal that even with this code was unmet, since certain aspects of the resulting site did not perform as expected.

The issue about modules had more to do with ensuring that the modules were installed and activated--not with the content for those modules (which my SQL load would have managed).

Thanks.
David

konsumer’s picture

The system table handles the modules, so using my code (the profile I made or the SQL example above,) if you do a full dump of every table, it will duplicate the site. I have tested this on several deployed fresh installs. It's perfect copy of the original, including modules, content-types, views, all content, variables, housekeeping, whatever. If the module is activated and installed on your development server, then it will be on the deployment server. This is not the most efficient, as the drupal installer creates a bunch of tables, then the SQL import deletes them and recreates them) but it will make a solid install that definitely works, exactly the same as the dev server.

It just adds a nice UI to making an exact site copy (sometimes I don't want to have to install phpmyadmin, or login to the server's cpanel and brave upload timeouts, or SSH and import sql scripts that you uploaded.) You can make a polished system, that seems to be "all setup" that you can hand off to the client with reasonable assurance that it will run, without you having to do anything. It's also great for backups that can be deployed by even a novice.

If you need upward-compatibility, this works great, just create the site, install the "Update status" module to get all new versions, and run Drupal's update.php to update to the newest versions (which uses each module's understanding of the database structure)

I think the profile generators are far superior to this method, if you need to make an install profile that will run on different versions, or have more control over what's going on. If you generate an install profile with the works enabled, and export your content to TSV, including nid's (and use node import to get the data back into the deployed site), there is little else that is needed to migrate (views come to mind, there are probably a few more)

I think the main downside to this is that you have to do alot of messing with things to ensure that it works right, and major API changes will render the cross-versioness of things stupid. It seems to me like most people are going to just copy their existing site to a server and have it work. The pure-SQL method works better for this (as long as both server's are running mysql, and the tables have the same name prefix, or are replaced with {table} tokens.) There are also pgsql converters around, and my script should work ok with their outputted stuff, as long as the actual outputted sql works (pgsql uses ";" too, after all)

sunfish62’s picture

I've run into some issues with loading the housekeeping files, which is why I've tried to omit them in dumps. It probably had something to do with buffer sizes, or PHP size limits on my host (1and1). Your approach, heavy-handed though it may be, is probably more reliable in the long run!

David

Stevenj007’s picture

Just saw this thread and I agree that the database export path is a a lot easier to maintain than the install profiles for scenarios where you can start with a clean slate. Konsumer, what does the profile script do, and why not just do a database restore?

Stevenj007’s picture

I'm guessing the answer to this is that just restoring the database would leave the php files on disk unconfigured and it is best to let the setup process run then overwrite the database.

yoloo’s picture

Here is site.profile file that loads whole sql dump during installation. The dump stored in dump.sql and should be placed in the same directory as .profile file


<?php

    define('PROFILE_NAME', 'site');
    define('PROFILE_DESCRIPTION', 'Custom site profile');
    function site_profile_modules() {
    return array();
    }

    function site_profile_details() {
    return array(
    'name' => PROFILE_NAME,
    'description' => PROFILE_DESCRIPTION,
    );
}

    function site_form_alter(&$form, $form_state, $form_id) { 
    if ($form_id == 'install_configure') {
    $form['#submit'][] = 'site_form_submit';
    }
}

    function site_form_submit($form, &$form_state) {
    $dump_file = dirname(__FILE__) . '/dump.sql';
    $success = import_dump($dump_file);
    if (!$success) {
    return;
    }
	
    variable_set('site_name', $form_state['values']['site_name']);
    variable_set('site_mail', $form_state['values']['site_mail']);
    variable_set('date_default_timezone', $form_state['values']['date_default_timezone']);
    variable_set('clean_url', $form_state['values']['clean_url']);
    variable_set('update_status_module', $form_state['values']['update_status_module']);
    variable_del('file_directory_temp');  
    $name = $form_state['values']['account']['name'];
    $pass = $form_state['values']['account']['pass'];
    $mail = $form_state['values']['account']['mail'];
    db_query("UPDATE {users} SET name = '%s', pass = MD5('%s'), mail = '%s' WHERE uid = 1", $name, $pass, $mail);
    user_authenticate(array('name' => $name, 'pass' => $pass));
    drupal_goto('<front>');
}

    function import_dump($filename) {
    if (!file_exists($filename) || !($fp = fopen($filename, 'r'))) {
    drupal_set_message(t('Unable to open dump %filename.', array('%filename' => $filename)), 'error');
    return FALSE;
    }

    foreach (list_tables() as $table) {
    db_query("DROP TABLE %s", $table);
    }
	
    $success = TRUE;
    $query = '';
    $new_line = TRUE;

     while (!feof($fp)) {
     $data = fgets($fp);
     if ($data === FALSE) {
     break;
     }
	 
     if ($new_line && ($data == "\n" || !strncmp($data, '--', 2) || !strncmp($data, '#', 1))) {
     continue;
     }

     $query .= $data;
     $len = strlen($data);
     if ($data[$len - 1] == "\n") {
     if ($data[$len - 2] == ';') {
     if (!_db_query($query, FALSE)) {
     $success = FALSE;
     }
     $query = '';
     }
     $new_line = TRUE;
     } else {
     $new_line = FALSE;
     }
}
     fclose($fp);

     if (!$success) {
     drupal_set_message(t('An error occured when importing the file %filename.', array('%filename' => $filename)), 'error');
     }
     return $success;
     }

     function list_tables() {
     global $db_prefix;
     $tables = array();
     if (is_array($db_prefix)) {
     $rx = '/^' . implode('|', array_filter($db_prefix)) . '/';
     }
     else if ($db_prefix != '') {
     $rx = '/^' . $db_prefix . '/';
     }

     switch ($GLOBALS['db_type']) {
     case 'mysql':
     case 'mysqli':
     $result = db_query("SHOW TABLES");
     break;
     case 'pgsql':
     $result = db_query("SELECT table_name FROM information_schema.tables WHERE table_schema = '%s'", 'public');
     break;
     }

     while ($table = db_fetch_array($result)) {
     $table = reset($table);
     if (is_array($db_prefix)) {
     if (preg_match($rx, $table, $matches)) {
     $table_prefix = $matches[0];
     $plain_table = substr($table, strlen($table_prefix));
     if ($db_prefix[$plain_table] == $table_prefix || $db_prefix['default'] == $table_prefix) {
     $tables[] = $table;
     }
   }
}
     else if ($db_prefix != '') {
     if (preg_match($rx, $table)) {
     $tables[] = $table;
     }
}
     else {
     $tables[] = $table;
     }
}
     return $tables;
}