It would be very useful to have a table duplication/copy method available.

A prime usecase is #323477: Increase simpletest speed by running on a simplified profile.

The key thing to watch out for is the users table which has a uid = 0 which causes problems with auto incrementing.

Comments

boombatower’s picture

boombatower commented

Here is an excerpt from how I accomplished this in the other patch. (This is not intended to be finished code, but at the time I had no better way to do it)

function simpletest_environment_copy_data($base, $prefix) {
  $destination_tables = db_find_tables($prefix . '%');

  foreach ($destination_tables as $destination_table) {
    $table = str_replace($prefix, '', $destination_table);
    $source_table = $base . $table;

    db_query('TRUNCATE TABLE ' . $destination_table);
    if ($table == 'users') {
      // Due to uid 0 the data connot be copied like the rest of tables.
      db_query('INSERT INTO ' . $destination_table . ' SELECT * FROM ' . $source_table . ' WHERE uid = 0');
      db_query('UPDATE ' . $destination_table . ' SET uid = uid - 1');
      db_query('INSERT INTO ' . $destination_table . ' SELECT * FROM ' . $source_table . ' WHERE uid > 0');
      continue;
    }
    db_query('INSERT INTO ' . $destination_table . ' SELECT * FROM ' . $source_table);
  }
}

This function assumes the tables schema has been copied. We probably should have either two methods or parameterize it to allow for the creation of just the schema and or data as well. In most cases we should just be able to create schema using schema API...so maybe just data?

damien tournoud’s picture

damien tournoud commented

Copying a whole table might be useful (it certainly is), but I think we should rather implement "whole schema" (in the Schema API sense) copying. This is the only way we could significantly speed-up tests on SQLite (by simply copying the whole database file).

Crell’s picture

Crell commented

Table-level cloning is also useful if we ever get around to implementing optimized databases for selected tables, such as system or the registry tables. For that, we'd need a clean "clone this table from this connection to this connection" operation.

boombatower’s picture

boombatower commented

For a number of cases in SimpleTest I need a clone table that copies data (preferably not schema as well), but I should write it differently. The main two things I am trying to accomplish are:
1) Tuning simpletest by creating skeleton databases and re-using them. (schema only initially)
2) Copying from a base database, could be a plain drupal install, or backed up version of live site for configuration testing (data only).

Currently I do all schema creation using the built in schema handling code (and seems sufficient), only time you may want raw copy of schema is if you do not know what the schema of the table is, which is not the problem in my case.

So if we could get a consistent copy data function that would be great.

In general, copy database functionality would work, but would be less tuned (depending on implementation) then only copying data. If we do things like copy database files I have no idea, but if done in queries then I would assume data only would be faster.

As a note: the code I have works, but as suggested (and I thought about) it should be in db layer.

Crell’s picture

Crell commented
Version: 7.x-dev » 8.x-dev

More robust connection handling will have to wait, sadly.

rfay’s picture

rfay
he,him
Primary language English
Location Palisade, CO, USA
commented

subscribe

neilnz’s picture

neilnz commented

Database duplication on Postgres (in a controlled environment) can be optimised using template DBs.

See http://www.postgresql.org/docs/8.3/interactive/manage-ag-templatedbs.html

Every DB in postgres is actually based on a template, but the default template is empty.

jhedstrom’s picture

jhedstrom
Primary language English
Location Portland, OR
commented
Version: 8.0.x-dev » 8.1.x-dev
Issue summary: View changes

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.0-beta1 was released on March 2, 2016, which means new developments and disruptive changes should now be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

smustgrave’s picture

smustgrave commented
Status: Active » Postponed (maintainer needs more info)
Issue tags: +stale-issue-cleanup

Thank you for sharing your idea for improving Drupal.

We are working to decide if this proposal meets the Criteria for evaluating proposed changes. There hasn't been any discussion here for over 8 years which suggests that this has either been implemented or there is no community support. Your thoughts on this will allow a decision to be made.

Since we need more information to move forward with this issue, the status is now Postponed (maintainer needs more info). If we don't receive additional information to help with the issue, it may be closed after three months.

Thanks!

smustgrave’s picture

smustgrave commented
Status: Postponed (maintainer needs more info) » Closed (outdated)

Since there's been no follow up and as a feature request going to close out. Can always be re-opened

Thanks all!

Now that this issue is closed, please review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, please credit people who helped resolve this issue.