Avoid Server Timeouts (perhaps perform backups in smaller pieces)

The module will not break up the import into smaller steps. I'd like to tackle this at some point, but the danger of running a partial import makes it a formidable challenge. Specifically, if the module imports only part of a backup file, it could leave the database in a state which prevents drupal from bootstrapping and then the rest of the import would fail.

Until I figure out a clean and safe way of doing partial imports, try using phpMyAdmin for restores. Backup and Migrate backup files should work fine in phpMyAdmin, and newer versions of that tool allow for partial import if there is a timeout.

Hope this helps
R

I'm setting this as a feature request, but it's going to have to be low on my priority list given the complexity of the task and the number of significantly lower hanging fruit awaiting my attention.

I'm guessing that the original author doesn't have access to the PHP.ini to override max_execution_time for PHP. Mine was originally set to 30 seconds and even though the DB wasn't that big (15Meg) it exceeded the time limit. Luckily I have the auth to go into .htaccess and override it

# PHP 5, Apache 1 and 2.
<IfModule mod_php5.c>
  # added new max_execution_time for MySQL dump 2008-0110 PPH  
  php_value max_execution_time              120
</IfModule>

I was wondering if the same effect could be achieved programmatically. If so it would be great if the admin could set a max_execution_time value via the modules interface. The main advantage beyond giving the power to those who can not update php.ini or a.htaccess would be that the increased limit would not be available to other modules which might have a problem and exceed normal execution time.

In conjunction with this it would be great to have a display that told us the execution time of a dump after it completes. This would enable us to fine tune our settings without accidentally using a value that was to large and might let other code run away.

I'll look into what can be done programatically and how that is affected by server lockdown. I should be able to use ini_set to raise php defaults for those users who are not in safe mode, but cannot (or don't want to) adjust their php settings themselves.

Thanks for the feedback.
R

Slightly improved title.

FWIW, I consider this a low priority. If your database is so big or the servers are so slow that it exceeds the max execution time you should look at alternate backup mechanisms.

I have added the increase in timeout via ini_set to the version 2 of the module. This should help those who are not in safe mode, but cannot edit their php settings for one reason or another.

Future improvements to help timeouts include:
Drush compatibility (medium term)
Batch compatibility for importing in smaller pieces (very long term, 6.x only)

Hope this helps

Subscribing.

+1 for a new feature batch importing in smaller pieces.

In the meantime the following alternative can handle very large databases such as 500MB or 1GB: BigDump http://drupal.org/node/43024

ronan: Maybe part of the BigDump's script can be use in Backup and Migrate? BigDump's code is GNU General Public License: http://www.ozerov.de/bigdump.php

Hope this helps,

Thanks for the info, I'll check out BigDump (tee-hee) and see if there's any knowledge to be gained their. The issue I have to deal with which BigDump doesn't is that my module needs drupal to bootstrap in order to work, so it can't leave the db in an indeterminate state. I'll probably solve this in 1 or both of the following ways:
1) Only allow partial/staggered restores to other databases (restoring to other dbs will soon be supported)
2) Do the restore to copies of the database tables and then once all is done, rename all of the tables to the real names.

The second one is riskier and more complex, but doesn't require a second copy of Drupal to be running on the same server (or at least with access to the same dbms). It would also allow for a sort of primitive 'undo' function.

I'd love to hear everybody else's ideas on this.