Closed (fixed)
Project:
Drush
Component:
SQL
Priority:
Normal
Category:
Feature request
Assigned:
Unassigned
Reporter:
Created:
20 Oct 2011 at 08:56 UTC
Updated:
19 Nov 2011 at 12:00 UTC
I wonder if it is possible tweak sql-sync to include innodb specific options, that significantly improves db import ( e.g. SET unique_checks=0; or SET foreign_key_checks=0;).
Any hooks to be implemented? am I missing something?
Thanks in advance.
Comments
Comment #1
moshe weitzman commentedI think this is a good idea. Any objections? Would be good to get a patch.
Comment #2
greg.1.anderson commentedYeah. I don't know anything at all about innodb, but it looks like it runs under mysql, and therefore uses mysqldump, etc.; I presume that there are cli options that you can pass to mysqldump to get those options to be inserted in your dump file?
We have this duplicate issue here: #518184: Allow Users To Set myqldump Arguments In drushrc.php (for drush sql dump). If that were implemented, it could be used to provide this functionality. Alternately, we could put innodb-specific options to drush sql-dump / sql-sync. A patch would be most welcome.
Comment #3
Remon commenteduhm, it sounds a bit stupid, but I guess that mysqldump is already adding these options to sql dump by default. It could be seen in mysql-specific comments:
Comment #4
greg.1.anderson commentedGood deal; marking as fixed.
Comment #5
alexfisher commentedDB imports still seem to take a long time. E.g. A 12MB dump file is taking me over 800 seconds to import. After doing some research, I found an article that mentions turning-off autocommit:
http://stackoverflow.com/questions/2167522/innodb-takes-over-an-hour-to-import-600mb-file-myisam-in-a-few-minutes/2167641#2167641
After adding a short SQL statement to the top of the dump the import took less than 30 seconds:
SET autocommit=0;In the article above, it did mention outputing the dump using "--opt" and the dump file should import faster, but after debugging Drush I saw it's already using this parameter. It wasn't improved until I added the code above.
Not sure if there are any negative side effects to this, but if not perhaps it would make sense to add to the dumps created by Drush? Someone with more experience have any input on this?
Comment #6
greg.1.anderson commentedI'm not really in favor of post-processing the dump file. Surely there must be a cli option or some other way to get mysqldump to put in the needed options.
Comment #7
jhaskins commentedThe "--no-autocommit" option for mysqldump should accomplish something similar to what was suggested.
Background info on autocommit for those who don't know:
By default, innodb commits changes after every statement (really not necessary when importing a dump). alexfisher's method would only commit once after the entire import is finished (not necessarily a bad thing). The mysqldump option provides a reasonable compromise between the two (commit once per table) that should still greatly improve import performance.
Comment #8
alexfisher commentedTo clarify, I wasn't specifically suggesting post-processing. Just wanted to explain the line of SQL had the desired effect of speeding import up. I think jkaskins suggestion is more along the lines of what I was envisioning.
Comment #9
moshe weitzman commentedcommitted --no-autocommit to 4.x and 5.x. thanks.