I setup a new sql dump today and had a couple minor glitches. It ended happily, but I think we have some improvements.

  1. !dump expects to be fed a full filename. I think it should optionally be a directory. When it is a directory, the name of the file is the one we usually construct when using the temp dir flow. The problem with specifying a filename like dump.sql is that you can start overwriting files from different source aliases when they all sync back down to developers local !dump directory.
  2. i could not get !drush to be used on a remote system. it was just using `php`. i had to specify !drush-script and then it used the specified path.
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

greg.1.anderson’s picture

Assigned: Unassigned » greg.1.anderson
greg.1.anderson’s picture

Based on the above, I will prepare a patch with the following changes:

1a. Allow the sql sync temp directory to be specified. I think it would be better to do this with a 'temp-dir' option, rather than relying on an ending '/' in !dump to distinguish between files and directories.
1b. Use a lock file during remote dumps to prevent problems caused by two different developers pulling down the same remote site at the same time. (I'm not sure about bothering to protect remote imports, though, as it is never good if two developers try to push to the same site at the same time...)
2. Bugfix for implied !drush.

Related to 1b, I know this isn't the situation that you reported, but it was something I thought of when reading your description, and figured it should be handled. While I do understand your request (1a), I'm not sure that I understand the specific situation that motivated it. Do you have two different live or staging sites that you are syncing with a single dev site, so the dev site "changes identity" based on who it is synced with last? (Odd...) Or do you have two different dev sites that both have the same exact '!dump' => '/path/dump.sql' specification? If the later, the best fix would be to use different dump filenames for each dev site, which avoids overwriting and also allows for more efficient use of the cache option and rsync optimizations. If you meant something else entirely, please let me know.

(A quick heads up -- I was sent to the ER for a minor injury yesterday, a time hit my already-crowded schedule really didn't need. I'm going to be less responsive on issues at least through the end of the calendar year, but will submit patches as soon as I can. I'll still monitor the issue queue so that I can assign sitealias / sync issues to myself; feel free to either reassign my issues to someone else, or on the flip side, go ahead and assign new issues directly to me for eventual processing if you so desire.)

moshe weitzman’s picture

'Or do you have two different dev sites that both have the same exact '!dump' => '/path/dump.sql' specification?'. Yes, this is what i was doing. I was planning to have all the dumps from all my clients to go into the same dir. And the names of those dumps would be generated based on the source alias. Now I se your point about cache and rsync optimization. I had not thought this all the way through.

I'm sorry to hear about your injury, and your busy schedule. We'll happily welcome you back when your load lightens. No worries.

greg.1.anderson’s picture

Status: Active » Needs review
FileSize
716 bytes

I had just a bit of spare time this evening to whip up a patch that addresses only issue #2 above.

When I first coded this up, DRUSH_COMMAND was defined to $GLOBALS['argv'][0], and I got the drush directory via dirname(DRUSH_COMMAND). Now, DRUSH_COMMAND is looked up via more complex logic, and dirname(DRUSH_COMMAND) no longer returns the right thing. This patch changes the drush directory lookup to dirname($GLOBALS['argv'][0]), which should put it back the way it was the last time I tested this function thoroughly.

This patch seems to work on my system, but please try it on different environments and see if it's okay for you too. A good way to test it is with drush sa --with-optional current, which will show you the default value for !drush. Of course, drush -s -r /srv/www/drupal/ -l mydrupal.com sync \!drush stage:\!drush is also a good test.

moshe weitzman’s picture

Status: Needs review » Reviewed & tested by the community

This might have changed yet again? Unless someone objects, I will probably commit this. Sorry that I overlooked this.

greg.1.anderson’s picture

I just gave this a quick test, and it still applies to HEAD and still works. Previously, it was dependent on other code (e.g. #586466: Drush.php is no longer directly executable (can interfere with backend invoke)), but currently is independent of code changes made there, so it is safe to commit.

(p.s. just launched http://dickensfair.com, a Drupal rework of an existing static html site. Getting good reviews so far. I should have time to get to my drush bugs soon--after I finish our family Christmas letter and such.)

(p.p.s. please also take a look at #648280: drush should find the drupal root correctly, even when symlinks are in use., which is not a duplicate of #599698: Load a drushrc.php from cwd but rather provides a useful facility with only a small code modification.)

moshe weitzman’s picture

committed. thx.

moshe weitzman’s picture

Status: Reviewed & tested by the community » Fixed
greg.1.anderson’s picture

Status: Fixed » Active

Thanks for the commit. I still owe you a patch for issue #1. If !dump is a directory, I will compose the filename from the alias name, but will not use the temporary file facility, so the filename comes out the same every time. With this addition, you will be able to set $options['!dump'] just once in a drushrc.php file, and not have to set !dump in each alias. !dump in a site alias will take precedence over !dump set in the context of a drushrc.php file. I may also support '!dump-local' and '!dump-remote', in case you want to use temporary files on the local machine, but follow a standard for the remote machines.

I'm going to work on #628996: Concurrently execute a single drush command on multiple sites first, but I should get to this pretty soon. Re-opening.

dman’s picture

( directed here from #670994: Site alias documentation )
I was sorta expecting that the dump filename would naturally be timestamped. At least that's the approach I'd always taken in the past when migrating sites.
Make backup db dump, named with date and source. transfer db dump. deploy db dump.

If this one is to be just a really ephemeral 'temp' file that the sysem just uses for temporary holding, that's probably not applicable, so never mind. Just a random thought.

greg.1.anderson’s picture

You are correct; if the file is a temporary file, then it will quickly be deleted. I suppose there might be some benefit to timestamping temporary files in case some bug prevented one from being deleted.

For dumps that are not temporary, timestamps are not appropriate; the filename must remain the same in order to take advantage of rsync's send-only-the-file-differences feature.

Of course, for backups, timestamps are entirely appropriate. Auto-timestamping might be a decent sql dump feature.

greg.1.anderson’s picture

Priority: Normal » Minor
Issue tags: +drush-3.0

Bump... proposing for 3.0, but optional. I'll try to get this in, but feel free to tag without it.

greg.1.anderson’s picture

Status: Active » Needs review
FileSize
3.12 KB

My most-put-off issue, finally done.

This patch allows you to specify a --dump-dir to sql-sync. If you do, persistent sql dump files named after the remote machine and database name will be stored there. Otherwise, temporary files are used as before.

moshe weitzman’s picture

Status: Needs review » Reviewed & tested by the community

typo: precidence

otherwise, rtbc

greg.1.anderson’s picture

Status: Reviewed & tested by the community » Fixed

Committed.

Status: Fixed » Closed (fixed)
Issue tags: -drush-3.0

Automatically closed -- issue fixed for 2 weeks with no activity.