Windows installer uses incompatible tar

Interestingly, I have run into exactly the same issue (on exactly the same day).

The root of the problem is that xNIX and windows treat single quotes in command line differently, so the easiest fix is simply use only double quotes when calling tar from drush on xNIX and windows. Attached file is a patch for archive-dump command.

Another issue is that MSI package doesn't include mysqldump which is required by archive-dump.

I tested that after applying this patch and adding mysqldump drush can create site archives on windows w/o any problems.

Please use drush_escapeshellarg() here. It is designed *specifically* for this purpose.

It seems we don't even need to call drush_escapeshellarg() manually as drush_shell_cd_and_exec() does it for us. Updated patch is attached.

Debug line file_put_contents('c:\tmp\full', print_r($full, true), FILE_APPEND); left in the patch.

We can't put $tar = drush_is_windows() ? "bsdtar" : "tar"; all around drush; if we really need to call a different tar based on the platform, then we should have a wrapper function so that tar can be called consistently wherever it is needed.

I did, however, think that the Windows installer was set up such that the 'tar' that was installed was a unix-like tar so that we did not have these problems. Perhaps this is an installation issue, where someone might have problems if they already have Windows 'tar' installed before they run the drush msi? Where does the "bsdtar" used in this patch come from? Is it installed by the drush msi? If we need to use "bsdtar" to avoid possible conflicts with other flavors of "tar", then we should make sure that the msi is installing it before we start using it.

@kulov, what do you think?

Oops, sorry, failed to notice the debug line.
Both tar and bsdtar are included into msi package. And the tar has the absolute archive path bug that is described in the Issue Summary. I'm absolutely sure that in my tests the instance of tar that comes with msi was called and no other installed in the system. And of course you are right that $tar = drush_is_windows() ? "bsdtar" : "tar"; should not be there.

Thanks for the clarification. It seems reasonable to me to basically take the approach of #4, as long as a wrapper function is created to select "bsdtar" vs. "tar". It would be great if @kulov could weigh in here, though.

Just a Windows user chiming in here. I was trying to make an archive-dump using 5.0-dev and it created the directory, but not the tar.

The patch in #4 fixes this issue, but it gave me a warning: 'file_put_contents(c:\tmp\full): failed to open stream: No such file or directory archive.drush.inc:172. I fixed the issue in my case by changing line 172:
file_put_contents($dest_dir . '\tmp', print_r($full, true), FILE_APPEND);

Line 172 is just for debugging; you can take it out if it's causing trouble.

Slightly improved patch with debug line removed and tar selecting code moved to a function

I bumped into this issue too but I took another approach to fix it.
For me it worked with windows tar by using the --force-local option.

I opened a separate issue here: http://drupal.org/node/1393672

Looks like another windows specific issue was recently introduced that causes drush to freeze on the archive-dump command. Problem arises when drush runs a command in a child drush process. I figured out that processes hang on blocking IO operations - parent process tries to read from child STDERR and blocks while the child is also blocked trying to write to STDIO. The issue seems to be the cmd.exe process that is running between these 2 processes. Specifying 'bypass_shell' option to the proc_open remedies the situation. Patch attached.

Is everyone using the latest msi installer when using Drush on Windows? It was my understanding that it installs bsdtar as "tar". If this is the case, then we should require that "tar" is "bsdtar" when running Drush on Windows.

Seems the I/O blocking problem should be addressed in a separate issue.

A new version of the Windows Installer was released yesterday. You can download it from http://drush.ws/drush_windows_installer. It contains the a dev version of Drush 5 from yesterday.

I'm reopening this because the GNU bsdtar on Windows does not support the -r option. This means that archive-dump output will not contain the sites or sql dump info.

Tested on Windows 7 with bsdtar/libarchive version 2.4.12.

Reported to gnuwin32 folks at:

https://sourceforge.net/p/gnuwin32/discussion/general/thread/1094eee7/

But I don't expect a fast response from them.

Since PhP 5.3 is a requirement now, perhaps it would be better to move away from the OS level commands for tar and use the Phar extension which is a built in module.

http://www.php.net/manual/en/book.phar.php
http://www.php.net/manual/en/phardata.buildfromdirectory.php (build a tar from a directory with regex exclusion)
http://www.php.net/manual/en/phar.converttoexecutable.php (convert from a phar to tar.gz)

Haven't looked to closely but it seems like the easiest way to do create tar.gz archives in an OS independent way. There is also an Archive_Tar pear library that might work.

This fix also breaks Drush for those of us that have been happily running bsdtar as tar on Windows as described in #330023: Install on Windows system.

The real fix would make this a (yet another) configurable setting that defaults to bsdtar for Windows and tar for *nix.

The quick-fix for this is to simply copy/paste tar.exe back to bsdtar.exe (so you'll have both a tar.exe and bsdtar.exe in your GnuWin32/bin directory). This is easiest to do from the Windows Explorer as you'll need to provide admin permissions to make the change.

FYI - There is another minor problem with the bsdtar coding. The current archive code uses the --dereference option to indicate following symlinks. However, bsdtar uses -h option for this (which is supported by tar as well). Not sure that it maters in Windows since few folks use NTFS joins.

Anyway, I've been playing around with Phar and it seems like a viable OS independent tar option. I don't know Drush internals enough to do a patch but attached is a "drush_tar" prototype that demonstrates this and is a good starting point for such a patch if the PTB want to go this route.

This provides the common tar features I saw used in Drush, IMHO, someone more knowledgeable about drush could easily use this to replace the tar shell calls.

I put in a fair amount of documentation with this to help any Drush devs who what to try/test/use this.

The drush_tar.inc file has the following main functions:

• drush_tar_build - create / add files to a tar with excludes and dereference support
• drush_tar_compress - compress the tar with gzip or bzip2
• drush_tar_extract - extract files from normal or compressed files.

I've include a little test program (drush_tar_test.php) I've been using to debug / validate it could be used. This does the following:

• Create a tar from a drupal site sans the sites directory (and .git directories)
• Add the sites/all directory to the tar
• Compresses it to tar.gz
• Extracts the tar.gz file into a test directory

I've wrapped all this in timer functions to see how much time it takes to do each step.

To run, just unzip the attached files in a new directory. Edit the setting at the top of the drush_tar_test.php file to point to a drupal site to test with (and valid the other settings). Then run it under PhP 5.3 which has Phar and the compression libs a standard built in mods.

Note: I got the following times from my test site which has about 130MB, 7,000 files, and 2,000 dirs (a lot of git submodule repos in this); creating the tar (core+sites) took about 40 sec; tar compression took about 35 sec;, and extracting took about 800s. So be patient during the last operation.

FYI - The Timing.inc class and utils.inc class are only needed for the test program.

Haven't tried the code above, but looks like a good effort. Drush has function drush_tarball_extract for extracting tarballs; archive-dump calls drush_get_tar_executable() and then calls it directly to create a tar archive.

This issue was marked closed (won't fix) because Drush has moved to Github.

If desired, you may copy this bug to our Github project and then post a link here to the new issue. Please also change the status of this issue to closed (duplicate).

Please ask support questions on Drupal Answers.