Clean up .hg and .git files
mfer - November 8, 2009 - 20:32
| Project: | Drush Make |
| Version: | 6.x-2.x-dev |
| Component: | Code |
| Category: | bug report |
| Priority: | critical |
| Assigned: | Unassigned |
| Status: | fixed |
Description
The attached patch adds Mercurial (hg) support to drush_make. Can this be added?
I've, also, attached a test.make file that downloads a dummy module from bitbucket.org for testing.
| Attachment | Size |
|---|---|
| drush_make_hg_support.patch | 1.7 KB |
| test.make_.txt | 265 bytes |
#1
#2
is there any command in hg that can export it, ie without the hidden directories hg adds? that would be preferable, but we haven't found one for git (and i'm not sure about bzr either).
#3
With regard to bzr, I think you're looking for the bzr export command. The updated patch has a change to "bzr export /local/path /path/to/repo". With this there is no .bzr directory, just the files. The updated test.make file has a config for materialized_view on the fourkitchens bzr server to test this.
The "svn export" equivalent command in hg is "hg archive" but, it only support local repos. If we use that here we would need to do an "hg clone" to a temp directory, a "hg archive" to the files destination, and then clean out the temp space. Unless someone know a better way. Thoughts?
For now I left the setup as "hg clone".
#4
thanks about the bzr. I've committed your patch for now, but I'm keeping it open for the following issues:
I'm not sure about exporting to a temporary directory. I know git could use that too.
what about something like this?
<?phpdrush_shell_exec("(cd %s; hg archive)", $this->project->tmp_path);
?>
something like this for git (maybe, needs testing)
<?phpdrush_shell_exec("(cd %s; rm -rf git)", $this->project->tmp_path);
?>
Not sure if
$this->project->tmp_pathis the right variable, but i think something like the above should work.Thanks!
#5
I think I see where you're going with git. Once you've checked out the project delete the .git directory containing the repo information. Isn't there a cleaner way to do this?
With "hg archive" the repo root cannot be it's destination. Maybe we could a hash for the repo name. Something like:
<?php
// Create an md5 hash to use as the repo name
$hash = md5($this->project->name);
// Clone the repo using the md5 hash for the name
drush_shell_exec("hg clone %s %s/%s", $this->project->download['url'], $this->project->tmp_path, $hash);
// Create an archive of the repo to the modules location.
drush_shell_exec("hg archive --repository %s/%s %s/%s", $this->project->tmp_path, $hash, $this->project->tmp_path, $this->project->name);
// Delete the repo.
drush_shell_exec("rm -Rf %s/$s", $this->project->tmp_path, $hash);
?>
This would feel cleaner to me if the system temp folder were used and if we were using a command that was windows compatible rather than "rm -Rf".
#6
With git, there's a very clean, very network-efficient option that will work sometimes, and a still-fairly-clean but network inefficient version that will work the rest of the time. Warning, pseudo-code!
<?php// Attempt the clean, but not-always-supported --remote option on git-archive
drush_shell_exec('git archive --format=tar --remote=%s | tar -xf - -C %s/%s', $this->project->download['url'], $this->project->tmp_path, $this->project->name);
?>
The git-archive --remote option is dependent on the server having enabled the capability to generate remotely-triggered archives. If it fails - which it will on github, for example - then you fall back to an approach pretty much like the one mfer's laid out in #5.
#7
mfer - we are using the system temp dir. something like /tmp/drush_make_XYZ and the site is in __build__. So it should be perfectly possible to do that.
sdboyer - is there really no cleaner solution? I'm not a git wizard myself so I wouldn't know ;-)
#8
Yeah...unfortunately not :( I mean, maybe I've missed something, but I definitely don't know another way.
The only way to reduce network traffic is to do a 'shallow' clone (available in versions >1.5) with --depth=1 . But the reduction isn't very significant...maybe around 10% a reduction in transfer size. Maybe. See http://blogs.gnome.org/simos/2009/04/18/git-clones-vs-shallow-git-clones/ on that.
Really, the --remote option on `git archive` is supposed to be the way you do this kind of thing, and that approach is quite efficient, if it's available. The kicker there is kinda more github than anything else - see http://groups.google.com/group/github/browse_thread/thread/cfcbcb1dc5f41f16 .
#9
When projects are grabbed from hg and git repos their repos come along with them. The attached patch cleans that up.
For hg/Mercurial the hg archive command is designed to work like cvs export/svn export. But, it does not work on external repos. Following the hg tips and tricks the best method is to just delete the .hg directory. See http://mercurial.selenic.com/wiki/TipsAndTricks#Make_a_clean_copy_of_a_s...
For git the archive command should be used. But, github does not support archiving from their remote repos. So, this patch does the same thing for git as for hg. It removes the .git folder.
The attached test.make file tests against git, hg, and bzr buy pulling from github, bitbucket, and launchpad.
#10
Done, thanks