Hi,

I started to rely on the concepts of distribution and installation profile for my drupal sites and I'd find useful to be able to download projects (either as an archive or as a directory) from the filesystem during the development phase.

To illustrate my use case, this is the structure a site project of mine:

my_distro/
|-- distro.make
`-- my_profile
    |-- my_profile.info
    |-- my_profile.install
    |-- my_profile.make
    |-- my_profile.profile
    |-- modules
    |   `-- custom
    |       |-- my_feature1
    |       |-- my_feature2
    |       |-- my_module1
    |       |-- my_module2
    |       `-- my_config
    `-- themes
        `-- custom
            `-- my_theme

And I would like to be able to fetch my_profile by just using file://my_profile in distro.make.

Right now I use this workaround, I put my_profile under git control, and I use this snippet in distro.make:

projects[my_profile][type] = profile
projects[my_profile][download][type] = git
projects[my_profile][download][url] = file://my_profile

but that has some drawbacks, I have to commit changes in my_profile in order for them to be pulled when I rebuild the project, and I can't easily distribute the whole project (I mean the whole my_distro dir) as a git repository without relying on git submodules.

What do you think about this feature? It is only me who feels the need for it?

Note that the issue is related to #840540: Specify a patch from local file system but that one talks about patches, ideally the code handling the file:// scheme can be shared between the two issues.

Thanks,
Antonio

CommentFileSizeAuthor
#6 make-get-from-filesystem.patch1.06 KBgreg.1.anderson
#5 Issue-1232466-drush-download-from-filesystem_experimental.patch1.74 KBao2

Comments

dmitrig01’s picture

dmitrig01 commented
Title: Download projects from the filesystem » Pull out a specified file/directory from a download

Yes. I think this has come up in several issues before. Ideally, as the title states, it should be possible to use just one file or directory from a whole download. This would significantly contribute to the other issue as well.

ao2’s picture

ao2 commented

@dmitrig01, my title was putting the accent on the WHERE we get the projects from: the filesystem as opposed to the net; while you are stressing on WHAT we pull in: an archive to be extracted, a single file as is, or a directory recursively.

I agree the two issues are connected for my final purpose, do you want to handle those in two distinct issues or you think is is better to solve here the WHAT first and then add in the capability to copy over from the filsystem?

Thanks,
Antonio

dmitrig01’s picture

dmitrig01 commented

The issue of what will need to be solved first. Then, this generalized version will be able to be easily applied to patches.

helmo’s picture

helmo commented
Title: Pull out a specified file/directory from a download » Copy a specified local directory instead of downloading
Project: Drush Make » Drush
Version: 6.x-3.x-dev »
Component: Code » Make

I think that the WHAT part is being addressed in #919224: Allow use of subtrees or individual file from archives instead of the whole archive

The WHERE part is more specific then just handling file://. Putting in projects[nodestream][download][url] = "file:///tmp/nodestream-7.x-2.0-alpha3.tar.gz" works OK.

It's when you reference an uncompressed directory that downloading fails.

ao2’s picture

ao2 commented
Title: Copy a specified local directory instead of downloading » "Download" files and directories from the local filesystem
StatusFileSize
new Issue-1232466-drush-download-from-filesystem_experimental.patch1.74 KB

Here is an experimental patch:

Works with this distro.make:

core = 7.x
api = 2

projects[drupal][type] = core
projects[drupal][version] = 7.14

projects[my_profile][type] = profile
projects[my_profile][download][type] = file
; path relative to the current directory
projects[my_profile][download][url] = my_profile

As said before, my use case for downloading something from the local filesystem is when I want to keep a distribution and a profile well separated, but still in the same git repository.

Any comments?

greg.1.anderson’s picture

greg.1.anderson commented
Status: Active » Needs review
StatusFileSize
new make-get-from-filesystem.patch1.06 KB

Yes, I think this is the right idea. I wanted to create a myprofile.make.bootstrap file in the same directory as myprofile.make, per common conventions. The bootstrap makefile then looks something like this:

; This file was auto-generated by drush make
core = 7.x

api = 2
projects[drupal][version] = "7.x"

projects[myprofile][type] = "profile"
projects[myprofile][download][type] = "get"
projects[myprofile][download][url] = .

This feature is useful in #1368242: drush generate-profile if the user simply wants to quickly make a profile on the local system for testing. The profile's download URL in the bootstrap makefile should be replaced with an absolute path before being pushed to a repository, of course.

I simplified the patch in #5 a bit; it is better to make_download_file instead of drush_download_file, as this allows us to easily replace the download-and-unpack operation with a simple drush_copy_dir operation.

Note that file:///tmp/nodestream-7.x-2.0-alpha3.tar.gz still works, because we only revert to a local filesystem copy when the target is a directory. The layout described in the OP and in #5 also still works here, as the URL can be either a relative or absolute path. Either 'file:///path' or just '/path' works for absolute paths.

jonhattan’s picture

jonhattan
Primary language Spanish
Location Plasencia
commented
Title: "Download" files and directories from the local filesystem » Get directories from the local filesystem in makefiles
Status: Needs review » Needs work

So it was already possible to get 'file:///tmp/something.tar' and '/tmp/something.tar' and this patch allows for local directories, ej 'file:///tmp/somedir' and '/tmp/somedir'.

We should have tests and documentation for both options.

+++ b/commands/make/make.download.inc
@@ -65,7 +65,15 @@ function make_download_pm($name, $download, $download_location) {
+  // the local filesystem rather than to a local or remote .tar.gz file,

It's just a comment but we support a variety of tarballs (and zip), not only .tar.gz

----

Other way to implement this is to treat everything local as local (directories and tarballs), and avoid the extra processing of the download mechanism. After copy or download we can rely on new function drush_file_is_tarball() to decide if extraction is needed.

jonhattan’s picture

jonhattan
Primary language Spanish
Location Plasencia
commented

Also, local method doesn't print any message. For this makefile:

$ cat test.make 
api = 2
core = 7

projects[a][type] = module
projects[a][download][type] = get
projects[a][download][url] = file:///home/jonhattan/a.tgz

projects[b][type] = module
projects[b][download][type] = get
projects[b][download][url] = file:///home/jonhattan/b

projects[c][type] = module
projects[c][download][type] = get
projects[c][download][url] = /home/jonhattan/c.tgz

projects[d][type] = module
projects[d][download][type] = get
projects[d][download][url] = /home/jonhattan/d

This is the output:

$ drush make --no-core test.make /tmp/dest
 >> c downloaded from /home/jonhattan/c.tgz.                                                                        [ok]
 >> a downloaded from file:///home/jonhattan/a.tgz.                                                                 [ok]
lotyrin’s picture

lotyrin commented

Ran into this.

I had to add a "tar -cf my_profile.tar my_profile/" to my build process to side step it.

greg.1.anderson’s picture

greg.1.anderson commented
Version: » 8.x-6.x-dev
Status: Needs work » Closed (won't fix)
Issue tags: +Needs migration

This issue was marked closed (won't fix) because Drush has moved to Github.

If this feature is still desired, you may copy it to our Github project. For best results, create a Pull Request that has been updated for the master branch. Post a link here to the PR, and please also change the status of this issue to closed (duplicate).

Please ask support questions on Drupal Answers.