This patch allow to store downloaded files in a specific folder in order not to re-download them again next time. I'ts a first attempt to see if someone else is interested.

At the moment, cache key is based on project tarball filename. It's fine for released project but not for 'dev' ones. I'll try to address this issue later.

I'm also planning to cache XML records from drupal.org update database based on it's age (ex. redownload the update record if cached one is 1 week old). This way, building a drupal installation from scratch can be very quick as every needed information are in cache.

CommentFileSizeAuthor
dl-cache-0.1.patch2.91 KBYoran

Comments

moshe weitzman’s picture

moshe weitzman commented
Status: Needs review » Postponed (maintainer needs more info)

Whats the use case for this? Seems a bit too specialized for drush core. Also see drush_make if you are downloading all the components for a site.

dman’s picture

dman commented

Hm, the patch indenting is a bit off.
A bit more explanation of your use-case, and exactly what advantage this provides would be quite important.
Are you running one server with many many sites all in their own distros? (Which is the only reason I can imagine it being wrthwhile for 'caching' reasons.

HOWEVER on Thursday I accidentally built a downloaded modules cache!
This is totally specific to my requirements this week, but I thought I'd describe my story.

I (as often happens) am called in to take an existing, possibly hacked, somewhat un-updated site and make it work better.
The site was built (by whoever, whenever) using stable downloads at the time, but I can't trust anything to be clean. Even more interesting, some of the site was checked into a competing CVS Repository.

Soo... I wrote a drush extension over here
That:

  1. Checks the alleged version of each installed module
  2. Downloads a clean copy of that package into a nearby cache dir
  3. Runs some diffs to check that the code hasn't been messed with
  4. Tells me what's safe to upgrade outright, and what needs attention
  5. Produce and log some diffs describing what's happened
  6. and then optionally drush dl all the pending updates
  7. ... and re-apply the changes if possible.
  8. An interesting feature was re-synching the NEW updated code with the OTHER repository. Normally using drush dl obliterated my ability to do that.

... The damn thing worked!
As I say, this is basically digging me out of a hole, and isn't neccessarily useful to anyone else, but it was a great exercise in leveraging drush to automate tests and things using bits of core drush (plus some commandline madness of my own).

I'm only mentioning it because it was interesting, and did entail me taking a local cache.

I saw #528016: Create ishacked drush command and this is related, somewhat, but my version.
It may address the problem described at #620926: How to download + svn commit with drush?

:-B

Yoran’s picture

Yoran commented

@moshe weitzman

I'll not really speak of "use case" here, but more advantages :
1/ When I'm building a new site, most of the modules are the same. So I so built some drush commands to add "feature" (a set of modules for a specific need) to any local site. As I need to be sure about those modules to work perfectly, I always take the same version for each of them. So, once again, I don't see the point of download everything every time.
2/ I have many local installations, I don't want to download N time the same module in order to update all installations (that are all different).
3/ I found very useful to have history of modules installed handy :-)

So with a local cache (for both xml records and tarballs), Drush act a bit like maven, with is "built on the flight" own modules repository that can be shared by a dev team.

Well, at least I find this usefull ;-) You tell me if this can be in your target vision. This is not a big deal for me as I can easily implement this as a package-handler (thank's for the design).

Yoran’s picture

Yoran commented

@dman funny, I have approximatively the same procedure here. The difference it is plain old perl :p

Actually I'm re-evaluating Drush (I tried it a too long time ago) in order to port all my messy build & dev process in a solid framework.

So, I'm a bit surprised because I don't see any other way (diff/patch) to update a module (and more, the all core). I have to look how its done and I'll probably use your code (if you don't mind).

moshe weitzman’s picture

moshe weitzman commented
Status: Postponed (maintainer needs more info) » Closed (won't fix)

Thanks for the info. I think this is too edge case for core drush, but it sounds like it could be a popular contrib module.

kenorb’s picture

kenorb commented
Version: » 7.x-5.x-dev
Component: Code » Base system (internal API)
Issue summary: View changes

See: https://github.com/drush-ops/drush/issues/413

kenorb’s picture

kenorb commented
Status: Closed (won't fix) » Closed (fixed)

Checked and actually drush cache the files now by default (including XML files).
E.g.

Downloading release history from http://updates.drupal.org/release-history/drupal/7.x                          
~/.drush/cache/download/http---updates.drupal.org-release-history-drupal-7.x retrieved from cache. 
Downloading project media to /tmp/drush_tmp_1410971245_5419b66d2d575 ...                                                       
~/.drush/cache/download/http---ftp.drupal.org-files-projects-media-7.x-2.0-alpha3.tar.gz retrieved from cache.

I'm using drush 6.3.0.

kenorb’s picture

kenorb commented
Version: 7.x-5.x-dev » 8.x-6.x-dev