git reference cache is never updated and doesn't contain commit objects [#1242200]

Two issues:

1) Debugging shows that the git reference cache is not fully populated; it looks like the --tags parameter limits the fetch from the cloned repo to only copy over the tags, but no commit objects:

Executing: git fetch views --tags

Executing: git clone  --reference H:/drush/.drush/gitcache --verbose --progress "sun@git.drupal.org:project/views.git" H:/drush/drush_tmp_1312745954/views
  Cloning into H:/drush/drush_tmp_1312745954/views...
  remote: Counting objects: 6471, done.
  Receiving objects: 100% (5388/5388), 979.52 KiB | 32 KiB/s, done.
  Resolving deltas: 100% (4333/4333), completed with 639 local objects. <----------------------------
  remote: Compressing objects: 100% (1771/1771), done.
  remote: Total 5388 (delta 4333), reused 4513 (delta 3597)

2) The git reference cache never seems to be updated, so if my cache contains a project from January, then each time I drush pm-clone, all new stuff since January is retrieved from scratch again.

Comment	File	Size	Author
#6	drush.cache-git.6.patch	3.93 KB	sun
#1	drush.cache-git.1.patch	3.26 KB	sun
	drush.gitcache-fetch.0.patch	1.91 KB	sun

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

sun

German

Karlsruhe

CreditAttribution: sun commented 7 August 2011 at 20:35

File	Size
drush.cache-git.1.patch	3.26 KB

Reconsidered. Attached patch implements separate git reference caches per project, and uses git clone --mirror, which seems to have been specifically designed for this job.

We're using a new cache directory to not have the new cache directories within the previous gitcache directory.

Comment #2

msonnabaum CreditAttribution: msonnabaum commented 7 August 2011 at 20:44

Status:

Needs review

» Reviewed & tested by the community

#1 is great. Solves our two major bottlenecks here in my testing.

If no one else objects I'll commit this to both 4 and 5.

The only downside I see is that projects cloned using the old method will need to be re-cloned, which sucks, but the benefit here is enough that I think it's warranted.

Comment #3

Damien Tournoud CreditAttribution: Damien Tournoud commented 7 August 2011 at 21:13

The only downside I see is that projects cloned using the old method will need to be re-cloned

That's a BC break. As a consequence, this should not go into 4.x.

Comment #4

msonnabaum CreditAttribution: msonnabaum commented 7 August 2011 at 23:32

Status:

Reviewed & tested by the community

» Needs work

Well, I don't know if it's that clear. What I should have said was project cloned using the old method would need to be re-cloned IF they want to take advantage of the improvements made in this patch.

This puts the new git caches in ~/.drush/cache/git, and doesn't do anything to the old gitcache dir, so the repo's cloned referencing gitcache should be fine, as long as you don't delete it.

Also, sdboyer mentioned that the git caches should really appended with .git since they are bare, and I agree.