Problem/Motivation
Our current AMIs contain a local copy of the Drupal core repo.
We could use this repo for checkouts instead of cloning the repo across the network.
Proposed resolution
- Experimentally add this behavior to the checkout plugin.
- Determine whether there is a benefit.
Remaining tasks
User interface changes
API changes
Data model changes
Original report
Checking somebody's comment, I took a look at the logs of the fastest testbots cloning contrib projects for contrib tests, and found that some clones were in fact taking 2-3 minutes, when they should take 5-10 seconds. I was baffled because a manual clone didn't take that long. However, when I added --reference /var/cache/git/reference, the checkout took twice as long for views. However, it took 3 minutes and 45 seconds for commerce.
It's possible that this is a regression in the (old) version of git we're pinning to because PIFR 6.x-2.x doesn't have the ability to provide a valid patch for current versions of git. I'll check that...
root@www:/tmp# time git clone git://git.drupal.org/project/views.git
Cloning into views...
remote: Counting objects: 52456, done.
remote: Compressing objects: 100% (12131/12131), done.
remote: Total 52456 (delta 38684), reused 52133 (delta 38472)
Receiving objects: 100% (52456/52456), 15.09 MiB | 875 KiB/s, done.
Resolving deltas: 100% (38684/38684), done.
real 0m21.716s
user 0m1.416s
sys 0m0.096s
root@www:/tmp# rm -rf views
root@www:/tmp# time git clone --reference /var/cache/git/reference git://git.drupal.org/project/views.git
Cloning into views...
remote: Counting objects: 50329, done.
remote: Compressing objects: 100% (11245/11245), done.
remote: Total 50329 (delta 37612), reused 49738 (delta 37374)
Receiving objects: 100% (50329/50329), 14.89 MiB | 438 KiB/s, done.
Resolving deltas: 100% (37612/37612), done.
real 0m46.655s
user 0m3.692s
sys 0m0.208s
Comments
Comment #1
rfayComment #2
rfayComment #3
rfayChanging the git version to 1.7.10.4 does not seem to affect this dynamic.
I suspect we should consider losing --reference altogether, or changing PIFR to populate the reference cache and then clone, which has significant disk space impact.
The tradeoff here is that core testing (drupal core only) currently clones in a couple of seconds, as opposed to 30 seconds without the reference dir.
Comment #4
isntall CreditAttribution: isntall at Drupal Association commentedComment #5
jthorson CreditAttribution: jthorson commentedWe don't currently have a reference repository in DrupalCI.
We should investigate whether we want/need one.
Comment #6
MixologicComment #7
MixologicIf there's a reference repo, its going to be baked into the AMI's. Moving to appropriate queue.
Comment #8
MixologicThere is a copy of drupal core baked into the AMI's that could be used as a reference repo (provided we did a git pull first).
We should explore this in the checkout plugin to see if it saves much.
Comment #9
Mile23