Meta-issue: Planning the migration to GIT

wrt repo structure: we have to decide first whether/how we want to support per-issue repos/branches before we settle on a repo structure for sandboxes. See some older thoughts for this there: http://groups.drupal.org/node/21157

yes.
We'd need a place to summarize thediscussion wrt personal branches/sandboxes somewhere else, though.
By the way, if we want to, we could deploy versioncontrol_api and versioncontrol_project with versioncontrol_cvs before we deploy git, if we want to stability-test versioncontrol_api+versioncontrol_project.

Myabe I'm missing something about how git works - but does it work well to add a totally unrelated branch to your sandbox? This sounds more like how bzr works. Trying to do this locally is not easy and git complains "warning: no common commits".

Seems like it would make more sense to add a per issue (+per user?) branch to the relevant project repo on demand - then anyone with that repo cloned can immediately fetch the issue branch with no extra setup. Or alternately, a user's sandbox should have a repo per core/contrib project?

Such working branches could be deleted once an issue is marked fixed just to avoid insane branch blot (well, for core it would be insane even with this).

Thinking about this does make me question a bit the choice of git over bzr...

@pwolanin, if I understand you correctly, you're talking about what is usually referred to as "topical branches", where you create a local branch to work on that doesn't get exposed to the world, and it's one of the most wonderful things about git... You can work away, under source control, on a separate branch in your local repository, without ever exposing what you're doing to the world, or polluting the world with an unnecessary branch.

please see http://drupal.org/node/714102 for issue-specific branches/repos/sandboxes.

@rfay - no that's not at all what I'm talking about. Rather, a git repo where on branch is drupal core, and another branch is a contrib module.

Just a quick note to point to the prototype handbook documentation for git: http://drupal.org/node/711070

(and to shamelessly subscribe and offer my help where i can.)

The the test bot is trivial to update to anything. I know certainly that there is a bot instance somewhere in the world ;) which uses bzr for example. All the bot needs is a git clone.

Any syncs between CVS and Git should probably be kept at a minimum because of added complexity (and I'm now also subscribed so that I can try and help whenever I can)

chx, can you make a separate infrastructure issue for that testbot clone into git stuff, preferably with pointers to the CVS and Bzr versions and a bit of hand-holding on how to proceed, so we could get someone to start working on that?

Angie, really, there is nothing to work on, you tell jimmy what command is necessary to get a fresh Drupal copy in a local directory (aka git://git.drupal.org/core/drupal.git) and that's it, check pifr code what it uses cvs for.

subscribing

Ah. git clone the command. Not "A clone of testbot that works with Git." Got it. :)

Eh. I'm a stickler for documentation anyway, so created #714436: Update testbot to Git. Hey, our first Novice task! :) If you notice any other "low hanging fruit" like that as you're going through, please tag them accordingly. There are lots of people who've voiced an interest in helping this along, but who lack either the access or the time for some of the really heavy lifting. And this would help the "core" infrastructure team focus on the hard stuff, while getting more people involved.

I don't have the privilege of editing the original issue here, but I created an issue for documentation here: http://drupal.org/node/714448 that should probably be linked from the "write documentation" bit up there.

As we financially supported the cvs updates and that work gave us an incalculable ROI I suppose I should cough up for this as well.

Anyone tell me where some funding might be needed for either kit or people?

Do you need help project planning?

Holy cow! Thank you, Fintan!!

I will defer to others on your question, but my sense is that it's a little early to know what exactly to sponsor, who will be doing the work, and how much it will all cost. We still ultimately need a sign-off from Dries for this whole thing to start, people are just very eager. :) I'm not sure of how you funded the CVS improvements -- if that was to the Drupal Association, or directly to Derek and Earl -- but I'm willing to put in a request for 2010 budget approval for this and help manage it on the Drupal Association side, if that makes it easier.

There are two projects that need development (which could possibly be furthered by sponsrships):

http://drupal.org/project/project

http://drupal.org/project/versioncontrol

I suggest you contact the respective project's authors to discuss funding.

We directly sponsored it but I am happy to do it via the association or direct with the guys.

I will get onto the guys and see where we can help....

Just a question about the Version Control API portion of this implementation plan.

Over in Ye Olde Monster Thread Of Doom, the very author of Version Control API argued against its use (granted, one of the reasons was lack of commercial support). But below that, both David and Sam make pretty strong cases for deep integration, rather than only integration with the subset of commands that apply across all version control systems.

Could someone comment on this?

hey @CorniI, thanks for pointing me here :-)

Install gitweb and set up a redesign-inspired CSS (important: enable the xss_protection mode)

I like the plan :-), just want to mention that we also need to determine which web browser.

I am restoring the title as it was set by webchick.

@#25:
The problem here is that the featureset which all VCS have in common is, compared to git's featureset, small, and so abstracting that will be hard.
One solution for us would be to merge versioncontrol and versioncontrol_git (both their 2.x branches) and turn that into a drupal_git module which implements everything needed for a d.o git + project* integration. This would hardcode everything to git as much as it is done for cvs currently, or even more.
Another solution would be to make versioncontrol_api a distributedversioncontrol_api, which then could be extended to support everything git needs, but with still keeping the possiblity of someone writing a bzr backend to drop in.
This would also keep (d)vcs_api more general, allowing a broader reuse than just d.o, we already have seen how much attention d.o-only modules get...

In general, vcs_api+vcs_git is the best available solution we currently have, and marvil07 and I are working to improve these modules, so that they could at least power a read-only solution. Everything beyond that is _a lot_ of work which needs to be done, regardless of the choices outlined above, and I hope others will chime in here as well and lend a hand.

My personal opinion is that we want to turn vcs_api into a dvcs_api and run that together with the git backend, project integration and maybe an extra module or two, if needed for our workflow (imagine github-like graphs of repositorys), on d.o

Something else to throw out there as a possibility: It looks like last year, Google Summer of Code was announced in early March. We should keep our eyes peeled for announcements related to that in the next couple of weeks, and see if there's anything with this initiative (a phase 2 or 3 item, since SoC takes place June => August) that makes sense to break off as a project for a prospective student to work on.

Google has in the past funded the development of SimpleTest, our automated testing infrastructure, the development of version control API module, and various other infrastructure improvements, all to wonderful results. But it's definitely a bit of a risk, because it effectively blocks anyone else from working on these projects, and we don't know whether the student will be able to do the job (unfortunately, we had a student fall through on the Project/Version Control API integration work one year).

GSoC projects work best when they're something we can live without, but would be tremendously kick-ass if it were done, and when there's someone who could do the work that might not have time to do it themselves, but does have time to answer questions for a few hours per week, and get someone else pointed in the right direction. So if you notice things like that, definitely set them aside for later.

about vcs_api

Like CorniI mentioned, we're thinking in ways to resolve the problem.. My actual choice would be implementing a dvcs_api which extends vcs_api, I mean another module in there :-) .. but really not sure about the time.

I see SoC as an option, but I would want to take advantage of all of the people who want to help, I mean I would prefer if some people get involved on the queue and we all make this project get realeased ASAP ;-)

could someone with the respective permissions please move the links to the issues #720664 and #720666 from Phase 3 to Phase 2?
We need the ssh keys of the project maintainers so they're able to commit to their project's repositorys.

On another matter:
webchick feared that if we deploy phase 2, but then have a long time gap until the deployment of phase 3, our community will scatter as github, gitorious etc. will become usable very easily, so that'll be hard to get these users back once phase 3 is deployed (which will take some time). This raises the question whether we want to hold back the deployment of phase 2 (except from staging/test servers) until we have phase 3 ready, so that d.o skips phase 2 altogether.

argh, it happened again...

It will be easy for people to move to github, but also easy for them to move back (especially once issue integration is working).

I think given the somewhat uncertain timing of phase 3, I would not support holding back phase 2 for it.

Phase 3 has a lot more scaling issues (as thousands more users add their ssh keys, etc) so I hope we can get to phase 2 even without needing a custom sshd, etc.

pwolanin: Yes, that's exactly the idea. That's why these are broken out into distinct phases, so all the "wouldn't it be nice" stuff gets tacked on to the end. ;)

Corni: Done.

Subscribing, to follow the action items for project and project_issue.

Question: instead of having this pseudo wiki page issue which no one but admins can edit, and we have to manually update whenever a new issue is posted, would it make sense to move to issue tags instead like "drupal.org git conversion phase 1/2/3"?

Deferring to Damien on this since he's kind of spear-heading this meta-organization stuff.

Re: organizing sub-tasks: Issue tags could be nice. However, I think this is a good use-case for a "Community initiative" section in the handbook, under http://drupal.org/community-initiatives

then, we get:

a) real wiki-like behavior, revisions, diffs, etc, that everyone can see
b) we can still use the issue nid [#xxxxxx] filter (unlike g.d.o)

until we have a more all-encompassing solution for #569552: Provide a mechanism for issue meta discussions I think the community-initiatives handbook pages are the best bet, especially for big, multi-pronged efforts like this. we could have sub-pages for various sub-areas of the migration (repo migration, packaging, issue queue, documentation, whatever)...

I think it's a great idea to use dww approach here

I was just about to create a child page on http://drupal.org/community-initiatives, but I think we need Damien and webchick approve before.

subscribing

Great. Then I think this particular issue has fulfilled its purpose. Meta-planning should happen at http://drupal.org/community-initiatives/git -- specific issues should be opened in the right queues and added to that page (or child pages of it, if it starts getting too unwieldy) for specific, actionable, self-contained tasks. Thanks, all.