Implement a git-shell wrapper with python-twisted

What happened for six months? We came to this *exact* same conclusion during the Git-sprint back in Drupalcon SF. I even started working on this and implementing a basic wrapper.

Oh, actually I know what happened. Deplorable decision making.

*Sigh*

Tagging for tracking purposes. Thanks a lot for the detailed write-up, Narayan!

As for "what happened," I'm not sure either. But if this was documented in the issue queue as a decision everyone reached, I must've missed it.

The first I _ever_ heard about using a twisted-based system was when David, Narayan and I discussed it at DC CPH. I did miss the initial discussion at DCSF, unfortunately, but the fact that a twisted-based solution was never mentioned in the big discussion (#714034: Determine the access control solution for git) and that the primary discussion throughout there was oriented towards gitolite leaves me very confused as to how this was at all decided six months ago. Moreoever, no one present at those discussions ever brought it up to me directly in the numerous personal conversations I've had on the topic since then.

I understand that you've felt disenfranchised by this process, Damien, but this vitriol you occasionally show up and shoot at us is hurtful, and in this case also seems baseless. We've done a boatload of communicating, and to the extent that it's possible used the tools available in the d.o community public space to come to the necessary decisions. There have been a number of people who've participated in the work and decisionmaking process quite a bit, enough that I feel pretty good about how open the process has been with respect to the decisions we've needed to take.

Truth is, I've avoided approaching you about most anything related to the migration because of how strongly you expressed your displeasure with the process when the contract was initially announced. If that was the wrong response on my part, and if you'd like to be more involved, I can try to loop you in more.

retagging the lost tag

My mistake! This is the implementation ticket. We're not going to get that done this sprint. :)

I just want to point out that while this is pretty much the exact same conclusion that was reached in SF, the real issue here is that the smart http transport option has been thoroughly evaluated and ruled it out as an option for drupal.org. The http option wasn't even considered in the discussion at SF, and I don't think that most of the documentation about it existed at the time.

Gonna try to get this done, or at least significant progress made, during this sprint.

When looking over this some questions came up. Can anyone answer these or point me to the person who knows the answers?

Do the CVS ACLs or equivalent already exist in Drupal.org's database or is migrating the current ACL system part of the git-ssh issue?

What kind of load is expected for git (are there any cvs stats available)?

Current CVS ACLs are only mapping projects and user IDs. There's nothing more fine-grained (like per-branch ACLs). All those live in the d.o DB.

#880818: Remove CVS access tab in favor of the project maintainers form

The data itself lives in the {cvs_project_maintainers} mapping table.

mysql> describe cvs_project_maintainers;
+-------+------------------+------+-----+---------+-------+
| Field | Type             | Null | Key | Default | Extra |
+-------+------------------+------+-----+---------+-------+
| nid   | int(10) unsigned | NO   | PRI | 0       |       | 
| uid   | int(10) unsigned | NO   | PRI | 0       |       | 
+-------+------------------+------+-----+---------+-------+
2 rows in set (0.00 sec)

The ACLs are enforced by the "xcvs" scripts that are included as part of the CVS Integration module.

Hope that helps. Let me know if you need any other info about the current CVS ACLs.

As for load, I'd expect it'll get heavier than CVS pretty quickly, and continue to increase as we expand the number of ways people can use git with d.o.

This is something I have been wondering about. Sam mentioned something about using varnish to cash the authentication calls.

Do we want to expose a web service that the twisted server checks against or do we want python to connect directly to the DBs?

I'm guessing we want to keep it as loosely coupled as possible and will have provide some kind of cacheable response. Is there a ticket for this anywhere yet?

Apparently this is already underway though not yet anywhere public for the community to participate in or review.

Assigning to me for follow up per Sprint 4 planning meeting.

subscribing

related #720664-34: Create a "ssh_key" module, to allow upload of SSH keys to drupal.org user profiles ?

Chizu and I each got working demos of this up and running and they are currently posted on github.

Both versions were based on the sample code nnarayan found. Each of our implementations had some strengths and some areas that still needed work so we are now in the process of combining them.

Chizu's version lives on his github account: Git Drupal

Mine lives on mine: Drupal.org Git Daemons and is authenticates against the placeholder module Project Git Auth.

While my code wasn't quite as tidy and needs more cleanup, it was closer to the eventual model we want to implement so we will be pushing my repository forward as we merge the best of the two implementations.

What we have working right now:

1. Creating or deleting a project fires an event off to beanstalkd which is handled by a very simple python daemon which creates or deletes the repo based on the project's shortname
2. A twisted based SSH daemon responds to git events, calls out to the companion drupal module, and authenticates the user if and only if they have registered that key in the SSH Keys module and are a maintainer of the corresponding project node
3. The authentication is done using a service that places the ssh key's fingerprint in a query string and which returns JSON and is completely cacheable in varnish allowing us to take some of the load off of the Drupal db
4. In Chizu's version the server will fail over to password authentication if the ssh key is not found!
5. The daemons are completely configurable in the .cnf file

With what we have done today we could do a sprint demo that walks all the way through creating a project and getting a repository, authenticating, adding other users and collaborating on development!

YAY TEAM!

Note: it's rare that projects are deleted. And even when they are, it's rarer still that we remove their directory from CVS. Do we ever want to be deleting Git repos? Maybe this code only needs to worry about create, not delete.

@dww : Yeah, I had considered that. For my own testing I wanted to be able to easily delete test data and get the git repos with it but I'm open to taking that back out.

Also Sam brought up a good point on IRC yesterday: Failing over to username and password is not an option because D.O allows for non-ssh safe usernames. As a result we'll have to scrap that. Everyone uses public keys and we sort out the training issues as they come up. Sorry rfay!

If we go with the currently planned proposal to deal with Git applications (basically, there isn't one, and people can create projects but not releases until approved), the ability to delete projects and repos gets more important.

@webchick: Well, we might want to differentiate. Approved projects don't get trashed but sandboxes can.

Also, ideally we should be able to spin up any number of developer sandboxes per user, right? We'd also need to have a provision for doing that mapping in the ssh daemon

Currently, sandboxes are slated for phase 3 (post-launch) because there's quite a bit of extra work to do there (as far as I know). But we should probably have an after-talk about this on our call today. #961144: Determine/finalize technical requirements for post-Git migration project approval process has brought this up again.

Regarding #24, chizu and I did some tests and the twisted ssh implementation doesn't seem to have trouble with most (any?) drupal.org usernames. We tested with some odd ones like Системаи нави-штан' and didn't have trouble. Are there other difficult names we should test?

So we should be able to deliver the desired password authentication functionality. Yay!

I think that spaces are a concern.

Spaces are not a problem for the twisted implementation of SSH nor for OpenSSH.

@halstead: Killer! We'll need to update the module to provide a service that the daemon can query against. I'll add that to the Drupal module this weekend.

I guess it'll be something like vcs-git-auth?user=[url encoded user name]&pass=[md5 hash] ?

We want to use query strings so that a varnish cash can sit in between twisted and Drupal (posts aren't supported).

Sending the password hashed isn't the most secure thing we could do but it seems marginally better than sending the cleartext and doing proper asymmetric encryption between python and drupal would raise the configuration barrier and be a pain.

As long as this is happening behind a firewall I suppose it shouldn't matter too much. For anyone using this stuff where both machines aren't behind a firewall they could access the drupal service over SSL (query strings are sent inside the tunnel, just not hostnames) or they could use something like pound to tunnel ssl traffic to varnish (varnish doesn't support SSL either).

How can we verify that all characters allowed in usernames (what passes in user_validate_name) will pass in SSH?

@tizzo the user name must be in ISO-10646 UTF-8 encoding [RFC3629] which works for any string user_validate_name() allows.

Tagging for consideration in git sprint 5, preferably as discreet issues.

This is implemented so we can call it done: https://github.com/tizzo/Drupal.org-Git-Daemons

It still needs some work, but the discrete follow up issues we have identified can be found at:

1. #981112: Properly daemonize python daemons
2. #981138: Provide more helpful git error messages by passing messages to our git scripts
3. #981146: Allow a generic 'git' user to resolve to specific user and authenticate based on key in addition to real username/password combo
4. #782610: Determine feedback to be provided back to clients when they push to d.o