[Policy, no patch] How to handle upstream fixes that don't get committed fast enough

Please also note that if we host those forks on drupal.org then the produced patches are automatically GPLv2+, which you can only contribute back upstream to MIT-licensed projects if all people involved in writing the patch approve/consent.

So keeping the forks on github under their original license sounds like a less painful approach.

I was going to say what Klausi said. Hosting forks on d.o is a total no-go from a licensing-safety standpoint.

The correct, IMO, approach is for Dries et al to get control of https://github.com/drupal, and we put our "local" forks there. Access can be handled however we want, and we can even file the PRs to upstream directly from the exact code we're using. Then we twiddle our composer.json file as needed.

I would say that's superior to just monkey-patching the in-repo vendor directory in all circumstances.

Note that this also remains exactly the same if we ever get around to taking the vendor libs back out of the main repository the way we should. :-)

Composer itself is a bit of an oddball, and right now the plan is basically the "subclass" strategy. I don't expect that to be typical, though, since other than PSR-4 support I don't think there's any Composer variation we'd need to think about, plus it's not "in" our code base in the first place, per se.

@webchick yes something like that. That'd let us pull in PRs from for example dawhener's fork, then have an 'official' upstream PR where follow-ups etc. could be handled centrally. And at the same time we'd be running the fork in core.

Also allows to run two patches at once (say we'd needed to hotfix both the tokenize change and trailing commas at the same).

IMO this needs to be reserved for critical bugs, because it's going to be tricky to keep track of multiple different PRs and forks, and we should concentrate efforts on getting the upstream fixes merged. But clearly there's going to be issues which are very critical for us, and less for others, and it's good to have something in place to handle it.

I think there's a much older issue about this somewhere, but couldn't find it.

Agreed that an upstream fork should be for exceptional cases, such as major performance changes (as in the most recent case), 0-day security issues (where we can't wait for upstream because it's already known), etc. Most of the time we should just work upstream and be polite but pushy about getting things merged. :-)

We could also give extra access to particular repos, so if we wanted to, e.g., give dawehner access to our doctrine fork to manage the PR, that can be done temporarily on GitHub without any impact on d.o's security or access settings.

I'm in favour of Crell's suggestion in #3.

Where we host is a red herring. This "get off the island" thing totally failed. The upstream projects stall us or move to new pastures and refuse to help us. The licenses are very permissive. We should take the (small) pieces from Doctrine Commons and Annotation we use, move it under the core/lib/Drupal folder -- we could, for example, finish the optimization and skip tokenizing altogether.

We should move phpunit in too. At this point, I literally can't write a strict mode (where method calls without expectations set on them cause failures) within our process because despite phpunit supports the necessary templating one needs to put the template in the phpunit directory itself and I am told the vendors directory is hands off. This morning the phpunit lead refused to work with us with something as small as dropping the PHPunit 3.8 requirements from 5.4.7 to 5.4.4 and instead will likely go 5.5 making it downright impossible to use 3.8 for D8 so we are left to our own devices.

In short: any other project that doesn't cooperate in a timely manner should be moved from vendors to lib.

Taking a bullying approach won't help us in the long run...

Next step here: Who wants to reach out to see about getting the GitHub group transferred?

What "bullying approach"? What's the point of adding maintenance burden for core committers of having to commit to multiple repositories? For the (real) enhancements listed in #8 we would need to fork those projects and if we need to fork why not fork into core and simplify?

1) We want to un-fork later, in most if not all cases. We're talking about temporary measures here, not total divorces.

2) These are stand-alone libraries. Keeping them as such forces us to think of them as such. Separation of concerns in the infrastructure helps reinforce separation of concerns in software design, which is an area Drupal 8 tries to get much better at.

See #2151829: Doctrine annotation parsing takes an unacceptable amount of time/memory on install for where we added a fork, and #2173719: Remove the fork of doctrine/common for where we removed it.

This process works.

I agree with #12, it should not be more complicated than that.

In that issue @dawehner hosted the fork on his github account. This might be better in a self hosted repo on git.drupal.org though, so that it is managed by the drupal association rather than by a volunteer.

Yeah I should have made a mirror on git.drupal.org though due to the fact that basically al the outside world is using github we will work with them via github forks.

In general I did not had any bad experience yet while working with other projects: symfony-CMF/symfony/doctrine. We tend to overthink the problem here.

I don't really mind if the forks are on github - it means we can have upstream pull requests from those same forks if we want.

I do think it'd be better if there's a Drupal organisation on github - especially for situations where we need to have two concurrent changes to the same library and etc, and it'll be easier for people looking through composer to tell why we're using a fork if they don't know who dawehner is.

There is a Drupal organization on github, owned by mikl. (Danish Drupal dev in good standing.) Can one of the committers just ping him and point to this issue, asking for ownership?

I saw that dawehner pinged him on twitter. https://twitter.com/mikl/status/425877106032406528

mikl: @da_wehner hand over? Do you think I stole it or something? It is (of course) available to the community if the devs want to use it.

Sorry, shouldn't have posted just one bit of the conversation out of context. It was quite amicable, and Mikl later said:

mikl: @da_wehner yeah, or perhaps the DA. If they want it, they can have it, no problem.

Looks like this is he https://drupal.org/user/58679

3 years later and we didn't run into big problems with this. Most of the time we got upstream fixes in time, rarely we used personal forks on Github.

So the conclusion of this issue is that we will continue to use temporary personal forks on Github for the rare cases.