Alternative(s) to php-cgi with runserver

Been digging into this and alternative approaches some more - not made any huge progress, but noting results here so we don't repeat ourselves. FIle uploads work fine with php-cgi.

- header() indeed does nothing but return 0 on php-cli (checked source). This is a real pain. Possible workarounds include:
--- using drupal_get_http_header() (which get most but not all) and coming up with the missing ones some other way
--- replacing the header() function with a wrapper via preg_replace in copies of the few files it occurs in (ugly^n), or perhaps runkit or APD extensions (somewhat defeats the point of runserver being easy to get started with)

- Another approach is to use add an alternative webserver running in "non-system-daemon" mode that can talk php with another SAPI - the only real candidate here is Apache httpd with mod_php (lighttpd, ngnix etc all use php-cgi). There is a rough version of this at https://github.com/zroger/drush-toolbox - the main downside of this is that it starts up a set of processes in the background, so is quite a different user experience than runserver with the php based server. I experimented with starting apache2 in the foreground with the -X flag, but then it is only single threaded, so I don't think that would work. I also think it would be quite a lot of work to make this work in a zero-config cross-platform way.

- One approach would be to use php-fgi which seems fairly common on some OS X builds - however unless there is a way we can make this run in a non-daemon mode (I couldn't find one) we would need to extend the php httpserver to open sockets in place of running php-cgi, as well as talk the FCGI protocol (not the same as CGI).

- Another option is to build in support for the lightweight web server that is built into php 5.4 (which looks very similar in function to ours). This is only a partial solution though, I think, as php 5.4 is only in alpha, and seems likely to break Drupal enough that it won't be feasible for a while.

- The other option is to check and (if necessary) extend the php build scripts for the major OS X packaging systems/packages (homebrew, MAMP, fink, macports, darwinports...?), contribute our changes upstream if possible, and document how to use for Drush users. I think I remember that it is problematic to build php as an Apache module and a cgi at the same time, hence the easy/dumb approach here is probably to clone the build script, alter the configure flags and change the install/deploy logic so that it only deploys just the php-cgi binary (not lib, share etc). Ideally the packaging would make a "php" package/group (for the lib, share etc), with a subtask to build the Apache module package/group, and we would just add a further subtask (perhaps controlled by an option) that reruns configure and make for php-cgi (potentially more efficient, since we don't end up rebuilding stuff). Hopefully much of the time there is existing code (from fcgi/cgi users such as fcgid, lighttpd, ngnix etc) that we could adapt as needed.

I am leaning towards the last option - open to ideas though.

I forgot to mention another option, which is for someone to build a (somewhat) static php-cgi binary and distribute just that file ourselves - I did experiment with this, and it looks like it would at least be dependent on system glibc (so not fully static), but that is probably not an issue with OS X. Even if it wasn't static at all, it seems there is a good chance it would still work within the same OS X and php extension API versions (the latter changes very rarely). Could be worth experimenting with as a fallback option perhaps.

Looking at the formula at https://github.com/adamv/homebrew-alt/blob/master/duplicates/php.rb I would think something like this (rough pseudocode) could be a reasonable approach:

    no_apache_argv = ARGV

    # Build Apache module by default [NOTE: moved from above]
    unless ARGV.include? '--with-fpm' or ARGV.include? '--without-apache'
      args.push "--with-apxs2=/usr/sbin/apxs"
      args.push "--libexecdir=#{prefix}/libexec"
    end

    unless ARGV.include? '--without-apache'
    #snip

    system "make"
    system "make install"

    if ARGV.include? '--with-cgi'
      ARGV = no_apache_argv
      args.push "--enable-cgi"
      system "./configure", *args
      system "make"
      cp build/path/php-cgi /system/path/bin # assuming there is no make install rule for just this
    end

To do it properly, the same could be done for php-fpm also.

Another option is https://github.com/indeyets/appserver-in-php (see http://vimeo.com/27071855 and https://github.com/freudenberg/drupal-on-aip) - no idea how much progress Stefan has made since the video (just e-mailed him), or how compatible this would be with the httpserver approach.

I think AiP could be very interesting from a performance point of view (unlike httpserver which is just orient around easy development), because it could do a bunch of bootstrap/preload activities before a connection starts (we discussed this concept some at the Drush sprint). I suspect it could also make multi-thread and ESI techniques much easier. The broken php-cli header() discussed above would likely still be an issue though, as would clearing any static caches that don't use the D7 static handler. I think running AiP/Drupal from inside Drush may be easier than standalone though, as we have the main environment issues figured out (my small patch above can reliably generate and serve anon pages).

Just had a play with the drupal-on-aip code - I couldn't get it quite working as is (couldn't find Superglobals middleware, and it constantly restarted workers without that). I did have a play with replacing the bootstrap sequence with Drush (see attached patch), which seemed to work roughly as well as the nocgi-hack above (which is to say that it can serve basic pages OK, but no login etc). It is a bit simpler that the current code though and I think may be worth pursuing some more, since the Drush bootstrap process is quite flexible. It would also probably be possible to borrow some ideas/code from the runserver-prepend.php file (that tweaks the Drupal environment to our needs without needing to alter the code).

In principle it should be possible to actually start from Drush (with no bootstrap) passing in the users runserver configuration, load AiG and start the workers and then (later on) bootstrap Drupal - I tried this also, but wasn't able to find a useful layer of abstraction in AiG where we could jump in without having to write out temporary config files (you can run the tasks without the config file, I think - but then it also reloads the config file later on).

I agree we are likely to keep hitting issues with core though. I did notice that D8 is proposing to use the Symphony http library for all request/response processing (see http://groups.drupal.org/node/167299), which should be able to handle this much much better, if not out of the box. If that work ends up being isolated to a small number of .inc files (bootstrap, common etc), it may even be possible to distribute a separate bootstrap for D7 based on this (or perhaps pressflow.

Had more of a play this evening, and ended up going in much the same direction as Stefan (ignore my previous patch, it doesn't make any sense) - attached current files. There are a few environment fixes, and I also dropped the ob (which was causing me problems) and replaced the delivery function with a non-printing version, which seems to work. I am able to log in, but not persist sessions between threads (I have some code and can set/save sessions with AiP, but figuring out where it should go in session.inc is hard!). I think the other main issues are header() of course, but also exit(), for example in drupal_goto(). I have been hacking these out one by one using alternate bootstrap/common.inc files (that call into the rest of core), but it is slow progress.

I wonder if it would be worth creating a separate d.o. project for the AiP code? It is pretty far off helping with this issue in any case :)

This is technically resolved because we have php 5.4 support, but I think it would still be interesting to look at aip now the Symfony kernel is in place (and actually being used now).