Call to a member function get_handlers() on a non-object in view.inc on line 422

I found out that for some months already the error logs are full of this error:

[warn] mod_fcgid: stderr: PHP Fatal error: Call to a member function get_handlers() on a non-object in /public_html/drupal-6/sites/all/modules/views/includes/view.inc on line 422

I noticed because the other day the site went away completely (white screen, WSOD) for about an hour and that error message was printed to the screen once I enabled error reporting. I tried to find the source of the problem, but it was really hard. Then, after I tried to disabled a couple of modules, I changed the theme (through the database). That made the site come back, but with some errors (i.e. panels erros). After some time I tried again with the usual theme and all the settings exactly as before - and the site was back. So I had tried a lot, but in the end nothing I did really seemed to have any effect. It was after that, that I noticed that the error is very frequent, although it doesn't usually kill the site.

The site is quite complex and has many modules activated. It uses Boost for the static cache and views cache for some views. In general, there are a lot of views in use.

I investigated and found some issues, but they don't seem to be related:
#724008: Call to a member function get_handlers() on a non-object in view.inc on line 372
#380560: Fatal error: Call to a member function get_handlers()
#962180: PHP Fatal error: Call to a member function get_handlers()
#778708: Fatal error: Call to a member function get_handlers() on a non-object

I also checked in my local copy of the site and didn't find any of those errors. So I guess it has to do with the server (in combination with an error?). The two servers are:

Live server:
System: Linux 2.6.32-bpo.5-686-bigmem #1 SMP Mon Jul 25 21:48:22 UTC 2011 i686
Build Date: Jan 7 2011 08:16:47
Server API: CGI/FastCGI
PHP Version: 5.2.17-0.dotdeb.0

Local server:
System: Linux jan 3.2.0-37-generic-pae #58-Ubuntu SMP Thu Jan 24 15:51:02 UTC 2013 i686
Build Date: Jan 18 2013 23:27:45
Server API: Apache 2.0 Handler
PHP Version: 5.3.10-1ubuntu3.5

Line 422 in view.inc is this function (starting with $this->$key):

  function _init_handler($key, $info) {
    // Load the requested items from the display onto the object.
    $this->$key = $this->display_handler->get_handlers($key);

    // This reference deals with difficult PHP indirection.
    $handlers = &$this->$key;

    // Run through and test for accessibility.
    foreach ($handlers as $id => $handler) {
      if (!$handler->access()) {
        unset($handlers[$id]);
      }
    }
  }