SimpleTest fails to detect a failing test

I almost got the PIFR client working, but the test "Detect a failing test" fails. I tested if error handling works using the following script:

function errorHandler($severity, $message, $file = NULL, $line = NULL) {
        echo "got error!";

}

set_error_handler('errorHandler');
$foo = 1 / 0;

I also got display_errors on and log_errors off. I am not sure if this is really a bug, but I hope to get instructions on how to find out what's wrong on the installation.

thanks,
Timo