I have some problems with Simpletest on a clean Drupal installation.

Simpletest.test fails on stable!

Simpletest can't get the user form on my drupal installation (clean, fresh one, just the simpletest module is here).

This is where I got the first fail:

Failed to set field name to simpletest_ZZOm	Other	simpletest.test	39	SimpleTestTestCase->setUp()

As we see, it could not find the name field in drupal user form. I looked a bit into the code, did see where it could bug. I also try to remove some paranoid apache rules, to have a more "normal" configuration, it always fails.

Then, I looked up higher, in passes just before, and I saw:

GET to http://drupal-6.guinevere/user, response is 1 bytes.	Browser	simpletest.test	39	SimpleTestTestCase->setUp()

It's not normal, the html response is only 1 byte! something is wrong with my installation, I cant find what.

Then, I also looked at apache's log, I found these (when simpletest is running):

127.0.0.1 - - [18/Jan/2009:01:38:14 +0100] "GET /user HTTP/1.1" 200 3886 "-" "-"
127.0.0.1 - - [18/Jan/2009:01:38:15 +0100] "GET /node HTTP/1.1" 200 5285 "-" "-"
127.0.0.1 - - [18/Jan/2009:01:38:15 +0100] "GET /install.php HTTP/1.1" 200 2422 "-" "-"
127.0.0.1 - - [18/Jan/2009:01:38:15 +0100] "GET /logout HTTP/1.1" 403 3833 "-" "-"
127.0.0.1 - - [18/Jan/2009:01:38:15 +0100] "GET /user HTTP/1.1" 200 3886 "-" "-"
127.0.0.1 - - [18/Jan/2009:01:38:15 +0100] "GET /user HTTP/1.1" 200 3886 "-" "-"
127.0.0.1 - - [18/Jan/2009:01:38:17 +0100] "GET /user HTTP/1.1" 200 3886 "-" "-"

Ok, you may say it's right, but you also should look at these:

127.0.0.1 - - [18/Jan/2009:01:44:40 +0100] "GET / HTTP/1.1" 200 4524 "http://drupal-6.guinevere/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5"
127.0.0.1 - - [18/Jan/2009:01:44:42 +0100] "GET /user HTTP/1.1" 200 4637 "http://drupal-6.guinevere/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5"

These last two were done by firefox. The weird thing is not the browser's identification (it sounds normal that curl may not identify itself), but at the preceding parameter which is "my domain" when requesting with firefox, but an empty "-" when requesting with simpletest & curl.

Is this a known bug ? (I read A LOT of issues and could not find something similar).

Batch and JS freezing the browser

And another anoying bug is with firefox 3.0.5, the js script when launching tests (batch api's js I suppose) just get mad and can't be stopped, it freezes the browser! (Firefox can stop it when it detects a script took to much time, and it said to me the error was in jquery's js file, probably deeper in the execution stack).

Additional notes

Just a note, I tested with drupal 6.9 with a lot of modules, then with drupal 6.9 in a clean installation, then also tested, in both configurations with simpletest 2.5 and simpletest 2.x-dev. I also tested with others gecko based browsers (epiphany-gecko).

My configuration is an Ubuntu desktop 8.10 / Apache2 2.9-7ubuntu3 / MySQL 5.0.67-0ubuntu6-log / Normal disto (with no further tuning).

Please, help! I can't develop anymore on any module which make intensive use of simpletest!

Comments

pounard’s picture

And my PHP version is 5.2.6-2ubuntu4.

Help!!!

lyricnz’s picture

1) The previous field in the apache logs is not "my domain", but is the referrer (ie: the page that linked to the page you're viewing). It's useful for traffic analysis, but is not used by Drupal for functionality. So don't worry about that one.

2) Make sure your PHP errors are set to show everything (in your php.ini): error_reporting = E_ALL. Restart apache.

3) Check your php error logs, this will normally contain the information about what's going wrong.

4) Dump out the entire page that SimpleTest is using, to see what it's seeing. This is not always what you expect! You can use a handy function like:

  /**
   * Write the given data to the given filename, relative to the Drupal
   * temporary directory. This method is really useful for determining what was
   * in the HTML page that DrupalWebTestCase actually saw.
   *
   * @param $filename
   *   The filename to write (defaults to out.html).
   * @param $data
   *   The data that will be written to the above file (defaults to the content
   *   of the current page).
   */
  function writeFile($filename = 'out.html', $data = NULL) {
    $data = ($data == NULL) ?  $this->drupalGetContent() : $data;
    $filename = file_directory_temp() . '/' . $filename;
    $this->assertTrue(file_put_contents($filename, $data), t('Wrote content to %filename', array('%filename' => $filename)));
    drupal_set_message("Wrote content to $filename");
  }

5) I've never had problems with the Batch API javascript - it's really pretty simple. Your browser might be reporting a timeout if the test is taking "too long" to execute. Enable LiveHttpHeaders or Firebug network monitor, to see what's actually happening on the wire.

6) Tail your webserver access logs AND error logs: tail -F /var/log/httpd/access_log /var/log/httpd/error_log (or wherever your files are)

pounard’s picture

Thanks,

I don't think I have to do more than dump the curl_exec return. It always return "1" in the file. When looking at PHP doc we ca see, for curl_exec:

Return Values

Returns TRUE on success or FALSE on failure. However, if the CURLOPT_RETURNTRANSFER option is set, it will return the result on success, FALSE on failure.

So it looks like my curl_exec() returns only TRUE, so I looked in drupal_web_test_case.php for this magic option and found:

  protected function curlConnect() {
    global $base_url, $db_prefix;
    if (!isset($this->ch)) {
      $this->ch = curl_init();
      $curl_options = $this->curl_options + array(
        CURLOPT_COOKIEJAR => $this->cookie_file,
        CURLOPT_URL => $base_url,
        CURLOPT_FOLLOWLOCATION => TRUE,
        CURLOPT_RETURNTRANSFER => TRUE,
        CURLOPT_SSL_VERIFYPEER => FALSE,
        CURLOPT_SSL_VERIFYHOST => FALSE,
        CURLOPT_HEADERFUNCTION => array($this, 'curlHeaderCallback'),
      );
      if (preg_match('/simpletest\d+/', $db_prefix)) {
        $curl_options[CURLOPT_USERAGENT] = $db_prefix;
      }
      if (!isset($curl_options[CURLOPT_USERPWD]) && ($auth = variable_get('simpletest_httpauth_username', ''))) {
        if ($pass = variable_get('simpletest_httpauth_pass', '')) {
          $auth .= ':' . $pass;
        }
        $curl_options[CURLOPT_USERPWD] = $auth;
      }
      curl_setopt_array($this->ch, $this->curl_options + $curl_options);
    }
  }

So, I did:

  protected function curlConnect() {
    global $base_url, $db_prefix;
    if (!isset($this->ch)) {
      $this->ch = curl_init();

      ...

      $this->assertTrue(TRUE, "Curl options has been set!");
    }
    else {
      $this->assertTrue(FALSE, "Curl options not set..");
    }
    $this->assertTrue(TRUE, "Making curlConnect");
  }

And, it's OK, my simpletest inits the curl_options, I have the assert TRUE the first call, then none the followings (which is normal because curl_options are inited only once).

I'm still looking, but I think this could be a broken curl on Ubuntu 8.10, what do you think ?

PS: In this method, we can see this:

$curl_options = $this->curl_options + array(

And then, some lines later, this is done:

curl_setopt_array($this->ch, $this->curl_options + $curl_options);

Is this a nonsens ? the $this->curl_options parameter is concatened twice with $curl_options? And thus, the first time, as the first element before the operator is $this->curl_options, if this object attributes carry the option CURLOPT_RETURNTRANSFER to FALSE, the setting won't be override.

pounard’s picture

And found an error!!!

When you do this:

  function drupalGet($path, $options = array()) {
    $options['absolute'] = TRUE;

    // We re-using a CURL connection here.  If that connection still has certain
    // options set, it might change the GET into a POST.  Make sure we clear out
    // previous options.
    $out = $this->curlExec(array(CURLOPT_HTTPGET => TRUE, CURLOPT_URL => url($path, $options)));
    $this->refreshVariables(); // Ensure that any changes to variables in the other thread are picked up.

    // Replace original page output with new output from redirected page(s).
    if (($new = $this->checkForMetaRefresh())) {
      $out = $new;
    }
    return $out;
  }

The CURLOPT_RETURNTRANSFER is not set! And the curlExec() function won't init it! Because in the init method I described higher, the $this->curl_options is never modified!!!

So, I had this:

- $out = $this->curlExec(array(CURLOPT_HTTPGET => TRUE, CURLOPT_URL => url($path, $options)));
+ $out = $this->curlExec(array(CURLOPT_RETURNTRANSFER => TRUE, CURLOPT_HTTPGET => TRUE, CURLOPT_URL => url($path, $options)));

And that works (I don't have my firefox freezing anymore!) Yay! But tests still won't pass.

But, that's better, my curl_exec now have a return when using the drupalGet() method.

I think this CURL_RETURNTRANSFER thing should be per default?

pounard’s picture

When I look at your code, the $this->ch is inited only once in curlConnect() method, and curl_setopt_array() is called with the good options the first time, so I can't figure it out why, but when it pass through the drupalGet() method, in my environement, it seems that the curl options have been erased somewhere, and the the CURL_RETURNTRANSFERT is not present anymore. Where it's bad is this should be set into the $this->curl_options, right? (Well I still doubt about that because in curlConnect() method $this->curl_options is used, to concatenate with new options, but never write back with those).

pounard’s picture

Title: Simpletest and curl, something is not working right » Bad curl_setopt() when open_basedir restriction is in effect.
Category: support » bug
Priority: Critical » Normal

curl_setopt can not set the CURLOPT_FOLLOWLOCATION option when open_basedir is in effectn see those two:

(thanks chx).

A workaround may be done in curlExec() method (http://www.php.net/manual/en/function.curl-setopt.php#85580).

boombatower’s picture

Project: SimpleTest » Drupal core
Version: 6.x-2.5 » 7.x-dev
Component: Code » simpletest.module

Drupal 7 related?

pounard’s picture

Good question, only saw the bug on D6 (did not tested D7).
But, this is a CURL related bug, not only Drupal's, so it has good chances to be in D7.

tic2000’s picture

On my Drupal 7 clean install I can't finish the tests. All errors have in common 2 things:
1. While running the tests at some point they stop and I get "Notice: Undefined index: HTTP_USER_AGENT..."
2. On the results page always the same exception: "curl_setopt_array(): open_basedir restriction in effect. File() is not within the allowed path(s)".

dave reid’s picture

I just had someone duplicate this error in #494682: At node deletion the browser hangs on the .../node/<nid>/delete page. I'll have the user report their server/config details here.

dimmie’s picture

Re #10, I'm this user.

I'm running debian lenny / testing

$ dpkg -l apache2 "*php*" | grep -v none | cut -c1-80
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend
||/ Name                                 Version                          Descri
+++-====================================-================================-======
ii  apache2                              2.2.11-5                         Apache
ii  libapache2-mod-php5                  5.2.9.dfsg.1-4                   server
ii  php-doc                              20081024-1                       Docume
ii  php5                                 5.2.9.dfsg.1-4                   server
ii  php5-cgi                             5.2.9.dfsg.1-4                   server
ii  php5-cli                             5.2.9.dfsg.1-4                   comman
ii  php5-common                          5.2.9.dfsg.1-4                   Common
ii  php5-curl                            5.2.9.dfsg.1-4                   CURL m
ii  php5-gd                              5.2.9.dfsg.1-4                   GD mod
ii  php5-mcrypt                          5.2.9.dfsg.1-4                   MCrypt
ii  php5-mysql                           5.2.9.dfsg.1-4                   MySQL 
ii  php5-pspell                          5.2.9.dfsg.1-4                   pspell
ii  phpmyadmin                           4:3.1.5-1                        MySQL

Also I am using open_basedir, and I suspect my ISP does too. I can turn it off on my home server, but I have no access to my ISP's setting.

-rg-

edited

  • Deinstalled simpletest 6.x-2.8, removed module source tree
  • Installed simpletest 6.x-2.x-dev, testing Simple Test still loops
  • removed open_basedir spec, Simple Test tests OK

Can provide a dump of the simpletest DB table (at forced termination) if required.

dimmie’s picture

@#6

I tried to be brave and slip the curl_redir_exec into the file simpletest/drupal_web_test_case.php and in the flow of things in the culrExec method, without other modifications.

The good news is that the unit testing of Simple test and XML sitemap did run to completion with an open_basedir directive in effeect.
The bad news is that this brainless insertion seemed to produce too many test mismatches to be of real use, even as a stopgap solution.

-rg-

dave reid’s picture

Status: Active » Closed (duplicate)

Hey look there's an existing SimpleTest issue for this. :) Marking as a duplicate. #323110: Remove the open_basedir requirement check