Problem/Motivation

I encountered this error running install.php on a Ubuntu 12.04 web server.

[Thu May 09 13:16:47 2013] [error] [client 10.10.10.10] PHP Fatal error: Call to undefined function Guzzle\\Http\\Curl\\curl_multi_init() in /var/www/mysite.org/drupal/core/vendor/guzzle/http/Guzzle/Http/Curl/CurlMulti.php on line 632, referer: http://mysite.org/drupal/core/install.php?langcode=en&profile=spark

Proposed resolution

This was resolved by installing the Ubuntu package php5-curl.

I'm filing a bug report because I hope this could be resolved for future users that run into this missing dependency by 1) dependency injection, 2) a check/warning during the install.php "Verify Requirements" steps, and/or 3) documentation at System Requirements.

Remaining tasks

  • Inject php5-curl dependency or add dependency check to install.php "Verify Requirements" step

edit: This is probably applicable upstream to D8-dev project (or already resolved there?).

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

pztrick’s picture

Project: Spark » Drupal core
Issue summary: View changes

added upstream note

Yaron Tal’s picture

Project: Drupal core » Spark
Version: 8.x-1.0-alpha8 » 8.x-1.x-dev

On a fresh D8 installation from the last git available atm:
PHP Fatal error: Call to undefined function Guzzle\\Http\\Curl\\curl_multi_init() in /var/www/virtual/drupal/core/vendor/guzzle/http/Guzzle/Http/Curl/CurlMulti.php on line 632, referer: http://drupal.yaron.dev/core/install.php?langcode=en&profile=standard
Installing curl fixed it.

Yaron Tal’s picture

Project: Spark » Drupal core
Version: 8.x-1.x-dev » 8.x-dev
Component: Code » documentation
Category: bug » task
Dave Reid’s picture

Title: Missing dependency php5-curl » Add curl extension to the requirements for core
Component: documentation » system.module
Category: task » bug

Guzzle used to be an optional thing that individual modules in core would need to add a dependency on (#1322890: Add a property to .info files to allow modules to specify required php extensions). But now we use Guzzle in the installer to download localize.d.org files, so we need to make cURL an official PHP extension requirement to core.

Dave Reid’s picture

The actual use is install_retrieve_file() needs to use Guzzle. This is an API function for all install profiles, therefore cURL is a dependency for all of Drupal.

cweagans’s picture

+1. It's way more effort and trouble to try to work around curl not being available than it is to just accept that curl is almost universally available and add it as a dependency.

klonos’s picture

Can we make php_curl a "soft" dependency (sort of what we do with clean URLs and the rewrite module in apache)? What happens to Drupal installation if curl is not available? Is it only used for downloading localization files? Perhaps we can get the basic Drupal setup working without it and show any optional component that needs curl in a disabled state.

In other words, during the installation steps we show disabled items along with some help text saying that in order to enable/install them cURL needs to be available. Also link to documentation where people can find ways to enable cURL or (such as in the case of localization files) instructions to manually download what is required.

Sounds reasonable?

Crell’s picture

I'm inclined to agree with Dave and cweagans. Let's just make it a requirement and call it a day. With language selection being the first step in the installer, we should just bail out early rather than try to deal with the rats nest that is the installer and adding even more workarounds to it.

Yaron Tal’s picture

So this would be the total patch?
There is no call to the Guzzle functions befor the requirements are checked so this + adding it to the documentation should do.

scor’s picture

Status: Active » Needs review
aspilicious’s picture

I have tiny sites on a cheap shared hosting. They don't support curl. So drupal8 will not work on the cheapest shared hostings? I'm fine with that but I'll cost us some users and websites.

Other way around, maybe this will force them to enable it :p

bojanz’s picture

So drupal8 will not work on the cheapest shared hostings? I'm fine with that but I'll cost us some users and websites.

That ship has sailed a long time ago.

On a related note, curl was always a requirement for Kickstart and we got no outrage over it.

David_Rothstein’s picture

I'm confused as to how Drupal 8 introduced a pluggable HTTP client implementation and in response we somehow have to change our global requirements. Isn't the whole point of pluggability that although a particular implementation might require curl, another one might not?

Requiring curl globally was also discussed in the original issue (#1447736: Adopt Guzzle library to replace drupal_http_request()) and in addition to cheap shared hosts (which I think is very important on its own) there's also the issue of sites in restricted environments where the server is not allowed to make HTTP requests to the outside world. A sysadmin in that kind of environment might reasonably balk at installing curl when the server by definition can never need it.

Anyway, isn't the most basic problem here just that the HTTP request is failing with a PHP fatal error? If curl isn't available, shouldn't a proper exception be thrown saying that the request couldn't be completed? Any code which makes an HTTP request already has to deal with this anyway, since anytime you try to make an HTTP request you can of course never guarantee it's going to be successful. For example, the code in the installer already deals with it:

function install_check_localization_server($uri) {
  try {
    $request = Drupal::httpClient()->head($uri);
    $response = $request->send();
    return TRUE;
  }
  catch (RequestException $e) {
    return FALSE;
  }
}
cweagans’s picture

So drupal8 will not work on the cheapest shared hostings? I'm fine with that but I'll cost us some users and websites.

I'm curious - which shared host is that? Last I checked, pretty much all of the major hosting companies have support for curl.

Godaddy: http://support.godaddy.com/help/article/285/do-you-support-curl
Site5: http://qa.site5.com/programming/php/is-php-compiled-with-support-for-htt...
Hostgator: http://support.hostgator.com/articles/pre-sales-questions/what-software-...
Dreamhost: http://wiki.dreamhost.com/CURL_PHP_tutorial

In fact, I'd make the assertion that any shared hosting that does not offer curl support is not worth bothering with. It's pretty standard for applications to check for updates these days - curl support is almost a must-have.

there's also the issue of sites in restricted environments where the server is not allowed to make HTTP requests to the outside world. A sysadmin in that kind of environment might reasonably balk at installing curl when the server by definition can never need it.

Maybe so, but in that instance, the site would need to provide some kind of no-op Guzzle backend anyways (to prevent other parts of the site from breaking). IMO, we should add a curl requirement to core and let contrib provide a no-op Guzzle backend for sites that need it. Those sorts of sites are not really representative of the majority of Drupal sites anyways.

aspilicious’s picture

#13 After your response I tried to find the specs for the hosting but I couldn't find them. So I started a chat with the helpdesk. And surprise surprise they support curl now.
(has been a while since I checked that to be honest)

So yeah even my 1.5euro/month host supports curl. (one.com)

Yaron Tal’s picture

Could we at least put the requirement in, since it is required now? Then if we agree that we should not require it someone can make a big patch to do exception handling, try alternative ways to do the same (fopen()?) and remove the requirement.

Liam Mitchell’s picture

Status: Needs review » Reviewed & tested by the community

cURL is now a requirement because of Guzzle library. #1447736: Adopt Guzzle library to replace drupal_http_request()

Patch prevents installer failing because cURL is missing.

catch’s picture

Assigned: Unassigned » Dries

I'm fine with just adding the requirement, at least for now to reflect the status quo.

Moving over to Dries in case he's got objections for any reason.

David_Rothstein’s picture

Title: Add curl extension to the requirements for core » Fatal error when installing Drupal on servers that don't have the cURL extension
Assigned: Dries » David_Rothstein
Status: Reviewed & tested by the community » Needs work

I think the status quo is that we have a simple bug and no one has explained why working around it (by adding a global requirement that doesn't reflect the reality of what Drupal actually requires) is preferable to simply fixing it :)

Assigning to myself to work on a direct fix instead.

We have no idea how many servers out there would be affected by this requirement. If we want actual data, we might be able to get it from our friends at Wordpress by the way (they collect this sort of data en masse)... I was at a talk by Andrew Nacin about a month ago that discussed this. I don't remember if he talked about cURL specifically, but some of the PHP environments out there in the actual real world were pretty crazy. For example, they have thousands (I think it may even have been tens of thousands) of sites reporting back to them from servers that don't even have the PHP hash extension turned on (an extension which is built in to PHP and which there doesn't seem like any good reason to go out of your way to turn off). So I can only imagine the number is much higher for something like cURL which is optional and which many Linux distros don't even have enabled by default, let alone PHP itself.

David_Rothstein’s picture

By the way, in starting to test this, the original bug report here actually doesn't occur during language selection; rather, it occurs at the end of the installer (when the Update Manager module tries to contact drupal.org, assuming you've selected the checkbox for it to do that).

If you choose a language other than English, you get a related error earlier in the installer too, of course.

So if you install in English and don't enable the Update Manager, you can actually install Drupal 8 on a server without cURL already.

catch’s picture

So if you install in English and don't enable the Update Manager, you can actually install Drupal 8 on a server without cURL already.

Yes this works. I thought I'd posted that here but either it went on a different issue or never pressed submit...

I can see adding some checks to the installer itself to ensure you're restricted to an English-only, update manager-free install if you don't have cURL installed. The intention when guzzle went in was to require cURL for features that needed it (but not system-wide), translation downloading in the installer wasn't considered at this point.

Feels like if we did that we'd need to warn people that they're not getting multilingual installer because they don't have cURL installed though not sure how far we should go with that.

David_Rothstein’s picture

Assigned: David_Rothstein » Unassigned
Status: Needs work » Needs review
FileSize
9.09 KB

Here's the patch. It actually comes in two parts:

  1. The actual fix for this bug, which like a a three-line change in core/vendor/guzzle/http/Guzzle/Http/Client.php. I'm including that here for testing purposes; to do it for real we'd submit it upstream (they already fixed a similar issue in https://github.com/guzzle/guzzle/pull/181 actually, but that's for people installing Guzzle directly, which isn't Drupal's use case). If it isn't accepted upstream for some reason, we could simply extend the class and override the __construct() method and do our check there.
  2. The second part of the patch is sort of an independent bug, but fixing it is required to fix this one. Basically, code throughout Drupal 8 is only catching certain types of Guzzle-related exceptions that the Guzzle library can throw (response-related exceptions), but we need it to catch more. In the patch I simply made it catch all of them (basically via a straight RequestException => GuzzleException substitution), which I wasn't sure about but after thinking about it for a while I think it's preferable since Drupal websites should never fail outright during an HTTP request; we just want to log the error no matter what it is. Someone developing a Drupal module with different requirements could handle it differently in their own HTTP requests, of course.
David_Rothstein’s picture

The intention when guzzle went in was to require cURL for features that needed it (but not system-wide)

Right, that made sense to me at the time (and still seems somewhat OK), although given that we have a pluggable HTTP client system it's theoretically wrong. But I don't think it would be the end of the world for the Aggregator module (for example) to never turn on if you don't have cURL.

However, given that on a practical level there is no real difference between a site that doesn't have cURL installed and one that does but can't make outgoing HTTP requests for some other reason, I'm not really sure there's a point... the Aggregator module (and any other) has to handle the second case at runtime anyway.

Feels like if we did that we'd need to warn people that they're not getting multilingual installer because they don't have cURL installed though not sure how far we should go with that.

So apparently we already do something like this (if you try to install in non-English and your site can't connect to the translation server for whatever reason, you get a requirements error on the screen), but the messaging is odd. Separate issue; I'm going to look for one and file it if it doesn't exist.

David_Rothstein’s picture

Alright, looks like #1912886: Improve installation language requirement descriptions and offline detection mostly covers the odd messaging. It could probably be expanded to specifically check for cURL (and have a dedicated message for that situation) if we really wanted to.

joates’s picture

Status: Needs review » Reviewed & tested by the community

patch is good !

note: there is an inconsistency where the exception variable is sometimes called $e and other times $exception but that is irrelevant to the actual functionality of the code

joates’s picture

tried d8 install after apt-get remove --purge php5-curl

this looks a lot like a Guzzle exception :)

screenshot..
issue-1991298-guzzle-exception.png

David_Rothstein’s picture

Thanks for marking this RTBC. I don't think it can be committed as is, though, because as mentioned it's making changes to the Guzzle library directly.

However, if this seems like an acceptable approach to others, a patch for the Guzzle changes could be submitted upstream at https://github.com/guzzle/guzzle, I believe, and see how it goes from there.

joates’s picture

Status: Needs work » Reviewed & tested by the community

@David_Rothstein

I just wanted to make it known that i preferred your solution to the other option of creating yet another hard dependency for the install process (i favor streamlining the on-boarding process -- e.g. installing Drupal for new users -- over increasing the complexity), i did think that the Guzzle exception was not the most helpful with its sparse message, but at least its not a fatal error as it was before.

"Upstreaming" issues is perhaps going to increasingly become a part of our process as we continue to incorporate external libraries (Symfony in this case), hopefully this won't hinder our ability to improve Drupal in the general sense, maybe we need a new status or tag "pushed upstream" or "pending upstream fix", but this is obviously not the place for that discussion ;-)

[ Edit: I learned that Guzzle is NOT in fact a Symfony component ]

alexpott’s picture

Status: Reviewed & tested by the community » Needs work

This totally needs to be fixed upstream in Guzzle...

ezheidtmann’s picture

Added upstream pull request containing the relevant changes from David_Rothstein's patch. Upstream already contains use ... RuntimeException, so my pull request only contains the conditional & throw.

https://github.com/guzzle/guzzle/pull/328

mtdowling’s picture

Curl is very commonly installed on shared hosts. I had a slide dedicated to outlining how common it is for someone to have curl installed on their system in my talk at Symfony Live. When not on a shared host, it's extremely easy (and possible ) to install curl.

If this issue becomes a blocker, take a look at how you can use Guzzle to send requests using PHP's stream wrapper. It doesn't have near as many options and I wouldn't recommend a fallback approach except in the most core and basic functions of Drupal.

http://guzzlephp.org/http-client/response.html#streaming-responses

ezheidtmann’s picture

Status: Reviewed & tested by the community » Needs work

The changes from comment #21 are now in upstream 'master', with a slightly different exception message.

https://github.com/guzzle/guzzle/commit/e3470f68606041172b08d7eab828bb92...

What is the next step to update Guzzle in Drupal?

myselfhimself’s picture

I confirm that installing php5-curl solves the issue.
Indeed, Guzzle should be updated within Drupal so as for Drupal to catch an exception and show it nicely...
Another solution would be for Drupal 8's default install profile to have a requirement on cURL for the server.

Yaron Tal’s picture

Since the issue itself isn't a drupal issue, can we close this issue (closed (won't fix))?
Then probably create a new issue to update Guzzle in Drupal?

tstoeckler’s picture

I think the goal should still be that Drupal is installable without cURL. We should alert the user that e.g. translation files cannot be downloaded, so that Drupal can only be installed in English, but we should not fail IMO.

Yaron Tal’s picture

Ok so use this issue to handle the Guzzle Exception in drupal, and create a seperate issue to update Guzzle in drupal?

tstoeckler’s picture

I think that would make sense, yes. This issue should probably be postponed on that, then.

Yaron Tal’s picture

Status: Needs work » Postponed

I created #2052923: Update Guzzle to latest version for the new issue. Postponing this issue for now.

timmillwood’s picture

I got the same error this morning with a fresh install of head.

[Wed Jul 31 09:58:17 2013] [error] [client 127.0.0.1] File does not exist: /var/www/system, referer: http://localhost/core/install.php?langcode=en&profile=standard
[Wed Jul 31 10:00:07 2013] [error] [client 127.0.0.1] PHP Fatal error:  Call to undefined function Guzzle\\Http\\Curl\\curl_multi_init() in /var/www/core/vendor/guzzle/http/Guzzle/Http/Curl/CurlMulti.php on line 632, referer: http://localhost/core/install.php?langcode=en&profile=standard

A simple page refresh seemed to resolve the issue.

Berdir’s picture

Status: Postponed » Active

The guzzle update is in, continue! :)

webchick’s picture

Priority: Normal » Major

Fatal errors are normally critical. If we consider servers without cURL an edge case, we can split the difference at major.

Yaron Tal’s picture

Am I missing something or are we wating for someone to rewrite the patch in #21 by @David_Rothstein, but without the Guzzle change?
Edit: and a test case I guess?

anavarre’s picture

Looks like Guzzle's Client.php file does have the changes David was talking about in #21 as @ezheidtmann got his pull request merged per Merge pull request #328 from ezheidtmann/exception-if-no-curl (see #31)

Should we update the issue summary to reflect that and move on with the remaining tasks?

pingers’s picture

Status: Active » Needs review
FileSize
378 bytes

I think this is all we need right?

(the only difference to the patch in comment 8 is maintaining alphabetical order of dependencies)

David_Rothstein’s picture

Besides the other reasons already mentioned above, I don't think that patch will even work. It would address the issue at the end of the installer, but likely not the one at the beginning of the installer (when a non-English language is selected) because doesn't that happen before the requirements check ever runs?

I think we need to continue with #21. Don't have time to work on it at the moment, but it should just need a reroll to remove the changes to the Guzzle library (by the way, thanks to the people who got that change into Guzzle!) and also check if there are other places in core that were added in the last few months that would need changes similar to those in the rest of the patch.

pingers’s picture

Here's a re-roll of #21 and a screenshot of the installer after selecting "Deutsch" on the first step (instead of English).

Run out of time today to take this further.

Status: Needs review » Needs work

The last submitted patch, curl-fatal-1991298-45.patch, failed testing.

pingers’s picture

Status: Needs work » Needs review
FileSize
1.88 KB
7.2 KB
157.75 KB

Made some progress in Prague airport...

pingers’s picture

Issue summary: View changes

formatted per guidelines

sun’s picture

There are a range of contributed modules that do not/cannot work at all without outbound HTTP requests.

If we allow Drupal to be installed without any way of performing outbound HTTP requests, then Core needs to supply a way for contributed modules to deny their installation.

Given that Guzzle is pluggable and that Guzzle may not even rely on cURL on a particular site, an abstract "Able to issue outbound HTTP requests" requirement check becomes even more necessary.

Otherwise, as a contrib/custom module developer, I have no idea at all anymore for how to check whether my module is able to work in the first place. That is, because wherever I previously checked for the exact requirements of drupal_http_request() (streams/sockets), the abstraction now requires me to check for "any possible thing."


The much more simple alternative: Require cURL to exist and to be functional.

In the grand scheme of things (i.e., where the web generally moving towards, which functionality requires outbound HTTP requests, and also, what competing applications are doing) it would be a good product management decision to simply require cURL.

Yaron Tal’s picture

Otherwise, as a contrib/custom module developer, I have no idea at all anymore for how to check whether my module is able to work in the first place. That is, because wherever I previously checked for the exact requirements of drupal_http_request() (streams/sockets), the abstraction now requires me to check for "any possible thing."

In the patch in #47 there is a check if (!extension_loaded('curl')) ..., a module maintainer could use that same check, right? Not sure how we could allow contrib modules to easily provide other libraries..

If we would agree on not making Curl a dependency for Drupal core, I think we should also change the requirement message to include that selecting "English" as the installation language would also solve the requirement.

A quote from the issue to add Guzzle to Drupal #1447736: Adopt Guzzle library to replace drupal_http_request():

The other concern was that Guzzle requires cURL, whereas drupal_http_request() does not. However, based on feedback so far it appears that cURL is far more common than it used to be, so making it a requirement for outgoing requests is not an onerous requirement. Arguably it's less onerous than Drupal's performance requirements, which effectively mandate APC already. The recommended resolution here is "meh, OK, you need cURL. Such is the modern web."

So I guess the choice was already made when Guzzle was added. Which would mean just adding curl to the dependencies in system_requirements() would be the fix we need.

David_Rothstein’s picture

I still don't see how adding curl as a requirement would even solve the bug; see #44. Nor will it provide a reliable way for modules to determine that the server can make outgoing HTTP requests. (There's no reliable way to do that, and never was; see #245990: HTTP request testing unreliable and may be undesirable for some fun history.)

All we have to do to fix the bug here is catch the exceptions that Guzzle is currently throwing.... Really, that's it :)

pingers’s picture

That was my understanding months ago :-)

Crell’s picture

mr.baileys’s picture

Status: Needs review » Needs work
Related issues: +#2052923: Update Guzzle to latest version
FileSize
8.85 KB

Re-rolled since the previous patch no longer applied to HEAD (update module was refactored in #1991298: Fatal error when installing Drupal on servers that don't have the cURL extension).

All we have to do to fix the bug here is catch the exceptions that Guzzle is currently throwing.... Really, that's it :)

Guzzle throws the RuntimeException (which implements GuzzleException) in the Client constructor. Since Guzzle is almost always instantiated as part of dependency injection, I'm not sure how/where to properly catch and handle the exception.

For example, with cURL disabled and the patch applied, the installer no longer fails (or at least it now provides a cleaner, more helpful, message when selecting non-English as language), but the update manager is broken (provided update checking was not disabled during installation). When I try to load *any* admin page, I'm greeted with Guzzle\Common\Exception\RuntimeException: The PHP cURL extension must be installed to use Guzzle. in Guzzle\Http\Client->__construct() (line 70 of core/vendor/guzzle/http/Guzzle/Http/Client.php).

damiankloip’s picture

Sorry, shouldn't we just wait for #2210637: Upgrade to Guzzle 4 which will basically fix this issue anyway?

catch’s picture

Status: Needs work » Closed (duplicate)

Yep. Marking this as duplicate of #2210637: Upgrade to Guzzle 4 and I bumped that issue to critical.