Github is now forcing all connections through SSL, and this causes a problem when downloading packages from there. This is a contrived example:

projects[jquery][type] = module
projects[jquery][download][type] = get
projects[jquery][download][url] = https://github.com/jquery/jquery/tarball/master

Curl errors out with code 60: unable to validate certificate.

One solution would be to force users to compile curl w/ certificate support, but this is not very user-friendly.

Another, probably not great solution is to not validate certificates by passing the -k option to curl.

Maybe another is to allow configurable curl options, so you could do either.

CommentFileSizeAuthor
#1 970160-drush-make-curl-nossl.patch1.3 KBjoshuajabbour

Comments

joshuajabbour’s picture

joshuajabbour commented
StatusFileSize
new 970160-drush-make-curl-nossl.patch1.3 KB

Here's a patch to make drush make ignore ssl certificates. I'm not necessarily recommending we do this (though I suppose it's not really less secure than accepting non-https urls anyway).

dmitrig01’s picture

dmitrig01 commented

Yeah. Is there any optimal solution here?

joshuajabbour’s picture

joshuajabbour commented

Not AFAIK. Users either have to install the certificates into curl themselves, or patch drush make like I did. Maybe we should have an optional flag to trigger the https ignore flag?

Also, I don't know if this is a problem for wget as well...

zoo33’s picture

zoo33 commented

Just the common "this worked for me" comment. No ideas of how to solve this properly, but maybe it would be possible to first attempt the secure way, and only when that fails, resort to the -k switch?

Thanks!

boombatower’s picture

boombatower commented

If one attempts to checkout from a github repository you have the same issue right? seems like either has to be done globally for all commands drush make uses or probably shouldn't bother. On suse all I need to do is install a package containing common root authorities (custom appliance that didn't have it) and I would assume the same can be said for most distros.

cossovich’s picture

cossovich commented

I'm dealing with this very issue at the moment. It would be nice to be able to set curl options via curl_setopt in a drushrc file. Also, I've got wget present but for some reason drush is using curl for a [download][type] = "get" directive.

walkah’s picture

walkah commented

I've just run into this as well on a project.

@dmitrig01 I think the optimal way would be to have a way to trigger -k in the makefile itself (i.e. libraries[foo][download][insecure] = true ). Since the problem seems to be largely with github certificates (or self-signed I imagine would be a problem). Best not to make everyone insecure by default.

boombatower’s picture

boombatower commented

As mentioned in #5, that will only solve curl...you will have same issue with git or anything else. It just makes it more inconsistent. Either need to go all the way or don't bother.

If you want github certs to work then install the root certs on your box. On openSUSE really easy

zypper search cert
Loading repository data...
Reading installed packages...

S | Name                              | Summary                                                                       | Type   
--+-----------------------------------+-------------------------------------------------------------------------------+--------
  | apache2-example-certificates      | Example certificates for the Apache 2 Web Server                              | package
i | ca-certificates                   | Utilities for system wide CA certificate installation                         | package
  | ca-certificates-cacert            | CAcert root certificates                                                      | package
i | ca-certificates-mozilla           | CA certificates for OpenSSL                                                   | package
  | check-create-certificate          | A non-interactive script that creates an SSL certificate if it does not exist | package
i | java-ca-certificates              | Utilities CA certificate import to gcj                                        | package
i | mozilla-nss-certs                 | CA certificates for NSS                                                       | package
i | mozilla-nss-certs-32bit           | CA certificates for NSS                                                       | package
  | mozilla-nss-certs-debuginfo       | Debug information for package mozilla-nss-certs                               | package
  | mozilla-nss-certs-debuginfo-32bit | Debug information for package mozilla-nss-certs                               | package

ca-certificates or ca-certificates-mozilla seem to do the trick. I would assume all distro have some sort of root certificate package(s) which seem to be installed by default on desktop versions, but not on servers.

dman’s picture

dman commented

Following a clue I found at stackoverflow, I used an alternate download location (that the github download link normally 301 redirects you to anyway) and this seems to get through without the SSL cert complaints.
Seeing as makefiles are all about ease of redistribution, it feels that any solution that requires users to do sysadmin stuff on their server that even I don't really understand is not going to be a solution.

This may only work for github - so might not be a global solution, I dunno. But it's a useful work-around if it works.

Here's my makefile that now works

api = 2
core = 6.x

; Libraries
libraries[arc2][download][type] = "get"
; Issues with SSL certificates mean https can be a problem.
; Use this alternate download location instead, it seems to skip the issue.
; @see http://drupal.org/node/970160
;libraries[arc2][download][url] = "https://github.com/semsol/arc2/tarball/master"
libraries[arc2][download][url] = "http://nodeload.github.com/semsol/arc2/tarball/master"

libraries[arc2][directory_name] = "ARC2/arc"
libraries[arc2][destination] = "libraries"

Strangely, I was getting mixed results when I first tried, and encountered the SSL complaint.
* When calling a single makefile directly, there was no problem.
* However, when running as a recursive make as part of a larger install, or when being included using [include] - this complaint caused failures. One of those quantum errors that went away when I looked at it specifically, but came back when running in larger batches.

FYI, I discovered the secondary download link by inspecting the http headers like so:

 wget -v -s http://github.com/semsol/arc2/tarball/master

--00:58:48--  http://github.com/semsol/arc2/tarball/master
           => `master.1'
Resolving github.com... 207.97.227.239
Connecting to github.com[207.97.227.239]:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://github.com/semsol/arc2/tarball/master [following]
--00:58:49--  https://github.com/semsol/arc2/tarball/master
           => `master.1'
Connecting to github.com[207.97.227.239]:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://nodeload.github.com/semsol/arc2/tarball/master [following]
--00:58:50--  https://nodeload.github.com/semsol/arc2/tarball/master
           => `master.1'
Resolving nodeload.github.com... 207.97.227.252
Connecting to nodeload.github.com[207.97.227.252]:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 101,796 [application/octet-stream]

100%[========================================================================================================================>] 101,796       64.09K/s
leenx’s picture

leenx commented

Simple way to get around this problem, was to enable insecure https downloads for curl.

echo "insecure" >> ~/.curlrc

or with a nice reminder ...

echo -e "\n#drush make https work around \ninsecure\n" >> ~/.curlrc

When you done, just edit the .curlrc file and hash out the insecure until next time.

Hope this helps
LeeT

helmo’s picture

helmo commented
Status: Active » Closed (won't fix)

[ Powered by #1115636: Issue Macros and Templates - Drush Make]

Drush make is being merged into drush core (discussed in issue:#1310130: Put drush make in drush core)
This means that the issue queue is also moving. The Drush project has a component called 'Make' for this purpose.

We would like to take this opportunity to leave behind old/obsolete issues, allowing us to focus on a stable make command in core. E.g. one of the major tasks ahead is making more use of the Drush core code for handling downloads and decompression.

If you feel that this issue is still relevant, feel free to re-open and move it to the Drush queue.

More information will be posted on the drush_make and drush project pages.

NOTE: Please re-test after #1267228: Drush Make should use Drush core's native download abilities concurrently lands.