From owner-freebsd-ports@freebsd.org Sat Oct 3 14:52:57 2015 Return-Path: Delivered-To: freebsd-ports@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3BFF2A0E9FF for ; Sat, 3 Oct 2015 14:52:57 +0000 (UTC) (envelope-from koobs.freebsd@gmail.com) Received: from mail-pa0-x234.google.com (mail-pa0-x234.google.com [IPv6:2607:f8b0:400e:c03::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0AA821D85 for ; Sat, 3 Oct 2015 14:52:57 +0000 (UTC) (envelope-from koobs.freebsd@gmail.com) Received: by padhy16 with SMTP id hy16so133129652pad.1 for ; Sat, 03 Oct 2015 07:52:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:reply-to:subject:references:to:from:message-id:date :user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=loCji7+R0e5TqPWaKLkV89Kg/PzlkcEukruB9PR8MDs=; b=HjKdTT0qByKQtUqmNoB7SgvauNpSWs0Ho0AZlr8g2KLp1u7f4Y2qs9PuQZTBOw09Io UfCxcD4SeusYwzOrzVBvtOCRjGtr9VXDyVPngy9ubPvnVf+99th3YjE1Q1c1Xa5rfwWb mwni/RT14UWQPZBxEHJ41pv1rfH7xsbkuNffILZn5kCZlWFVKCOhQ/LoxwXgGjZd3CrB EEJbhUSLHB43wcHYk28aTV1RFzWFjeQwMSPXrOH18F3vUFIhwSsuOr2eSuuvbcmph3ZJ GkRITasEK0LcUqCrB6VuOgGJiryb79cG0d+tKGEHjKT9eVdiZjPk8LBKTd2st4rcaoq4 aK5g== X-Received: by 10.66.131.10 with SMTP id oi10mr27457102pab.87.1443883976599; Sat, 03 Oct 2015 07:52:56 -0700 (PDT) Received: from ?IPv6:2001:44b8:31ae:7b01::8? (2001-44b8-31ae-7b01-0000-0000-0000-0008.static.ipv6.internode.on.net. [2001:44b8:31ae:7b01::8]) by smtp.gmail.com with ESMTPSA id kw10sm17914229pbc.25.2015.10.03.07.52.53 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 03 Oct 2015 07:52:56 -0700 (PDT) Sender: Kubilay Kocak Reply-To: koobs@FreeBSD.org Subject: Re: Porting S3QL and ca-root-nss.crt: Python unable to find needed certificates References: <560FD601.8030502@kulturflatrate.net> To: Niklaas Baudet von Gersdorff , freebsd-ports@freebsd.org From: Kubilay Kocak X-Enigmail-Draft-Status: N1110 Message-ID: <560FEBBB.1080203@FreeBSD.org> Date: Sun, 4 Oct 2015 00:52:43 +1000 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:41.0) Gecko/20100101 Thunderbird/41.0 MIME-Version: 1.0 In-Reply-To: <560FD601.8030502@kulturflatrate.net> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Oct 2015 14:52:57 -0000 On 3/10/2015 11:20 PM, Niklaas Baudet von Gersdorff wrote: > Hi, > > I'm porting S3QL, see https://bitbucket.org/nikratio/s3ql/overview. It > creates a mountable filesystem on a Amazon S3 bucket. I already came > this far: https://github.com/niklaas/freebsd-port-s3ql which also > requires two python modules there are so far no ports for too: > > llfuse: https://github.com/niklaas/freebsd-port-llfuse > > dugong: https://github.com/niklaas/freebsd-port-dugong > > `poudriere testport` works fine. I installed the port on a FreeBSD > system successfully and it seems to work fine. S3QL connects to the S3 > bucket via SSL. But the connection only works if I use the following > command, explicitly stating the location of ca-root-nss.crt: > > mkfs.s3ql --backend-options > ssl-ca-path=/usr/local/share/certs/ca-root-nss.crt s3:// > > Not doing so causes the following errors: > >> Traceback (most recent call last): >> File "/usr/local/bin/mount.s3ql", line 9, in >> load_entry_point('s3ql==2.15', 'console_scripts', 'mount.s3ql')() >> File "/usr/local/lib/python3.4/site-packages/s3ql/mount.py", line 120, in main >> options.authfile, options.compress) >> File "/usr/local/lib/python3.4/site-packages/s3ql/common.py", line 340, in get_backend_factory >> backend.fetch('s3ql_passphrase') >> File "/usr/local/lib/python3.4/site-packages/s3ql/backends/common.py", line 351, in fetch >> return self.perform_read(do_read, key) >> File "/usr/local/lib/python3.4/site-packages/s3ql/backends/common.py", line 107, in wrapped >> return method(*a, **kw) >> File "/usr/local/lib/python3.4/site-packages/s3ql/backends/common.py", line 314, in perform_read >> fh = self.open_read(key) >> File "/usr/local/lib/python3.4/site-packages/s3ql/backends/common.py", line 107, in wrapped >> return method(*a, **kw) >> File "/usr/local/lib/python3.4/site-packages/s3ql/backends/s3c.py", line 302, in open_read >> resp = self._do_request('GET', '/%s%s' % (self.prefix, key)) >> File "/usr/local/lib/python3.4/site-packages/s3ql/backends/s3c.py", line 437, in _do_request >> query_string=query_string, body=body) >> File "/usr/local/lib/python3.4/site-packages/s3ql/backends/s3c.py", line 668, in _send_request >> self.conn.send_request(method, path, body=body, headers=headers) >> File "/usr/local/lib/python3.4/site-packages/dugong/__init__.py", line 508, in send_request >> self.timeout) >> File "/usr/local/lib/python3.4/site-packages/dugong/__init__.py", line 1396, in eval_coroutine >> if not next(crt).poll(timeout=timeout): >> File "/usr/local/lib/python3.4/site-packages/dugong/__init__.py", line 535, in co_send_request >> self.connect() >> File "/usr/local/lib/python3.4/site-packages/dugong/__init__.py", line 444, in connect >> self._sock = self.ssl_context.wrap_socket(self._sock, server_hostname=server_hostname) >> File "/usr/local/lib/python3.4/ssl.py", line 365, in wrap_socket >> _context=self) >> File "/usr/local/lib/python3.4/ssl.py", line 583, in __init__ >> self.do_handshake() >> File "/usr/local/lib/python3.4/ssl.py", line 810, in do_handshake >> self._sslobj.do_handshake() >> ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600) > > I did some research and found this *fixed* bug which is more or less recent: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=196431 > > I was wondering whether there are still some issues with > security/ca_root_nss or whether I forgot to specify something in the > Makefile? Hi Niklaas, I'm not aware of any reported issues. You could confirm this by reducing the test case to exclude your port, and attempting to test python/ssl functionality alone with something like: import socket, ssl context = ssl.SSLContext(ssl.PROTOCOL_TLSv1) context.verify_mode = ssl.CERT_REQUIRED context.check_hostname = True context.load_default_certs() s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) ssl_sock = context.wrap_socket(s, server_hostname='www.verisign.com') ssl_sock.connect(('www.verisign.com', 443)) This works for me (no errors) on my local Python 3.4 (from ports) installation. > How can I find out where python is looking for the certificates? This > would enable me to create a symlink which could be added to the final > version of my port of net/s3ql then too. (Note: I don't know python.) As per the commit messages for the bugzilla issue you mentioned, python, unlike other software (like curl iirc) uses an OpenSSL function, *not* an internal list of paths, to derive CA certificate path locations. See this commit message for full details: https://svnweb.freebsd.org/ports?view=revision&revision=378720 Note: The path locations used by this function are determined at OpenSSL *build time*. The changes made in: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=196431 and https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=189811 ensured that symlinks were created so that the certs could be found at those build-time path locations. If you're not using OpenSSL from *ports* (ie, from base) then you'll want to ensure the ETCSYMLINK option is enabled when installing ca_root_nss from ports. Note: The package for ca_root_nss has this option enabled by default, which is the exact and only change bug #189811 fixed. Jump in #freebsd-python on IRC (freenode) if your investigations turn up any issues regarding Python/SSL certificate verification. ./koobs