Date: Sun, 4 Oct 2015 00:52:43 +1000 From: Kubilay Kocak <koobs@FreeBSD.org> To: Niklaas Baudet von Gersdorff <niklaas@kulturflatrate.net>, freebsd-ports@freebsd.org Subject: Re: Porting S3QL and ca-root-nss.crt: Python unable to find needed certificates Message-ID: <560FEBBB.1080203@FreeBSD.org> In-Reply-To: <560FD601.8030502@kulturflatrate.net> References: <560FD601.8030502@kulturflatrate.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 3/10/2015 11:20 PM, Niklaas Baudet von Gersdorff wrote: > Hi, > > I'm porting S3QL, see https://bitbucket.org/nikratio/s3ql/overview. It > creates a mountable filesystem on a Amazon S3 bucket. I already came > this far: https://github.com/niklaas/freebsd-port-s3ql which also > requires two python modules there are so far no ports for too: > > llfuse: https://github.com/niklaas/freebsd-port-llfuse > > dugong: https://github.com/niklaas/freebsd-port-dugong > > `poudriere testport` works fine. I installed the port on a FreeBSD > system successfully and it seems to work fine. S3QL connects to the S3 > bucket via SSL. But the connection only works if I use the following > command, explicitly stating the location of ca-root-nss.crt: > > mkfs.s3ql --backend-options > ssl-ca-path=/usr/local/share/certs/ca-root-nss.crt s3://<bucket-name> > > Not doing so causes the following errors: > >> Traceback (most recent call last): >> File "/usr/local/bin/mount.s3ql", line 9, in <module> >> load_entry_point('s3ql==2.15', 'console_scripts', 'mount.s3ql')() >> File "/usr/local/lib/python3.4/site-packages/s3ql/mount.py", line 120, in main >> options.authfile, options.compress) >> File "/usr/local/lib/python3.4/site-packages/s3ql/common.py", line 340, in get_backend_factory >> backend.fetch('s3ql_passphrase') >> File "/usr/local/lib/python3.4/site-packages/s3ql/backends/common.py", line 351, in fetch >> return self.perform_read(do_read, key) >> File "/usr/local/lib/python3.4/site-packages/s3ql/backends/common.py", line 107, in wrapped >> return method(*a, **kw) >> File "/usr/local/lib/python3.4/site-packages/s3ql/backends/common.py", line 314, in perform_read >> fh = self.open_read(key) >> File "/usr/local/lib/python3.4/site-packages/s3ql/backends/common.py", line 107, in wrapped >> return method(*a, **kw) >> File "/usr/local/lib/python3.4/site-packages/s3ql/backends/s3c.py", line 302, in open_read >> resp = self._do_request('GET', '/%s%s' % (self.prefix, key)) >> File "/usr/local/lib/python3.4/site-packages/s3ql/backends/s3c.py", line 437, in _do_request >> query_string=query_string, body=body) >> File "/usr/local/lib/python3.4/site-packages/s3ql/backends/s3c.py", line 668, in _send_request >> self.conn.send_request(method, path, body=body, headers=headers) >> File "/usr/local/lib/python3.4/site-packages/dugong/__init__.py", line 508, in send_request >> self.timeout) >> File "/usr/local/lib/python3.4/site-packages/dugong/__init__.py", line 1396, in eval_coroutine >> if not next(crt).poll(timeout=timeout): >> File "/usr/local/lib/python3.4/site-packages/dugong/__init__.py", line 535, in co_send_request >> self.connect() >> File "/usr/local/lib/python3.4/site-packages/dugong/__init__.py", line 444, in connect >> self._sock = self.ssl_context.wrap_socket(self._sock, server_hostname=server_hostname) >> File "/usr/local/lib/python3.4/ssl.py", line 365, in wrap_socket >> _context=self) >> File "/usr/local/lib/python3.4/ssl.py", line 583, in __init__ >> self.do_handshake() >> File "/usr/local/lib/python3.4/ssl.py", line 810, in do_handshake >> self._sslobj.do_handshake() >> ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600) > > I did some research and found this *fixed* bug which is more or less recent: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=196431 > > I was wondering whether there are still some issues with > security/ca_root_nss or whether I forgot to specify something in the > Makefile? Hi Niklaas, I'm not aware of any reported issues. You could confirm this by reducing the test case to exclude your port, and attempting to test python/ssl functionality alone with something like: import socket, ssl context = ssl.SSLContext(ssl.PROTOCOL_TLSv1) context.verify_mode = ssl.CERT_REQUIRED context.check_hostname = True context.load_default_certs() s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) ssl_sock = context.wrap_socket(s, server_hostname='www.verisign.com') ssl_sock.connect(('www.verisign.com', 443)) This works for me (no errors) on my local Python 3.4 (from ports) installation. > How can I find out where python is looking for the certificates? This > would enable me to create a symlink which could be added to the final > version of my port of net/s3ql then too. (Note: I don't know python.) As per the commit messages for the bugzilla issue you mentioned, python, unlike other software (like curl iirc) uses an OpenSSL function, *not* an internal list of paths, to derive CA certificate path locations. See this commit message for full details: https://svnweb.freebsd.org/ports?view=revision&revision=378720 Note: The path locations used by this function are determined at OpenSSL *build time*. The changes made in: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=196431 and https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=189811 ensured that symlinks were created so that the certs could be found at those build-time path locations. If you're not using OpenSSL from *ports* (ie, from base) then you'll want to ensure the ETCSYMLINK option is enabled when installing ca_root_nss from ports. Note: The package for ca_root_nss has this option enabled by default, which is the exact and only change bug #189811 fixed. Jump in #freebsd-python on IRC (freenode) if your investigations turn up any issues regarding Python/SSL certificate verification. ./koobs
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?560FEBBB.1080203>