Date: Mon, 11 Aug 2014 19:20:56 +0000 (UTC) From: Peter Wemm <peter@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r269832 - in vendor/serf/serf-1.3.7: . buckets Message-ID: <53e91798.2aac.617d7d2c@svn.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: peter Date: Mon Aug 11 19:20:56 2014 New Revision: 269832 URL: http://svnweb.freebsd.org/changeset/base/269832 Log: Tag serf-1.3.7 import Added: vendor/serf/serf-1.3.7/ - copied from r268959, vendor/serf/dist/ Replaced: vendor/serf/serf-1.3.7/CHANGES - copied unchanged from r269831, vendor/serf/dist/CHANGES vendor/serf/serf-1.3.7/buckets/ssl_buckets.c - copied unchanged from r269831, vendor/serf/dist/buckets/ssl_buckets.c vendor/serf/serf-1.3.7/serf.h - copied unchanged from r269831, vendor/serf/dist/serf.h Copied: vendor/serf/serf-1.3.7/CHANGES (from r269831, vendor/serf/dist/CHANGES) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/serf/serf-1.3.7/CHANGES Mon Aug 11 19:20:56 2014 (r269832, copy of r269831, vendor/serf/dist/CHANGES) @@ -0,0 +1,283 @@ +Serf 1.3.7 [2014-08-11, from /tags/1.3.7, r2411] + Handle NUL bytes in fields of an X.509 certificate. (r2393, r2399) + +Serf 1.3.6 [2014-06-09, from /tags/1.3.6, r2372] + Revert r2319 from serf 1.3.5: this change was making serf call handle_response + multiple times in case of an error response, leading to unexpected behavior. + +Serf 1.3.5 [2014-04-27, from /tags/1.3.5, r2355] + Fix issue #125: no reverse lookup during Negotiate authentication for proxies. + Fix a crash caused by incorrect reuse of the ssltunnel CONNECT request (r2316) + Cancel request if response parsing failed + authn callback set (r2319) + Update the expired certificates in the test suite. + + +Serf 1.3.4 [2014-02-08, from /tags/1.3.4, r2310] + Fix issue #119: Endless loop during ssl tunnel setup with Negotiate authn + Fix issue #123: Can't setup ssl tunnel which sends Connection close header + Fix a race condition when initializing OpenSSL from multiple threads (r2263) + Fix issue #138: Incorrect pkg-config file when GSSAPI isn't configured + + +Serf 1.3.3 [2013-12-09, from /tags/1.3.3, r2242] + Fix issue 129: Try more addresses of multihomed servers + Handle X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE correctly (r2225) + Return APR_TIMEUP from poll() to enable detecting connection timeouts (r2183) + + +Serf 1.3.2 [2013-10-04, from /tags/1.3.2, r2195] + Fix issue 130: HTTP headers should be treated case-insensitively + Fix issue 126: Compilation breaks with Codewarrior compiler + Fix crash during cleanup of SSL buckets in apr_terminate() (r2145) + Fix Windows build: Also export functions with capital letters in .def file + Fix host header when url contains a username or password (r2170) + Ensure less TCP package fragmentation on Windows (r2145) + Handle authentication for responses to HEAD requests (r2178,-9) + Improve serf_get: add option to add request headers, allow url with query, + allow HEAD requests (r2143,r2175,-6) + Improve RFC conformance: don't expect body for certain responses (r2011,-2) + Do not invoke progress callback when no data was received (r2144) + And more test suite fixes and build warning cleanups + SCons-related fixes: + Fix build when GSSAPI not in default include path (2155) + Fix OpenBSD build: always map all LIBPATH entries into RPATH (r2156) + Checksum generation in Windows shared libraries for release builds (2162) + Mac OS X: Use MAJOR version only in dylib install name (r2161) + Use both MAJOR and MINOR version for the shared library name (2163) + Fix the .pc file when installing serf in a non-default LIBDIR (r2191) + + +Serf 1.3.1 [2013-08-15, from /tags/1.3.1, r2138] + Fix issue 77: Endless loop if server doesn't accept Negotiate authentication. + Fix issue 114: ssl/tls renegotiation fails + Fix issue 120: error with ssl tunnel over proxy with KeepAlive off and + Basic authentication. + Fixed bugs with authentication (r2057,2115,2118) + SCons-related fixes: + Fix issue 111: add flag to set custom library path + Fix issue 112: add soname + Fix issue 113: add gssapi libs in the serf pc file + Fix issue 115: Setting RPATH on Solaris broken in SConstruct + Fix issue 116: scons check should return non-zero exit staths + Fix issue 121: make CFLAGS, LIBS, LINKFLAGS and CPPFLAGS take a space- + separated list of flags. + Fix issue 122: make scons PREFIX create the folder if it doesn't exist + Mac OS X: Fix scons --install-sandbox + Solaris: Fix build with cc, don't use unsupported compiler flags + Require SCons version 2.3.0 or higher now (for the soname support). + + +Serf 1.3.0 [2013-07-23, from /tags/1.3.0, r2074] + Fix issue 83: use PATH rather than URI within an ssltunnel (r1952) + Fix issue 108: improved error reporting from the underlying socket (r1951) + NEW: Switch to the SCons build system; retire serfmake, serf.mak, autotools + Improved Basic and Digest authentication: + - remember credentials on a per-server basis + - properly manage authentication realms + - continue functioning when a server sets KeepAlive: off + Windows: add support for NTLM authentication + Improved 2617 compliance: always use strongest authentication (r1968,1971) + Fixed bugs with proxy authentication and SSL tunneling through a proxy + Fixed bugs the response parser (r2032,r2036) + SSL connection performance improvements + Huge expansion of the test suite + + +Serf 1.2.1 [2013-06-03, from /tags/1.2.1, r1906] + Fix issue 95: add gssapi switches to configure (r1864, r1900) + Fix issue 97: skip mmap bucket if APR_HAS_MMAP is undefined (r1877) + Fix issue 100: building against an old Windows Platform SDK (r1881) + Fix issue 102: digest authentication failures (r1885) + Improve error return values in SSPI authentication (r1804) + Ensure serf-1.pc is constructed by serfmake (r1865) + Optimize SPNego authentication processing (r1868) + Reject certs that application does not like (r1794) + Fix possible endless loop in serf_linebuf_fetch() (r1816) + Windows build: dereference INTDIR in serf.mak (r1882) + + +Serf 1.2.0 [2013-02-22, from /tags/1.2.0, r1726] + Fixed issue 94: Serf can enter an infinite loop when server aborts conn. + Fixed issue 91: Serf doesn't handle an incoming 408 Timeout Request + Fixed issue 80: Serf is not handling Negotiate authentication correctly + Fixed issue 77: Endless loop if server doesn't accept Negotiate authn + Fixed issue 93: cleanup-after-fork interferes with parent (r1714) + Fixed most of issue 89: Support REAL SPNEGO authentication + Enable Negotiate/Kerberos support for proxy servers. + Return error when C-L, chunked, gzip encoded response bodies were + truncated (due to aborted connection) (r1688) + Add a logging mechanism that can be enabled at compile-time. + Don't lookup server address if a proxy was configured. (r1706) + Fix an off-by-one in buffer sizing (r1695) + Disable SSL compression by default + API to enable it (r1692) + New serf_connection_get_latency() for estimated network latency (r1689) + New error code and RFC compliance for the HTTPS tunnel (r1701, r1644) + Handle EINTR when a user suspends and then backgrounds the app (r1708) + Minor fixes and test suite improvements. + + +Serf 1.1.1 [2012-10-04, from /tags/1.1.1, r1657] + Fixed issue 86: ensure requeued requests are correctly handled. + This fixes: + - infinite loop with multiple connection resets or SIGPIPE errors + - "connection" hang where we would not re-queue requests that are + held after we re-connect + Fixed issue 74: test_all goes in an endless loop + Fix memleak when conn. is closed explicitly/due to pool cleanups (r1623) + Windows: Fix https connection aborts (r1628..-30,-33,-34,-37) + Add new error codes for the SSL bucket + + +Serf 1.1.0 [2012-06-07, from /tags/1.1.0, r1617] + New: serf_bucket_request_set_CL() for C-L based, non-chunked requests + New: serf_ssl_server_cert_chain_callback_set() for full-chain validation + + +Serf 1.0.3 [2012-03-20, from /tags/1.0.3, r1586] + Map more OpenSSL errors into SERF_SSL_CERT_UNKNOWNCA (r1573) + + +Serf 1.0.2 + Not released. + + +Serf 1.0.1 [2012-02-15, from /tags/1.0.1, r1569] + FreeBSD fixes in the test suite (r1560, r1565) + Minor build fixes + + +Serf 1.0.0 [2011-07-15, from /tags/1.0.0, r1540] + Fixed issue 38: enable builds using non-GNU make + Fixed issue 49: support SSL tunnels for HTTPS via a proxy + Fixed issue 56: allow Subject Alternative Name, and enable SNI + Fixed issue 61: include order dependencies + Fixed issue 66: improved error reporting when creating install dirs + Fixed issue 71: handle ECONNREFUSED on Windows + Fixed issue 79: destroy the APR allocator, if we create one + Fixed issue 81: build failed on APR 0.9.x + Major performance improvements and bug fixes for SSL buckets/handling (r1462) + Add a new "iovec" bucket type (r1434) + Minimize network packet writes based on ra_serf analysis (r1467, r1471) + Fix out of order issue with multiple priority requests (r1469) + Work around broken WSAPoll() impl on Windows introduced in APR 1.4.0 (r1506) + Fix 100% CPU usage with many pipelined requests (r1456) + Corrected contents of build/serf.def; it now includes bucket types (r1512) + Removed "snapshot" feature from buckets (r1503) + Various improvements to the test system + Various memory leak fixes + + +Serf 0.7.2 [2011-03-12, from /tags/0.7.2, r1452] + Actually disable Nagle when creating a connection (r1441) + Return error when app asks for HTTPS over proxy connection (r1433) + + +Serf 0.7.1 [2011-01-25, from /tags/0.7.1, r1432] + Fix memory leak when using SSL (r1408, r1416) + Fix build for blank apr-util directory (r1421) + + +Serf 0.7.0 [2010-08-25, from /tags/0.7.0, r1407] + Fix double free abort when destroying request buckets + Fix test server in unit test framework to avoid random test failures + Allow older Serf programs which don't use the new authn framework to still + handle authn without forcing them to switch to the new framework. (r1401) + Remove the SERF_DECLARE macros, preferring a .DEF file for Windows + Barrier buckets now pass read_iovec to their wrapped bucket + Fix HTTP header parsing to allow for empty header values + + +Serf 0.6.1 [2010-05-14, from /tags/0.6.1, r1370] + Generally: this release fixes problems with the 0.4.0 packaging + Small compilation fix in outgoing.c for Windows builds + + +Serf 0.6.0 + Not released. + + +Serf 0.5.0 + Not released. + + +Serf 0.4.0 + WITHDRAWN: this release misstated itself as 0.5.0; use a later release + + Provide authn framework, supporting Basic, Digest, Kerberos (SSPI, GSS), + along with proxy authn using Basic or Digest + Added experimental listener framework, along with test_server.c + Improvements and fixes to SSL support, including connection setup changes + Experimental support for unrequested, arriving ("async") responses + Experimental BWTP support using the async arrival feature + Headers are combined on read (not write), to ease certian classes of parsing + Experimental feature on aggregate buckets for a callback-on-empty + Fix the bucket allocator for when APR is using its pool debugging features + Proxy support in the serf_get testing utility + Fix to include the port number in the Host header + serf_get propagates errors from the response, instead of aborting (Issue 52) + Added serf_lib_version() for runtime version tests + + +Serf 0.3.1 [2010-02-14, from /tags/0.3.1, r1322] + Fix loss of error on request->setup() callback. (Issue 47) + Support APR 2.x. (Issue 48) + Fixed slowdown in aggregate bucket with millions of child buckets + Avoid hang in apr_pollset_poll() by unclosed connections after fork() + + +Serf 0.3.0 [2009-01-26, from /tags/0.3.0, r1217] + Support LTFLAGS override as a config-time env. variable (Issue 44) + Fix CUTest test harness compilation on Solaris (Issue 43) + Fix small race condition in OpenSSL initialization (Issue 39) + Handle content streams larger than 4GB on 32-bit OSes (Issue 41) + Fix test_ssl.c compilation with mingw+msys + Fix conn close segfault by explicitly closing conn when pool is destroyed + Expose the depth of the SSL certificate so the validator can use that info + Fix socket address family issue when opening a connection to a proxy + Provide new API to take snapshots of buckets + Implement snapshot API for simple and aggregate buckets + Build with bundled apr and apr-util VPATH builds + Build with bundled OpenSSL builds + + +Serf 0.2.0 [2008-06-06, from /tags/0.2.0, r1189] + Enable use of external event loop: serf_create_context_ex + Enable adding new requests at the beginning of the request queue + Handle 'Connection:close' headers + Enable limiting the number of outstanding requests + Add readline function to simple buckets + Concatenate repeated headers using comma as separator, as per RFC 2616, + section 4.2. (Issue 29) + Add proxy server support + Add progress feedback support. (Issue 11) + Provide new API to simplify use of proxy and progress feedback support + Add callback to validate SSL server certificates. (Issue 31) + Add new test framework + Send current version string in the test programs (Issue 21) + Bugfixes: + Fix segfault with epoll when removing a NULL socket + Reset OpenSSL thread-safety callbacks when apr_terminate() called + Do not remove the socket from the pollset on pool cleanup + Do not issue double close on skt w/second one being close(-1) (Issue 33) + + +Serf 0.1.2 [2007-06-18, from /tags/0.1.2, r1115] + Enable thread-safety with OpenSSL (Issue 19) + Teach serfmake to install headers into include/serf-0 + Be more tolerant when servers close the connection without telling us + Do not open the connection until we have requests to deliver + Fix serfmake to produce the library that corresponds to the minor version + Fix a memory leak with the socket bucket (Issue 14) + Fix uninitialized branch in serf_spider (Issue 15) + + +Serf 0.1.1 [2007-05-12, from /tags/0.1.1, r1105] + Add SSL client certificate support + Implement optimized iovec reads for header buckets + Fix up 'make clean' and 'make distclean' (Issues 9, 10) + Add SERF_VERSION_AT_LEAST macro + Remove abort() calls (Issue 13) + + +Serf 0.1.0 [2006-12-14, from /tags/0.1.0, r1087] + Initial packaged release Copied: vendor/serf/serf-1.3.7/buckets/ssl_buckets.c (from r269831, vendor/serf/dist/buckets/ssl_buckets.c) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/serf/serf-1.3.7/buckets/ssl_buckets.c Mon Aug 11 19:20:56 2014 (r269832, copy of r269831, vendor/serf/dist/buckets/ssl_buckets.c) @@ -0,0 +1,1898 @@ +/* Copyright 2002-2004 Justin Erenkrantz and Greg Stein + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * ---- + * + * For the OpenSSL thread-safety locking code: + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Originally developed by Aaron Bannert and Justin Erenkrantz, eBuilt. + */ + +#include <apr_pools.h> +#include <apr_network_io.h> +#include <apr_portable.h> +#include <apr_strings.h> +#include <apr_base64.h> +#include <apr_version.h> +#include <apr_atomic.h> + +#include "serf.h" +#include "serf_private.h" +#include "serf_bucket_util.h" + +#include <openssl/bio.h> +#include <openssl/ssl.h> +#include <openssl/err.h> +#include <openssl/pkcs12.h> +#include <openssl/x509v3.h> + +#ifndef APR_VERSION_AT_LEAST /* Introduced in APR 1.3.0 */ +#define APR_VERSION_AT_LEAST(major,minor,patch) \ + (((major) < APR_MAJOR_VERSION) \ + || ((major) == APR_MAJOR_VERSION && (minor) < APR_MINOR_VERSION) \ + || ((major) == APR_MAJOR_VERSION && (minor) == APR_MINOR_VERSION && \ + (patch) <= APR_PATCH_VERSION)) +#endif /* APR_VERSION_AT_LEAST */ + +#ifndef APR_ARRAY_PUSH +#define APR_ARRAY_PUSH(ary,type) (*((type *)apr_array_push(ary))) +#endif + + +/* + * Here's an overview of the SSL bucket's relationship to OpenSSL and serf. + * + * HTTP request: SSLENCRYPT(REQUEST) + * [context.c reads from SSLENCRYPT and writes out to the socket] + * HTTP response: RESPONSE(SSLDECRYPT(SOCKET)) + * [handler function reads from RESPONSE which in turn reads from SSLDECRYPT] + * + * HTTP request read call path: + * + * write_to_connection + * |- serf_bucket_read on SSLENCRYPT + * |- serf_ssl_read + * |- serf_databuf_read + * |- common_databuf_prep + * |- ssl_encrypt + * |- 1. Try to read pending encrypted data; If available, return. + * |- 2. Try to read from ctx->stream [REQUEST bucket] + * |- 3. Call SSL_write with read data + * |- ... + * |- bio_bucket_read can be called + * |- bio_bucket_write with encrypted data + * |- store in sink + * |- 4. If successful, read pending encrypted data and return. + * |- 5. If fails, place read data back in ctx->stream + * + * HTTP response read call path: + * + * read_from_connection + * |- acceptor + * |- handler + * |- ... + * |- serf_bucket_read(SSLDECRYPT) + * |- serf_ssl_read + * |- serf_databuf_read + * |- ssl_decrypt + * |- 1. SSL_read() for pending decrypted data; if any, return. + * |- 2. Try to read from ctx->stream [SOCKET bucket] + * |- 3. Append data to ssl_ctx->source + * |- 4. Call SSL_read() + * |- ... + * |- bio_bucket_write can be called + * |- bio_bucket_read + * |- read data from ssl_ctx->source + * |- If data read, return it. + * |- If an error, set the STATUS value and return. + * + */ + +typedef struct bucket_list { + serf_bucket_t *bucket; + struct bucket_list *next; +} bucket_list_t; + +typedef struct { + /* Helper to read data. Wraps stream. */ + serf_databuf_t databuf; + + /* Our source for more data. */ + serf_bucket_t *stream; + + /* The next set of buckets */ + bucket_list_t *stream_next; + + /* The status of the last thing we read. */ + apr_status_t status; + apr_status_t exhausted; + int exhausted_reset; + + /* Data we've read but not processed. */ + serf_bucket_t *pending; +} serf_ssl_stream_t; + +struct serf_ssl_context_t { + /* How many open buckets refer to this context. */ + int refcount; + + /* The pool that this context uses. */ + apr_pool_t *pool; + + /* The allocator associated with the above pool. */ + serf_bucket_alloc_t *allocator; + + /* Internal OpenSSL parameters */ + SSL_CTX *ctx; + SSL *ssl; + BIO *bio; + + serf_ssl_stream_t encrypt; + serf_ssl_stream_t decrypt; + + /* Client cert callbacks */ + serf_ssl_need_client_cert_t cert_callback; + void *cert_userdata; + apr_pool_t *cert_cache_pool; + const char *cert_file_success; + + /* Client cert PW callbacks */ + serf_ssl_need_cert_password_t cert_pw_callback; + void *cert_pw_userdata; + apr_pool_t *cert_pw_cache_pool; + const char *cert_pw_success; + + /* Server cert callbacks */ + serf_ssl_need_server_cert_t server_cert_callback; + serf_ssl_server_cert_chain_cb_t server_cert_chain_callback; + void *server_cert_userdata; + + const char *cert_path; + + X509 *cached_cert; + EVP_PKEY *cached_cert_pw; + + apr_status_t pending_err; + + /* Status of a fatal error, returned on subsequent encrypt or decrypt + requests. */ + apr_status_t fatal_err; +}; + +typedef struct { + /* The bucket-independent ssl context that this bucket is associated with */ + serf_ssl_context_t *ssl_ctx; + + /* Pointer to the 'right' databuf. */ + serf_databuf_t *databuf; + + /* Pointer to our stream, so we can find it later. */ + serf_bucket_t **our_stream; +} ssl_context_t; + +struct serf_ssl_certificate_t { + X509 *ssl_cert; + int depth; +}; + +static void disable_compression(serf_ssl_context_t *ssl_ctx); +static char * + pstrdup_escape_nul_bytes(const char *buf, int len, apr_pool_t *pool); + +#if SSL_VERBOSE +/* Log all ssl alerts that we receive from the server. */ +static void +apps_ssl_info_callback(const SSL *s, int where, int ret) +{ + const char *str; + int w; + w = where & ~SSL_ST_MASK; + + if (w & SSL_ST_CONNECT) + str = "SSL_connect"; + else if (w & SSL_ST_ACCEPT) + str = "SSL_accept"; + else + str = "undefined"; + + if (where & SSL_CB_LOOP) { + serf__log(SSL_VERBOSE, __FILE__, "%s:%s\n", str, + SSL_state_string_long(s)); + } + else if (where & SSL_CB_ALERT) { + str = (where & SSL_CB_READ) ? "read" : "write"; + serf__log(SSL_VERBOSE, __FILE__, "SSL3 alert %s:%s:%s\n", + str, + SSL_alert_type_string_long(ret), + SSL_alert_desc_string_long(ret)); + } + else if (where & SSL_CB_EXIT) { + if (ret == 0) + serf__log(SSL_VERBOSE, __FILE__, "%s:failed in %s\n", str, + SSL_state_string_long(s)); + else if (ret < 0) { + serf__log(SSL_VERBOSE, __FILE__, "%s:error in %s\n", str, + SSL_state_string_long(s)); + } + } +} +#endif + +/* Returns the amount read. */ +static int bio_bucket_read(BIO *bio, char *in, int inlen) +{ + serf_ssl_context_t *ctx = bio->ptr; + const char *data; + apr_status_t status; + apr_size_t len; + + serf__log(SSL_VERBOSE, __FILE__, "bio_bucket_read called for %d bytes\n", + inlen); + + if (ctx->encrypt.status == SERF_ERROR_WAIT_CONN + && BIO_should_read(ctx->bio)) { + serf__log(SSL_VERBOSE, __FILE__, + "bio_bucket_read waiting: (%d %d %d)\n", + BIO_should_retry(ctx->bio), BIO_should_read(ctx->bio), + BIO_get_retry_flags(ctx->bio)); + /* Falling back... */ + ctx->encrypt.exhausted_reset = 1; + BIO_clear_retry_flags(bio); + } + + status = serf_bucket_read(ctx->decrypt.pending, inlen, &data, &len); + + ctx->decrypt.status = status; + + serf__log(SSL_VERBOSE, __FILE__, "bio_bucket_read received %d bytes (%d)\n", + len, status); + + if (!SERF_BUCKET_READ_ERROR(status)) { + /* Oh suck. */ + if (len) { + memcpy(in, data, len); + return len; + } + if (APR_STATUS_IS_EOF(status)) { + BIO_set_retry_read(bio); + return -1; + } + } + + return -1; +} + +/* Returns the amount written. */ +static int bio_bucket_write(BIO *bio, const char *in, int inl) +{ + serf_ssl_context_t *ctx = bio->ptr; + serf_bucket_t *tmp; + + serf__log(SSL_VERBOSE, __FILE__, "bio_bucket_write called for %d bytes\n", + inl); + + if (ctx->encrypt.status == SERF_ERROR_WAIT_CONN + && !BIO_should_read(ctx->bio)) { + serf__log(SSL_VERBOSE, __FILE__, + "bio_bucket_write waiting: (%d %d %d)\n", + BIO_should_retry(ctx->bio), BIO_should_read(ctx->bio), + BIO_get_retry_flags(ctx->bio)); + /* Falling back... */ + ctx->encrypt.exhausted_reset = 1; + BIO_clear_retry_flags(bio); + } + + tmp = serf_bucket_simple_copy_create(in, inl, + ctx->encrypt.pending->allocator); + + serf_bucket_aggregate_append(ctx->encrypt.pending, tmp); + + return inl; +} + +/* Returns the amount read. */ +static int bio_file_read(BIO *bio, char *in, int inlen) +{ + apr_file_t *file = bio->ptr; + apr_status_t status; + apr_size_t len; + + BIO_clear_retry_flags(bio); + + len = inlen; + status = apr_file_read(file, in, &len); + + if (!SERF_BUCKET_READ_ERROR(status)) { + /* Oh suck. */ + if (APR_STATUS_IS_EOF(status)) { + BIO_set_retry_read(bio); + return -1; + } else { + return len; + } + } + + return -1; +} + +/* Returns the amount written. */ +static int bio_file_write(BIO *bio, const char *in, int inl) +{ + apr_file_t *file = bio->ptr; + apr_size_t nbytes; + + BIO_clear_retry_flags(bio); + + nbytes = inl; + apr_file_write(file, in, &nbytes); + + return nbytes; +} + +static int bio_file_gets(BIO *bio, char *in, int inlen) +{ + return bio_file_read(bio, in, inlen); +} + +static int bio_bucket_create(BIO *bio) +{ + bio->shutdown = 1; + bio->init = 1; + bio->num = -1; + bio->ptr = NULL; + + return 1; +} + +static int bio_bucket_destroy(BIO *bio) +{ + /* Did we already free this? */ + if (bio == NULL) { + return 0; + } + + return 1; +} + +static long bio_bucket_ctrl(BIO *bio, int cmd, long num, void *ptr) +{ + long ret = 1; + + switch (cmd) { + default: + /* abort(); */ + break; + case BIO_CTRL_FLUSH: + /* At this point we can't force a flush. */ + break; + case BIO_CTRL_PUSH: + case BIO_CTRL_POP: + ret = 0; + break; + } + return ret; +} + +static BIO_METHOD bio_bucket_method = { + BIO_TYPE_MEM, + "Serf SSL encryption and decryption buckets", + bio_bucket_write, + bio_bucket_read, + NULL, /* Is this called? */ + NULL, /* Is this called? */ + bio_bucket_ctrl, + bio_bucket_create, + bio_bucket_destroy, +#ifdef OPENSSL_VERSION_NUMBER + NULL /* sslc does not have the callback_ctrl field */ +#endif +}; + +static BIO_METHOD bio_file_method = { + BIO_TYPE_FILE, + "Wrapper around APR file structures", + bio_file_write, + bio_file_read, + NULL, /* Is this called? */ + bio_file_gets, /* Is this called? */ + bio_bucket_ctrl, + bio_bucket_create, + bio_bucket_destroy, +#ifdef OPENSSL_VERSION_NUMBER + NULL /* sslc does not have the callback_ctrl field */ +#endif +}; + +typedef enum san_copy_t { + EscapeNulAndCopy = 0, + ErrorOnNul = 1, +} san_copy_t; + + +static apr_status_t +get_subject_alt_names(apr_array_header_t **san_arr, X509 *ssl_cert, + san_copy_t copy_action, apr_pool_t *pool) +{ + STACK_OF(GENERAL_NAME) *names; + + /* assert: copy_action == ErrorOnNul || (san_arr && pool) */ + + if (san_arr) { + *san_arr = NULL; + } + + /* Get subjectAltNames */ + names = X509_get_ext_d2i(ssl_cert, NID_subject_alt_name, NULL, NULL); + if (names) { + int names_count = sk_GENERAL_NAME_num(names); + int name_idx; + + if (san_arr) + *san_arr = apr_array_make(pool, names_count, sizeof(char*)); + for (name_idx = 0; name_idx < names_count; name_idx++) { + char *p = NULL; + GENERAL_NAME *nm = sk_GENERAL_NAME_value(names, name_idx); + + switch (nm->type) { + case GEN_DNS: + if (copy_action == ErrorOnNul && + strlen(nm->d.ia5->data) != nm->d.ia5->length) + return SERF_ERROR_SSL_CERT_FAILED; + if (san_arr && *san_arr) + p = pstrdup_escape_nul_bytes((const char *)nm->d.ia5->data, + nm->d.ia5->length, + pool); + break; + default: + /* Don't know what to do - skip. */ + break; + } + + if (p) { + APR_ARRAY_PUSH(*san_arr, char*) = p; + } + } + sk_GENERAL_NAME_pop_free(names, GENERAL_NAME_free); + } + + return APR_SUCCESS; +} + +static apr_status_t validate_cert_hostname(X509 *server_cert, apr_pool_t *pool) +{ + char buf[1024]; + int length; + apr_status_t ret; + + ret = get_subject_alt_names(NULL, server_cert, ErrorOnNul, NULL); + if (ret) { + return ret; + } else { + /* Fail if the subject's CN field contains \0 characters. */ + X509_NAME *subject = X509_get_subject_name(server_cert); + if (!subject) + return SERF_ERROR_SSL_CERT_FAILED; + + length = X509_NAME_get_text_by_NID(subject, NID_commonName, buf, 1024); + if (length != -1) + if (strlen(buf) != length) + return SERF_ERROR_SSL_CERT_FAILED; + } + + return APR_SUCCESS; +} + +static int +validate_server_certificate(int cert_valid, X509_STORE_CTX *store_ctx) +{ + SSL *ssl; + serf_ssl_context_t *ctx; + X509 *server_cert; + int err, depth; + int failures = 0; + apr_status_t status; + + ssl = X509_STORE_CTX_get_ex_data(store_ctx, + SSL_get_ex_data_X509_STORE_CTX_idx()); + ctx = SSL_get_app_data(ssl); + + server_cert = X509_STORE_CTX_get_current_cert(store_ctx); + depth = X509_STORE_CTX_get_error_depth(store_ctx); + + /* If the certification was found invalid, get the error and convert it to + something our caller will understand. */ + if (! cert_valid) { + err = X509_STORE_CTX_get_error(store_ctx); + + switch(err) { + case X509_V_ERR_CERT_NOT_YET_VALID: + failures |= SERF_SSL_CERT_NOTYETVALID; + break; + case X509_V_ERR_CERT_HAS_EXPIRED: + failures |= SERF_SSL_CERT_EXPIRED; + break; + case X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT: + case X509_V_ERR_SELF_SIGNED_CERT_IN_CHAIN: + failures |= SERF_SSL_CERT_SELF_SIGNED; + break; + case X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY: + case X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT: + case X509_V_ERR_CERT_UNTRUSTED: + case X509_V_ERR_INVALID_CA: + case X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE: + failures |= SERF_SSL_CERT_UNKNOWNCA; + break; + case X509_V_ERR_CERT_REVOKED: + failures |= SERF_SSL_CERT_REVOKED; + break; + default: + failures |= SERF_SSL_CERT_UNKNOWN_FAILURE; + break; + } + } + + /* Validate hostname */ + status = validate_cert_hostname(server_cert, ctx->pool); + if (status) + failures |= SERF_SSL_CERT_UNKNOWN_FAILURE; + + /* Check certificate expiry dates. */ + if (X509_cmp_current_time(X509_get_notBefore(server_cert)) >= 0) { + failures |= SERF_SSL_CERT_NOTYETVALID; + } + else if (X509_cmp_current_time(X509_get_notAfter(server_cert)) <= 0) { + failures |= SERF_SSL_CERT_EXPIRED; + } + + if (ctx->server_cert_callback && + (depth == 0 || failures)) { + serf_ssl_certificate_t *cert; + apr_pool_t *subpool; + + apr_pool_create(&subpool, ctx->pool); + + cert = apr_palloc(subpool, sizeof(serf_ssl_certificate_t)); + cert->ssl_cert = server_cert; + cert->depth = depth; + + /* Callback for further verification. */ + status = ctx->server_cert_callback(ctx->server_cert_userdata, + failures, cert); + if (status == APR_SUCCESS) + cert_valid = 1; + else { + /* Even if openssl found the certificate valid, the application + told us to reject it. */ + cert_valid = 0; + /* Pass the error back to the caller through the context-run. */ + ctx->pending_err = status; + } + apr_pool_destroy(subpool); + } + + if (ctx->server_cert_chain_callback + && (depth == 0 || failures)) { + STACK_OF(X509) *chain; + const serf_ssl_certificate_t **certs; + int certs_len; + apr_pool_t *subpool; + + apr_pool_create(&subpool, ctx->pool); + + /* Borrow the chain to pass to the callback. */ + chain = X509_STORE_CTX_get_chain(store_ctx); + + /* If the chain can't be retrieved, just pass the current + certificate. */ + /* ### can this actually happen with _get_chain() ? */ + if (!chain) { + serf_ssl_certificate_t *cert = apr_palloc(subpool, sizeof(*cert)); + + cert->ssl_cert = server_cert; + cert->depth = depth; + + /* Room for the server_cert and a trailing NULL. */ + certs = apr_palloc(subpool, sizeof(*certs) * 2); + certs[0] = cert; + + certs_len = 1; + } else { + int i; + + certs_len = sk_X509_num(chain); + + /* Room for all the certs and a trailing NULL. */ + certs = apr_palloc(subpool, sizeof(*certs) * (certs_len + 1)); + for (i = 0; i < certs_len; ++i) { + serf_ssl_certificate_t *cert; + + cert = apr_palloc(subpool, sizeof(*cert)); + cert->ssl_cert = sk_X509_value(chain, i); + cert->depth = i; + + certs[i] = cert; + } + } + certs[certs_len] = NULL; + + /* Callback for further verification. */ + status = ctx->server_cert_chain_callback(ctx->server_cert_userdata, + failures, depth, + certs, certs_len); + if (status == APR_SUCCESS) { + cert_valid = 1; + } else { + /* Even if openssl found the certificate valid, the application + told us to reject it. */ + cert_valid = 0; + /* Pass the error back to the caller through the context-run. */ + ctx->pending_err = status; + } + + apr_pool_destroy(subpool); + } + + /* Return a specific error if the server certificate is not accepted by + OpenSSL and the application has not set callbacks to override this. */ + if (!cert_valid && + !ctx->server_cert_chain_callback && + !ctx->server_cert_callback) + { + ctx->pending_err = SERF_ERROR_SSL_CERT_FAILED; + } + + return cert_valid; +} + +/* This function reads an encrypted stream and returns the decrypted stream. */ +static apr_status_t ssl_decrypt(void *baton, apr_size_t bufsize, + char *buf, apr_size_t *len) +{ + serf_ssl_context_t *ctx = baton; + apr_size_t priv_len; + apr_status_t status; + const char *data; + int ssl_len; + + if (ctx->fatal_err) + return ctx->fatal_err; + + serf__log(SSL_VERBOSE, __FILE__, "ssl_decrypt: begin %d\n", bufsize); + + /* Is there some data waiting to be read? */ + ssl_len = SSL_read(ctx->ssl, buf, bufsize); + if (ssl_len > 0) { + serf__log(SSL_VERBOSE, __FILE__, + "ssl_decrypt: %d bytes (%d); status: %d; flags: %d\n", + ssl_len, bufsize, ctx->decrypt.status, + BIO_get_retry_flags(ctx->bio)); + *len = ssl_len; + return APR_SUCCESS; + } + + status = serf_bucket_read(ctx->decrypt.stream, bufsize, &data, &priv_len); + + if (!SERF_BUCKET_READ_ERROR(status) && priv_len) { + serf_bucket_t *tmp; + + serf__log(SSL_VERBOSE, __FILE__, + "ssl_decrypt: read %d bytes (%d); status: %d\n", + priv_len, bufsize, status); + + tmp = serf_bucket_simple_copy_create(data, priv_len, + ctx->decrypt.pending->allocator); + + serf_bucket_aggregate_append(ctx->decrypt.pending, tmp); *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?53e91798.2aac.617d7d2c>