From owner-svn-src-projects@freebsd.org Thu Sep 20 18:39:02 2018 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CB3A010A55C4 for ; Thu, 20 Sep 2018 18:39:02 +0000 (UTC) (envelope-from bjkfbsd@gmail.com) Received: from mail-ot1-x330.google.com (mail-ot1-x330.google.com [IPv6:2607:f8b0:4864:20::330]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5C68879705; Thu, 20 Sep 2018 18:39:02 +0000 (UTC) (envelope-from bjkfbsd@gmail.com) Received: by mail-ot1-x330.google.com with SMTP id n5-v6so10462672otl.5; Thu, 20 Sep 2018 11:39:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=/SAqThKArF2gNz1hxBWhTIoJQFzBX/onOEgSljEeRtM=; b=GBus5H55sM/sfU8EMJa79Ebt7N5+DBWbkdDtak4qKn1lBeOkFICoNiwIHyzmJ0qZeZ vwXEdp8Wa7oC39mFUeLNL8kpAS/BiVSazMTLnSGs+w68F5CiODua57znhDbCwJ2H1JO7 zshI0yXA4sumP0KFVQmRY+WxFv50/MeCZ04XNNZ5rlz3kAJ6Nj8B5F7DyHL26RtZa2Gq fMEP0HLH8imyrs2Eq8QBn2gb+1HMQhSDUVc/01ZZPu4+J948a95fDJRz4dCYr6Tu6Ubj JlsbTQ+edShGXJSKsAQzf5OgMuA1HQrYyIp5NDA0kS/Vlx7ZUUC4+dvhJWyvpYfzIQp9 OTNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=/SAqThKArF2gNz1hxBWhTIoJQFzBX/onOEgSljEeRtM=; b=WihDtk0NLmCWvSTv3+lRI3dYtYAXSZ2/wbwTKTeUXGGbG8NZC8YaikWP2GkIgawaDS Nh5X8ZDqWrP+jxmjM3rQB6U97aANLDCMFmLjV2PC2EPspNVYi5Wqj9S/hH1zVAv9x25/ /GjqBjyjZaRkZkxuD/EZOvDdrsumdG5sTWX4zn7CJTToxyHDn5dkVqOF8P4LsuDbCQMf HU2SzeNYFDI49bvp+pN8FPjVtzT8MO0YNSAftydH9tQ9b6S46ZRQTBQDDoeY+wNqXKGM LJj/VIJcjHJWAqENVtfiBOWid4DMRIigtdt2ssK4d4NmBh9JNPZR1LChGDf6bEP56Tbk uENw== X-Gm-Message-State: APzg51C4G2K53ZAzWqBcwnBiyShgSM21Dgt7X1+dYvLr0AebxaRpd2lp auyxPl+VaWL8BK89ao2fgpMsv74fNHQaVLN59S4= X-Google-Smtp-Source: ANB0VdYfPzYGE3YlmM1xnUmfpyk5ciVJAdXLVgKRTSUrnyhW0BH2nx+QnoYtr+b7Swmrr153kRIY1kymj2zpxpBSfo8= X-Received: by 2002:a9d:5241:: with SMTP id q1-v6mr23955227otg.18.1537468741523; Thu, 20 Sep 2018 11:39:01 -0700 (PDT) MIME-Version: 1.0 References: <201809200020.w8K0K45h000526@repo.freebsd.org> <20180920100939.GU3161@kib.kiev.ua> <20180920131952.GZ3161@kib.kiev.ua> <08a628fe-e640-7804-7c54-de9fdc407c7d@FreeBSD.org> In-Reply-To: <08a628fe-e640-7804-7c54-de9fdc407c7d@FreeBSD.org> From: Benjamin Kaduk Date: Thu, 20 Sep 2018 13:38:50 -0500 Message-ID: Subject: Re: svn commit: r338816 - in projects/openssl111: secure/lib/libcrypto share/mk To: Konstantin Belousov Cc: svn-src-projects@freebsd.org, src-committers , John Baldwin , jkim@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2018 18:39:03 -0000 On Thu, Sep 20, 2018 at 12:41 PM Jung-uk Kim wrote: > On 18. 9. 20., Konstantin Belousov wrote: > > On Thu, Sep 20, 2018 at 08:16:02AM -0500, Benjamin Kaduk wrote: > >> On Thu, Sep 20, 2018 at 5:10 AM Konstantin Belousov < > kostikbel@gmail.com> > >> wrote: > >> > >>> On Thu, Sep 20, 2018 at 12:20:04AM +0000, Jung-uk Kim wrote: > >>>> Author: jkim > >>>> Date: Thu Sep 20 00:20:04 2018 > >>>> New Revision: 338816 > >>>> URL: https://svnweb.freebsd.org/changeset/base/338816 > >>>> > >>>> Log: > >>>> Link libcrypto with pthread. > >>> Why ? > >>> > >>> > >> It uses pthread_once and pthread locks. > > So what ? libc provides the stubs. > > Historically, OpenSSL was okay without pthread because native > implementation wasn't provided and users had to provide native callback > functions instead. > > https://www.openssl.org/docs/man1.0.2/crypto/threads.html > > In fact, "objdump -T /lib/libcrypto.so.8 | grep pthread_" returns > nothing. dwmalone discovered it and committed r127643 about 14 years ago. > > https://svnweb.freebsd.org/changeset/base/127643 > > Now OpenSSL 1.1 actually uses POSIX pthread. Please see the blog post > for the rationale: > > https://www.openssl.org/blog/blog/2017/02/21/threads/ > > Unfortunately, our stubs are not enough or broken somehow, i.e., some > functions malfunction without pthread. > > FYI, OpenSSL 1.1.1 requires the following functions now: > > pthread_atfork(3) > pthread_equal(3) > pthread_getspecific(3) > pthread_key_create(3) > pthread_key_delete(3) > pthread_once(3) > pthread_rwlock_destroy(3) > pthread_rwlock_init(3) > pthread_rwlock_rdlock(3) > pthread_rwlock_unlock(3) > pthread_rwlock_wrlock(3) > pthread_self(3) > pthread_setspecific(3) > > If you have a functional patch for libc stubs, I'll be more than happy > to revert it. > > kib's reply is taking a long time to arrive, so let me just paste in the relevant bits from another source: % From the list, I think the possible candidates are % pthread_once(), % pthread_atfork(), % pthread_key*/setspecific(). % Other should work with stubs as is, key/setspecific currently just fail. % pthread_once() and pthread_atfork() silently do nothing, is it your % problem ? % % The story about pthread_once() is known, there are some high-profile % programs depending on pthread_once() failing in single-threaded environment % (AKA gcc). pthread_atfork() is just used to increment a generation counter for the CSPRNG in the child, but pthread_once() and pthread_key*/setspecific() are heavily used for important functionality. The list of ciphers, digests, error strings, etc., are initialized in pthread_once(). pthread_setspecific() is used to provide per-thread DRBG instances (so the default RNG seems likely to fail if that function is a noop stub). In short, openssl is now a heavy pthread consumer (on Unix). -Ben