From owner-freebsd-current@FreeBSD.ORG Mon Nov 14 01:30:05 2011 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85384106564A; Mon, 14 Nov 2011 01:30:05 +0000 (UTC) (envelope-from das@freebsd.org) Received: from zim.MIT.EDU (ZIM.MIT.EDU [18.95.3.101]) by mx1.freebsd.org (Postfix) with ESMTP id 50A238FC12; Mon, 14 Nov 2011 01:30:05 +0000 (UTC) Received: from zim.MIT.EDU (localhost [127.0.0.1]) by zim.MIT.EDU (8.14.5/8.14.2) with ESMTP id pAE1U4tA053862; Sun, 13 Nov 2011 20:30:04 -0500 (EST) (envelope-from das@freebsd.org) Received: (from das@localhost) by zim.MIT.EDU (8.14.5/8.14.2/Submit) id pAE1U4lB053861; Sun, 13 Nov 2011 20:30:04 -0500 (EST) (envelope-from das@freebsd.org) Date: Sun, 13 Nov 2011 20:30:04 -0500 From: David Schultz To: Andrey Chernov , current@freebsd.org, secteam@freebsd.org Message-ID: <20111114013004.GA53392@zim.MIT.EDU> Mail-Followup-To: Andrey Chernov , current@freebsd.org, secteam@freebsd.org References: <20080916140319.GA34447@nagual.pp.ru> <20080916201932.GA59781@zim.MIT.EDU> <20111112102241.GA75396@vniz.net> <20111112154135.GA21512@zim.MIT.EDU> <20111112171531.GA83419@vniz.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111112171531.GA83419@vniz.net> Cc: Subject: Re: Is fork() hook ever possible? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Nov 2011 01:30:05 -0000 On Sat, Nov 12, 2011, Andrey Chernov wrote: > On Sat, Nov 12, 2011 at 10:41:35AM -0500, David Schultz wrote: > > On Sat, Nov 12, 2011, Andrey Chernov wrote: > > > On Tue, Sep 16, 2008 at 04:19:32PM -0400, David Schultz wrote: > > > > secteam@ already agreed to the idea of solving the fork problem as > > > > in OpenBSD over a month ago. > > > > > > On Wed, Sep 17, 2008 at 12:50:25PM +0400, Andrey Chernov wrote: > > > > I agree with your patch (BTW you can remove unneded #define RANDOMDEV). > > > > > > The question remains: why you don't commit this patch all that 3 > > > years, having secteam@ and mine agreements too? > > > > Sorry, but in the three years that have intervened, my brain has > > paged out the relevant context. As I recall, there were issues > > with some of your changes to arc4random() and I proposed tracking > > OpenBSD's implementation more closely. > > I can't say for secteam@ side (it was you who said that they agree), but > personally me still agree with your proposal and still see security > problem in our current implementation, like the same-generated tmp names > after fork in son and parent. > > > If everyone's in agreement on that, please go ahead and commit the changes. > > I can't... It seems I reach dead end talking to our @secteam. In few > words, they: > 1) Explicitly disallow my commits in all 'random' areas until their > review. > 2) They never do that review (I must to mention again that 3 years > passed since they promise it). > Being particular, I suggest them to use your patch at the end. Nothing > happens. > Hope you'll get more luck with them committing it by yourself. I don't have those patches anymore, but I redid them from scratch using the latest revision from OpenBSD. The patch at http://www.freebsd.org/~das/patches/vshead.diff syncs our arc4random.c with OpenBSD's to the extent possible, style bugs and all. It seems prudent to treat it as a vendor source and avoid gratuitous differences: Unlike our version, all the changes to OpenBSD's arc4random.c were vetted by several people. Switching to OpenBSD's version fixes the bug where a parent and child process see the same random sequence. The diff http://www.freebsd.org/~das/patches/vsobsd.diff shows the differences between our version and OpenBSD's after applying the patch. Most of the remaining differences are in arc4_stir(), where we use /dev/random instead of a sysctl to get entropy for the IV. In arc4_stir(), I also fixed the bug where the wrong buffer size was being passed to arc4_addrandom(), resulting in entropy loss. That change should be committed separately. When adding the XXX comment in the patch, I noticed that it isn't clearly documented what arc4random() is supposed to provide, and there seems to be a substantial amount of confusion on this point. I can think of three reasonable kinds of RNGs: 1. Low-quality pseudorandomness, e.g., for test generation. These are only "random" w.r.t. certain statistical tests. 2. High-quality, reproducible pseudorandomness, e.g., for deterministic encryption, or test generation with stronger statistical requirements. 3. High-quality, unpredictable pseudorandomness, e.g., for key generation. David Mazieres' original arc4random() implementation was intended to provide a less useful variant of #2, but was extended to provide #3 in OpenBSD. FreeBSD's version provides #3 most of the time, unless something goes wrong -- e.g., you forget to load random.ko or you run it in a jail where there's no /dev -- and then you get #2. It seems that most people think arc4random() is giving them #3, and if that's the case, then arc4random() should be fail-fast: it should abort if it's unable to obtain a high-quality IV. There's similar confusion evident in the revision history for random(), which really only provides #1. > On Tue, Sep 16, 2008 at 04:19:32PM -0400, David Schultz wrote: > > If getpid() really winds up being a serious problem, we can solve [...] > I run some tests but can't come to conclusion, is overhead is significant > or not for real life tasks (which usually don't call arc4random() very > often in the loop). I have a bunch of tests for floating point that use arc4random(), and they are substantially slower with the getpid() changes. Realistically they should be using mrand48(), though. Here's a TODO list of worthwhile improvements that I don't have time for. None of them seem terribly pressing. - Document what arc4random() actually provides. (The OpenBSD manpage is pretty good, albeit a bit technical.) - Like OpenBSD, implement and use a sysctl to get the IV in arc4random() instead of /dev/random. This is likely to be faster, and there's less than can go wrong (e.g., jails without a working /dev). - Make arc4random() abort if it can't get any cryptographic-quality entropy. Make sure it's still possible to boot in single-user mode without random.ko. - Either optimize getpid() or replace its use with a fork hook, so that it isn't necessary to do a syscall on every arc4random() call. If there's no speed advantage to using arc4random(), people might as well just be reading from /dev/random all the time.