From owner-freebsd-arch@FreeBSD.ORG Mon Mar 2 17:41:17 2015 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3F7EC9D4; Mon, 2 Mar 2015 17:41:17 +0000 (UTC) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B8F8B930; Mon, 2 Mar 2015 17:41:16 +0000 (UTC) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id t22HfGCB050191 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 2 Mar 2015 09:41:16 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id t22HfFPZ050190; Mon, 2 Mar 2015 09:41:15 -0800 (PST) (envelope-from jmg) Date: Mon, 2 Mar 2015 09:41:15 -0800 From: John-Mark Gurney To: Alfred Perlstein Subject: Re: locks and kernel randomness... Message-ID: <20150302174115.GR32329@funkthat.com> References: <20150224012026.GY46794@funkthat.com> <1824482166.23183751.1425303073196.JavaMail.zimbra@stormshield.eu> <54F47C98.2080505@freebsd.org> <20150302171025.GO32329@funkthat.com> <1D168865-DF1B-4A08-BB42-FB26B4D88D6E@mu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1D168865-DF1B-4A08-BB42-FB26B4D88D6E@mu.org> X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Mon, 02 Mar 2015 09:41:16 -0800 (PST) Cc: Emeric POUPON , "arch@freebsd.org" , Alfred Perlstein X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 17:41:17 -0000 Alfred Perlstein wrote this message on Mon, Mar 02, 2015 at 12:19 -0500: > > On Mar 2, 2015, at 12:10 PM, John-Mark Gurney wrote: > > > > Alfred Perlstein wrote this message on Mon, Mar 02, 2015 at 10:07 -0500: > >>> On 3/2/15 8:31 AM, Emeric POUPON wrote: > >>> About arc4random, we have noticed significant contention in that function on multi CPU systems when ciphering a lot of packets in the IPsec stack. > >>> This is indeed due to the mutex that is being used in the arc4rand function. > >>> > >>> Actually randomness is required by the IV used in the forged output packets. > >>> However, making a separate random generator per CPU might be more complicated than expected. > >>> The RFC 6027 (http://www.ietf.org/rfc/rfc6027.txt) reminds that the IV must not be repeated : > >>> --- > >>> 3.7.1. Outbound SAs Using Counter Modes > >>> > >>> For SAs involving counter mode ciphers such as Counter Mode (CTR) > >>> ([RFC3686]) or Galois/Counter Mode (GCM) ([RFC4106]) there is yet > >>> another complication. The initial vector for such modes MUST NOT be > >>> repeated, and senders use methods such as counters or linear feedback > >>> shift registers (LFSRs) to ensure this [...] > >>> --- > >>> > >>> What do you think? > >> > >> If you can not have multiple random sources then what do you think of > >> having a thread that pre-fetches batches of random values and queues it > >> to each cpu? If you have the queue be pretty large then you shouldn't > >> bottleneck on it. > >> > >> Sort of like UMA for random data. > >> > >> Sorry if this is a daft idea, not sure about this code path in general, > >> but this struck me as a potential workaround. > > > > I'd say that's needlessly complex... You'd still need a lock, or play > > w/ the scheduler (sched_bind) to serialize access to the PCPU random > > pool... > > John, that is how you break down a lock. Do you have a lock free or per CPU solution? > > Using a strategy like this is very typical when trying to scale across CPU. > > Do you have alternate idea? Did people not at ALL read the original email? I said in the original email: I already have another patch that converts arc4rand and friends over to ChaCha. This patch does use PCPU data and sched_pin to help eliminate locks, but this does need more study. Please, read the original email, and then ask questions.. If you jump into the middle of the thread, it's you're responsibility to reread the thread... Does my original email answer your questions? If not, feel free to ask additional questions... Thanks. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."