From owner-svn-src-all@freebsd.org Wed Dec 7 19:41:29 2016 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0E36CC6C2F9; Wed, 7 Dec 2016 19:41:29 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebi.us (glebi.us [96.95.210.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebi.us", Issuer "cell.glebi.us" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 7C1001170; Wed, 7 Dec 2016 19:41:28 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebi.us (localhost [127.0.0.1]) by cell.glebi.us (8.15.2/8.15.2) with ESMTPS id uB7JfLUb021130 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 7 Dec 2016 11:41:21 -0800 (PST) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebi.us (8.15.2/8.15.2/Submit) id uB7JfLVh021129; Wed, 7 Dec 2016 11:41:21 -0800 (PST) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebi.us: glebius set sender to glebius@FreeBSD.org using -f Date: Wed, 7 Dec 2016 11:41:21 -0800 From: Gleb Smirnoff To: Mateusz Guzik Cc: Fabien Thomas , svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org Subject: Re: svn commit: r309143 - head/sys/libkern Message-ID: <20161207194121.GM27748@FreeBSD.org> References: <201611251349.uAPDnX09042110@repo.freebsd.org> <20161125160010.GA3307@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20161125160010.GA3307@dft-labs.eu> User-Agent: Mutt/1.7.0 (2016-08-17) X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Dec 2016 19:41:29 -0000 On Fri, Nov 25, 2016 at 05:00:10PM +0100, Mateusz Guzik wrote: M> On Fri, Nov 25, 2016 at 01:49:33PM +0000, Fabien Thomas wrote: M> > Author: fabient M> > Date: Fri Nov 25 13:49:33 2016 M> > New Revision: 309143 M> > URL: https://svnweb.freebsd.org/changeset/base/309143 M> > M> > Log: M> > In a dual processor system (2*6 cores) during IPSec throughput tests, M> > we see a lot of contention on the arc4 lock, used to generate the IV M> > of the ESP output packets. M> > M> > The idea of this patch is to split this mutex in order to reduce the M> > contention on this lock. M> > M> > +MALLOC_DEFINE(M_ARC4RANDOM, "arc4random", "arc4random structures"); M> > M> > -static u_int8_t arc4_randbyte(void); M> > +struct arc4_s { M> > + u_int8_t i, j; M> > + int numruns; M> > + u_int8_t sbox[256]; M> > + time_t t_reseed; M> > + M> > + struct mtx mtx; M> > +}; M> > + M> M> Why is the lock at the end? Right now you got false sharing with the M> next entry. M> M> That said, I suggest: M> 1. moving thelock to the beginning M> 2. annotating the struct with __aligned(CACHE_LINE_SIZE) It could be even better not to allocate them with regular malloc at all, but to put them into the per cpu areas. I haven't tried, but looks like the DPCPU mechanism defined in sys/pcpu.h would fit that well. -- Totus tuus, Glebius.