From owner-svn-src-stable@FreeBSD.ORG Mon Aug 26 06:26:16 2013 Return-Path: Delivered-To: svn-src-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id CDD72BB8; Mon, 26 Aug 2013 06:26:16 +0000 (UTC) (envelope-from jmg@h2.funkthat.com) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8F8BF2B4F; Mon, 26 Aug 2013 06:26:16 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id r7Q6QAmT058689 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 25 Aug 2013 23:26:10 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id r7Q6QA6H058688; Sun, 25 Aug 2013 23:26:10 -0700 (PDT) (envelope-from jmg) Date: Sun, 25 Aug 2013 23:26:10 -0700 From: John-Mark Gurney To: Ollivier Robert Subject: Re: svn commit: r254859 - stable/9/sys/geom/eli Message-ID: <20130826062610.GC29777@funkthat.com> References: <201308251429.r7PETlY7035520@svn.freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201308251429.r7PETlY7035520@svn.freebsd.org> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Sun, 25 Aug 2013 23:26:10 -0700 (PDT) Cc: svn-src-stable@FreeBSD.org, svn-src-all@FreeBSD.org, src-committers@FreeBSD.org, svn-src-stable-9@FreeBSD.org X-BeenThere: svn-src-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for all the -stable branches of the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Aug 2013 06:26:16 -0000 Ollivier Robert wrote this message on Sun, Aug 25, 2013 at 14:29 +0000: > Author: roberto > Date: Sun Aug 25 14:29:47 2013 > New Revision: 254859 > URL: http://svnweb.freebsd.org/changeset/base/254859 > > Log: > MFC r226840: AESNI-related commit, from a long time ago: > > Before this change when GELI detected hardware crypto acceleration it will > start only one worker thread. For software crypto it will start by default > N worker threads where N is the number of available CPUs. > > This is not optimal if hardware crypto is AES-NI, which uses CPU for AES > calculations. > > Change that to always start one worker thread for every available CPU. > Number of worker threads per GELI provider can be easly reduced with > kern.geom.eli.threads sysctl/tunable and even for software crypto it > should be reduced when using more providers. > > While here, when number of threads exceeds number of CPUs avilable don't > reduce this number, assume the user knows what he is doing. > > Submitted by: Michael Moll I'll just point out that if you have a large number of geli volumes, that this turns out to hurt performance rather than help it.. At least in my testing on a 6 core machine w/ an 8 disk zfs pool where all 8 disks are encrypted... I think the problem is that we launch threads on a per geli/disk instead of just having a generic set of threads that can work with any geli volume... When you have 17 geli volumes (like I do on my machine) * 6 threads, thats a lot of threads, imagine if the box had 32 or 64 cores? You're looking at 500-1000 threads just for encryption... In case someone asks, 8 zfs disks + 2 spares, swap, l2arc * 2, zil * 2 and zfs mirrored root all encrypted volumes... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."