From owner-freebsd-arch@FreeBSD.ORG Tue Feb 11 18:56:47 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0DADED42; Tue, 11 Feb 2014 18:56:47 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C46881499; Tue, 11 Feb 2014 18:56:46 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s1BIueiC054785 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 11 Feb 2014 10:56:40 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s1BIueUM054784; Tue, 11 Feb 2014 10:56:40 -0800 (PST) (envelope-from jmg) Date: Tue, 11 Feb 2014 10:56:39 -0800 From: John-Mark Gurney To: security@FreeBSD.org, arch@FreeBSD.org Subject: CFR: unifing sha256 userland/kernel implementation... Message-ID: <20140211185639.GK34851@funkthat.com> Mail-Followup-To: security@FreeBSD.org, arch@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Tue, 11 Feb 2014 10:56:40 -0800 (PST) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Feb 2014 18:56:47 -0000 I did some performance testing on sha256, and found that the libmd version is significantly faster, ~20%, than the kernel version. Even if you enable SHA2_UNROLL_TRANSFORM (which isn't the default), the version in libmd is still faster. So, this patch moves libmd's sha256c.c and sha256.h into the kernel, and adapts the userland to pull the version from the kernel. This change removes sha256 from the existing sha2.c file, and does some minor cleanup of types in sha2. I have tested this w/ ZFS using sha256 checksums, and a ZFS made pre-patch is read fine by a kernel post patch. I have also run the tests in lib/libmd and they all pass fine. Passes buildworld/buildkernel/installkernel/reboot/installworld/reboot/test. Patch: https://www.funkthat.com/~jmg/sha256.kern.patch Following stats are in seconds to digest 100000 10000-byte blocks, calculated using sha256 -t: $ ministat soft.times kernsoft.times x soft.times + kernsoft.times +------------------------------------------------------------------------------+ |x xx xx +++ + +| | |___________AM_________| |_______M_____A______________| | +------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 5 6.775387 8.279581 7.848128 7.792094 0.60912664 + 5 8.997429 10.768921 9.090787 9.4359144 0.75040822 Difference at 95.0% confidence 1.64382 +/- 0.99674 21.096% +/- 12.7917% (Student's t, pooled s = 0.683428) This is in preperation of bringing in an SSE4 accelerated version of sha256 (for both userland and kernel) that sees a 2x performance increase. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."