From owner-freebsd-questions@FreeBSD.ORG Thu Jan 15 17:08:33 2009 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2D4961065672 for ; Thu, 15 Jan 2009 17:08:33 +0000 (UTC) (envelope-from rsmith@xs4all.nl) Received: from smtp-vbr17.xs4all.nl (smtp-vbr17.xs4all.nl [194.109.24.37]) by mx1.freebsd.org (Postfix) with ESMTP id B5CEC8FC26 for ; Thu, 15 Jan 2009 17:08:32 +0000 (UTC) (envelope-from rsmith@xs4all.nl) Received: from slackbox.xs4all.nl (slackbox.xs4all.nl [213.84.242.160]) by smtp-vbr17.xs4all.nl (8.13.8/8.13.8) with ESMTP id n0FH8Vme052785; Thu, 15 Jan 2009 18:08:31 +0100 (CET) (envelope-from rsmith@xs4all.nl) Received: by slackbox.xs4all.nl (Postfix, from userid 1001) id B99D2BA8F; Thu, 15 Jan 2009 18:08:30 +0100 (CET) Date: Thu, 15 Jan 2009 18:08:30 +0100 From: Roland Smith To: RW Message-ID: <20090115170830.GA34713@slackbox.xs4all.nl> References: <20090114175954.GC97086@slackbox.xs4all.nl> <20090114225538.66e001de@gumby.homeunix.com> <20090114232054.GB6422@slackbox.xs4all.nl> <20090115020658.3b93a3d3@gumby.homeunix.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="nFreZHaLTZJo0R7j" Content-Disposition: inline In-Reply-To: <20090115020658.3b93a3d3@gumby.homeunix.com> X-GPG-Fingerprint: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 X-GPG-Key: http://www.xs4all.nl/~rsmith/pubkey.txt X-GPG-Notice: If this message is not signed, don't assume I sent it! User-Agent: Mutt/1.5.18 (2008-05-17) X-Virus-Scanned: by XS4ALL Virus Scanner Cc: freebsd-questions@freebsd.org Subject: Re: freebsd encrypted hard disk? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Jan 2009 17:08:33 -0000 --nFreZHaLTZJo0R7j Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jan 15, 2009 at 02:06:58AM +0000, RW wrote: > On Thu, 15 Jan 2009 00:20:54 +0100 > Roland Smith wrote: >=20 > > On Wed, Jan 14, 2009 at 10:55:38PM +0000, RW wrote: >=20 > > > Not just in reduced transfer rates, but also in terms of CPU cycles > > > used - a sustained geli to geli file copy makes things really slow > > > for me. > >=20 > > That's probably because two geli kernel threads are competing for time > > on a single core. I've had problems with that as well (geli-encrypted > > USB drive stalling). > >=20 > > Since I've switched to a multi-core machine (where the number of cores > > should be at least equal to the number of geli-encrypted devices), CPU > > load for gele has dropped to barely noticable. > >=20 >=20 > I find that puzzling; have you measured that on sustained geli to > geli transfers (with GB size files). Not really. My previous single core machine kept stalling the drive. So instead of storing dumps for my biggest (/home) partition, I've turned to rsync-ing that, and using dumps for /, /usr and /var. I just did a simple test: copying a 5249634304 byte file from a non-encryp= ted disk to an encrypted one. This took 3:22 with the CPU load hovering at 10-13% at full speed. That's around 24.8 MiB/s. Copying the same file to another location on the unencrypted disk takes 2:57 with the CPU at around 5% at about 3/4 speed. That comes down to 28.3 MiB/s.=20 Not a big difference IMHO. In normal usage the encrypted partition doesn't feel slower, because there are still at least two cores fully available for other jobs so interactivity doesn't suffer. A single-core machine would obviously feel different. The only thing that takes a while is compiling ports and big LaTeX jobs (compiling 300 pages five times with makeindex and bibtex runs in between). > The reason I'm a bit sceptical is that dd'ing /dev/random to /dev/null > runs at about 20MBytes/s on my single core (verses 700MBytes/s > for /dev/zero). File copies into geli run at about 15Mbytes/s, openssl > enc -aes-256-cbs runs at about the same ballpark figure.=20 I get: dd if=3D/dev/random of=3D/dev/null bs=3D64k count=3D10000 10000+0 records in 10000+0 records out 655360000 bytes transferred in 10.693598 secs (61285266 bytes/sec) say 58 MiB/s dd if=3D/dev/zero of=3D/dev/null bs=3D64k count=3D10000 10000+0 records in 10000+0 records out 655360000 bytes transferred in 0.281247 secs (2330194568 bytes/sec) or around 2200 MiB/s And that is obviously running on _one_ core (CPU at approx. 25%). > Even if I had > multi-cores I would still be cpu-limited to 20MB/s, and that would fully > occupy two cores on geli to geli transfers. Your cores are probably > faster, but I'd expect a factor of two or so would be swallowed-up by > faster transfers. I don't see how cpu usage would be negligible unless > your individual cores are an order of magnitude faster than that. It turns out that on a multi-core machine a geli thread is started on each core for each disk (4 cores, two disks): ps -xa | grep g_eli 92 ?? DL 5:31.75 [g_eli[0] ad4s1g] 93 ?? DL 6:33.76 [g_eli[1] ad4s1g] 94 ?? DL 6:26.02 [g_eli[2] ad4s1g] 95 ?? DL 7:41.18 [g_eli[3] ad4s1g] 98 ?? DL 1:49.60 [g_eli[0] ad6s1g] 99 ?? DL 1:30.30 [g_eli[1] ad6s1g] 100 ?? DL 1:53.87 [g_eli[2] ad6s1g] 101 ?? DL 1:50.64 [g_eli[3] ad6s1g] This can be tuned with the sysctl 'kern.geom.eli.threads'. > Just out of curiosity what rate do you get on =20 > dd if=3D/dev/random of=3D/dev/null bs=3D64k count=3D10000 See above. =20 Roland --=20 R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) --nFreZHaLTZJo0R7j Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAklvbY4ACgkQEnfvsMMhpyU55wCePgDYlLzfaDs98mn3iqLucwv8 FEkAn3kIt3Qt0akKsc8AInuAulw28uJe =afIF -----END PGP SIGNATURE----- --nFreZHaLTZJo0R7j--