From owner-freebsd-questions@FreeBSD.ORG Tue May 18 02:57:02 2010 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A1BA91065672 for ; Tue, 18 May 2010 02:57:02 +0000 (UTC) (envelope-from fireduck@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3820C8FC0A for ; Tue, 18 May 2010 02:57:01 +0000 (UTC) Received: by fxm19 with SMTP id 19so1758203fxm.13 for ; Mon, 17 May 2010 19:57:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=sYYrbprAe7mtUUUeo22/WV0bG0nzUKKfxlZPahwuyBs=; b=LLKRWNUGX0lmduwL+glKWtD7KQfXRsYX8CtMfkZwqHQGynpVL5o5K47+ZZ0Pzh36K9 nUq3YFyg2NCPtIkQzFAS6pSjNcvn0Bcc/BAH5pPlMSjaO2BZSeLqCohAJIKUbCZ3LtFm RBvLU7NLtkkY2Hb8nLAuWqN+kWtWIy3Zms+mE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=Sjrn6PYVXpzDvG7jqu248zHo5+2S4nBBHKDflDzmZ7gD+9mui+BoloYdxsMpNu1jUh 1LC/Y0fZGa6FARkTO48jHlcA2x5ZbN1FM5EFUKHg7Vk9IyvC7M9THG79gFI0s1QLn8jj StlqhW9x+uH/wkQHvXPF4vWjytFwbhdwSZLPA= MIME-Version: 1.0 Received: by 10.223.25.74 with SMTP id y10mr7265824fab.81.1274149955805; Mon, 17 May 2010 19:32:35 -0700 (PDT) Received: by 10.223.116.193 with HTTP; Mon, 17 May 2010 19:32:35 -0700 (PDT) Date: Mon, 17 May 2010 19:32:35 -0700 Message-ID: From: Joseph Gleason To: freebsd-questions@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Where has my gbde write performance gone? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 May 2010 02:57:02 -0000 Sometime between FreeBSD 7.2-RELEASE-p4 and 8.0-RELEASE write performance of gbde encrypted devices seems to have dropped significantly. A system I have running 7.2 seems to run gbde drives at or near the drive max rate (30-40MB/s) while I am seeing less than 10% of that on 8.0 systems. I get the same slow writes on 8.0-RELEASE-p2 as well as 8.0-RELEASE. Here is an example on a fresh 8.0 install which shows gbde taking the drive write performance of 40 MB/s down to 2.6 MB/s: lab# uname -a FreeBSD lab.int.fireduck.com 8.0-RELEASE FreeBSD 8.0-RELEASE #0: Sat Nov 21 15:02:08 UTC 2009 root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 lab# dd if=/dev/urandom of=/dev/ad4s1d bs=32k count=32k 32768+0 records in 32768+0 records out 1073741824 bytes transferred in 25.130537 secs (42726577 bytes/sec) lab# gbde init /dev/ad4s1d Enter new passphrase: Reenter new passphrase: lab# gbde attach /dev/ad4s1d Enter passphrase: lab# dd if=/dev/urandom of=/dev/ad4s1d.bde bs=32k count=32k 32768+0 records in 32768+0 records out 1073741824 bytes transferred in 401.097004 secs (2677013 bytes/sec) iostat from while that last 'dd' was running: tty ad4 cpu tin tout KB/t tps MB/s us ni sy in id 0 22 5.67 483 2.67 0 0 4 1 96 0 66 5.67 509 2.82 0 0 4 1 95 0 22 5.69 514 2.86 0 0 6 1 94 0 22 5.67 506 2.80 0 0 6 1 93 0 22 5.67 472 2.61 0 0 4 1 95 iostat on a FreeBSD 7.2-RELEASE-p4 box doing a similar operation: tin tout KB/t tps MB/s us ni sy in id 0 22 29.54 1208 34.86 3 0 56 2 39 0 22 29.56 1177 33.97 3 0 57 1 39 0 22 29.54 1201 34.64 3 0 58 2 37 0 22 29.57 1144 33.04 2 0 51 3 44 0 22 29.56 1126 32.52 3 0 54 2 42 0 22 29.53 1179 34.01 3 0 53 2 42 0 22 29.57 1165 33.65 2 0 58 2 38 One thing I notice is the larger block size the 7.2 writes but I don't imagine that would be that significant. I've been using FreeBSD in various amateurish and wrong ways since 2.2, so I wouldn't rule out me doing something stupid. If so, I'd love to know what.