From owner-freebsd-performance@FreeBSD.ORG Tue Aug 11 06:47:54 2009 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 28F7F106566B for ; Tue, 11 Aug 2009 06:47:54 +0000 (UTC) (envelope-from freebsd@sopwith.solgatos.com) Received: from sopwith.solgatos.com (pool-173-50-131-130.ptldor.fios.verizon.net [173.50.131.130]) by mx1.freebsd.org (Postfix) with ESMTP id F1E8E8FC35 for ; Tue, 11 Aug 2009 06:47:52 +0000 (UTC) Received: by sopwith.solgatos.com (Postfix, from userid 66) id 9E416B64F; Mon, 10 Aug 2009 23:16:16 -0700 (PDT) Received: from localhost by sopwith.solgatos.com (8.8.8/6.24) id GAA06250; Tue, 11 Aug 2009 06:37:02 GMT Message-Id: <200908110637.GAA06250@sopwith.solgatos.com> To: freebsd-performance@freebsd.org In-reply-to: Your message of "Mon, 10 Aug 2009 23:03:00 EDT." Date: Mon, 10 Aug 2009 23:37:02 PDT From: Dieter Subject: Re: RELENG_7 heavy disk = system crawls X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Aug 2009 06:47:54 -0000 > > hardware crypto accellerators > > The soekris ones work and are cheap. I thougt I saw posts that show > openssl -speed on today's fast cpu's being faster than the accel > cards. Disk crypto is symmetric, not initial pki session setup. The accel cards probably don't get updated as often as cpus do. Then there is the question of how many cards would you need and having enough free slots to plug them into. And the cards aren't going to help with zfs. Does anyone make a disk controller with crypto built in? Then it would scale with the number of drives. And not need extra slots. > > throw lots of cpu (e.g. phenom2 x4) at it > > That would only help it get done busying out the system sooner, not > balance things out while actively under load. Which as a file server, > it always is. > > Well, ok, it will help after the system is able to do spindle -> > geli -> fs -> process at the max sustained read/write speed of the > spindles. Which is about 56MiB/sec reading in this case. Which is > over 10x faster than I'm getting now. Which means I'd need maybe > 10 x 1.8GHz worth of cpu before I have any free cycles to devote > to the user interface :) Well, you could look into mainboards with 2 or 4 CPU sockets. Tyan probably has something. > Maybe I'm just clueless this month and the list is too busy to beat > me about the head with it. Anyone running geli+zfs is sitting there waiting for the user interface to update their screen. :-) >>> The data disks are hanging off a dumb ata133 pdc20269 card Ok, here is a REALLY UGLY way to slow the disks down: for disk in ad2 ad4 ... do atacontrol mode disk UDMA66 done Not a good solution. If one transfer is eating your user interface, with 8 drives you'd have to set the speed very very low. You really want a way to set the priority. BTW, if your data is so important that encryption is required, "checksum all the way back to the uberblock", etc. etc. you might want to upgrade the disks from pata to at least sata. Pata doesn't do error detection on the control info, so it could happily write your bits to the wrong sector. Sata checks the control info as well as the data. Current 7200 rpm sata speeds: dd reading the bare drive, no filesystem, at fast end of the drive: extended device statistics device r/s w/s kr/s kw/s wait svc_t %b ad18 1871.1 0.0 118303.1 0.0 2 0.8 98 dd reading a file from FFS: 28125720000 bytes transferred in 264.921699 secs (106166162 bytes/sec) Or you could go with the green disks that spin slower.