From owner-freebsd-hardware@FreeBSD.ORG Tue Aug 28 23:08:01 2012 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AD618106564A for ; Tue, 28 Aug 2012 23:08:01 +0000 (UTC) (envelope-from josh@tcbug.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by mx1.freebsd.org (Postfix) with ESMTP id 783818FC15 for ; Tue, 28 Aug 2012 23:08:01 +0000 (UTC) Received: from compute5.internal (compute5.nyi.mail.srv.osa [10.202.2.45]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 6FCAA208FA; Tue, 28 Aug 2012 19:07:54 -0400 (EDT) Received: from frontend1.nyi.mail.srv.osa ([10.202.2.160]) by compute5.internal (MEProxy); Tue, 28 Aug 2012 19:07:54 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:date:from:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding; s=smtpout; bh=f9DdGfcTyRSULsehHNmQUy svJPY=; b=ZdbkBueLnvUUWtiNIgvqaFFUcpV6sr0icw2bIU5qwjpLJiibo4cbJx JKRrMX8YP6oRujMSjVASl+3IWZt5EIav0m64rBjkrSpppOroHD+gU1PXln39EcVO l3XVVf3JDpJgOQQ++LJYsi22QDRG4kkg8zvhZOP2mcULZQI1pNw2U= X-Sasl-enc: DVzBaFQihnH8BbwgN1MQW5v4/TwsO7V5HeqOIzch9CEu 1346195274 Received: from commander.ixsystems.com (unknown [206.40.55.65]) by mail.messagingengine.com (Postfix) with ESMTPA id EC1118E0205; Tue, 28 Aug 2012 19:07:53 -0400 (EDT) Message-ID: <503D4F48.9020108@tcbug.org> Date: Tue, 28 Aug 2012 16:07:52 -0700 From: Josh Paetzel User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:13.0) Gecko/20120621 Thunderbird/13.0.1 MIME-Version: 1.0 To: Andy Young References: <503D3026.6010901@ixsystems.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hardware@freebsd.org Subject: Re: Support for Fusion IO drives? X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2012 23:08:01 -0000 >> -------- Original Message -------- >> Subject: Support for Fusion IO drives? >> Date: Tue, 28 Aug 2012 16:46:00 -0400 >> From: Andy Young >> To: freebsd-hardware@freebsd.org >> >> >> We are investigating adding SSDs as ZIL devices to boost our ZFS write >> performance. I read an article a while ago about iX Systems teaming up >> with >> Fusion IO to integrate their hardware with FreeBSD. Does anyone know >> anything about supported drivers for Fusion IO's iodrives? >> >> Thanks! >> >> Andy >> _______________________________________________ >> freebsd-hardware@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hardware >> To unsubscribe, send any mail to >> "freebsd-hardware-unsubscribe@freebsd.org" >> >> >> >> I'll put on my iXsystems hat here, as well as my fast storage, ZFS and Fusion-I/O hat. The ZFS filesystem supports dedicated ZIL devices, which can accelerate certain types of write requests, notably related to fsync. The VMWare NFS client issues a sync with every write, and most databases do as well. In those types of environments having a fast dedicated ZIL device is almost essential. In other environments the benefits of a dedicated ZIL range from non-existent to substantial. A good dedicated ZIL device is all about latency. It doesn't need to be large, in fact it will only ever handle 10 seconds of writes, so 10x network bandwidth is worst case. (In most environments this means 20GB is larger than needed). Fusion-I/O cards are far too large to be cost effective ZIL devices. Even though they do rock at I/O latency, the really fast ones are also fairly large, so the $/GB on them isn't so attractive. There are better options for ZIL devices. Another consideration is the Fusion-I/O driver is fairly memory hungry, which competes with memory ZFS wants to use for read caching. Now as an L2ARC device, that's a whole different can of worms. Command line used: iozone -r 4k -s 96g -i 0 -i 1 -i 2 -t 8 Parent sees throughput for 8 readers = 1712399.95 KB/sec L2 ARC Breakdown: 197.45m Hit Ratio: 98.61% 194.71m L2 ARC Size: (Adaptive) 771.13 GiB ARC Efficiency: 683.40m Actual Hit Ratio: 71.09% 485.82m ~ 800GB test data, all served from cache. If you are considering Fusion-I/O, the FreeBSD driver is generally not released to the general public by Fusion-I/O, but can be obtained from various partners. (I believe iXsystems is the only FreeBSD friendly fusion-i/o partner but could be wrong about that) -- Thanks, Josh Paetzel