From owner-freebsd-current@FreeBSD.ORG Fri Jul 31 20:22:27 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 06541106566B for ; Fri, 31 Jul 2009 20:22:27 +0000 (UTC) (envelope-from james-freebsd-current@jrv.org) Received: from mail.jrv.org (adsl-70-243-84-13.dsl.austtx.swbell.net [70.243.84.13]) by mx1.freebsd.org (Postfix) with ESMTP id BCA278FC12 for ; Fri, 31 Jul 2009 20:22:26 +0000 (UTC) (envelope-from james-freebsd-current@jrv.org) Received: from kremvax.housenet.jrv (kremvax.housenet.jrv [192.168.3.124]) by mail.jrv.org (8.14.3/8.14.3) with ESMTP id n6VKMQBd091149 for ; Fri, 31 Jul 2009 15:22:26 -0500 (CDT) (envelope-from james-freebsd-current@jrv.org) Authentication-Results: mail.jrv.org; domainkeys=pass (testing) header.from=james-freebsd-current@jrv.org DomainKey-Signature: a=rsa-sha1; s=enigma; d=jrv.org; c=nofws; q=dns; h=message-id:date:from:user-agent:mime-version:to:subject: content-type:content-transfer-encoding; b=lgatfZF6Mf0XhAUVROgQkzAUvj8CcGk+5dYEiSlzQU/rTEKFC0CTMSL09E0fhXn0C 13CtC+9VNWmeWFjD8d+DxWtISVRJ5w3fMCcmLyKQJ4IXhyfeOYSbPzTm9jCJn8VEjt6 JtvIintOVFX2ZUhk5z8AH1lvjITozLaNYzybK48= Message-ID: <4A735282.8040502@jrv.org> Date: Fri, 31 Jul 2009 15:22:26 -0500 From: "James R. Van Artsdalen" User-Agent: Thunderbird 2.0.0.22 (Macintosh/20090605) MIME-Version: 1.0 To: FreeBSD Current Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: SIIS/CAM driver performance X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 31 Jul 2009 20:22:27 -0000 I sent this to mav earlier but thought it would be of wider interest... My first attempt at measuring raw throughput with the SIIS driver (Silicon Image) got 875 MByte/sec with no tuning or changes to an old dual Opteron 244 (1.8GHz single core) Tyan K8W motherboard using two 3124 controllers and ten drives in four port-multiplier enclosures. I used 128KB blocks in the sequential reads since that's what ZFS seems to do by default. This same combo peaked at 375 MByte/sec with the ATA driver with my (badly done) hacks. SIIS appears stable. SIIS should clearly do better than 1 GByte/sec with modern hardware. One unknown is how well PCI-Express cards with the 3124 perform: the 3124 is a PCI-X part, and PCI-Express cards using the 3124 use the Intel 41110 Serial to Parallel PCI bridge which may add some overhead. mav has thrown down the GB/s gauntlet and we'll see which filesystems can pick it up. :-) This is exciting because it allows FreeBSD to be the OS of choice for cheap storage servers or perhaps even high-speed data capture. A 50 TB server should price well under $10k, a price/performance point hard to hit any other way. Issues: The Silicon Image 3132 is a native 2-port PCI-Express controller. but it appears to have problems as I am not aware of any driver on any OS capable of coaxing more than 150MByte/sec total out of it. The chip works but is very slow. Use the 3124 for performance. The SIIS driver doesn't support staggered spin-up yet. Anyone wanting to build a 50 TB storage array with SIIS needs to keep in mind that spin-up power spike for now.