From owner-freebsd-questions@FreeBSD.ORG Sat Oct 19 19:31:43 2013 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id ED14FEA0 for ; Sat, 19 Oct 2013 19:31:43 +0000 (UTC) (envelope-from aurfalien@gmail.com) Received: from mail-ob0-x22c.google.com (mail-ob0-x22c.google.com [IPv6:2607:f8b0:4003:c01::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A39412C7A for ; Sat, 19 Oct 2013 19:31:43 +0000 (UTC) Received: by mail-ob0-f172.google.com with SMTP id gq1so742133obb.17 for ; Sat, 19 Oct 2013 12:31:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:mime-version:content-type:subject:date:in-reply-to:to :references:message-id; bh=63ACLzmNr4etvmDlqQjgZHn/+47zeqZNddcDgrV3ijs=; b=g4Hmm5oyGQv/UQhVJl5eyf8GYnTIzRT0gOM+yYsuSjg8hxQgnTPK6X9mn/4hplcorx KCscQ0RBUoFzhS65XtKnPSvaysiMiZO/wYStSKB59DqF77QGqqHfz37wnCFwPh0tEa29 DZEcjhKfNWbIyQYuVbQm5mWsU+ifIyeLqcMteOGYKqKKkto8gu62ePfzuEGBdrnJ3hNs Mk8pZAB8R7E/hQxzFT9tMf3U5AC+IDv/y4jUd/L4KSgSclfvG8uls1YeuXUXeDZbcAgK 5J+pX1K+MRxd2KIpsuTmU5wxUlYxQp/A1TtWlWh60Uza+lUA2D6Il43BE31EdQLRe8PT GGcw== X-Received: by 10.182.131.196 with SMTP id oo4mr1460266obb.50.1382211102954; Sat, 19 Oct 2013 12:31:42 -0700 (PDT) Received: from [192.168.1.74] (75-63-29-182.lightspeed.irvnca.sbcglobal.net. [75.63.29.182]) by mx.google.com with ESMTPSA id hs4sm18481732obb.5.2013.10.19.12.31.40 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 19 Oct 2013 12:31:41 -0700 (PDT) From: aurfalien Mime-Version: 1.0 (Apple Message framework v1085) Subject: Re: FreeBSD, Centos and ZFS Date: Sat, 19 Oct 2013 12:31:39 -0700 In-Reply-To: <525D143B.50202@gmail.com> To: FreeBSD Mailing List References: <1381600575.18393.33219025.5D7B78D0@webmail.messagingengine.com> <8CA809B3-1692-4760-A63F-9D7451EB49BD@gmail.com> <525D143B.50202@gmail.com> Message-Id: X-Mailer: Apple Mail (2.1085) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Oct 2013 19:31:44 -0000 On Oct 15, 2013, at 3:08 AM, Johan Hendriks wrote: > aurfalien wrote: >> On Oct 12, 2013, at 10:56 AM, Mark Felder wrote: >>=20 >>> On Sat, Oct 12, 2013, at 10:53, aurfalien wrote: >>>> Hi, >>>>=20 >>>> I would like to first say that by no means is this a "hey, why is = my Mac >>>> faster then my PC" kind of email. >>>>=20 >>>> I'm really hoping its an LSI driver issue. >>>>=20 >>> It may very well be an LSI firmware issue. What are the firmwares = for >>> those HBAs? >> Upon doing this; >>=20 >> sysctl -a | grep mps >>=20 >> I get this; >>=20 >> dev.mps.0.driver_version: 14.00.00.01-fbsd >>=20 >> LSIs site mentions the latest drives at being 17.00.00.00 >>=20 >> I'll go ahead and install the latest to see what happens. >>=20 >> Whats the best way to do this, I assume build it and load via = loader.conf? >>=20 >> - aurf >> _______________________________________________ >> freebsd-questions@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-questions >> To unsubscribe, send any mail to = "freebsd-questions-unsubscribe@freebsd.org" > The way i do it is from FreeBSD itself. > I did it on FreeBSD 9.0 and 9.2, never did it on a 9.1 or 10 Current. = I also never did it on cards other then the 9211-8i LSI cards.=20 > So do not hold me responsible if things go wrong :D > If things go wrong it can damage the card so it has to be put in for = RMA > Download the firmware, and the sas2flash file for FreeBSD=20 >=20 > Then i do the following > =20 >=20 > myrootonzfs ~ # ./sas2flash -listall >=20 >=20 Rather then up the driver in FreeBSD 9, (I tried and it didn't go too = well anyways) I left it at v14 and down graded the firmware. While the = Intel rebranded LSI controllers were stuck at an older Intel version, I = decided to make them all v14 of the LSI firmware. Throughput has improved a bit and it seems more consistent in terms of = test results. So the key here in FreeBSD is to match firmware with driver versions. = Linux didn't seem to care but all OS's have there nuances so I'm not = concerned. In terms of flashing the firmware, I had to do it via EFI using the = -force flag. Even though v14 didn't have an EFI tool , v17 did so I = used it while specifying v14 firmware file. But a note on the Linux comparison; While I mimicked my values on both platforms, I did not mimic all = values. ZoL comes with a little interesting feature; arc2 compression, which = does seem to enhance performance all around. I disabled this in CentOS = to level the playing field. There are several other things I saw on ZoL which seem very new and more = in line with ZFS on FreeBSD 10. So I installed FreeBSD 10, set up my prefs as before but performance = were very close to FreeBSD 9, which were both lower to ZoL in CentOS. I've attached my IOzone results in zipped xls format if any one is = interested. I'm still baffled as to why CentOS has substantially better = results. * Concerning the graphs; Scales are all set equal to allow for easy comparisons. Values to the right are avg file sizes in my env; 8MB 12MB 16MB 30MB 53MB 2GB At any rate, I'm hoping some one could help solve this performance = discrepancy. I suppose a dump of my hardware list and ZFS properties are in order but = only if any one is interested as I don't like typing in vein :) Thanks in advance, - aurf