From owner-freebsd-scsi@FreeBSD.ORG Mon Jan 19 17:29:30 2015 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C538C130 for ; Mon, 19 Jan 2015 17:29:30 +0000 (UTC) Received: from mail-pd0-f179.google.com (exprod7og101.obsmtp.com [64.18.2.155]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 79469C83 for ; Mon, 19 Jan 2015 17:29:29 +0000 (UTC) Received: from mail-pd0-f179.google.com ([209.85.192.179]) (using TLSv1) by exprod7ob101.postini.com ([64.18.6.12]) with SMTP ID DSNKVL0+8xDSpE5ilPMbTH9uJIN6QZm71QXA@postini.com; Mon, 19 Jan 2015 09:29:30 PST Received: by mail-pd0-f179.google.com with SMTP id v10so19473703pde.10 for ; Mon, 19 Jan 2015 09:29:22 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to:content-type :content-transfer-encoding; bh=bn3nCbxImju6m8oO5R1sSeNHQnAygkEdRcCDNdJsQE8=; b=dv3fHGf0d6TIf7PnHOHoeYSxkiblxziH1zrUxEqwY71zKvhMszf6OZwfmBgQ41F1IE vK8RGEX+25YCydouHT5O+g2acc59bSyCfEM95ZCELVcuzXxctoKSfjz+Cv5ipYyBhedp XMM/0IFfrgVgle2EJlw0UKWYh0CbhKPNSfTW2vfwA1wmk3Pfv+ao+sk1j7wm1MnjlHDQ gTQnt5sPy88v0A29VRvKD8wI2QX34HUXyNGs2VBBIisPvfrGiqClqft7O93vHgNH7/q/ gw2mfaYLGOUXMVEBbnicdrTSbpZafuoost7WIDmB4F6ovTv4Bwx6IER+3MP+KxlEbXTr K0Ng== X-Received: by 10.70.48.201 with SMTP id o9mr7710335pdn.4.1421686785531; Mon, 19 Jan 2015 08:59:45 -0800 (PST) X-Gm-Message-State: ALoCoQl5cOBySPbF8zLv2MDniOLciPdXSWmjHCtujZ05G3eEB/KY+1CcsO93r6iD7wTczkOoDXyO9uL9gDMfVyVaSTbH6Xaxp0jtKonDzg6GRB3vuKvlWHTVjn5OpBDMA7z7zAhs5Q/dOLw1qFAjV15wBdnNl8286g== X-Received: by 10.70.48.201 with SMTP id o9mr7710300pdn.4.1421686785323; Mon, 19 Jan 2015 08:59:45 -0800 (PST) From: Stephen Mcconnell References: <54BAAC24.5030704@gmail.com> In-Reply-To: <54BAAC24.5030704@gmail.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQJk6WPRlUxXN5JyaIsLV56GHSNz4JuePZHQ Date: Mon, 19 Jan 2015 09:59:44 -0700 Message-ID: <7e011d51e0792ca8fd0b02d03acc18dd@mail.gmail.com> Subject: RE: LSI 9300-16i supported? To: Tobias Oberstein , freebsd-scsi@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jan 2015 17:29:31 -0000 Hi Tobias, Officially, Avago (LSI) hasn't done ANY testing on 10.1 yet and I personall= y have not used the 9300-16i card. So, officially, no cards are supported on 10.1 yet. Unofficially, my guess is that they both work just fine, but you'll probably have to do your own validation until Avago does its 10.1 testing. Stephen McConnell -----Original Message----- From: owner-freebsd-scsi@freebsd.org [mailto:owner-freebsd-scsi@freebsd.org= ] On Behalf Of Tobias Oberstein Sent: Saturday, January 17, 2015 11:38 AM To: freebsd-scsi@freebsd.org Subject: LSI 9300-16i supported? Hi, The FreeBSD 10.1 hardware compatibility list https://www.freebsd.org/releases/10.1R/hardware.html for the LSI mpr(4) driver mentions these 3 cards: * LSI SAS 3004 (4 Port SAS) * LSI SAS 3008 (8 Port SAS) * LSI SAS 3108 (8 Port SAS) It does _not_ mention the Avago (LSI) 9300-16i http://www.lsi.com/products/host-bus-adapters/Pages/sas-9300-16i.aspx which is quite new, and features _two_ SAS 3008 chips (Fusion MPT=E2=84=A2 = 2.5). =3D=3D Is this chip supported? Any practical experience? From the FreeBSD source tree (not as a binary blob from LSI .. they mentio= n FreeBSD support for that card, but I don't want to depend on them)? I want to deploy _three_ of those cards to drive 48 SAS disks in a quite high-end server. I need to be really confident that it works, since the box is for a customer and costs 100k Euro. If there is any doubt, I probably go with the LSI SAS 9201-16i http://www.lsi.com/products/host-bus-adapters/pages/lsi-sas-9201-16i.aspx as this seems to be well supported for quite some time. Please correct me i= f I'm wrong and should look for something else .. Thanks a lot for any hints! /Tobias _______________________________________________ freebsd-scsi@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-scsi To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" From owner-freebsd-scsi@FreeBSD.ORG Mon Jan 19 19:14:02 2015 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 03C55BC3 for ; Mon, 19 Jan 2015 19:14:02 +0000 (UTC) Received: from mail-wi0-x22d.google.com (mail-wi0-x22d.google.com [IPv6:2a00:1450:400c:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 86DD1A87 for ; Mon, 19 Jan 2015 19:14:01 +0000 (UTC) Received: by mail-wi0-f173.google.com with SMTP id r20so18007354wiv.0 for ; Mon, 19 Jan 2015 11:13:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=pGBecn2bhrDrP8Ea1FrnsBBqleJTOqcEUSOHHAGlI+E=; b=FvbcHXMVjApR8rY/SWeLnWyeeh8RcSuaLUc3x+VS6hOZxvwdnlNEgi3X/hrPvX1vHJ DD7K2mf5Q2c4gGP3eCDfsjU9VFvZactE0X8xcBC2Uoi+actgW8gnhkHdpSvPQXOCW7// paeIjJHs5l2f5xMDcj69emGKQ1VE9/GulpVFwYY9Lgk3eNUwyVuuIzcUWpIHCD18a/Jh gpg1zu8ZGMZStqSVGvK/SUtz54XE0KHlJyeqd5rTZTbFM+hCAyivoPhzQwyCDb1vvoA6 gF5ehINm3fRLUXwpJ8P79pD9ZRis+D3xZNxRR28F4pCu2oVWQlSgFvP46cf9TKvj84mr VfnQ== X-Received: by 10.180.211.169 with SMTP id nd9mr37737334wic.4.1421694839585; Mon, 19 Jan 2015 11:13:59 -0800 (PST) Received: from [192.168.1.130] (ppp-88-217-56-194.dynamic.mnet-online.de. [88.217.56.194]) by mx.google.com with ESMTPSA id i3sm101045wie.23.2015.01.19.11.13.58 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Jan 2015 11:13:58 -0800 (PST) Message-ID: <54BD5774.4010203@gmail.com> Date: Mon, 19 Jan 2015 20:13:56 +0100 From: Tobias Oberstein User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: Stephen Mcconnell , freebsd-scsi@freebsd.org Subject: Re: LSI 9300-16i supported? References: <54BAAC24.5030704@gmail.com> <7e011d51e0792ca8fd0b02d03acc18dd@mail.gmail.com> In-Reply-To: <7e011d51e0792ca8fd0b02d03acc18dd@mail.gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jan 2015 19:14:02 -0000 Hi Stephen, thanks a lot for your info! Btw: is Avago/LSI contributing all it's driver code for FreeBSD to be included in the FreeBSD source tree? E.g. LSI has downloads for FreeBSD drivers: http://www.lsi.com/products/host-bus-adapters/pages/lsi-sas-9201-16i.aspx#tab/tab4 Why is that? Is that different code than the one included in FreeBSD source tree? I'd prefer to rely on standard FreeBSD source tree (definitely no vendor blobs .. but it seems the download above is source anyway). > Officially, Avago (LSI) hasn't done ANY testing on 10.1 yet and I personally > have not used the 9300-16i card. So, officially, no cards are supported on > 10.1 yet. Unofficially, my guess is that they both work just fine, but > you'll probably have to do your own validation until Avago does its 10.1 > testing. I see. I guess I go with the LSI SAS 9201-16i for the time being - probably "less risk". The fact that the 9300 has _two_ 3008 chips on one card could be a source of additional pain .. and then we have 3 of those cards. My problem is: I'd love to use the newer card (more IOPS pushed etc), but this box needs to go to production real quickly, and costs some money. There are enough potential sources of surprise already, like the box will have 8 Intel P3700 PCIe 2TB SSDs. We'll find out what FreeBSD/ZFS/PostgreSQL makes out of these;) Cheers, /Tobias > > Stephen McConnell > > -----Original Message----- > From: owner-freebsd-scsi@freebsd.org [mailto:owner-freebsd-scsi@freebsd.org] > On Behalf Of Tobias Oberstein > Sent: Saturday, January 17, 2015 11:38 AM > To: freebsd-scsi@freebsd.org > Subject: LSI 9300-16i supported? > > Hi, > > The FreeBSD 10.1 hardware compatibility list > > https://www.freebsd.org/releases/10.1R/hardware.html > > for the LSI mpr(4) driver mentions these 3 cards: > > * LSI SAS 3004 (4 Port SAS) > * LSI SAS 3008 (8 Port SAS) > * LSI SAS 3108 (8 Port SAS) > > It does _not_ mention the Avago (LSI) 9300-16i > > http://www.lsi.com/products/host-bus-adapters/Pages/sas-9300-16i.aspx > > which is quite new, and features _two_ SAS 3008 chips (Fusion MPT™ 2.5). > > == > > Is this chip supported? Any practical experience? > > From the FreeBSD source tree (not as a binary blob from LSI .. they mention > FreeBSD support for that card, but I don't want to depend on them)? > > I want to deploy _three_ of those cards to drive 48 SAS disks in a quite > high-end server. I need to be really confident that it works, since the box > is for a customer and costs 100k Euro. > > If there is any doubt, I probably go with the LSI SAS 9201-16i > > http://www.lsi.com/products/host-bus-adapters/pages/lsi-sas-9201-16i.aspx > > as this seems to be well supported for quite some time. Please correct me if > I'm wrong and should look for something else .. > > Thanks a lot for any hints! > > /Tobias > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" > From owner-freebsd-scsi@FreeBSD.ORG Mon Jan 19 19:39:13 2015 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6AFD6528 for ; Mon, 19 Jan 2015 19:39:13 +0000 (UTC) Received: from mail-pa0-f47.google.com (exprod7og120.obsmtp.com [64.18.2.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1D3BAD07 for ; Mon, 19 Jan 2015 19:39:12 +0000 (UTC) Received: from mail-pa0-f47.google.com ([209.85.220.47]) (using TLSv1) by exprod7ob120.postini.com ([64.18.6.12]) with SMTP ID DSNKVL1dWmZPnOjTZ6g/OYs+dAUOYGIf+X7C@postini.com; Mon, 19 Jan 2015 11:39:13 PST Received: by mail-pa0-f47.google.com with SMTP id kq14so40601087pab.6 for ; Mon, 19 Jan 2015 11:39:06 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to:content-type :content-transfer-encoding; bh=t6UKImCVwVAnClh2e+fAUJU8OA/UKqnJ+8Q6H85mnTY=; b=biGfkuR+hqiBErzC8WX8LzAB6piec3TDyYu5vmAW3A8f2LHILZYnb9pYzOqB+KMKEl JZ9hlihgoOMaxTm/SwHZe7FQqSuNYLMxk0Jwhrte9UZTLTFxcWSQeDrz0TWCfUCOpbmz ahpEKCM8i9EU8wAxGSpswG5KgSQC3S5nuMPaq8ftlPEOiy5o7VUwtwIHGtkWiqT6znG3 dqUNx/7woAkyHypU7XlHVa4f8PfxQCGMVnN1RtzcV4uScoDnWXqYphSqj9lZfpeWwRrM IQ+xL7y0cyzE48gdk5SMxlGxMKd5aBAh7l51wf9iRK0Fcp1rTt8kwnMz/Uow2SYwTVA3 KMkA== X-Gm-Message-State: ALoCoQkh09Eq8cyaR0a9qv1cgf4q/D/rF7sSoBP8+sLr0FWehjHONxGBwhYx4uUPRz3JTjnb8+txR0l9kBuXYOeplNHyhM2X/OH3MbqUdSnRVR4wQZJhSWn2z8JOiHo1FsUGu/qxFwU/81JiTaJAcHhVUjdOXwHQ0g== X-Received: by 10.66.141.7 with SMTP id rk7mr25760757pab.15.1421696346614; Mon, 19 Jan 2015 11:39:06 -0800 (PST) X-Received: by 10.66.141.7 with SMTP id rk7mr25760725pab.15.1421696346332; Mon, 19 Jan 2015 11:39:06 -0800 (PST) From: Stephen Mcconnell References: <54BAAC24.5030704@gmail.com> <7e011d51e0792ca8fd0b02d03acc18dd@mail.gmail.com> <54BD5774.4010203@gmail.com> In-Reply-To: <54BD5774.4010203@gmail.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQJk6WPRlUxXN5JyaIsLV56GHSNz4AD/7ZRtAkGpdGKbhF5wkA== Date: Mon, 19 Jan 2015 12:39:05 -0700 Message-ID: Subject: RE: LSI 9300-16i supported? To: Tobias Oberstein , freebsd-scsi@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jan 2015 19:39:13 -0000 > -----Original Message----- > From: Tobias Oberstein [mailto:tobias.oberstein@gmail.com] > Sent: Monday, January 19, 2015 12:14 PM > To: Stephen Mcconnell; freebsd-scsi@freebsd.org > Subject: Re: LSI 9300-16i supported? > > Hi Stephen, > > thanks a lot for your info! > > Btw: is Avago/LSI contributing all it's driver code for FreeBSD to be > included in > the FreeBSD source tree? > > E.g. LSI has downloads for FreeBSD drivers: > > http://www.lsi.com/products/host-bus-adapters/pages/lsi-sas-9201- > 16i.aspx#tab/tab4 > > Why is that? Is that different code than the one included in FreeBSD > source tree? Yes, the FreeBSD tree is a bit different. Avago has out of box drivers tha= t are maintained, and we try to keep the FreeBSD tree up to date with the in-box changes. You should be OK to go with the in-box driver. > > I'd prefer to rely on standard FreeBSD source tree (definitely no vendor > blobs .. > but it seems the download above is source anyway). > > > Officially, Avago (LSI) hasn't done ANY testing on 10.1 yet and I > personally > > have not used the 9300-16i card. So, officially, no cards are > > supported on > > 10.1 yet. Unofficially, my guess is that they both work just fine, > > but you'll probably have to do your own validation until Avago does > > its 10.1 testing. > > I see. > > I guess I go with the LSI SAS 9201-16i for the time being - probably "les= s > risk". > The fact that the 9300 has _two_ 3008 chips on one card could be a source > of > additional pain .. and then we have 3 of those cards. > > My problem is: I'd love to use the newer card (more IOPS pushed etc), but > this > box needs to go to production real quickly, and costs some money. > > There are enough potential sources of surprise already, like the box will > have 8 > Intel P3700 PCIe 2TB SSDs. We'll find out what FreeBSD/ZFS/PostgreSQL > makes > out of these;) > > Cheers, > /Tobias > > > > > Stephen McConnell > > > > -----Original Message----- > > From: owner-freebsd-scsi@freebsd.org > > [mailto:owner-freebsd-scsi@freebsd.org] > > On Behalf Of Tobias Oberstein > > Sent: Saturday, January 17, 2015 11:38 AM > > To: freebsd-scsi@freebsd.org > > Subject: LSI 9300-16i supported? > > > > Hi, > > > > The FreeBSD 10.1 hardware compatibility list > > > > https://www.freebsd.org/releases/10.1R/hardware.html > > > > for the LSI mpr(4) driver mentions these 3 cards: > > > > * LSI SAS 3004 (4 Port SAS) > > * LSI SAS 3008 (8 Port SAS) > > * LSI SAS 3108 (8 Port SAS) > > > > It does _not_ mention the Avago (LSI) 9300-16i > > > > http://www.lsi.com/products/host-bus-adapters/Pages/sas-9300-16i.aspx > > > > which is quite new, and features _two_ SAS 3008 chips (Fusion MPT=E2=84= =A2 2.5). > > > > =3D=3D > > > > Is this chip supported? Any practical experience? > > > > From the FreeBSD source tree (not as a binary blob from LSI .. they > > mention FreeBSD support for that card, but I don't want to depend on > > them)? > > > > I want to deploy _three_ of those cards to drive 48 SAS disks in a > > quite high-end server. I need to be really confident that it works, > > since the box is for a customer and costs 100k Euro. > > > > If there is any doubt, I probably go with the LSI SAS 9201-16i > > > > http://www.lsi.com/products/host-bus-adapters/pages/lsi-sas-9201-16i.a > > spx > > > > as this seems to be well supported for quite some time. Please correct > > me if I'm wrong and should look for something else .. > > > > Thanks a lot for any hints! > > > > /Tobias > > _______________________________________________ > > freebsd-scsi@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-scsi > > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" > > From owner-freebsd-scsi@FreeBSD.ORG Fri Jan 23 05:01:39 2015 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 241D13F0 for ; Fri, 23 Jan 2015 05:01:39 +0000 (UTC) Received: from mwork.nabble.com (mwork.nabble.com [162.253.133.43]) by mx1.freebsd.org (Postfix) with ESMTP id 0ED0DAD9 for ; Fri, 23 Jan 2015 05:01:38 +0000 (UTC) Received: from msam.nabble.com (unknown [162.253.133.85]) by mwork.nabble.com (Postfix) with ESMTP id 402471171962 for ; Thu, 22 Jan 2015 21:01:40 -0800 (PST) Date: Thu, 22 Jan 2015 22:01:37 -0700 (MST) From: skis4hire To: freebsd-scsi@freebsd.org Message-ID: <1421989297990-5983034.post@n5.nabble.com> Subject: CAM_MAX_HIGHPOWER MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jan 2015 05:01:39 -0000 CAM_MAX_HIGHPOWER is a kernel option for the CAM module to specify how many disks can spin up at once, default is 4. Is there an equivalent run-time/boot-time method to set this, like via loader.conf? As far as I can tell this is a compile-time option only but I would like to avoid re-compiling the kernel. I plan to set this to 1 or 2 in conjunction with PUIS to reduce peak boot power for a multi disk system. -- View this message in context: http://freebsd.1045724.n5.nabble.com/CAM-MAX-HIGHPOWER-tp5983034.html Sent from the freebsd-scsi mailing list archive at Nabble.com. From owner-freebsd-scsi@FreeBSD.ORG Fri Jan 23 09:16:33 2015 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 19837A55 for ; Fri, 23 Jan 2015 09:16:33 +0000 (UTC) Received: from mail-lb0-x22c.google.com (mail-lb0-x22c.google.com [IPv6:2a00:1450:4010:c04::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8FF5E699 for ; Fri, 23 Jan 2015 09:16:32 +0000 (UTC) Received: by mail-lb0-f172.google.com with SMTP id l4so6012231lbv.3 for ; Fri, 23 Jan 2015 01:16:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=t1DmQEDLNH2BVeU0syrpU7UK1mjhmAhTm+hvqp9Uqeg=; b=ml2/O0FJ+CMdhA3Al1aFO6Qr2dkpSPtAZRWFDUwNIRlkclhkVyD+3TlbQkrIBn1l3E xzrIw35hpgfJVfhvx9qHEWM9iabGnXE44V3x9ST5pNaZSeNsYY/jpv+SYksi+Kawr9FB 6J62OgxH0goziyCOjM6weTiNuXeK//82p6RmEXdH1mB3mwMPbJz6gAbF+NpFGTZ5pCYo y8tazl5wZuggsAZfeozki7xt/qLmXHfbs59kej02d5CGkBS6aRF1tB0aUc8GG8+S8OIZ bXjjNT7ErdkYMvwmjKTCvT44eT2TOu+f2OHed9tbQuPqWCIoORg3hj14wroLP4rL04BP dVWA== X-Received: by 10.152.45.4 with SMTP id i4mr6225311lam.74.1422004590432; Fri, 23 Jan 2015 01:16:30 -0800 (PST) MIME-Version: 1.0 Received: by 10.25.163.73 with HTTP; Fri, 23 Jan 2015 01:16:09 -0800 (PST) From: Ahmed Kamal Date: Fri, 23 Jan 2015 11:16:09 +0200 Message-ID: Subject: Dell branded storage controller for ZFS file-server | Advice requested To: freebsd-scsi@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jan 2015 09:16:33 -0000 Hi everyone, I'm looking to build a ZFS file-server, unfortunately I'm locked into Dell branded hardware only. I know Dell sometimes ships LSI branded cards, but that too probably won't work for me, as I live outside the US and the local Dell rep is clueless (so whatever I want to buy, has to be on Dell's website). I am considering Dell R720xd or R730xd. I will use Intel i350 NICs, and ECC RAM, but the biggest questions are around the SAS HBA Per my research, I have the following notes and questions. Appreciating your comments on them * H310 Card supports syspd (jbod) mode per http://svnweb.freebsd.org/base?view=revision&revision=254906 so it is probably the best card for what I want to do, right? * H310 is a low end card, it has very limited queue depth (25 I believe) .. Is this actually a problem when using the disks (20 x SAS 10k disks) in syspd/jbod mode? what about if I attach a couple of SSD disks? * H310 can be flashed to IT mode (LSI firmware), and is supported by Illumos kernel (if I'll ever need that) .. so overall it looks like a good option. Any "watch-outs" I should be aware of ? * H710/p have no way of exposing JBOD/syspd in any way .. If using those cards, the best I can do, is to create a raid-0 per disk, and put zfs on top of that. How bad is this? Is the only problem that I'm locked into H710 firmware? Would the nvram and bbu on H710 improve performance significantly vs H310? * H330 and H730, seem supported by mrsas driver. Is this driver able to expose jbod/syspd mode? If yes, what's the overall advise, should I go with one of those, or the "trusty" H310 ? Your help and advice is most appreciated From owner-freebsd-scsi@FreeBSD.ORG Fri Jan 23 10:21:21 2015 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8DE19530 for ; Fri, 23 Jan 2015 10:21:21 +0000 (UTC) Received: from mail-pa0-f43.google.com (mail-pa0-f43.google.com [209.85.220.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 60864DB9 for ; Fri, 23 Jan 2015 10:21:20 +0000 (UTC) Received: by mail-pa0-f43.google.com with SMTP id eu11so7824904pac.2 for ; Fri, 23 Jan 2015 02:21:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=WwwLr39lD77pOEqm2z5QAAxTSt/0d5svUmctiAybN7s=; b=StIFvWhW7/s7G+jueugWOjxCxDvFp9sG8C3jo4As08zT3SuSChVdvTR0jQbsUlarIH SuxOrzYlccY5437cHrApQ9klhctxzTf9PM6MdAbnu/4Zg7z7bJPHuvtWgvcxX09AVD8m OZ79M1s9yu54uG+awcM4ZnncYUbYAuZeq5igoD13rrUAYdgNVSXhGa3hRZJLPM4Oe/hN hVzlh0y6A9qM67Ayqkt8wGDhPub+z+z4kUi2zY2JxLS/fYMcLzp7fG9yTY3pDqM17TKJ PszFm32uKbPXEOvZnVQHwbzHu7R2lwceDlfpKfb0gNVm2hBv9yjrQAcIabguGwzyUW+L uwow== X-Gm-Message-State: ALoCoQliY7KpOjxYSFPTnyqmoIwQ9hNfpzdU0J+7F1Y+Iqqdqbc7yZQasTIxh2RIyAYjR3T+/A/S X-Received: by 10.68.131.3 with SMTP id oi3mr9890207pbb.27.1422008474606; Fri, 23 Jan 2015 02:21:14 -0800 (PST) Received: from thinmac.home.animata.net (60-241-41-69.static.tpgi.com.au. [60.241.41.69]) by mx.google.com with ESMTPSA id pk11sm1468325pdb.5.2015.01.23.02.21.11 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 23 Jan 2015 02:21:13 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\)) Subject: Re: Dell branded storage controller for ZFS file-server | Advice requested From: David Gwynne In-Reply-To: Date: Fri, 23 Jan 2015 20:21:05 +1000 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Ahmed Kamal X-Mailer: Apple Mail (2.1993) Cc: freebsd-scsi@freebsd.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jan 2015 10:21:21 -0000 > On 23 Jan 2015, at 7:16 pm, Ahmed Kamal = wrote: >=20 > Hi everyone, >=20 > I'm looking to build a ZFS file-server, unfortunately I'm locked into = Dell > branded hardware only. I know Dell sometimes ships LSI branded cards, = but > that too probably won't work for me, as I live outside the US and the = local > Dell rep is clueless (so whatever I want to buy, has to be on Dell's > website). I am considering Dell R720xd or R730xd. I will use Intel = i350 > NICs, and ECC RAM, but the biggest questions are around the SAS HBA >=20 > Per my research, I have the following notes and questions. = Appreciating > your comments on them >=20 > * H310 Card supports syspd (jbod) mode per > http://svnweb.freebsd.org/base?view=3Drevision&revision=3D254906 so it = is > probably the best card for what I want to do, right? no. > * H310 is a low end card, it has very limited queue depth (25 I = believe) .. > Is this actually a problem when using the disks (20 x SAS 10k disks) = in > syspd/jbod mode? what about if I attach a couple of SSD disks? there are two queue limits here, one for the controller and one for each = disk attached to it. h310s have a controller queue depth of 31, ie, they can only support 31 = commands at a time. if you have 24 disks and distribute the controllers = command slots between them, thats about 1 command per disk. > * H310 can be flashed to IT mode (LSI firmware), and is supported by > Illumos kernel (if I'll ever need that) .. so overall it looks like a = good > option. Any "watch-outs" I should be aware of ? yes. if you flash the h310 to become an sas hba, the pci product/vendor = and subproduct/subvendor ids on the controller change. the dell bios = will refuse to boot if it detects an unsupported controller in its = storage slot. you'll brick the machine until you remove the h310. = because it is on a non standard mezzanine connector you will not be able = to flash it back to a h310, and you'll have a useless chunk of silicon = to put on your desk as a monument to how annoying vendors are. > * H710/p have no way of exposing JBOD/syspd in any way .. If using = those > cards, the best I can do, is to create a raid-0 per disk, and put zfs = on > top of that. How bad is this? Is the only problem that I'm locked into = H710 > firmware? Would the nvram and bbu on H710 improve performance = significantly > vs H310? i wouldnt run zfs on that. > * H330 and H730, seem supported by mrsas driver. Is this driver able = to > expose jbod/syspd mode? If yes, what's the overall advise, should I go = with > one of those, or the "trusty" H310 ? thats a workable option. h330 should be fine if all you want to do is = jbod. >=20 > Your help and advice is most appreciated > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to = "freebsd-scsi-unsubscribe@freebsd.org" From owner-freebsd-scsi@FreeBSD.ORG Fri Jan 23 11:41:49 2015 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 164034FD for ; Fri, 23 Jan 2015 11:41:49 +0000 (UTC) Received: from mail-lb0-x235.google.com (mail-lb0-x235.google.com [IPv6:2a00:1450:4010:c04::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 79BCC84C for ; Fri, 23 Jan 2015 11:41:48 +0000 (UTC) Received: by mail-lb0-f181.google.com with SMTP id u10so6524829lbd.12 for ; Fri, 23 Jan 2015 03:41:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=wDJFZ3GOZ2T3vCnwfsQhIq0w3EW/mK946BoyDqhrnh8=; b=VvkiFWXU2mtK62JyLXhD1I1VrQzue25GSGtPQuMZMtcm3jGnDruVnLLlCvqnt7MFg7 224kl8kPsAkLQOcPaDNz/3fHPDz3DuHMFBCbXzhTzxFAolv4qOPppMFkCCmvCBIoPWC+ gkm2fu7g4Ix8y2O3PjC18Rh7O52DqifIszBggpiDe0d2XhlBdEs5SSacRX7VviyHU0y5 ON6gzrcXR3YRrC34klVdbjhQwZmlfQM1YyvJnZe+A7UAOvkjD8qiiy3YPH9rnyHCb4w+ QZjHZpE4dHgqmJWibomwYJKxdQCmogEbJt2VY5hRRX8YYiETQzAxblqjjkq60gGBGZST 8iOw== X-Received: by 10.152.5.226 with SMTP id v2mr6872714lav.34.1422013306228; Fri, 23 Jan 2015 03:41:46 -0800 (PST) MIME-Version: 1.0 Received: by 10.25.163.73 with HTTP; Fri, 23 Jan 2015 03:41:25 -0800 (PST) In-Reply-To: References: From: Ahmed Kamal Date: Fri, 23 Jan 2015 13:41:25 +0200 Message-ID: Subject: Re: Dell branded storage controller for ZFS file-server | Advice requested To: David Gwynne Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-scsi@freebsd.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jan 2015 11:41:49 -0000 Thanks a lot for your reply .. I've got a couple of questions inline On Fri, Jan 23, 2015 at 12:21 PM, David Gwynne wrote: > > > On 23 Jan 2015, at 7:16 pm, Ahmed Kamal > wrote: > > > > Hi everyone, > > > > I'm looking to build a ZFS file-server, unfortunately I'm locked into > Dell > > branded hardware only. I know Dell sometimes ships LSI branded cards, but > > that too probably won't work for me, as I live outside the US and the > local > > Dell rep is clueless (so whatever I want to buy, has to be on Dell's > > website). I am considering Dell R720xd or R730xd. I will use Intel i350 > > NICs, and ECC RAM, but the biggest questions are around the SAS HBA > > > > Per my research, I have the following notes and questions. Appreciating > > your comments on them > > > > * H310 Card supports syspd (jbod) mode per > > http://svnweb.freebsd.org/base?view=revision&revision=254906 so it is > > probably the best card for what I want to do, right? > > no. > > > * H310 is a low end card, it has very limited queue depth (25 I believe) > .. > > Is this actually a problem when using the disks (20 x SAS 10k disks) in > > syspd/jbod mode? what about if I attach a couple of SSD disks? > > there are two queue limits here, one for the controller and one for each > disk attached to it. > > h310s have a controller queue depth of 31, ie, they can only support 31 > commands at a time. if you have 24 disks and distribute the controllers > command slots between them, thats about 1 command per disk. > My initial line of thought was that in pass-through (jbod) mode, the HBA does *not* buffer commands and proxy disk responses internally. Which meant that in jbod mode, the HBA basically becomes a communications channel connecting the OS to disks, making its internal buffering limitations not-applicable. Is this not correct ? > > > * H310 can be flashed to IT mode (LSI firmware), and is supported by > > Illumos kernel (if I'll ever need that) .. so overall it looks like a > good > > option. Any "watch-outs" I should be aware of ? > > yes. if you flash the h310 to become an sas hba, the pci product/vendor > and subproduct/subvendor ids on the controller change. the dell bios will > refuse to boot if it detects an unsupported controller in its storage slot. > you'll brick the machine until you remove the h310. because it is on a non > standard mezzanine connector you will not be able to flash it back to a > h310, and you'll have a useless chunk of silicon to put on your desk as a > monument to how annoying vendors are. > pff nightmare scenario .. thanks for pointing this out. I stumbled across web posts where people successfully flashed h310, but meh, now I'm scared to buy that configuration > > > * H710/p have no way of exposing JBOD/syspd in any way .. If using those > > cards, the best I can do, is to create a raid-0 per disk, and put zfs on > > top of that. How bad is this? Is the only problem that I'm locked into > H710 > > firmware? Would the nvram and bbu on H710 improve performance > significantly > > vs H310? > > i wouldnt run zfs on that. > > > * H330 and H730, seem supported by mrsas driver. Is this driver able to > > expose jbod/syspd mode? If yes, what's the overall advise, should I go > with > > one of those, or the "trusty" H310 ? > > thats a workable option. h330 should be fine if all you want to do is jbod. > Now that you're pointing me to H330 being my best option given the situation, got some more questions: * "if all you want is jbod" .. with zfs, that's all we need right ? just wondering if I'm missing something important * Any idea if jbod configuration on h330 is done on bios level, or using some freebsd tool (how?) * mrsas man page, mentions the driver appeared in 10.1 .. Does that mean if I decide to use FreeNAS (9.x based) it won't be there ? * If I go the h330 path, it'll be a r730xd server. Worst case if the h330 gives me trouble, can I throw it out and plug a LSI 9207-8i in its place (same pci slot, cables ..etc) ? Thanks! > > > > > Your help and advice is most appreciated > > _______________________________________________ > > freebsd-scsi@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-scsi > > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" > > From owner-freebsd-scsi@FreeBSD.ORG Fri Jan 23 11:57:52 2015 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DBA96CA9 for ; Fri, 23 Jan 2015 11:57:52 +0000 (UTC) Received: from mail-pa0-f51.google.com (mail-pa0-f51.google.com [209.85.220.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AD4F299C for ; Fri, 23 Jan 2015 11:57:52 +0000 (UTC) Received: by mail-pa0-f51.google.com with SMTP id fb1so8216252pad.10 for ; Fri, 23 Jan 2015 03:57:46 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=7ellI1AyIPrEp8zjd+ETOKORrWXVc73y0UhKskZGios=; b=I2DynzhDMPtctFouhEi21bEdlcUxQwQOYD4MnK6lyZBH3goQ2UUa4zhzkiQXFIRNIZ rQyTALBKNIVpfK04jJkwzDASPQPXfNt+9EZe4QV2fGjvgt7h5Xs4gUuBGqJIL5Y9HtlI D361Dse1Ap6yia40WSNfsTLt6CVPX/kEEy8rK9nxv8f7N6/YGWI04G88WrB0aHuK3VYN LSo/pA/VDVc+KH+WmNrOfkTuPEIj0yhW3aO4CHntWeOSV7lY4WlOLm7QZE0Y4oY9nbSW gxXgP/i/5D3CS8ssk7nsjx2kp1/TPMzbDRkA7QFhxUgdmyKpWtxMkKjM7SV6NkjEy0Ig WeYg== X-Gm-Message-State: ALoCoQkVAeDi+b+5NizIzQ+6CSAdnceb9a8U2GtQb1VQGWxmuqvDqv73+8yaavT4VgbcJFnOIJjJ X-Received: by 10.70.100.202 with SMTP id fa10mr5105002pdb.131.1422014266561; Fri, 23 Jan 2015 03:57:46 -0800 (PST) Received: from ?IPv6:2001:388:e000:ba00:86a:91f9:4155:c482? ([2001:388:e000:ba00:86a:91f9:4155:c482]) by mx.google.com with ESMTPSA id w9sm1752157pbt.10.2015.01.23.03.57.43 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 23 Jan 2015 03:57:45 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\)) Subject: Re: Dell branded storage controller for ZFS file-server | Advice requested From: David Gwynne In-Reply-To: Date: Fri, 23 Jan 2015 21:57:38 +1000 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Ahmed Kamal X-Mailer: Apple Mail (2.1993) Cc: freebsd-scsi@freebsd.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jan 2015 11:57:52 -0000 > On 23 Jan 2015, at 9:41 pm, Ahmed Kamal = wrote: >=20 > Thanks a lot for your reply .. I've got a couple of questions inline >=20 > On Fri, Jan 23, 2015 at 12:21 PM, David Gwynne = wrote: >=20 > > On 23 Jan 2015, at 7:16 pm, Ahmed Kamal = wrote: > > > > Hi everyone, > > > > I'm looking to build a ZFS file-server, unfortunately I'm locked = into Dell > > branded hardware only. I know Dell sometimes ships LSI branded = cards, but > > that too probably won't work for me, as I live outside the US and = the local > > Dell rep is clueless (so whatever I want to buy, has to be on Dell's > > website). I am considering Dell R720xd or R730xd. I will use Intel = i350 > > NICs, and ECC RAM, but the biggest questions are around the SAS HBA > > > > Per my research, I have the following notes and questions. = Appreciating > > your comments on them > > > > * H310 Card supports syspd (jbod) mode per > > http://svnweb.freebsd.org/base?view=3Drevision&revision=3D254906 so = it is > > probably the best card for what I want to do, right? >=20 > no. >=20 > > * H310 is a low end card, it has very limited queue depth (25 I = believe) .. > > Is this actually a problem when using the disks (20 x SAS 10k disks) = in > > syspd/jbod mode? what about if I attach a couple of SSD disks? >=20 > there are two queue limits here, one for the controller and one for = each disk attached to it. >=20 > h310s have a controller queue depth of 31, ie, they can only support = 31 commands at a time. if you have 24 disks and distribute the = controllers command slots between them, thats about 1 command per disk. >=20 > My initial line of thought was that in pass-through (jbod) mode, the = HBA does *not* buffer commands and proxy disk responses internally. = Which meant that in jbod mode, the HBA basically becomes a = communications channel connecting the OS to disks, making its internal = buffering limitations not-applicable. Is this not correct ? the h310 physically doesnt have cache, so it cant buffer your I/O even = if it wanted to. so yes, in your use case it would just be a hba, but a very constrained = one. even most sata controllers (eg, ahci or the silicon image ones) can = keep more commands up in the air since they can have up 32 command slots = per port instead of 31 for the whole controller. im also not sure if dell will let you order an r720xd with a h310. i = dont think they consider that a supported config. > =20 >=20 > > * H310 can be flashed to IT mode (LSI firmware), and is supported by > > Illumos kernel (if I'll ever need that) .. so overall it looks like = a good > > option. Any "watch-outs" I should be aware of ? >=20 > yes. if you flash the h310 to become an sas hba, the pci = product/vendor and subproduct/subvendor ids on the controller change. = the dell bios will refuse to boot if it detects an unsupported = controller in its storage slot. you'll brick the machine until you = remove the h310. because it is on a non standard mezzanine connector you = will not be able to flash it back to a h310, and you'll have a useless = chunk of silicon to put on your desk as a monument to how annoying = vendors are. >=20 > pff nightmare scenario .. thanks for pointing this out. I stumbled = across web posts where people successfully flashed h310, but meh, now = I'm scared to buy that configuration there are h310s that are normal pci-e cards that go into dells lower end = servers. the ones in the r720s sit on the custom mezzanine connector. > > * H710/p have no way of exposing JBOD/syspd in any way .. If using = those > > cards, the best I can do, is to create a raid-0 per disk, and put = zfs on > > top of that. How bad is this? Is the only problem that I'm locked = into H710 > > firmware? Would the nvram and bbu on H710 improve performance = significantly > > vs H310? >=20 > i wouldnt run zfs on that. >=20 > > * H330 and H730, seem supported by mrsas driver. Is this driver able = to > > expose jbod/syspd mode? If yes, what's the overall advise, should I = go with > > one of those, or the "trusty" H310 ? >=20 > thats a workable option. h330 should be fine if all you want to do is = jbod. >=20 > Now that you're pointing me to H330 being my best option given the = situation, got some more questions: > * "if all you want is jbod" .. with zfs, that's all we need right ? = just wondering if I'm missing something important nope. > * Any idea if jbod configuration on h330 is done on bios level, or = using some freebsd tool (how?) you can do it in the bios. > * mrsas man page, mentions the driver appeared in 10.1 .. Does that = mean if I decide to use FreeNAS (9.x based) it won't be there ? i have no idea :) > * If I go the h330 path, it'll be a r730xd server. Worst case if the = h330 gives me trouble, can I throw it out and plug a LSI 9207-8i in its = place (same pci slot, cables ..etc) ? the h330 sits on a mezzanine connector. you probably can get that card = and drive the disks with it, but i havent done it with an r730 or 730xd = so i cant say for sure. i have done that with an r720 though, and it = mostly works. we had to order those systems with a h310 and add the lsi card = ourselves, and then we discovered that the disks they shipped in the = system do not spin up when theyre powered on. the raid controller (h310) = explicitly spins them up before issuing io to them. if you attach those disks to a straight sas hba or use them via syspd on = the h310, you cant boot off them because the machines bios (not the raid = controller) blindly issues io to them without spinning them up first. fun times. dlg > =20 >=20 > Thanks! > =20 >=20 > > > > Your help and advice is most appreciated > > _______________________________________________ > > freebsd-scsi@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-scsi > > To unsubscribe, send any mail to = "freebsd-scsi-unsubscribe@freebsd.org" >=20 >=20 From owner-freebsd-scsi@FreeBSD.ORG Fri Jan 23 12:57:07 2015 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 72E0EE4A for ; Fri, 23 Jan 2015 12:57:07 +0000 (UTC) Received: from mail-wi0-f171.google.com (mail-wi0-f171.google.com [209.85.212.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 08142F99 for ; Fri, 23 Jan 2015 12:57:06 +0000 (UTC) Received: by mail-wi0-f171.google.com with SMTP id l15so2624734wiw.4 for ; Fri, 23 Jan 2015 04:56:59 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=mNTpLwvFtcfkja3l8IEHXAvV2HfONlCHeIeE4ACnhIU=; b=aZE5/sPsdCUq2ukFyoXHYLAU+cdtbgX5+UuBW0tWhF+xMbZVSmuPFOchKFUB6ztYr4 +5wn0xQUfTKGAdNOPPZY3ESPW+QUmnpmiENf7oFYKA5ZZOHfnnY1jRoZkBK8hDMVUhhA z1PIH5GCAN78q0IPbuy0u7BR5YwYxF5RiVe2PxSwr2uwV+ICBsjXs6axVaZxiYgXwWyB Rr1cfERMxSxSxmUsD81eXVWBofIbcdWL7n/MWVf2HspetGbUcAWpN8wFZX1DJvIlA9IC h0g64TD2kbnKdhcgRBYGheamsisJNgqZyUz/KqClyMdCUqmcO0hENb5bt262TCMa+KIJ qSkg== X-Gm-Message-State: ALoCoQmc6mM435UTdnQs2dY05p8e1mhg05Ccbat9R+qlMo45qvcclMXkLN56TfpQOe7lz2oo6vpL X-Received: by 10.194.189.77 with SMTP id gg13mr10534588wjc.81.1422017333046; Fri, 23 Jan 2015 04:48:53 -0800 (PST) Received: from [10.10.1.68] (82-69-141-170.dsl.in-addr.zen.co.uk. [82.69.141.170]) by mx.google.com with ESMTPSA id vs8sm2107934wjc.6.2015.01.23.04.48.51 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 23 Jan 2015 04:48:52 -0800 (PST) Message-ID: <54C243A4.6070703@multiplay.co.uk> Date: Fri, 23 Jan 2015 12:50:44 +0000 From: Steven Hartland User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: freebsd-scsi@freebsd.org Subject: Re: Dell branded storage controller for ZFS file-server | Advice requested References: In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jan 2015 12:57:07 -0000 Dells c6000 chassis have the option of LSI controllers, however their midplane is troublesome and will cause issues at 6Gbps speed, so be careful. Regards Steve On 23/01/2015 11:57, David Gwynne wrote: >> On 23 Jan 2015, at 9:41 pm, Ahmed Kamal wrote: >> >> Thanks a lot for your reply .. I've got a couple of questions inline >> >> On Fri, Jan 23, 2015 at 12:21 PM, David Gwynne wrote: >> >>> On 23 Jan 2015, at 7:16 pm, Ahmed Kamal wrote: >>> >>> Hi everyone, >>> >>> I'm looking to build a ZFS file-server, unfortunately I'm locked into Dell >>> branded hardware only. I know Dell sometimes ships LSI branded cards, but >>> that too probably won't work for me, as I live outside the US and the local >>> Dell rep is clueless (so whatever I want to buy, has to be on Dell's >>> website). I am considering Dell R720xd or R730xd. I will use Intel i350 >>> NICs, and ECC RAM, but the biggest questions are around the SAS HBA >>> >>> Per my research, I have the following notes and questions. Appreciating >>> your comments on them >>> >>> * H310 Card supports syspd (jbod) mode per >>> http://svnweb.freebsd.org/base?view=revision&revision=254906 so it is >>> probably the best card for what I want to do, right? >> no. >> >>> * H310 is a low end card, it has very limited queue depth (25 I believe) .. >>> Is this actually a problem when using the disks (20 x SAS 10k disks) in >>> syspd/jbod mode? what about if I attach a couple of SSD disks? >> there are two queue limits here, one for the controller and one for each disk attached to it. >> >> h310s have a controller queue depth of 31, ie, they can only support 31 commands at a time. if you have 24 disks and distribute the controllers command slots between them, thats about 1 command per disk. >> >> My initial line of thought was that in pass-through (jbod) mode, the HBA does *not* buffer commands and proxy disk responses internally. Which meant that in jbod mode, the HBA basically becomes a communications channel connecting the OS to disks, making its internal buffering limitations not-applicable. Is this not correct ? > the h310 physically doesnt have cache, so it cant buffer your I/O even if it wanted to. > > so yes, in your use case it would just be a hba, but a very constrained one. even most sata controllers (eg, ahci or the silicon image ones) can keep more commands up in the air since they can have up 32 command slots per port instead of 31 for the whole controller. > > im also not sure if dell will let you order an r720xd with a h310. i dont think they consider that a supported config. > >> >> >>> * H310 can be flashed to IT mode (LSI firmware), and is supported by >>> Illumos kernel (if I'll ever need that) .. so overall it looks like a good >>> option. Any "watch-outs" I should be aware of ? >> yes. if you flash the h310 to become an sas hba, the pci product/vendor and subproduct/subvendor ids on the controller change. the dell bios will refuse to boot if it detects an unsupported controller in its storage slot. you'll brick the machine until you remove the h310. because it is on a non standard mezzanine connector you will not be able to flash it back to a h310, and you'll have a useless chunk of silicon to put on your desk as a monument to how annoying vendors are. >> >> pff nightmare scenario .. thanks for pointing this out. I stumbled across web posts where people successfully flashed h310, but meh, now I'm scared to buy that configuration > there are h310s that are normal pci-e cards that go into dells lower end servers. the ones in the r720s sit on the custom mezzanine connector. > >>> * H710/p have no way of exposing JBOD/syspd in any way .. If using those >>> cards, the best I can do, is to create a raid-0 per disk, and put zfs on >>> top of that. How bad is this? Is the only problem that I'm locked into H710 >>> firmware? Would the nvram and bbu on H710 improve performance significantly >>> vs H310? >> i wouldnt run zfs on that. >> >>> * H330 and H730, seem supported by mrsas driver. Is this driver able to >>> expose jbod/syspd mode? If yes, what's the overall advise, should I go with >>> one of those, or the "trusty" H310 ? >> thats a workable option. h330 should be fine if all you want to do is jbod. >> >> Now that you're pointing me to H330 being my best option given the situation, got some more questions: >> * "if all you want is jbod" .. with zfs, that's all we need right ? just wondering if I'm missing something important > nope. > >> * Any idea if jbod configuration on h330 is done on bios level, or using some freebsd tool (how?) > you can do it in the bios. > >> * mrsas man page, mentions the driver appeared in 10.1 .. Does that mean if I decide to use FreeNAS (9.x based) it won't be there ? > i have no idea :) > >> * If I go the h330 path, it'll be a r730xd server. Worst case if the h330 gives me trouble, can I throw it out and plug a LSI 9207-8i in its place (same pci slot, cables ..etc) ? > the h330 sits on a mezzanine connector. you probably can get that card and drive the disks with it, but i havent done it with an r730 or 730xd so i cant say for sure. i have done that with an r720 though, and it mostly works. > > we had to order those systems with a h310 and add the lsi card ourselves, and then we discovered that the disks they shipped in the system do not spin up when theyre powered on. the raid controller (h310) explicitly spins them up before issuing io to them. > > if you attach those disks to a straight sas hba or use them via syspd on the h310, you cant boot off them because the machines bios (not the raid controller) blindly issues io to them without spinning them up first. > > fun times. > > dlg > > > >> >> >> Thanks! >> >> >>> Your help and advice is most appreciated >>> _______________________________________________ >>> freebsd-scsi@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-scsi >>> To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" >> > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" From owner-freebsd-scsi@FreeBSD.ORG Fri Jan 23 13:02:54 2015 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9990DFC for ; Fri, 23 Jan 2015 13:02:54 +0000 (UTC) Received: from mail-la0-x234.google.com (mail-la0-x234.google.com [IPv6:2a00:1450:4010:c03::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0994DEF for ; Fri, 23 Jan 2015 13:02:54 +0000 (UTC) Received: by mail-la0-f52.google.com with SMTP id ge10so2982494lab.11 for ; Fri, 23 Jan 2015 05:02:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=6nxbQY+Va2Rs8SXj/+biSvkhc+M2XP4UVOyjz/2U+Tw=; b=u7v1pEjt230P2gfqFEkz2R9GSihI7LrjEENbO5W0cwEAkg1wjYx/JBcBlk60Pp1y5X +sMz6Rxm2Z7MD4pxSo5a9OCad72VuNy4wQW+NzxQ/hRG8w9zn48GMrOdLTHKAkfzyPo3 /eZS2uSvcuhjgz1IHv03D52FgtJMgs7ZfAhJ9REFHeg7T3mmz5qVFj1KPcu73KVS+9zx fCXxF2glAKg9Z58DZdnV6EEAQAIE8XvSK+GYijBNeHrDXiU2PC5zL1Oe5SaWSUNya10O iIw5EYWs8ARv3Xd9zbOSuwHP21vUB9GXTx92YK32F+aS2v9kESBkSiXyBsZrPbvdDjqy x63A== X-Received: by 10.112.38.4 with SMTP id c4mr7261116lbk.46.1422018171927; Fri, 23 Jan 2015 05:02:51 -0800 (PST) MIME-Version: 1.0 Received: by 10.25.163.73 with HTTP; Fri, 23 Jan 2015 05:02:31 -0800 (PST) In-Reply-To: References: From: Ahmed Kamal Date: Fri, 23 Jan 2015 15:02:31 +0200 Message-ID: Subject: Re: Dell branded storage controller for ZFS file-server | Advice requested To: David Gwynne Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-scsi@freebsd.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jan 2015 13:02:54 -0000 comments down below On Fri, Jan 23, 2015 at 1:57 PM, David Gwynne wrote: > > > On 23 Jan 2015, at 9:41 pm, Ahmed Kamal > wrote: > > > > Thanks a lot for your reply .. I've got a couple of questions inline > > > > On Fri, Jan 23, 2015 at 12:21 PM, David Gwynne > wrote: > > > > > On 23 Jan 2015, at 7:16 pm, Ahmed Kamal < > email.ahmedkamal@googlemail.com> wrote: > > > > > > Hi everyone, > > > > > > I'm looking to build a ZFS file-server, unfortunately I'm locked into > Dell > > > branded hardware only. I know Dell sometimes ships LSI branded cards, > but > > > that too probably won't work for me, as I live outside the US and the > local > > > Dell rep is clueless (so whatever I want to buy, has to be on Dell's > > > website). I am considering Dell R720xd or R730xd. I will use Intel i350 > > > NICs, and ECC RAM, but the biggest questions are around the SAS HBA > > > > > > Per my research, I have the following notes and questions. Appreciating > > > your comments on them > > > > > > * H310 Card supports syspd (jbod) mode per > > > http://svnweb.freebsd.org/base?view=revision&revision=254906 so it is > > > probably the best card for what I want to do, right? > > > > no. > > > > > * H310 is a low end card, it has very limited queue depth (25 I > believe) .. > > > Is this actually a problem when using the disks (20 x SAS 10k disks) in > > > syspd/jbod mode? what about if I attach a couple of SSD disks? > > > > there are two queue limits here, one for the controller and one for each > disk attached to it. > > > > h310s have a controller queue depth of 31, ie, they can only support 31 > commands at a time. if you have 24 disks and distribute the controllers > command slots between them, thats about 1 command per disk. > > > > My initial line of thought was that in pass-through (jbod) mode, the HBA > does *not* buffer commands and proxy disk responses internally. Which meant > that in jbod mode, the HBA basically becomes a communications channel > connecting the OS to disks, making its internal buffering limitations > not-applicable. Is this not correct ? > > the h310 physically doesnt have cache, so it cant buffer your I/O even if > it wanted to. > > so yes, in your use case it would just be a hba, but a very constrained > one. even most sata controllers (eg, ahci or the silicon image ones) can > keep more commands up in the air since they can have up 32 command slots > per port instead of 31 for the whole controller. > > im also not sure if dell will let you order an r720xd with a h310. i dont > think they consider that a supported config. > > > > > > > > * H310 can be flashed to IT mode (LSI firmware), and is supported by > > > Illumos kernel (if I'll ever need that) .. so overall it looks like a > good > > > option. Any "watch-outs" I should be aware of ? > > > > yes. if you flash the h310 to become an sas hba, the pci product/vendor > and subproduct/subvendor ids on the controller change. the dell bios will > refuse to boot if it detects an unsupported controller in its storage slot. > you'll brick the machine until you remove the h310. because it is on a non > standard mezzanine connector you will not be able to flash it back to a > h310, and you'll have a useless chunk of silicon to put on your desk as a > monument to how annoying vendors are. > > > > pff nightmare scenario .. thanks for pointing this out. I stumbled > across web posts where people successfully flashed h310, but meh, now I'm > scared to buy that configuration > > there are h310s that are normal pci-e cards that go into dells lower end > servers. the ones in the r720s sit on the custom mezzanine connector. > > > > * H710/p have no way of exposing JBOD/syspd in any way .. If using > those > > > cards, the best I can do, is to create a raid-0 per disk, and put zfs > on > > > top of that. How bad is this? Is the only problem that I'm locked into > H710 > > > firmware? Would the nvram and bbu on H710 improve performance > significantly > > > vs H310? > > > > i wouldnt run zfs on that. > > > > > * H330 and H730, seem supported by mrsas driver. Is this driver able to > > > expose jbod/syspd mode? If yes, what's the overall advise, should I go > with > > > one of those, or the "trusty" H310 ? > > > > thats a workable option. h330 should be fine if all you want to do is > jbod. > > > > Now that you're pointing me to H330 being my best option given the > situation, got some more questions: > > * "if all you want is jbod" .. with zfs, that's all we need right ? just > wondering if I'm missing something important > > nope. > > > * Any idea if jbod configuration on h330 is done on bios level, or using > some freebsd tool (how?) > > you can do it in the bios. > > > * mrsas man page, mentions the driver appeared in 10.1 .. Does that mean > if I decide to use FreeNAS (9.x based) it won't be there ? > > i have no idea :) > > > * If I go the h330 path, it'll be a r730xd server. Worst case if the > h330 gives me trouble, can I throw it out and plug a LSI 9207-8i in its > place (same pci slot, cables ..etc) ? > > the h330 sits on a mezzanine connector. you probably can get that card and > drive the disks with it, but i havent done it with an r730 or 730xd so i > cant say for sure. i have done that with an r720 though, and it mostly > works. > > we had to order those systems with a h310 and add the lsi card ourselves, > and then we discovered that the disks they shipped in the system do not > spin up when theyre powered on. the raid controller (h310) explicitly spins > them up before issuing io to them. > > if you attach those disks to a straight sas hba or use them via syspd on > the h310, you cant boot off them because the machines bios (not the raid > controller) blindly issues io to them without spinning them up first. > scary stuff .. so just to be clear: - LSI SAS hba => disks DONT spin up - h310 in jbod mode => disks DONT spin up - h330 in jbod mode => disks work fine ? Note that currently dell's web configurator, only sells h330 with r730(xd) not r720(xd) .. The r720 comes with h310 or h710 afaict Note 2, Illumos people are able to pass a driver parameter to spin up crappy dell disks .. check http://www.listbox.com/member/archive/182179/2013/06/sort/from/page/4/entry/15:430/20130617155143:52F0F7EA-D787-11E2-84A9-87C6F7D228D8/ > > fun times. > > dlg > > > > > > > > > Thanks! > > > > > > > > > > Your help and advice is most appreciated > > > _______________________________________________ > > > freebsd-scsi@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-scsi > > > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org > " > > > > > > From owner-freebsd-scsi@FreeBSD.ORG Fri Jan 23 13:21:47 2015 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E42BE690 for ; Fri, 23 Jan 2015 13:21:47 +0000 (UTC) Received: from mail-pd0-f177.google.com (mail-pd0-f177.google.com [209.85.192.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B520F350 for ; Fri, 23 Jan 2015 13:21:47 +0000 (UTC) Received: by mail-pd0-f177.google.com with SMTP id y13so8523488pdi.8 for ; Fri, 23 Jan 2015 05:21:41 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=yzKQI1l59tAJ5WQWjv/l8fPnaTBLvY7pfCwOvYDbcE4=; b=hbHwsSbRs/mWApQd4axNKHeIvr1bmq+2uv7KZ6KWdywATTVeThj6N1ZyD3y13086Xc 0shTMn7qriRRQuoNq20IkMDItT0olDTkmw6CQE9WvplBKszJxln5rgag0kZ9uFsw9J6y waxtasP2dp+tipx41/m8HX32SqbTX7ftF/AOReegE2Urarzbyb9+keh8+Mo2nyJGYHY6 Ed6t6RL2k9zqw6u4XJDFqwtHh5uCXtzYqzWJqRLp6T/NRVulAvNNbMrJE8FZJAGZyk1Z toRRrznFHjZzxuwaO/XwI8lqZYSrdW66LePiSBaPkiwsEP3jQKEzWahtG9T4pFUVQkKh OMaA== X-Gm-Message-State: ALoCoQkSAEkTRgOcgnXJy5hQN6rSHvzMfySz6JWqaXZUkN2FntZVn5EgTTTuT09FftTR/80dlA2Y X-Received: by 10.70.61.197 with SMTP id s5mr11122429pdr.110.1422019301479; Fri, 23 Jan 2015 05:21:41 -0800 (PST) Received: from ?IPv6:2001:388:e000:ba00:86a:91f9:4155:c482? ([2001:388:e000:ba00:86a:91f9:4155:c482]) by mx.google.com with ESMTPSA id mz7sm1985026pdb.3.2015.01.23.05.21.38 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 23 Jan 2015 05:21:40 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\)) Subject: Re: Dell branded storage controller for ZFS file-server | Advice requested From: David Gwynne In-Reply-To: Date: Fri, 23 Jan 2015 23:21:32 +1000 Content-Transfer-Encoding: quoted-printable Message-Id: <5BA9C549-9159-4A85-8C37-C30D700C8E4A@gwynne.id.au> References: To: Ahmed Kamal X-Mailer: Apple Mail (2.1993) Cc: freebsd-scsi@freebsd.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jan 2015 13:21:48 -0000 > On 23 Jan 2015, at 11:02 pm, Ahmed Kamal = wrote: >=20 > scary stuff .. so just to be clear: >=20 > - LSI SAS hba =3D> disks DONT spin up > - h310 in jbod mode =3D> disks DONT spin up > - h330 in jbod mode =3D> disks work fine ? h330 doing jbod is a supported config from dell, so if it doesnt work = should be able to have a little tantrum and get them to fix it or = replace bits till it does work. >=20 > Note that currently dell's web configurator, only sells h330 with = r730(xd) not r720(xd) .. The r720 comes with h310 or h710 afaict yes. > Note 2, Illumos people are able to pass a driver parameter to spin up = crappy dell disks .. check=20 > = http://www.listbox.com/member/archive/182179/2013/06/sort/from/page/4/entr= y/15:430/20130617155143:52F0F7EA-D787-11E2-84A9-87C6F7D228D8/ you can patch your kernels as much as you want to, but you cant fix the = bios in the machine. if you want to boot off disks like that in a jbod = config you'll need ones that spin up when they get power. dlg= From owner-freebsd-scsi@FreeBSD.ORG Fri Jan 23 14:50:32 2015 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 99B82B9B for ; Fri, 23 Jan 2015 14:50:32 +0000 (UTC) Received: from mail-wi0-f177.google.com (mail-wi0-f177.google.com [209.85.212.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2CFCFEE9 for ; Fri, 23 Jan 2015 14:50:31 +0000 (UTC) Received: by mail-wi0-f177.google.com with SMTP id r20so3315172wiv.4 for ; Fri, 23 Jan 2015 06:50:30 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=HHy/dN+OXvfqybrkkMGWe615LtmJjkjAafGLWJOeCrI=; b=O6z2fmqoIrgeeOQW14N2mc1RRQRrYzepkOZ13cYyQhcIkd3ff+zyNM2KRiBxr0WcsE g4HVKuzvdHylV4mLf71t/RR5d5RDlTKdVSdoNXLd0b4RofYfQY6xy+p98HTLnKVHdDys CKjHO7jF+r3CrzLy+qKqTpdbEi5k14fNLCyBDgbd4KmIx150/OHMqmIzW5+llVZN8dkB DRVRY6xarWySA5Xm3WeW+OYK8MScn0lqHhNU3AsL27PKaSPOIsgxt6i+6ejfkJ9GQdlc Cj1NtgWb7LDgT7AckYxP4YVtqdf/gYF6AdLZQpZZsT/8wOC7g1O4xMUNT3V1mFQNbPPq rPjQ== X-Gm-Message-State: ALoCoQmyBsm06XE4aagpFoVe0UtH8RnHAe1AsExXlgkjLVOQNf03Yyvk91UDt+vAX12l+NsphnYn X-Received: by 10.180.37.197 with SMTP id a5mr4490799wik.2.1422024245534; Fri, 23 Jan 2015 06:44:05 -0800 (PST) Received: from [10.10.1.68] (82-69-141-170.dsl.in-addr.zen.co.uk. [82.69.141.170]) by mx.google.com with ESMTPSA id fo17sm2474664wjc.19.2015.01.23.06.44.04 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 23 Jan 2015 06:44:04 -0800 (PST) Message-ID: <54C25EA4.1070504@multiplay.co.uk> Date: Fri, 23 Jan 2015 14:45:56 +0000 From: Steven Hartland User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: freebsd-scsi@freebsd.org Subject: Re: Dell branded storage controller for ZFS file-server | Advice requested References: <5BA9C549-9159-4A85-8C37-C30D700C8E4A@gwynne.id.au> In-Reply-To: <5BA9C549-9159-4A85-8C37-C30D700C8E4A@gwynne.id.au> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jan 2015 14:50:32 -0000 I believe h330 is a LSI 3008 underneath with 256 QD which is classed my mrsas as Invader/Fury 12Gb/s MR controller so you should get decent performance from it, as with anything though only testing will answer the question categorically ;-) On 23/01/2015 13:21, David Gwynne wrote: >> On 23 Jan 2015, at 11:02 pm, Ahmed Kamal wrote: >> >> scary stuff .. so just to be clear: >> >> - LSI SAS hba => disks DONT spin up >> - h310 in jbod mode => disks DONT spin up >> - h330 in jbod mode => disks work fine ? > h330 doing jbod is a supported config from dell, so if it doesnt work should be able to have a little tantrum and get them to fix it or replace bits till it does work. > >> Note that currently dell's web configurator, only sells h330 with r730(xd) not r720(xd) .. The r720 comes with h310 or h710 afaict > yes. > >> Note 2, Illumos people are able to pass a driver parameter to spin up crappy dell disks .. check >> http://www.listbox.com/member/archive/182179/2013/06/sort/from/page/4/entry/15:430/20130617155143:52F0F7EA-D787-11E2-84A9-87C6F7D228D8/ > you can patch your kernels as much as you want to, but you cant fix the bios in the machine. if you want to boot off disks like that in a jbod config you'll need ones that spin up when they get power. > > dlg > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" From owner-freebsd-scsi@FreeBSD.ORG Fri Jan 23 16:16:28 2015 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 32F19B87 for ; Fri, 23 Jan 2015 16:16:28 +0000 (UTC) Received: from mail-we0-f177.google.com (mail-we0-f177.google.com [74.125.82.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B908AC18 for ; Fri, 23 Jan 2015 16:16:27 +0000 (UTC) Received: by mail-we0-f177.google.com with SMTP id l61so8432791wev.8 for ; Fri, 23 Jan 2015 08:16:20 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=cXI6ESxceLVWZL+1eOs37I5P+/JWMSn/GJPb9QOpiSw=; b=AS5xTW+EKUIrQP0TqA1LeBP1sWoOlI5mDMZ38dlA3L1rUcq8sMfG2Ty+14vILOF0+r RZbGPS2zH+RvhEgRB3u4ahPSm1lzDB9+wUS6msiYsG9cm9X3qArhMXLLfLuQG4IghZwJ qPAPEKlavtQG+IyBkz8M8gvHnMB94V3/MR7Rg/vKy79b+1Px8BT4bLlfz3ZpnrVlg4XN hPC+FHK/vrRMrXarNzZLUmRJ2aNcQjV+vuzpsDQW3gamCJAWAe53o4x6YXIyJIJNT80E 26VJLQeiSQFlkhXIBUZI+c5Ie/RScHo0NAzKk94ibPYvf8Q6V5FAEUpVhpAre5Q74SUG CQtQ== X-Gm-Message-State: ALoCoQmei43QM5CNBgesSNdsVTgrnOmPNKYvBn/ny7Y/zKZo3uGanftrGvj9+trpzowE7whrDf0P X-Received: by 10.194.2.240 with SMTP id 16mr15959178wjx.108.1422029780048; Fri, 23 Jan 2015 08:16:20 -0800 (PST) Received: from [10.10.1.68] (82-69-141-170.dsl.in-addr.zen.co.uk. [82.69.141.170]) by mx.google.com with ESMTPSA id cy8sm2427724wib.9.2015.01.23.08.16.18 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 23 Jan 2015 08:16:19 -0800 (PST) Message-ID: <54C27442.7040001@multiplay.co.uk> Date: Fri, 23 Jan 2015 16:18:10 +0000 From: Steven Hartland User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: kpneal@pobox.com Subject: Re: Dell branded storage controller for ZFS file-server | Advice requested References: <5BA9C549-9159-4A85-8C37-C30D700C8E4A@gwynne.id.au> <54C25EA4.1070504@multiplay.co.uk> <20150123151327.GA1526@neutralgood.org> In-Reply-To: <20150123151327.GA1526@neutralgood.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-scsi@freebsd.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jan 2015 16:16:28 -0000 On 23/01/2015 15:13, kpneal@pobox.com wrote: > On Fri, Jan 23, 2015 at 02:45:56PM +0000, Steven Hartland wrote: >> I believe h330 is a LSI 3008 underneath with 256 QD which is classed my >> mrsas as Invader/Fury 12Gb/s MR controller so you should get decent >> performance from it, as with anything though only testing will answer >> the question categorically ;-) > Just curious: Are jbod disks all bootable? Or does only one show up in the > bios' boot list? This is BIOS dependent, most provide support for a number of boot disks. > If only one, what happens if that drive mostly fails but still shows up > to the bios? How about if it totally fails and doesn't show up? How about > if the boot process starts but fails after control is passed to the boot > code loaded from disk? > > These questions are part of the reason I boot my Dells from the memory > card. > > Personally, I wish Dell would stop screwing around with the firmware used > with LSI chips. Generally we'll use the on-board controller to provide the boot disk pool in a simple mirror then configure a large RAIDZ for data, however that depends on how the physical disks are attached. This avoids the issue where the disk being chosen to boot from in the BIOS is not actually bootable.