From owner-freebsd-scsi@freebsd.org Mon Nov 16 09:09:52 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 03640A3096B; Mon, 16 Nov 2015 09:09:52 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from cu01176b.smtpx.saremail.com (cu01176b.smtpx.saremail.com [195.16.151.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AE4A01E05; Mon, 16 Nov 2015 09:09:51 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop01.sare.net (Postfix) with ESMTPSA id 422FC9DCA79; Mon, 16 Nov 2015 10:00:35 +0100 (CET) Subject: Re: LSI SAS2008 mps driver preferred firmware version Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: <20151114143104.GA41119@in-addr.com> Date: Mon, 16 Nov 2015 10:00:32 +0100 Cc: Kai Gallasch , freebsd-scsi@freebsd.org, Royce Williams , freebsd-stable Content-Transfer-Encoding: quoted-printable Message-Id: <7710CBCC-E68F-4454-9E29-E50ED1C6B511@sarenet.es> References: <5644FF09.9090200@free.de> <56472686.5030301@free.de> <20151114143104.GA41119@in-addr.com> To: Gary Palmer X-Mailer: Apple Mail (2.1283) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2015 09:09:52 -0000 On Nov 14, 2015, at 3:31 PM, Gary Palmer wrote: > You can do thinks in /boot/loader.conf to hard code bus and drive > assignments. =20 >=20 > e.g. >=20 > hint.da.0.at=3D"scbus0" > hint.da.0.target=3D"19" > hint.da.0.unit=3D"0" > hint.da.1.at=3D"scbus0" > hint.da.1.target=3D"18" > hint.da.1.unit=3D"0" Beware, the targer number assignment is not predictable. There's no = guarantee especially if you replace a disk. Borja. From owner-freebsd-scsi@freebsd.org Mon Nov 16 19:36:22 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6CCF8A3023A; Mon, 16 Nov 2015 19:36:22 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-ob0-x232.google.com (mail-ob0-x232.google.com [IPv6:2607:f8b0:4003:c01::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2FA87132A; Mon, 16 Nov 2015 19:36:22 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by obdgf3 with SMTP id gf3so130932275obd.3; Mon, 16 Nov 2015 11:36:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=NCqONzSCvI/Il6YjpEVTiDVTeyfq96vcXjBL6GdnQmc=; b=dIDYvnbT8HSTBfiA7HrnqBVXlupkND9CQYjlQaduPnQ8UOHYP5/VhkIIAxYMRnHTPl uEy7no/gIDK4MTT6UlvlmLOuXzSUnITc6TUUDzg9rQiqpv5nrcmQc9IcC6tEFGsmXsxa C0a0Jo2C57NparL4V/kdAl/3YwuoToCl8atlGlosrxTxI0+pXZNiPjmGu1JI4L5Gq5fk 2Y0vyV4n3Q4c/0giCpxbnKIDEKutHVpqCa/mvGCPJ7+qkZCKosUM5PBroBlnmmO94e7g weGkzUsAVWQREPJkaOHsh9/x9U44rIjshk8TBtw+xFSAtscbNTxxll1qiOKUtRRSJ2o5 DGiA== MIME-Version: 1.0 X-Received: by 10.182.111.196 with SMTP id ik4mr24217131obb.60.1447702581324; Mon, 16 Nov 2015 11:36:21 -0800 (PST) Sender: kob6558@gmail.com Received: by 10.202.98.131 with HTTP; Mon, 16 Nov 2015 11:36:21 -0800 (PST) In-Reply-To: <7710CBCC-E68F-4454-9E29-E50ED1C6B511@sarenet.es> References: <5644FF09.9090200@free.de> <56472686.5030301@free.de> <20151114143104.GA41119@in-addr.com> <7710CBCC-E68F-4454-9E29-E50ED1C6B511@sarenet.es> Date: Mon, 16 Nov 2015 11:36:21 -0800 X-Google-Sender-Auth: DnSvn7GLpkMPhfkWQsJBPxzeGgs Message-ID: Subject: Re: LSI SAS2008 mps driver preferred firmware version From: Kevin Oberman To: Borja Marcos Cc: Gary Palmer , freebsd-scsi@freebsd.org, Royce Williams , freebsd-stable , Kai Gallasch Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2015 19:36:22 -0000 On Mon, Nov 16, 2015 at 1:00 AM, Borja Marcos wrote: > > On Nov 14, 2015, at 3:31 PM, Gary Palmer wrote: > > > You can do thinks in /boot/loader.conf to hard code bus and drive > > assignments. > > > > e.g. > > > > hint.da.0.at="scbus0" > > hint.da.0.target="19" > > hint.da.0.unit="0" > > hint.da.1.at="scbus0" > > hint.da.1.target="18" > > hint.da.1.unit="0" > > Beware, the target number assignment is not predictable. There's no > guarantee especially if you replace > a disk. > > > > > > Borja. > As already mentioned, unless you are using zfs, use gpart to label you file systems/disks. Then use the /dev/gpt/LABEL as the mount device in fstab. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 From owner-freebsd-scsi@freebsd.org Mon Nov 16 19:40:14 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 025B3A3037E; Mon, 16 Nov 2015 19:40:14 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-ob0-x244.google.com (mail-ob0-x244.google.com [IPv6:2607:f8b0:4003:c01::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B9A5F1770; Mon, 16 Nov 2015 19:40:13 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by obbbj7 with SMTP id bj7so13219112obb.3; Mon, 16 Nov 2015 11:40:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=kdG2KizSVUPL+J7KLCanem6IYZHMCJCm25+YsXbqAcE=; b=VNjPofSo2aKJgaRvyW/SBDRctXFiqyBqh3XtusrjynkfOXK7pzC6Nw+xBnWHKofci4 APBO91bad4985i3smghf6fGbOF8Ek1uYAowzmEHrOU6MV2aBIiviGuAnxab8CxjQlr4v OO5ewcUyqfHPxkvPgHgtumUgwGz3xicjVJXnOU8tE8RzbxFPwDUIsHeFAMe7YL9zf+kP JfEQuZBX1SIYbBh9RDzCeg7c7WNgu8wYOstVCZAovhY4rvUuZx9IGPWt7Nm3g3M5U6Qv XJiS5qJbeiseqlOGUiW2hTGsWdsZ9QUdgIEQqm5u+EnPnRHDIHMrSvKUWnbfh6oto30r O3GA== MIME-Version: 1.0 X-Received: by 10.182.91.6 with SMTP id ca6mr2519502obb.24.1447702813038; Mon, 16 Nov 2015 11:40:13 -0800 (PST) Received: by 10.76.29.74 with HTTP; Mon, 16 Nov 2015 11:40:12 -0800 (PST) In-Reply-To: References: <5644FF09.9090200@free.de> <56472686.5030301@free.de> <20151114143104.GA41119@in-addr.com> <7710CBCC-E68F-4454-9E29-E50ED1C6B511@sarenet.es> Date: Mon, 16 Nov 2015 11:40:12 -0800 Message-ID: Subject: Re: LSI SAS2008 mps driver preferred firmware version From: Freddie Cash To: Kevin Oberman Cc: Borja Marcos , freebsd-scsi@freebsd.org, Royce Williams , freebsd-stable , Kai Gallasch Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2015 19:40:14 -0000 On Mon, Nov 16, 2015 at 11:36 AM, Kevin Oberman wrote= : > On Mon, Nov 16, 2015 at 1:00 AM, Borja Marcos wrote: > > > > > On Nov 14, 2015, at 3:31 PM, Gary Palmer wrote: > > > > > You can do thinks in /boot/loader.conf to hard code bus and drive > > > assignments. > > > > > > e.g. > > > > > > hint.da.0.at=3D"scbus0" > > > hint.da.0.target=3D"19" > > > hint.da.0.unit=3D"0" > > > hint.da.1.at=3D"scbus0" > > > hint.da.1.target=3D"18" > > > hint.da.1.unit=3D"0" > > > > Beware, the target number assignment is not predictable. There's no > > guarantee especially if you replace > > a disk. > > > > > > > > > > > > Borja. > > > > As already mentioned, unless you are using zfs, use gpart to label you fi= le > systems/disks. Then use the /dev/gpt/LABEL as the mount device in fstab. > =E2=80=8BEven if you are using ZFS, labelling the drives with the location = of the disk in the system (enclosure, column, row, whatever) makes things so much easier to work with when there are disk-related issues. Just create a single partition that covers the whole disk, label it, and use the label to create the vdevs in the pool.=E2=80=8B --=20 Freddie Cash fjwcash@gmail.com From owner-freebsd-scsi@freebsd.org Mon Nov 16 20:57:41 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B8A8EA30828; Mon, 16 Nov 2015 20:57:41 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 703B91812; Mon, 16 Nov 2015 20:57:41 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1ZyQqE-000Obi-ST; Mon, 16 Nov 2015 23:57:34 +0300 Date: Mon, 16 Nov 2015 23:57:34 +0300 From: Slawa Olhovchenkov To: Freddie Cash Cc: Kevin Oberman , freebsd-scsi@freebsd.org, Royce Williams , freebsd-stable , Borja Marcos , Kai Gallasch Subject: Re: LSI SAS2008 mps driver preferred firmware version Message-ID: <20151116205734.GM48728@zxy.spb.ru> References: <5644FF09.9090200@free.de> <56472686.5030301@free.de> <20151114143104.GA41119@in-addr.com> <7710CBCC-E68F-4454-9E29-E50ED1C6B511@sarenet.es> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2015 20:57:41 -0000 On Mon, Nov 16, 2015 at 11:40:12AM -0800, Freddie Cash wrote: > On Mon, Nov 16, 2015 at 11:36 AM, Kevin Oberman wrote: > > > On Mon, Nov 16, 2015 at 1:00 AM, Borja Marcos wrote: > > > > > > > > On Nov 14, 2015, at 3:31 PM, Gary Palmer wrote: > > > > > > > You can do thinks in /boot/loader.conf to hard code bus and drive > > > > assignments. > > > > > > > > e.g. > > > > > > > > hint.da.0.at="scbus0" > > > > hint.da.0.target="19" > > > > hint.da.0.unit="0" > > > > hint.da.1.at="scbus0" > > > > hint.da.1.target="18" > > > > hint.da.1.unit="0" > > > > > > Beware, the target number assignment is not predictable. There's no > > > guarantee especially if you replace > > > a disk. > > > > > > > > > > > > > > > > > > Borja. > > > > > > > As already mentioned, unless you are using zfs, use gpart to label you file > > systems/disks. Then use the /dev/gpt/LABEL as the mount device in fstab. > > > > ​Even if you are using ZFS, labelling the drives with the location of the > disk in the system (enclosure, column, row, whatever) makes things so much > easier to work with when there are disk-related issues. > > Just create a single partition that covers the whole disk, label it, and > use the label to create the vdevs in the pool.​ Bad idea. Re-placed disk in different bay don't relabel automaticly. Other issuse where disk placed in bay some remotely hands in data center -- I am relay don't know how disk distributed by bays. Best way for identify disk -- uses enclouse services. I have many sites with ZFS on whole disk and some sites with ZFS on GPT partition. ZFS on GPT more heavy for administration. From owner-freebsd-scsi@freebsd.org Mon Nov 16 21:19:57 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5A7F4A30EA9; Mon, 16 Nov 2015 21:19:57 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-ob0-x235.google.com (mail-ob0-x235.google.com [IPv6:2607:f8b0:4003:c01::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 10595189B; Mon, 16 Nov 2015 21:19:57 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by obdgf3 with SMTP id gf3so133127158obd.3; Mon, 16 Nov 2015 13:19:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=DbhmXduNBQvMEXo+tgkVysvcxT4EF9WC4VYxdBB5FYk=; b=YzFMQCxcuVNE6leuw7bTbw2Coni/5NaeRJTbc82XHjv48I6KLwiuzT3jnT6TspMMTG Xn8rFqSQghYDN4onDzQ58/GN69w/5fpN7aFpdedIdN1F7hIzUchVufmE6DEEyX4W6IcB cnvqqQglaiA2CexMzl/9Z6Jj0SuPlAcQQHELrYALN4c0Rs6Uj2lST48VUm7gWR6TM+Ag xT2EWjDoGs9yzwOdx2X+lmvctRCGw6iV5SKFx/CGR8zi5GLj8qJ+GE67OFrDqkRs5Hvg DtqMCvtio1cjaYpWpIBgh0xcVgBxwzXZ/5HPBvrMpOn6uN3e89biO8jX0kT9c+Ph1SPR JVFQ== MIME-Version: 1.0 X-Received: by 10.60.76.42 with SMTP id h10mr22713812oew.13.1447708796286; Mon, 16 Nov 2015 13:19:56 -0800 (PST) Received: by 10.76.29.74 with HTTP; Mon, 16 Nov 2015 13:19:55 -0800 (PST) In-Reply-To: <20151116205734.GM48728@zxy.spb.ru> References: <5644FF09.9090200@free.de> <56472686.5030301@free.de> <20151114143104.GA41119@in-addr.com> <7710CBCC-E68F-4454-9E29-E50ED1C6B511@sarenet.es> <20151116205734.GM48728@zxy.spb.ru> Date: Mon, 16 Nov 2015 13:19:55 -0800 Message-ID: Subject: Re: LSI SAS2008 mps driver preferred firmware version From: Freddie Cash To: Slawa Olhovchenkov Cc: Kevin Oberman , freebsd-scsi@freebsd.org, Royce Williams , freebsd-stable , Borja Marcos , Kai Gallasch Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2015 21:19:57 -0000 On Mon, Nov 16, 2015 at 12:57 PM, Slawa Olhovchenkov wrote= : > On Mon, Nov 16, 2015 at 11:40:12AM -0800, Freddie Cash wrote: > > > On Mon, Nov 16, 2015 at 11:36 AM, Kevin Oberman > wrote: > > > As already mentioned, unless you are using zfs, use gpart to label yo= u > file > > > systems/disks. Then use the /dev/gpt/LABEL as the mount device in > fstab. > > > > > > > =E2=80=8BEven if you are using ZFS, labelling the drives with the locat= ion of the > > disk in the system (enclosure, column, row, whatever) makes things so > much > > easier to work with when there are disk-related issues. > > > > Just create a single partition that covers the whole disk, label it, an= d > > use the label to create the vdevs in the pool.=E2=80=8B > > Bad idea. > Re-placed disk in different bay don't relabel automaticly. > =E2=80=8BDid the original disk get labelled automatically? No, you had to = do that when you first started using it. So, why would you expect a replaced disk to get labelled automatically? Offline the dead/dying disk. Physically remove the disk. Insert the new disk. Partition / label the new disk. "zfs replace" using the new label to get it into the pool.=E2=80=8B > Other issuse where disk placed in bay some remotely hands in data > center -- I am relay don't know how disk distributed by bays. > =E2=80=8BYou label the disks as they are added to the system the first time= . That way, you always know where each disk is located, and you only deal with the labels. Then, when you need to replace a disk (or ask someone in a remote location to replace it) it's a simple matter: the label on the disk itself tells you where the disk is physically located. And it doesn't change if the controller decides to change the direction it enumerates devices. Which is easier to tell someone in a remote location: Replace disk enc0a6 (meaning enclosure 0, column A, row 6)? or Replace the disk called da36?=E2=80=8B =E2=80=8Bor Find the disk with serial number XXXXXXXX? or Replace the disk where the light is (hopefully) flashing (but I can't tell you which enclosure, front or back, or anything else like that)? The first one lets you know exactly where the disk is located physically. The second one just tells you the name of the device as determined by the OS, but doesn't tell you anything about where it is located. And it can change with a kernel update, driver update, or firmware update! The third requires you to pull every disk in turn to read the serial number off the drive itself. In order for the second or third option to work, you'd have to write down the device names and/or serial numbers and stick that onto the drive bay itself.=E2=80=8B > Best way for identify disk -- uses enclouse services. > =E2=80=8BOnly if your enclosure services are actually working (or even enab= led). I've yet to work on a box where that actually works (we custom-build our storage boxes using OTS hardware). Best way, IMO, is to use the physical location of the device as the actual device name itself. That way, there's never any ambiguity at the physical layer, the driver layer, the OS layer, or the ZFS pool layer.=E2=80=8B > I have many sites with ZFS on whole disk and some sites with ZFS on > GPT partition. ZFS on GPT more heavy for administration. > =E2=80=8BIt's 1 extra step: partition the drive, supplying the location of= the drive as the label for the partition. Everything else works exactly the same. I used to do everything with whole drives and no labels. Did that for about a month, until 2 separate drives on separate controllers died (in a 24-bay setup) and I couldn't figure out where they were located as a BIOS upgrade changed which controller loaded first. And then I had to work on a server that someone else configured with direct-attach bays (24 cables) that were connected almost at random. Then I used glabel(8) to label the entire disk, and things were much better. But that didn't always play well with 4K drives, and replacing drives that were the same size didn't always work as the number of sectors in each disk was different (ZFS plays better with this now). Then I started to GPT partition things, and life has been so much simpler. All the partitions are aligned to 1 MB, and I can manually set the size of the partition to work around different physical sector counts. All the partitions are labelled using the physical location of the disk (originally just row/column naming like a spreadsheet, but now I'm adding enclosure name as well as we expand to multiple enclosures per system). It's so much simpler now, ESPECIALLY when I have to get someone to do something remotely. :) =E2=80=8BEveryone has their own way to manage things. I just haven't seen = any better setup than labelling the drives themselves using their physical location.=E2=80=8B --=20 Freddie Cash fjwcash@gmail.com From owner-freebsd-scsi@freebsd.org Tue Nov 17 08:08:08 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5A0F8A319F4; Tue, 17 Nov 2015 08:08:08 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from kagate.punkt.de (kagate.punkt.de [217.29.33.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C6CAA1051; Tue, 17 Nov 2015 08:08:07 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from hugo10.ka.punkt.de (hugo10.ka.punkt.de [217.29.44.10]) by gate2.intern.punkt.de with ESMTP id tAH87v6K076985; Tue, 17 Nov 2015 09:07:57 +0100 (CET) Received: from [217.29.44.157] ([217.29.44.157]) by hugo10.ka.punkt.de (8.14.2/8.14.2) with ESMTP id tAH87vtO031155; Tue, 17 Nov 2015 09:07:57 +0100 (CET) (envelope-from hausen@punkt.de) Subject: ZFS on labelled partitions (was: Re: LSI SAS2008 mps driver preferred firmware version) Mime-Version: 1.0 (Mac OS X Mail 9.1 \(3096.5\)) Content-Type: multipart/signed; boundary="Apple-Mail=_1B2B703B-9A6A-4088-B7E0-8CC3E339EA61"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.6b2 From: "Patrick M. Hausen" In-Reply-To: Date: Tue, 17 Nov 2015 09:08:12 +0100 Cc: freebsd-stable , freebsd-scsi@freebsd.org Message-Id: <878D84A9-4553-413B-ACA4-5ABF499F28C1@punkt.de> References: <5644FF09.9090200@free.de> <56472686.5030301@free.de> <20151114143104.GA41119@in-addr.com> <7710CBCC-E68F-4454-9E29-E50ED1C6B511@sarenet.es> <20151116205734.GM48728@zxy.spb.ru> To: Freddie Cash X-Mailer: Apple Mail (2.3096.5) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2015 08:08:08 -0000 --Apple-Mail=_1B2B703B-9A6A-4088-B7E0-8CC3E339EA61 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi, all, > Am 16.11.2015 um 22:19 schrieb Freddie Cash : >=20 > =E2=80=8BYou label the disks as they are added to the system the first = time. That > way, you always know where each disk is located, and you only deal = with the > labels. we do the same for obvious reasons. But I always wonder about the = possible downsides, because ZFS documentation explicitly states: ZFS operates on raw devices, so it is possible to create a = storage pool comprised of logical volumes, either software or hardware. This configuration is not = recommended, as ZFS works best when it uses raw physical devices. Using logical volumes = might sacrifice performance, reliability, or both, and should be avoided. (from http://docs.oracle.com/cd/E19253-01/819-5461/gbcik/index.html) Can anyone shed some lght on why not using raw devices might sacrifice performance or reliability? Or is this just outdated folklore? Thanks, Patrick -- punkt.de GmbH * Kaiserallee 13a * 76133 Karlsruhe Tel. 0721 9109 0 * Fax 0721 9109 100 info@punkt.de http://www.punkt.de Gf: J=C3=BCrgen Egeling AG Mannheim 108285 --Apple-Mail=_1B2B703B-9A6A-4088-B7E0-8CC3E339EA61 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJWSuBsAAoJEJBvLuLt2olcScMH/1VvysKjTcZqVOIltP1FyZWn LzaS3uGKVPNJKYKmL6fl0oluynUhFdupvPqLT8qI5ViMe6lpA6ay9dCZ2uhdcOhS mu2qT8orRTSNWr4DZmpuH0lxulYypEcBGN0oxwokcTuv3imVE1iMSBFnUoNWS1NT cf/jBREsixk3YFr2Az7cKG80Jcnp/T2sPrTMQV1UWvkz4qe1JEUCU5VflUhPEH56 CJpp5U1647nrlvqmsv5XZgENtA3QGN2WrhFei56C1wGRkrmqDGy81WQzQAaVsPbD t6JgaOJaFKB+m4S1T3Ar0Qf96rp6/mm+/f96fCaurzK4oBuHSuuaJPs5cHUPke4= =MTIy -----END PGP SIGNATURE----- --Apple-Mail=_1B2B703B-9A6A-4088-B7E0-8CC3E339EA61-- From owner-freebsd-scsi@freebsd.org Tue Nov 17 08:23:07 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6876AA31DC0; Tue, 17 Nov 2015 08:23:07 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 247DA1A2D; Tue, 17 Nov 2015 08:23:06 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id DBF7A28431; Tue, 17 Nov 2015 09:22:57 +0100 (CET) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id C13682842B; Tue, 17 Nov 2015 09:22:56 +0100 (CET) Message-ID: <564AE3E0.9060209@quip.cz> Date: Tue, 17 Nov 2015 09:22:56 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:35.0) Gecko/20100101 Firefox/35.0 SeaMonkey/2.32 MIME-Version: 1.0 To: "Patrick M. Hausen" , Freddie Cash CC: freebsd-scsi@freebsd.org, freebsd-stable Subject: Re: ZFS on labelled partitions References: <5644FF09.9090200@free.de> <56472686.5030301@free.de> <20151114143104.GA41119@in-addr.com> <7710CBCC-E68F-4454-9E29-E50ED1C6B511@sarenet.es> <20151116205734.GM48728@zxy.spb.ru> <878D84A9-4553-413B-ACA4-5ABF499F28C1@punkt.de> In-Reply-To: <878D84A9-4553-413B-ACA4-5ABF499F28C1@punkt.de> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2015 08:23:07 -0000 Patrick M. Hausen wrote on 11/17/2015 09:08: > Hi, all, > >> Am 16.11.2015 um 22:19 schrieb Freddie Cash : >> >> ​You label the disks as they are added to the system the first time. That >> way, you always know where each disk is located, and you only deal with the >> labels. > > we do the same for obvious reasons. But I always wonder about the possible > downsides, because ZFS documentation explicitly states: > > ZFS operates on raw devices, so it is possible to create a storage pool comprised of logical > volumes, either software or hardware. This configuration is not recommended, as ZFS works > best when it uses raw physical devices. Using logical volumes might sacrifice performance, > reliability, or both, and should be avoided. > > (from http://docs.oracle.com/cd/E19253-01/819-5461/gbcik/index.html) > > Can anyone shed some lght on why not using raw devices might sacrifice > performance or reliability? Or is this just outdated folklore? It was on Solaris but not on FreeBSD. If you were using partitions on Solaris the drive cache was disabled (or something like that, I am not 100% sure) Miroslav Lachman From owner-freebsd-scsi@freebsd.org Tue Nov 17 08:23:09 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B9AE0A31DDF; Tue, 17 Nov 2015 08:23:09 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: from mail-wm0-x22d.google.com (mail-wm0-x22d.google.com [IPv6:2a00:1450:400c:c09::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 473261A44; Tue, 17 Nov 2015 08:23:09 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: by wmww144 with SMTP id w144so14247000wmw.0; Tue, 17 Nov 2015 00:23:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=gDedhX817bOHm9pQoLUg0E3hYQEtgvcFTGwPzmhQOyA=; b=yCl0BKOE8rNrACU7UxTzWcIvXvC0Xi/C3jso+rhQJiAUV2XIHFaqd3UkuzcnCydSQS udUS9V86QVUDJXRkdWHJy2ecE1UKOm4UoG+tVix5ylT9Wn9/QGFi7xuQLTqPFn8yHk0U u3bS1UlvbJa9nnOjpjXag/jqDqAvcEauZ0eIutQS7FDY6kkJIxE/LUjBCmJ4ROpqR0LW k1CCwmdekUXYrBCEYSdyY4Fbosms0kmzrtRchhvOPy/7VdOD6Wx8xV/TVF6pFBFPTCY1 xdlZ0QOloHmP+pFeSYWQvuca5923NcHmKDJQpLShpoJGkUM5XFbulrb5wZxYPLo52GZd wjag== MIME-Version: 1.0 X-Received: by 10.194.239.104 with SMTP id vr8mr40013392wjc.64.1447748586845; Tue, 17 Nov 2015 00:23:06 -0800 (PST) Received: by 10.28.181.213 with HTTP; Tue, 17 Nov 2015 00:23:06 -0800 (PST) In-Reply-To: <878D84A9-4553-413B-ACA4-5ABF499F28C1@punkt.de> References: <5644FF09.9090200@free.de> <56472686.5030301@free.de> <20151114143104.GA41119@in-addr.com> <7710CBCC-E68F-4454-9E29-E50ED1C6B511@sarenet.es> <20151116205734.GM48728@zxy.spb.ru> <878D84A9-4553-413B-ACA4-5ABF499F28C1@punkt.de> Date: Tue, 17 Nov 2015 08:23:06 +0000 Message-ID: Subject: Re: ZFS on labelled partitions (was: Re: LSI SAS2008 mps driver preferred firmware version) From: krad To: "Patrick M. Hausen" Cc: Freddie Cash , freebsd-scsi@freebsd.org, freebsd-stable Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2015 08:23:09 -0000 >From what i remember its a control thing. If you have another layer below zfs, be it software based or hardware based, zfs cant be sure what is going on, therefore cant guarantee anything. This is quite a big thing when it comes to data integrity which is a big reason to use zfs. I remember having to be very careful with some external caching arrays and making sure that they flushed correctly as often they ignore the scsi flush commands. This is one reason why I would always use the IT based firmware rather then the RAID one, as its less likely to lead to issues. On 17 November 2015 at 08:08, Patrick M. Hausen wrote: > Hi, all, > > > Am 16.11.2015 um 22:19 schrieb Freddie Cash : > > > > =E2=80=8BYou label the disks as they are added to the system the first = time. > That > > way, you always know where each disk is located, and you only deal with > the > > labels. > > we do the same for obvious reasons. But I always wonder about the possibl= e > downsides, because ZFS documentation explicitly states: > > ZFS operates on raw devices, so it is possible to create a storag= e > pool comprised of logical > volumes, either software or hardware. This configuration is not > recommended, as ZFS works > best when it uses raw physical devices. Using logical volumes > might sacrifice performance, > reliability, or both, and should be avoided. > > (from http://docs.oracle.com/cd/E19253-01/819-5461/gbcik/index.html) > > Can anyone shed some lght on why not using raw devices might sacrifice > performance or reliability? Or is this just outdated folklore? > > Thanks, > Patrick > -- > punkt.de GmbH * Kaiserallee 13a * 76133 Karlsruhe > Tel. 0721 9109 0 * Fax 0721 9109 100 > info@punkt.de http://www.punkt.de > Gf: J=C3=BCrgen Egeling AG Mannheim 108285 > > From owner-freebsd-scsi@freebsd.org Tue Nov 17 08:32:36 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ECE5DA2E0F4; Tue, 17 Nov 2015 08:32:35 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: from mail-wm0-x231.google.com (mail-wm0-x231.google.com [IPv6:2a00:1450:400c:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 81F271061; Tue, 17 Nov 2015 08:32:35 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: by wmvv187 with SMTP id v187so214865460wmv.1; Tue, 17 Nov 2015 00:32:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=js8gPnZ617uyojxHsa6RMFJvhn/HmboCYLYIvFMcRew=; b=EWzuCFqaoIVxWIOW9WQ/MnPyMeSg9HUFedBjygjgyJo8CaJZVQW+5W/XOoGgO5vOs4 cVv9cw1jtsSfsMdMdK4HT5+9mCIuD+ajn3Ez3OvH3MvWeHtuNLFZc1fl54lIRnMF40uL WL/VuoEIyirmTYP+V5618viV5vgEYK/zDB9ncTqT+F1qJUxhIFKuuahnFgsnTXqc83/4 wxtwTm47RQQdURMatb24zumkQGVGKiy9+RcdqIoY5FJAPuehyFPcZCc/1l2MUrcMLPx6 YkH+q9p/yG77FSeoMdayqefvGDKB21a6SpRw9NLetqF61jC81uFS1triw8QMJDtPt73V t6mQ== MIME-Version: 1.0 X-Received: by 10.194.185.173 with SMTP id fd13mr41841837wjc.54.1447749153937; Tue, 17 Nov 2015 00:32:33 -0800 (PST) Received: by 10.28.181.213 with HTTP; Tue, 17 Nov 2015 00:32:33 -0800 (PST) In-Reply-To: <564AE3E0.9060209@quip.cz> References: <5644FF09.9090200@free.de> <56472686.5030301@free.de> <20151114143104.GA41119@in-addr.com> <7710CBCC-E68F-4454-9E29-E50ED1C6B511@sarenet.es> <20151116205734.GM48728@zxy.spb.ru> <878D84A9-4553-413B-ACA4-5ABF499F28C1@punkt.de> <564AE3E0.9060209@quip.cz> Date: Tue, 17 Nov 2015 08:32:33 +0000 Message-ID: Subject: Re: ZFS on labelled partitions From: krad To: Miroslav Lachman <000.fbsd@quip.cz> Cc: "Patrick M. Hausen" , Freddie Cash , freebsd-scsi@freebsd.org, freebsd-stable Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2015 08:32:36 -0000 It was a control thing again, if you were using a partition another application could be using the drive on another partition, therefore zfs couldn't guarantee exclusive use of the disk so had to be more careful in the way it operated the drive. I think this meant I went into write through mode like you say. On 17 November 2015 at 08:22, Miroslav Lachman <000.fbsd@quip.cz> wrote: > Patrick M. Hausen wrote on 11/17/2015 09:08: > >> Hi, all, >> >> Am 16.11.2015 um 22:19 schrieb Freddie Cash : >>> >>> =E2=80=8BYou label the disks as they are added to the system the first = time. >>> That >>> way, you always know where each disk is located, and you only deal with >>> the >>> labels. >>> >> >> we do the same for obvious reasons. But I always wonder about the possib= le >> downsides, because ZFS documentation explicitly states: >> >> ZFS operates on raw devices, so it is possible to create a >> storage pool comprised of logical >> volumes, either software or hardware. This configuration is not >> recommended, as ZFS works >> best when it uses raw physical devices. Using logical volumes >> might sacrifice performance, >> reliability, or both, and should be avoided. >> >> (from http://docs.oracle.com/cd/E19253-01/819-5461/gbcik/index.html) >> >> Can anyone shed some lght on why not using raw devices might sacrifice >> performance or reliability? Or is this just outdated folklore? >> > > It was on Solaris but not on FreeBSD. If you were using partitions on > Solaris the drive cache was disabled (or something like that, I am not 10= 0% > sure) > > Miroslav Lachman > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-scsi@freebsd.org Tue Nov 17 08:37:16 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 952C1A2E2B1; Tue, 17 Nov 2015 08:37:16 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: from mail-wm0-x242.google.com (mail-wm0-x242.google.com [IPv6:2a00:1450:400c:c09::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 23AA5124E; Tue, 17 Nov 2015 08:37:16 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: by wmec201 with SMTP id c201so2973252wme.3; Tue, 17 Nov 2015 00:37:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=UPVJFo7Ljh4RXtH263iyf1LDRVM3hGJbihvLXbBwdwo=; b=dyPWJB0EUUPdg9+ZK2MZ9fzJrVmgKLIhxTiD9yrI9UViFNrVq22gJoT6Hj3EEFg39h URIURBDKUQ27kpEql59bghXQXzkhcWolFfHOMyBpxYAk7tXGcASgKhtQRx/+5//oPFJI 7WU2vJX1q1tmnj9tk81E7fXUsu5IvN3NSeg3+FRzK16z48v1AUrQTa+xLD2HdrL9mc0V dQ4Z8xZnfbMo3CWiz1Xbm3KzMRTwJaQGjSQEfg+FtpYKAu0ZK9kfpyjogwwQSpZcNBjc B0uVwuaJT7cahEw2fweOhsN4dI5P9wdo94N9zQBwMeDVodjHDW1HLAVwWvK5tNOCwo2l oojg== MIME-Version: 1.0 X-Received: by 10.194.134.3 with SMTP id pg3mr46399390wjb.63.1447749434500; Tue, 17 Nov 2015 00:37:14 -0800 (PST) Received: by 10.28.181.213 with HTTP; Tue, 17 Nov 2015 00:37:14 -0800 (PST) In-Reply-To: References: <5644FF09.9090200@free.de> <56472686.5030301@free.de> <20151114143104.GA41119@in-addr.com> <7710CBCC-E68F-4454-9E29-E50ED1C6B511@sarenet.es> <20151116205734.GM48728@zxy.spb.ru> Date: Tue, 17 Nov 2015 08:37:14 +0000 Message-ID: Subject: Re: LSI SAS2008 mps driver preferred firmware version From: krad To: Freddie Cash Cc: Slawa Olhovchenkov , Royce Williams , freebsd-stable , Borja Marcos , Kai Gallasch , Kevin Oberman , freebsd-scsi@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2015 08:37:16 -0000 I disagree, get the remote hands to copy the serial number to an easily visible location on the drive when its in the enclosure. Then label the drives with the serial number (or a compatible version of it). That way the label is tied to the drive, and you dont have to rely on the remote hands 100%. Better still do the physical labelling yourself On 16 November 2015 at 21:19, Freddie Cash wrote: > On Mon, Nov 16, 2015 at 12:57 PM, Slawa Olhovchenkov > wrote: > > > On Mon, Nov 16, 2015 at 11:40:12AM -0800, Freddie Cash wrote: > > > > > On Mon, Nov 16, 2015 at 11:36 AM, Kevin Oberman > > wrote: > > > > As already mentioned, unless you are using zfs, use gpart to label > you > > file > > > > systems/disks. Then use the /dev/gpt/LABEL as the mount device in > > fstab. > > > > > > > > > > =E2=80=8BEven if you are using ZFS, labelling the drives with the loc= ation of > the > > > disk in the system (enclosure, column, row, whatever) makes things so > > much > > > easier to work with when there are disk-related issues. > > > > > > Just create a single partition that covers the whole disk, label it, > and > > > use the label to create the vdevs in the pool.=E2=80=8B > > > > Bad idea. > > Re-placed disk in different bay don't relabel automaticly. > > > > =E2=80=8BDid the original disk get labelled automatically? No, you had t= o do that > when you first started using it. So, why would you expect a replaced dis= k > to get labelled automatically? > > Offline the dead/dying disk. > Physically remove the disk. > Insert the new disk. > Partition / label the new disk. > "zfs replace" using the new label to get it into the pool.=E2=80=8B > > > > Other issuse where disk placed in bay some remotely hands in data > > center -- I am relay don't know how disk distributed by bays. > > > > =E2=80=8BYou label the disks as they are added to the system the first ti= me. That > way, you always know where each disk is located, and you only deal with t= he > labels. > > Then, when you need to replace a disk (or ask someone in a remote locatio= n > to replace it) it's a simple matter: the label on the disk itself tells > you where the disk is physically located. And it doesn't change if the > controller decides to change the direction it enumerates devices. > > Which is easier to tell someone in a remote location: > Replace disk enc0a6 (meaning enclosure 0, column A, row 6)? > or > Replace the disk called da36?=E2=80=8B > =E2=80=8Bor > Find the disk with serial number XXXXXXXX? > or > Replace the disk where the light is (hopefully) flashing (but I can't > tell you which enclosure, front or back, or anything else like that)? > > The first one lets you know exactly where the disk is located physically. > > The second one just tells you the name of the device as determined by the > OS, but doesn't tell you anything about where it is located. And it can > change with a kernel update, driver update, or firmware update! > > The third requires you to pull every disk in turn to read the serial numb= er > off the drive itself. > > In order for the second or third option to work, you'd have to write down > the device names and/or serial numbers and stick that onto the drive bay > itself.=E2=80=8B > > > > Best way for identify disk -- uses enclouse services. > > > > =E2=80=8BOnly if your enclosure services are actually working (or even en= abled). > I've yet to work on a box where that actually works (we custom-build our > storage boxes using OTS hardware). > > Best way, IMO, is to use the physical location of the device as the actua= l > device name itself. That way, there's never any ambiguity at the physica= l > layer, the driver layer, the OS layer, or the ZFS pool layer.=E2=80=8B > > > > I have many sites with ZFS on whole disk and some sites with ZFS on > > GPT partition. ZFS on GPT more heavy for administration. > > > > =E2=80=8BIt's 1 extra step: partition the drive, supplying the location = of the > drive as the label for the partition. > > Everything else works exactly the same. > > I used to do everything with whole drives and no labels. Did that for > about a month, until 2 separate drives on separate controllers died (in a > 24-bay setup) and I couldn't figure out where they were located as a BIOS > upgrade changed which controller loaded first. And then I had to work on= a > server that someone else configured with direct-attach bays (24 cables) > that were connected almost at random. > > Then I used glabel(8) to label the entire disk, and things were much > better. But that didn't always play well with 4K drives, and replacing > drives that were the same size didn't always work as the number of sector= s > in each disk was different (ZFS plays better with this now). > > Then I started to GPT partition things, and life has been so much simpler= . > All the partitions are aligned to 1 MB, and I can manually set the size o= f > the partition to work around different physical sector counts. All the > partitions are labelled using the physical location of the disk (original= ly > just row/column naming like a spreadsheet, but now I'm adding enclosure > name as well as we expand to multiple enclosures per system). It's so mu= ch > simpler now, ESPECIALLY when I have to get someone to do something > remotely. :) > > =E2=80=8BEveryone has their own way to manage things. I just haven't see= n any > better setup than labelling the drives themselves using their physical > location.=E2=80=8B > > -- > Freddie Cash > fjwcash@gmail.com > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-scsi@freebsd.org Tue Nov 17 16:07:49 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F1FCDA31067; Tue, 17 Nov 2015 16:07:48 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-oi0-x232.google.com (mail-oi0-x232.google.com [IPv6:2607:f8b0:4003:c06::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B23F41BE1; Tue, 17 Nov 2015 16:07:48 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by oige206 with SMTP id e206so7523395oig.2; Tue, 17 Nov 2015 08:07:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=jGimzqf72d8yTZww+w/pVpMX50a7lefVSCwrOX4Vv8Y=; b=WsU42+dyVzDatxadD1Cin2n+8iwHV75oAlezbagLLloRNqvjegSxiFzw07EbynV+Cg 3/5wBBgz8QeMcYqZ8Bzsdjg9RF2UYXxhuxO31+iVvxHYlza/6QjFYtWGp7BNoUqUNpDW PjgAFB0n5v6a3S4SLHsLB/u+2e6H4LSo9IybKfRQoufHmOvF7erv8Z2ZhdXzgbEppYfj CUAvtaj2veZ9TZGGn4llNKsZw4fKYN5jHXJVmNMiBxQnSD/fEGbQnr0b9Q6YwjukCP6E jTOuuYXL83Be/d3Nb9riq8upOfYpSSzwt4QNCTiiK2KDELpNUrhKWWG6S4nijuHuTKkU Osnw== MIME-Version: 1.0 X-Received: by 10.202.207.12 with SMTP id f12mr24682300oig.101.1447776467368; Tue, 17 Nov 2015 08:07:47 -0800 (PST) Received: by 10.76.29.74 with HTTP; Tue, 17 Nov 2015 08:07:47 -0800 (PST) In-Reply-To: <878D84A9-4553-413B-ACA4-5ABF499F28C1@punkt.de> References: <5644FF09.9090200@free.de> <56472686.5030301@free.de> <20151114143104.GA41119@in-addr.com> <7710CBCC-E68F-4454-9E29-E50ED1C6B511@sarenet.es> <20151116205734.GM48728@zxy.spb.ru> <878D84A9-4553-413B-ACA4-5ABF499F28C1@punkt.de> Date: Tue, 17 Nov 2015 08:07:47 -0800 Message-ID: Subject: Re: ZFS on labelled partitions (was: Re: LSI SAS2008 mps driver preferred firmware version) From: Freddie Cash To: "Patrick M. Hausen" Cc: freebsd-stable , freebsd-scsi@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2015 16:07:49 -0000 On Tue, Nov 17, 2015 at 12:08 AM, Patrick M. Hausen wrote= : > Hi, all, > > > Am 16.11.2015 um 22:19 schrieb Freddie Cash : > > > > =E2=80=8BYou label the disks as they are added to the system the first = time. > That > > way, you always know where each disk is located, and you only deal with > the > > labels. > > we do the same for obvious reasons. But I always wonder about the possibl= e > downsides, because ZFS documentation explicitly states: > > ZFS operates on raw devices, so it is possible to create a storag= e > pool comprised of logical > volumes, either software or hardware. This configuration is not > recommended, as ZFS works > best when it uses raw physical devices. Using logical volumes > might sacrifice performance, > reliability, or both, and should be avoided. > > (from http://docs.oracle.com/cd/E19253-01/819-5461/gbcik/index.html) > > Can anyone shed some lght on why not using raw devices might sacrifice > performance or reliability? Or is this just outdated folklore? > =E2=80=8BOn Solaris, using raw devices allows ZFS to enable the caches on t= he disks themselves, while using any kind of partitioning on the disk forces the caches to be disabled. This is not an issue on FreeBSD due to the way GEOM works. Caches on disks are enabled regardless of how the disk is accessed (raw, dd-partitioned, MBR-partitioned, GPT-partitioned, gnop, geli, whatever). This is a common misconception and FAQ with ZFS on FreeBSD and one reason to not take any Sun/Oracle documentation at face value, as it doesn't always apply to FreeBSD. There were several posts from pjd@ about this back in the 7.x days when ZFS was first imported to FreeBSD. --=20 Freddie Cash fjwcash@gmail.com From owner-freebsd-scsi@freebsd.org Wed Nov 18 09:21:39 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 14C11A31341 for ; Wed, 18 Nov 2015 09:21:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 00BFA1A47 for ; Wed, 18 Nov 2015 09:21:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id tAI9LcuK083758 for ; Wed, 18 Nov 2015 09:21:38 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-scsi@FreeBSD.org Subject: [Bug 204646] 10.2 iSCSI backed zpool shows imporper warnings about non-native block sizes that 10.1 doesn't show Date: Wed, 18 Nov 2015 09:21:38 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-scsi@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2015 09:21:39 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204646 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-scsi@FreeBSD.org -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-scsi@freebsd.org Wed Nov 18 09:21:51 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E03A0A3137E for ; Wed, 18 Nov 2015 09:21:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CC2DC1BCC for ; Wed, 18 Nov 2015 09:21:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id tAI9LpXs085745 for ; Wed, 18 Nov 2015 09:21:51 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-scsi@FreeBSD.org Subject: [Bug 204642] 10.2 iSCSI connected initiator to a 10.1 iSCSI Target generates excessive UNMAP/TRIM commands, system unresponsive. Date: Wed, 18 Nov 2015 09:21:51 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-scsi@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2015 09:21:52 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204642 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-scsi@FreeBSD.org -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-scsi@freebsd.org Wed Nov 18 10:25:08 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7152BA325EC; Wed, 18 Nov 2015 10:25:08 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 27EB7113F; Wed, 18 Nov 2015 10:25:08 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1ZyzvC-0002G2-HL; Wed, 18 Nov 2015 13:25:02 +0300 Date: Wed, 18 Nov 2015 13:25:02 +0300 From: Slawa Olhovchenkov To: Freddie Cash Cc: Royce Williams , freebsd-stable , Borja Marcos , Kai Gallasch , Kevin Oberman , freebsd-scsi@freebsd.org Subject: Re: LSI SAS2008 mps driver preferred firmware version Message-ID: <20151118102502.GO48728@zxy.spb.ru> References: <5644FF09.9090200@free.de> <56472686.5030301@free.de> <20151114143104.GA41119@in-addr.com> <7710CBCC-E68F-4454-9E29-E50ED1C6B511@sarenet.es> <20151116205734.GM48728@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2015 10:25:08 -0000 On Mon, Nov 16, 2015 at 01:19:55PM -0800, Freddie Cash wrote: > On Mon, Nov 16, 2015 at 12:57 PM, Slawa Olhovchenkov wrote: > > > On Mon, Nov 16, 2015 at 11:40:12AM -0800, Freddie Cash wrote: > > > > > On Mon, Nov 16, 2015 at 11:36 AM, Kevin Oberman > > wrote: > > > > As already mentioned, unless you are using zfs, use gpart to label you > > file > > > > systems/disks. Then use the /dev/gpt/LABEL as the mount device in > > fstab. > > > > > > > > > > ​Even if you are using ZFS, labelling the drives with the location of the > > > disk in the system (enclosure, column, row, whatever) makes things so > > much > > > easier to work with when there are disk-related issues. > > > > > > Just create a single partition that covers the whole disk, label it, and > > > use the label to create the vdevs in the pool.​ > > > > Bad idea. > > Re-placed disk in different bay don't relabel automaticly. > > > > ​Did the original disk get labelled automatically? No, you had to do that > when you first started using it. So, why would you expect a > replaced disk Initial labeling is problem too. For new chassis with 36 identical disk (already installed) -- what is simple way to labeling disks? > to get labelled automatically? Consistency keeping is another problem. > Offline the dead/dying disk. > Physically remove the disk. > Insert the new disk. > Partition / label the new disk. > "zfs replace" using the new label to get it into the pool.​ New disk can be inserted in free bay. This is may be done by remote hand. And I can be miss information where disk is placed. > > Other issuse where disk placed in bay some remotely hands in data > > center -- I am relay don't know how disk distributed by bays. > > > > ​You label the disks as they are added to the system the first time. That > way, you always know where each disk is located, and you only deal with the > labels. > > Then, when you need to replace a disk (or ask someone in a remote location > to replace it) it's a simple matter: the label on the disk itself tells > you where the disk is physically located. And it doesn't change if the > controller decides to change the direction it enumerates devices. > > Which is easier to tell someone in a remote location: "Replace disk in bay with blinked led" Author: bapt Date: Sat Sep 5 00:06:01 2015 New Revision: 287473 URL: https://svnweb.freebsd.org/changeset/base/287473 Log: Add a new sesutil(8) utility This is an utility for managing SCSI Enclosure Services (SES) device. For now only one command is supported "locate" which will change the test of the external LED associated to a given disk. Usage if the following: sesutil locate disk [on|off] Disk can be a device name: "da12" or a special keyword: "all". > Replace disk enc0a6 (meaning enclosure 0, column A, row 6)? > or > Replace the disk called da36?​ > ​or > Find the disk with serial number XXXXXXXX? > or > Replace the disk where the light is (hopefully) flashing (but I can't > tell you which enclosure, front or back, or anything else like that)? > > The first one lets you know exactly where the disk is located physically. > > The second one just tells you the name of the device as determined by the > OS, but doesn't tell you anything about where it is located. And it can > change with a kernel update, driver update, or firmware update! > > The third requires you to pull every disk in turn to read the serial number > off the drive itself. Usaly serial number can be read w/o pull disk (for SuperMicro cases this is true, remote hand replaced disk by S/N for me w/o pull every disk). > In order for the second or third option to work, you'd have to write down > the device names and/or serial numbers and stick that onto the drive bay > itself.​ > > > > Best way for identify disk -- uses enclouse services. > > > > ​Only if your enclosure services are actually working (or even enabled). > I've yet to work on a box where that actually works (we custom-build our > storage boxes using OTS hardware). > > Best way, IMO, is to use the physical location of the device as the actual > device name itself. That way, there's never any ambiguity at the physical > layer, the driver layer, the OS layer, or the ZFS pool layer.​ > > > > I have many sites with ZFS on whole disk and some sites with ZFS on > > GPT partition. ZFS on GPT more heavy for administration. > > > > ​It's 1 extra step: partition the drive, supplying the location of the > drive as the label for the partition. > > Everything else works exactly the same. > > I used to do everything with whole drives and no labels. Did that for > about a month, until 2 separate drives on separate controllers died (in a > 24-bay setup) and I couldn't figure out where they were located as a BIOS > upgrade changed which controller loaded first. And then I had to work on a > server that someone else configured with direct-attach bays (24 cables) > that were connected almost at random. All currently used by me servers have some randoms in detecting and reporting controllers and HDDs. No problem for ZFS and/or replacing by remote hands (by S/N). From owner-freebsd-scsi@freebsd.org Wed Nov 18 13:48:19 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 24084A30B2D for ; Wed, 18 Nov 2015 13:48:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0F6851213 for ; Wed, 18 Nov 2015 13:48:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id tAIDmI08085963 for ; Wed, 18 Nov 2015 13:48:18 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-scsi@FreeBSD.org Subject: [Bug 204642] 10.2 iSCSI connected initiator to a 10.1 iSCSI Target generates excessive UNMAP/TRIM commands, system unresponsive. Date: Wed, 18 Nov 2015 13:48:18 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: chris@acsi.ca X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-scsi@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2015 13:48:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204642 --- Comment #1 from Christopher Forgeron --- Please note Bug 204641, where is is noted that the new ctl in 10.2 doesn't allow UNMAP on a file based iSCSI - Perhaps something is overlooked, and it's not completely disabled. I'll be running the tests with a zvol backed iSCSI connection today and update once I have results. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-scsi@freebsd.org Wed Nov 18 16:08:18 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BBC45A2BD41 for ; Wed, 18 Nov 2015 16:08:18 +0000 (UTC) (envelope-from ken@kdm.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 9B7FA1C05 for ; Wed, 18 Nov 2015 16:08:18 +0000 (UTC) (envelope-from ken@kdm.org) Received: by mailman.ysv.freebsd.org (Postfix) id 97CAAA2BD3B; Wed, 18 Nov 2015 16:08:18 +0000 (UTC) Delivered-To: scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 97333A2BD3A; Wed, 18 Nov 2015 16:08:18 +0000 (UTC) (envelope-from ken@kdm.org) Received: from mithlond.kdm.org (mithlond.kdm.org [96.89.93.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5BA211C03; Wed, 18 Nov 2015 16:08:17 +0000 (UTC) (envelope-from ken@kdm.org) Received: from mithlond.kdm.org (localhost [127.0.0.1]) by mithlond.kdm.org (8.15.2/8.14.9) with ESMTPS id tAIG53lo003284 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 18 Nov 2015 11:05:03 -0500 (EST) (envelope-from ken@mithlond.kdm.org) Received: (from ken@localhost) by mithlond.kdm.org (8.15.2/8.14.9/Submit) id tAIG53oX003283; Wed, 18 Nov 2015 11:05:03 -0500 (EST) (envelope-from ken) Date: Wed, 18 Nov 2015 11:05:03 -0500 From: "Kenneth D. Merry" To: scsi@freebsd.org, current@freebsd.org Subject: Re: async pass(4) patches available Message-ID: <20151118160503.GA3095@mithlond.kdm.org> References: <20150330222358.GA46342@mithlond.kdm.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="pWyiEgJYm5f9v55/" Content-Disposition: inline In-Reply-To: <20150330222358.GA46342@mithlond.kdm.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [127.0.0.1]); Wed, 18 Nov 2015 11:05:03 -0500 (EST) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS autolearn=ham autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mithlond.kdm.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2015 16:08:18 -0000 --pWyiEgJYm5f9v55/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline I have updated the asynchronous pass(4) changes, and fixed a number of bugs in camdd(8). The new patches are here: FreeBSD/head as of SVN revision 290970: http://people.freebsd.org/~ken/async_pass.head.20151117.1.txt FreeBSD stable/10 as of SVN revision 290899: http://people.freebsd.org/~ken/async_pass.stable10.20151117.1.txt And a description / draft commit message, this time updated to include all the files that have changed: http://people.freebsd.org/~ken/async_pass_commitmsg.20151118.txt I have also attached the description to this email. At this point I think I've fixed enough bugs and it is stable enough to go into the tree. That will allow others to more easily use the code and add enhancements. Ken On Mon, Mar 30, 2015 at 16:23:58 -0600, Kenneth D. Merry wrote: > > I have put patches to add an asynchronous interface to the pass(4) driver > and add a new camdd(8) utility here: > > FreeBSD/head as of SVN revision 280857: > > http://people.freebsd.org/~ken/async_pass.head.20150330.1.txt > > FreeBSD stable/10 as of SVN revision 280856: > > http://people.freebsd.org/~ken/async_pass.stable_10.20150330.1.txt > > And the description / draft commit message: > > http://people.freebsd.org/~ken/async_pass_commitmsg.20150330.txt > > I have also attached the description and draft commit message to this > email. > > The asynchronous changes to the pass(4) driver allow queueing and fetching > CAM CCBs via two new ioctls. Notification of completed I/O can come via > kqueue(2), poll(2), select(2), etc. > > The camdd(8) utility is intended as a simple data transfer utility, > benchmark, and an in-tree example of how to use the asynchronous pass(4) > interface. > > camdd(8) is still a work in progress. It needs to be cleaned up a bit and > streamlined. > > There is one known arrival and departure bug with the pass(4) driver > changes. We've reproduced it with our tests at Spectra, but I haven't yet > tracked it down. > > There are many more arrival and departure bugs in FreeBSD/head, however. > We have fixed quite a few in our local tree, but the test (called devad2) > that triggers all of the problems uses the asynchronous pass(4) interface. > So this is a prerequisite for fixing/verifying those bugs. > > Comments and testing are welcome! As I said, camdd(8) in particular is a > work in progress. It could use some cleanup and there are some more useful > features that could be added there. > > Part of the reason for camdd(8) was as a test facility for the new > interface. But, it also serves as a useful demonstration of the > asynchronous pass(4) functionality, given that the original application > that used the API doesn't make sense to go into FreeBSD. (It is > Spectra-specific, and not generally useful.) > > Ken > -- > Kenneth Merry > ken@FreeBSD.ORG > Add asynchronous command support to the pass(4) driver, and the new > camdd(8) utility. > > CCBs may be queued to the driver via the new CAMIOQUEUE ioctl, and > completed CCBs may be retrieved via the CAMIOGET ioctl. User > processes can use poll(2) or kevent(2) to get notification when > I/O has completed. > > While the existing CAMIOCOMMAND blocking ioctl interface only > supports user virtual data pointers in a CCB (generally only > one per CCB), the new CAMIOQUEUE ioctl supports user virtual and > physical address pointers, as well as user virtual and physical > scatter/gather lists. This allows user applications to have more > flexibility in their data handling operations. > > Kernel memory for data transferred via the queued interface is > allocated from the zone allocator in MAXPHYS sized chunks, and user > data is copied in and out. This is likely faster than the > vmapbuf()/vunmapbuf() method used by the CAMIOCOMMAND ioctl in > configurations with many processors (there are more TLB shootdowns > caused by the mapping/unmapping operation) but may not be as fast > as running with unmapped I/O. > > The new memory handling model for user requests also allows > applications to send CCBs with request sizes that are larger than > MAXPHYS. The pass(4) driver now limits queued requests to the I/O > size listed by the SIM driver in the maxio field in the Path > Inquiry (XPT_PATH_INQ) CCB. > > There are some things things would be good to add: > > 1. Come up with a way to do unmapped I/O on multiple buffers. > Currently the unmapped I/O interface operates on a struct bio, > which includes only one address and length. It would be nice > to be able to send an unmapped scatter/gather list down to > busdma. This would allow eliminating the copy we currently do > for data. > > 2. Add an ioctl to list currently outstanding CCBs in the various > queues. > > 3. Add an ioctl to cancel a request, or use the XPT_ABORT CCB to do > that. > > 4. Test physical address support. Virtual pointers and scatter > gather lists have been tested, but I have not yet tested > physical addresses or scatter/gather lists. > > 5. Investigate multiple queue support. At the moment there is one > queue of commands per pass(4) device. If multiple processes > open the device, they will submit I/O into the same queue and > get events for the same completions. This is probably the right > model for most applications, but it would be good to make sure > that there is not really a case for multiple queues before > pushing this code upstream. > > Also, add a new utility, camdd(8) that uses the asynchronous pass(4) > driver interface. > > This utility is intended to be a basic data transfer/copy utility, > a simple benchmark utility, and an example of how to use the > asynchronous pass(4) interface. > > It can copy data to and from pass(4) devices using any target queue > depth, starting offset and blocksize for the input and ouptut devices. > It currently only supports SCSI devices, but could be easily extended > to support ATA devices. > > It can also copy data to and from regular files, block devices, tape > devices, pipes, stdin, and stdout. It does not support queueing > multiple commands to any of those targets, since it uses the standard > read(2)/write(2)/writev(2)/readv(2) system calls. > > The I/O is done by two threads, one for the reader and one for the > writer. The reader thread sends completed read requests to the > writer thread in strictly sequential order, even if they complete > out of order. That could be modified later on for random I/O patterns > or slightly out of order I/O. > > camdd(8) uses kqueue(2)/kevent(2) to get I/O compeltion events from > the pass(4) driver and also to send request notifications internally. > > For pass(4) devcies, camdd(8) uses a single buffer (CAM_DATA_VADDR) > per CAM CCB on the reading side, and a scatter/gather list > (CAM_DATA_SG) on the writing side. In addition to testing both > interfaces, this makes any potential reblocking of I/O easier. No > data is copied between the reader and the writer, but rather the > reader's buffers are split into multiple I/O requests or combined > into a single I/O request depending on the input and output blocksize. > > For the file I/O path, camdd(8) also uses a single buffer (read(2), > write(2), pread(2) or pwrite(2)) on reads, and a scatter/gather list > (readv(2), writev(2), preadv(2), pwritev(2)) on writes. > > Things that would be nice to do for camdd(8) eventually: > > 1. Add support for I/O pattern generation. Patterns like all > zeros, all ones, LBA-based patterns, random patterns, etc. Right > Now you can always use /dev/zero, /dev/random, etc. > > 2. Add support for a "sink" mode, so we do only reads with no > writes. Right now, you can use /dev/null. > > 3. Add support for automatic queue depth probing, so that we can > figure out the right queue depth on the input and output side > for maximum throughput. At the moment it defaults to 6. > > 4. Add support for SATA device passthrough I/O. > > 5. Add support for random LBAs and/or lengths on the input and > side. > > 6. Track average per-I/O latency and busy time. The busy time > and latency could also feed in to the automatic queue depth > determination. > > sys/cam/scsi/scsi_pass.h: > Define two new ioctls, CAMIOQUEUE and CAMIOGET, that queue > and fetch asynchronous CAM CCBs respectively. > > Although these ioctls do not have a declared argument, they > both take a union ccb pointer. If we declare a size here, > the ioctl code in sys/kern/sys_generic.c will malloc and free > a buffer for either the CCB or the CCB pointer (depending on > how it is declared). Since we have to keep a copy of the > CCB (which is fairly large) anyway, having the ioctl malloc > and free a CCB for each call is wasteful. > > sys/cam/scsi/scsi_pass.c: > Add asynchronous CCB support. > > Add two new ioctls, CAMIOQUEUE and CAMIOGET. > > CAMIOQUEUE adds a CCB to the incoming queue. The CCB is > executed immediately (and moved to the active queue) if it > is an immediate CCB, but otherwise it will be executed > in passstart() when a CCB is available from the transport layer. > > When CCBs are completed (because they are immediate or > passdone() if they are queued), they are put on the done > queue. > > If we get the final close on the device before all pending > I/O is complete, all active I/O is moved to the abandoned > queue and we increment the peripheral reference count so > that the peripheral driver instance doesn't go away before > all pending I/O is done. > > The new passcreatezone() function is called on the first > call to the CAMIOQUEUE ioctl on a given device to allocate > the UMA zones for I/O requests and S/G list buffers. This > may be good to move off to a taskqueue at some point. > The new passmemsetup() function allocates memory and > scatter/gather lists to hold the user's data, and copies > in any data that needs to be written. For virtual pointers > (CAM_DATA_VADDR), the kernel buffer is malloced from the > new pass(4) driver malloc bucket. For virtual > scatter/gather lists (CAM_DATA_SG), buffers are allocated > from a new per-pass(9) UMA zone in MAXPHYS-sized chunks. > Physical pointers are passed in unchanged. We have support > for up to 16 scatter/gather segments (for the user and > kernel S/G lists) in the default struct pass_io_req, so > requests with longer S/G lists require an extra kernel malloc. > > The new passcopysglist() function copies a user scatter/gather > list to a kernel scatter/gather list. The number of elements > in each list may be different, but (obviously) the amount of data > stored has to be identical. > > The new passmemdone() function copies data out for the > CAM_DATA_VADDR and CAM_DATA_SG cases. > > The new passiocleanup() function restores data pointers in > user CCBs and frees memory. > > Add new functions to support kqueue(2)/kevent(2): > > passreadfilt() tells kevent whether or not the done > queue is empty. > > passkqfilter() adds a knote to our list. > > passreadfiltdetach() removes a knote from our list. > > Add a new function, passpoll(), for poll(2)/select(2) > to use. > > Add devstat(9) support for the queued CCB path. > > usr.sbin/camdd/Makefile: > Add a makefile for camdd(8). > > usr.sbin/camdd/camdd.8: > Man page for camdd(8). > > usr.sbin/camdd/camdd.c: > The new camdd(8) utility. > -- Kenneth Merry ken@FreeBSD.ORG --pWyiEgJYm5f9v55/ Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="async_pass_commitmsg.20151118.txt" Add asynchronous command support to the pass(4) driver, and the new camdd(8) utility. CCBs may be queued to the driver via the new CAMIOQUEUE ioctl, and completed CCBs may be retrieved via the CAMIOGET ioctl. User processes can use poll(2) or kevent(2) to get notification when I/O has completed. While the existing CAMIOCOMMAND blocking ioctl interface only supports user virtual data pointers in a CCB (generally only one per CCB), the new CAMIOQUEUE ioctl supports user virtual and physical address pointers, as well as user virtual and physical scatter/gather lists. This allows user applications to have more flexibility in their data handling operations. Kernel memory for data transferred via the queued interface is allocated from the zone allocator in MAXPHYS sized chunks, and user data is copied in and out. This is likely faster than the vmapbuf()/vunmapbuf() method used by the CAMIOCOMMAND ioctl in configurations with many processors (there are more TLB shootdowns caused by the mapping/unmapping operation) but may not be as fast as running with unmapped I/O. The new memory handling model for user requests also allows applications to send CCBs with request sizes that are larger than MAXPHYS. The pass(4) driver now limits queued requests to the I/O size listed by the SIM driver in the maxio field in the Path Inquiry (XPT_PATH_INQ) CCB. There are some things things would be good to add: 1. Come up with a way to do unmapped I/O on multiple buffers. Currently the unmapped I/O interface operates on a struct bio, which includes only one address and length. It would be nice to be able to send an unmapped scatter/gather list down to busdma. This would allow eliminating the copy we currently do for data. 2. Add an ioctl to list currently outstanding CCBs in the various queues. 3. Add an ioctl to cancel a request, or use the XPT_ABORT CCB to do that. 4. Test physical address support. Virtual pointers and scatter gather lists have been tested, but I have not yet tested physical addresses or scatter/gather lists. 5. Investigate multiple queue support. At the moment there is one queue of commands per pass(4) device. If multiple processes open the device, they will submit I/O into the same queue and get events for the same completions. This is probably the right model for most applications, but it would be good to make sure that there is not really a case for multiple queues before pushing this code upstream. Also, add a new utility, camdd(8) that uses the asynchronous pass(4) driver interface. This utility is intended to be a basic data transfer/copy utility, a simple benchmark utility, and an example of how to use the asynchronous pass(4) interface. It can copy data to and from pass(4) devices using any target queue depth, starting offset and blocksize for the input and ouptut devices. It currently only supports SCSI devices, but could be easily extended to support ATA devices. It can also copy data to and from regular files, block devices, tape devices, pipes, stdin, and stdout. It does not support queueing multiple commands to any of those targets, since it uses the standard read(2)/write(2)/writev(2)/readv(2) system calls. The I/O is done by two threads, one for the reader and one for the writer. The reader thread sends completed read requests to the writer thread in strictly sequential order, even if they complete out of order. That could be modified later on for random I/O patterns or slightly out of order I/O. camdd(8) uses kqueue(2)/kevent(2) to get I/O completion events from the pass(4) driver and also to send request notifications internally. For pass(4) devcies, camdd(8) uses a single buffer (CAM_DATA_VADDR) per CAM CCB on the reading side, and a scatter/gather list (CAM_DATA_SG) on the writing side. In addition to testing both interfaces, this makes any potential reblocking of I/O easier. No data is copied between the reader and the writer, but rather the reader's buffers are split into multiple I/O requests or combined into a single I/O request depending on the input and output blocksize. For the file I/O path, camdd(8) also uses a single buffer (read(2), write(2), pread(2) or pwrite(2)) on reads, and a scatter/gather list (readv(2), writev(2), preadv(2), pwritev(2)) on writes. Things that would be nice to do for camdd(8) eventually: 1. Add support for I/O pattern generation. Patterns like all zeros, all ones, LBA-based patterns, random patterns, etc. Right Now you can always use /dev/zero, /dev/random, etc. 2. Add support for a "sink" mode, so we do only reads with no writes. Right now, you can use /dev/null. 3. Add support for automatic queue depth probing, so that we can figure out the right queue depth on the input and output side for maximum throughput. At the moment it defaults to 6. 4. Add support for SATA device passthrough I/O. 5. Add support for random LBAs and/or lengths on the input and side. 6. Track average per-I/O latency and busy time. The busy time and latency could also feed in to the automatic queue depth determination. sys/cam/scsi/scsi_pass.h: Define two new ioctls, CAMIOQUEUE and CAMIOGET, that queue and fetch asynchronous CAM CCBs respectively. Although these ioctls do not have a declared argument, they both take a union ccb pointer. If we declare a size here, the ioctl code in sys/kern/sys_generic.c will malloc and free a buffer for either the CCB or the CCB pointer (depending on how it is declared). Since we have to keep a copy of the CCB (which is fairly large) anyway, having the ioctl malloc and free a CCB for each call is wasteful. sys/cam/scsi/scsi_pass.c: Add asynchronous CCB support. Add two new ioctls, CAMIOQUEUE and CAMIOGET. CAMIOQUEUE adds a CCB to the incoming queue. The CCB is executed immediately (and moved to the active queue) if it is an immediate CCB, but otherwise it will be executed in passstart() when a CCB is available from the transport layer. When CCBs are completed (because they are immediate or passdone() if they are queued), they are put on the done queue. If we get the final close on the device before all pending I/O is complete, all active I/O is moved to the abandoned queue and we increment the peripheral reference count so that the peripheral driver instance doesn't go away before all pending I/O is done. The new passcreatezone() function is called on the first call to the CAMIOQUEUE ioctl on a given device to allocate the UMA zones for I/O requests and S/G list buffers. This may be good to move off to a taskqueue at some point. The new passmemsetup() function allocates memory and scatter/gather lists to hold the user's data, and copies in any data that needs to be written. For virtual pointers (CAM_DATA_VADDR), the kernel buffer is malloced from the new pass(4) driver malloc bucket. For virtual scatter/gather lists (CAM_DATA_SG), buffers are allocated from a new per-pass(9) UMA zone in MAXPHYS-sized chunks. Physical pointers are passed in unchanged. We have support for up to 16 scatter/gather segments (for the user and kernel S/G lists) in the default struct pass_io_req, so requests with longer S/G lists require an extra kernel malloc. The new passcopysglist() function copies a user scatter/gather list to a kernel scatter/gather list. The number of elements in each list may be different, but (obviously) the amount of data stored has to be identical. The new passmemdone() function copies data out for the CAM_DATA_VADDR and CAM_DATA_SG cases. The new passiocleanup() function restores data pointers in user CCBs and frees memory. Add new functions to support kqueue(2)/kevent(2): passreadfilt() tells kevent whether or not the done queue is empty. passkqfilter() adds a knote to our list. passreadfiltdetach() removes a knote from our list. Add a new function, passpoll(), for poll(2)/select(2) to use. Add devstat(9) support for the queued CCB path. sys/cam/ata/ata_da.c: Add support for the BIO_VLIST bio type. sys/cam/cam_ccb.h: Add a new enumeration for the xflags field in the CCB header. (This doesn't change the CCB header, just adds an enumeration to use.) sys/cam/cam_xpt.c: Add a new function, xpt_setup_ccb_flags(), that allows specifying CCB flags. sys/cam/cam_xpt.h: Add a prototype for xpt_setup_ccb_flags(). sys/cam/scsi/scsi_da.c: Add support for BIO_VLIST. sys/dev/md/md.c: Add BIO_VLIST support to md(4). sys/geom/geom_disk.c: sys/kern/subr_bus_dma.c: Change _bus_dmamap_load_vlist() to take a starting offset and length. Add a new function, _bus_dmamap_load_pages(), that will load a list of physical pages starting at an offset. Update _bus_dmamap_load_bio() to allow loading BIO_VLIST bios. Allow unmapped I/O to start at an offset. sys/kern/subr_uio.c: Add two new functions, physcopyin_vlist() and physcopyout_vlist(). sys/sys/bio.h: Add a new bio flag, BIO_VLIST. sys/sys/uio.h: Add prototypes for physcopyin_vlist() and physcopyout_vlist(). share/man/man4/pass.4: Document the CAMIOQUEUE and CAMIOGET ioctls. usr.sbin/Makefile: Add camdd. usr.sbin/camdd/Makefile: Add a makefile for camdd(8). usr.sbin/camdd/camdd.8: Man page for camdd(8). usr.sbin/camdd/camdd.c: The new camdd(8) utility. --pWyiEgJYm5f9v55/-- From owner-freebsd-scsi@freebsd.org Wed Nov 18 16:15:17 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2CEC3A3206D; Wed, 18 Nov 2015 16:15:17 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-ob0-x234.google.com (mail-ob0-x234.google.com [IPv6:2607:f8b0:4003:c01::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DEC7C1164; Wed, 18 Nov 2015 16:15:16 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by obdgf3 with SMTP id gf3so37409354obd.3; Wed, 18 Nov 2015 08:15:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=2KE4uk0bTK3DXRF/8MdHwvPzUzVV7xcqo/KJVoAxPqo=; b=AfKhaiHyQpJTWXhcegPHNC2Du0OkmLxCKPe5/N7wCvNttmWkAAL/lYDYCW4N4tg45D y0SN9+Y4dFRxEQMJK9V/hXoW7fhpZEZCXNZn9z3BQHw4plspSk+lH2k29XRSt4ic79PX zsR7MpTMWg/dubkMN5MrKmzQH+Hn2Tctna0+vlKKlRzLEXnrdD3Mub/x91enEm06PGFu pCr7t+ljNqnwWDzkPujB5Cj9Mpl6vXLlvR0UxSdU1qogtNPKXEFhOUmUoNofii+nblbk YfVfceUr8Zt+uHkZi/BY7VDPGALdUA7TDtzhS4rWzEL/VYu04lmta2Ruz5d49DUhq3AE uJPA== MIME-Version: 1.0 X-Received: by 10.60.76.42 with SMTP id h10mr1543574oew.13.1447863316098; Wed, 18 Nov 2015 08:15:16 -0800 (PST) Received: by 10.76.80.229 with HTTP; Wed, 18 Nov 2015 08:15:15 -0800 (PST) In-Reply-To: <20151118102502.GO48728@zxy.spb.ru> References: <5644FF09.9090200@free.de> <56472686.5030301@free.de> <20151114143104.GA41119@in-addr.com> <7710CBCC-E68F-4454-9E29-E50ED1C6B511@sarenet.es> <20151116205734.GM48728@zxy.spb.ru> <20151118102502.GO48728@zxy.spb.ru> Date: Wed, 18 Nov 2015 08:15:15 -0800 Message-ID: Subject: Re: LSI SAS2008 mps driver preferred firmware version From: Freddie Cash To: Slawa Olhovchenkov Cc: freebsd-stable , freebsd-scsi@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2015 16:15:17 -0000 On Wed, Nov 18, 2015 at 2:25 AM, Slawa Olhovchenkov wrote: > On Mon, Nov 16, 2015 at 01:19:55PM -0800, Freddie Cash wrote: > > > On Mon, Nov 16, 2015 at 12:57 PM, Slawa Olhovchenkov > wrote: > > =E2=80=8BDid the original disk get labelled automatically? No, you had= to do > that > > when you first started using it. So, why would you expect a > > replaced disk > > Initial labeling is problem too. > For new chassis with 36 identical disk (already installed) -- what is > simple way to labeling disks? > =E2=80=8BThat's the easy part. Boot with all the drives pulled out a bit, = so they aren't connected/detected. Insert first disk, wait for it to be detected and get a /dev node, then partition/label it. Repeat for each disk. Takes about 5 minutes to label a 45-bay JBOD chassis. No different than how you would get the serial number off each disk before inserting them into the chassis, so you'd know for sure which slot they're in. "Replace disk in bay with blinked led" > > Author: bapt > Date: Sat Sep 5 00:06:01 2015 > =E2=80=8BAnd, how did you manage to do that before Sep 5, 2015?=E2=80=8B Usaly serial number can be read w/o pull disk (for SuperMicro cases > this is true, remote hand replaced disk by S/N for me w/o pull every disk= ). > =E2=80=8BHow? We have all SuperMicro storage chassis (SC2xx, SC8xx, and JB= ODs) and server chassis in our data centre here. None of them allow you to read the serial number off the physical disk without pulling the disk out completely.=E2=80=8B You'd have to manually label each bay with the serial= number before inserting the disk into the chassis ... which is no different from labelling the device in the OS. Except it's much faster to find a 3D co-ordinate (enc0a6) than to scan every bay looking for a specific serial number. But, to each their own. :) Everyone has their "perfect" system that works for them. :D --=20 Freddie Cash fjwcash@gmail.com From owner-freebsd-scsi@freebsd.org Wed Nov 18 16:54:20 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 55711A32A46; Wed, 18 Nov 2015 16:54:20 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0B47B1D9F; Wed, 18 Nov 2015 16:54:20 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1Zz5zs-000Ahq-TV; Wed, 18 Nov 2015 19:54:16 +0300 Date: Wed, 18 Nov 2015 19:54:16 +0300 From: Slawa Olhovchenkov To: Freddie Cash Cc: freebsd-stable , freebsd-scsi@freebsd.org Subject: Re: LSI SAS2008 mps driver preferred firmware version Message-ID: <20151118165416.GS31314@zxy.spb.ru> References: <56472686.5030301@free.de> <20151114143104.GA41119@in-addr.com> <7710CBCC-E68F-4454-9E29-E50ED1C6B511@sarenet.es> <20151116205734.GM48728@zxy.spb.ru> <20151118102502.GO48728@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2015 16:54:20 -0000 On Wed, Nov 18, 2015 at 08:15:15AM -0800, Freddie Cash wrote: > On Wed, Nov 18, 2015 at 2:25 AM, Slawa Olhovchenkov wrote: > > > On Mon, Nov 16, 2015 at 01:19:55PM -0800, Freddie Cash wrote: > > > > > On Mon, Nov 16, 2015 at 12:57 PM, Slawa Olhovchenkov > > wrote: > > > ​Did the original disk get labelled automatically? No, you had to do > > that > > > when you first started using it. So, why would you expect a > > > replaced disk > > > > Initial labeling is problem too. > > For new chassis with 36 identical disk (already installed) -- what is > > simple way to labeling disks? > > > > ​That's the easy part. Boot with all the drives pulled out a bit, so they > aren't connected/detected. > > Insert first disk, wait for it to be detected and get a /dev node, then > partition/label it. Repeat for each disk. Takes about 5 minutes to label > a 45-bay JBOD chassis. Hmm, from me to server more then 1700km, how I can do this? > No different than how you would get the serial number off each disk before > inserting them into the chassis, so you'd know for sure which slot they're > in. This is do by manufacturer. Or in DC after service ordering. I am don't assemble servers, in general. And I am don't see servers and don't know how they look. > "Replace disk in bay with blinked led" > > > > Author: bapt > > Date: Sat Sep 5 00:06:01 2015 > > > > ​And, how did you manage to do that before Sep 5, 2015?​ Deteched disk don't blink activity LED. > Usaly serial number can be read w/o pull disk (for SuperMicro cases > > this is true, remote hand replaced disk by S/N for me w/o pull every disk). > > > > ​How? We have all SuperMicro storage chassis (SC2xx, SC8xx, and JBODs) and > server chassis in our data centre here. None of them allow you to read the > serial number off the physical disk without pulling the disk out > completely.​ You'd have to manually label each bay with the serial number > before inserting the disk into the chassis ... which is no different from > labelling the device in the OS. Except it's much faster to find a 3D > co-ordinate (enc0a6) than to scan every bay looking for a specific serial > number. For SC847A this do for me in NL DC (as I understand -- through holes at an angle). > But, to each their own. :) Everyone has their "perfect" system that works > for them. :D > > -- > Freddie Cash > fjwcash@gmail.com From owner-freebsd-scsi@freebsd.org Wed Nov 18 17:13:12 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 736D4A32EA0 for ; Wed, 18 Nov 2015 17:13:12 +0000 (UTC) (envelope-from ken@kdm.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 53389194E for ; Wed, 18 Nov 2015 17:13:12 +0000 (UTC) (envelope-from ken@kdm.org) Received: by mailman.ysv.freebsd.org (Postfix) id 5181CA32E9E; Wed, 18 Nov 2015 17:13:12 +0000 (UTC) Delivered-To: scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 373C1A32E9C; Wed, 18 Nov 2015 17:13:12 +0000 (UTC) (envelope-from ken@kdm.org) Received: from mithlond.kdm.org (mithlond.kdm.org [96.89.93.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0BC5C194B; Wed, 18 Nov 2015 17:13:11 +0000 (UTC) (envelope-from ken@kdm.org) Received: from mithlond.kdm.org (localhost [127.0.0.1]) by mithlond.kdm.org (8.15.2/8.14.9) with ESMTPS id tAIHD97A004267 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 18 Nov 2015 12:13:09 -0500 (EST) (envelope-from ken@mithlond.kdm.org) Received: (from ken@localhost) by mithlond.kdm.org (8.15.2/8.14.9/Submit) id tAIHD9fw004266; Wed, 18 Nov 2015 12:13:09 -0500 (EST) (envelope-from ken) Date: Wed, 18 Nov 2015 12:13:09 -0500 From: "Kenneth D. Merry" To: scsi@freebsd.org, current@freebsd.org Subject: CAM Shingled Disk support patches available Message-ID: <20151118171309.GA3564@mithlond.kdm.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [127.0.0.1]); Wed, 18 Nov 2015 12:13:09 -0500 (EST) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS autolearn=ham autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mithlond.kdm.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2015 17:13:12 -0000 I have work in progress patches to add SMR (Shingled Magnetic Recording) support to CAM and GEOM here: FreeBSD/head as of SVN revision 290997: https://people.freebsd.org/~ken/cam_smr.head.20151117.1.txt FreeBSD stable/10 as of SVN revision 290995: https://people.freebsd.org/~ken/cam_smr.stable10.20151117.1.txt This includes support for Host Managed, Host Aware and Drive Managed SMR drives that are either SCSI (ZBC) or ATA (ZAC) attached via a SAS controller. This does not include support for SMR ATA drives attched via an ATA controller. Also, I have not yet figured out how to properly detect a Host Managed ATA drive, so this code won't do that. The big drive vendors are moving to SMR for at least some of their drives. The primary challenge with SMR is that it requires writing a relatively large zone sequentially starting at the beginning of the zone. The usual zone size is 256MB. It is conceptually almost like having a 256MB sector size. We (Spectra Logic) are working on ZFS changes that will use this CAM and GEOM infrastructure to make ZFS play well with SMR drives. Those changes aren't yet done. The patches linked above include: o A new 'camcontrol zone' command that allows displaying and managing drive zones via SCSI/ATA passthrough. o A new zonectl(8) utility that uses the new DIOCZONECMD ioctl to display and manage zones via the da(4) (and later ada(4)) driver. o Changes to diskinfo -v to display the zone mode of a drive. o A new disk zone API, sys/sys/disk_zone.h. o A new bio type, BIO_ZONE, and modifications to GEOM to support it. This new bio will allow filesystems to query zone support in a drive and manage zoned drives. o Extensive modifications to the da(4) driver to handle probing SCSI and SATA behind SAS SMR drives. o Additional CAM CDB building functions for zone commands. The current issues that need to be addressed are: o The da(4) driver now has 6 additional probe states, 5 of which are needed for probing ATA drives behind SAS controllers. I have not yet added support for BIO_ZONE bios to ada(4), but it will be very similar to the da(4) driver version. The ATA probe code needs to be pulled out of the da(4) driver and changed into a form that will allow it to work for either the ada(4) or da(4) driver. Otherwise we'll have a fair amount of code duplication between the two drivers. o There is a reasonable amount of code duplication between 'camcontrol zone' and zonectl(8). This was done for speed / expediency's sake, but it may be possible to make more of the code common there. o In order to add the new BIO_ZONE bio command, I had to change the bio_cmd field in struct bio from a uint8_t to a uint16_t. This will cause binary compatibility problems with any 3rd party loadable modules. Advice on how to handle this would be welcome. o In the process of developing these changes, I discovered that the dxfer_len paramter for scsi_ata_pass_16() was too small (uint16_t, and it needed to be uint32_t). I increased it, but that will potentially cause a binary incompatibility problem with any existing applications that use the current API via libcam. Advice on how to handle that would be welcome. If you look through the code, you'll notice that the disk_zone.h API is separate from the SCSI and ATA APIs. The intent is to allow filesystems and other consumers of the API to just talk to the disk zone API without dealing with the SCSI and ATA specifics. Another reason behind all of this is that even though the SCSI ZBC and ATA ZAC specs were developed in concert, and are intended to be functionally identical, they are still SCSI and ATA. As usual, SCSI is big endian and ATA is little endian. So to present a common API to the filesystem, we give all of the zone data back in native byte order, regardless of the underlying device protocol. Another thing to note is the extensive use of ATA passthrough in the da(4) driver. This is necessary because the SCSI SAT (SCSI to ATA Translation) specification has not yet caught up with translating SCSI zone commands (ZBC) to ATA zone commands (ZAC). So, until the spec is updated and LSI and other vendors update their SCSI to ATA translation layers, we'll have to use the ATA version of the commands when talking to ATA drives via SAS controllers. I have only tested the code so far with Seagate SATA Drive Managed and Host Aware drives. I would appreciate testing with any drives. (And testing to make sure that the patches don't cause problems with existing hardware.) Right now, all you can really do is manage the zones manually using camcontrol(8) or zonectl(8). Automatic management will come with the ZFS changes. (Or changes to other filesysems if people want to do it.) If you have a SATA Host Aware drive, in theory camcontrol(8) should allow you to manage the drive if you have it attached to a SATA controller. Here is an example of some of the commands. Get general zoning information: [root@sm4u-1 ~]# zonectl -c params -d /dev/da21 Zone Mode: Host Aware Command support: Report Zones, Open, Close, Finish, Reset Write Pointer Unrestricted Read in Sequential Write Required Zone (URSWRZ): No Optimal Number of Open Sequential Write Preferred Zones: 128 Optimal Number of Non-Sequentially Written Sequential Write Preferred Zones: 8 Maximum Number of Open Sequential Write Required Zones: Unlimited Look at information from the da(4) driver: [root@sm4u-1 ~]# sysctl kern.cam.da.21 kern.cam.da.21.delete_method: NONE kern.cam.da.21.delete_max: 1081344 kern.cam.da.21.minimum_cmd_size: 6 kern.cam.da.21.sort_io_queue: -1 kern.cam.da.21.zone_mode: Host Aware kern.cam.da.21.zone_support: Report Zones, Open, Close, Finish, Reset Write Pointer kern.cam.da.21.optimal_seq_zones: 128 kern.cam.da.21.optimal_nonseq_zones: 8 kern.cam.da.21.max_seq_zones: 4294967295 kern.cam.da.21.error_inject: 0 Display all of the zones with zonectl(8): [root@sm4u-1 ~]# zonectl -d /dev/da21 -c rz 29809 zones, Maximum LBA 0x3a3812aaf (15628053167) Zone lengths and types may vary Start LBA Length WP LBA Zone Type Condition Sequential Reset 0, 524288, 0x80000, Conventional, NWP, Sequential, No Reset Needed 0x80000, 524288, 0x100000, Conventional, NWP, Sequential, No Reset Needed 0x100000, 524288, 0x180000, Conventional, NWP, Sequential, No Reset Needed 0x180000, 524288, 0x200000, Conventional, NWP, Sequential, No Reset Needed 0x200000, 524288, 0x280000, Conventional, NWP, Sequential, No Reset Needed 0x280000, 524288, 0x300000, Conventional, NWP, Sequential, No Reset Needed 0x300000, 524288, 0x380000, Conventional, NWP, Sequential, No Reset Needed 0x380000, 524288, 0x400000, Conventional, NWP, Sequential, No Reset Needed 0x400000, 524288, 0x480000, Conventional, NWP, Sequential, No Reset Needed 0x480000, 524288, 0x500000, Conventional, NWP, Sequential, No Reset Needed 0x500000, 524288, 0x580000, Conventional, NWP, Sequential, No Reset Needed 0x580000, 524288, 0x600000, Conventional, NWP, Sequential, No Reset Needed 0x600000, 524288, 0x680000, Conventional, NWP, Sequential, No Reset Needed 0x680000, 524288, 0x700000, Conventional, NWP, Sequential, No Reset Needed 0x700000, 524288, 0x780000, Conventional, NWP, Sequential, No Reset Needed [ ... ] 0x1f00000, 524288, 0x1f80000, Conventional, NWP, Sequential, No Reset Needed 0x1f80000, 524288, 0x2000000, Conventional, NWP, Sequential, No Reset Needed 0x2000000, 524288, 0x2080000, Seq Preferred, Full, Sequential, No Reset Needed 0x2080000, 524288, 0x2100000, Seq Preferred, Full, Sequential, No Reset Needed 0x2100000, 524288, 0x2180000, Seq Preferred, Full, Sequential, No Reset Needed 0x2180000, 524288, 0x2200000, Seq Preferred, Full, Sequential, No Reset Needed 0x2200000, 524288, 0x2280000, Seq Preferred, Full, Sequential, No Reset Needed 0x2280000, 524288, 0x2300000, Seq Preferred, Full, Sequential, No Reset Needed [ ... ] Use camcontrol zone to reset the write pointer for the first shingled zone listed above: [root@sm4u-1 ~]# camcontrol zone da21 -v -c rwp -l 0x2000000 Use camcontrol zone to ask the drive to report empty zones: [root@sm4u-1 ~]# camcontrol zone da21 -v -c rz -o empty 1 zones, Maximum LBA 0x3a3812aaf (15628053167) Zone lengths and types may vary Start LBA Length WP LBA Zone Type Condition Sequential Reset 0x2000000, 524288, 0x2000000, Seq Preferred, Empty, Sequential, No Reset Needed Get information on a Host Aware drive: root@sm4u-1 ~]# diskinfo -v da21 da21 512 # sectorsize 8001563222016 # mediasize in bytes (7.3T) 15628053168 # mediasize in sectors 4096 # stripesize 0 # stripeoffset 972801 # Cylinders according to firmware. 255 # Heads according to firmware. 63 # Sectors according to firmware. Z84008NY # Disk ident. enc@5003048001f311fd/elmtype@array_device/slot@22 # Physical path Host Aware # Zone Mode Get information on a drive managed drive: [root@sm4u-1 ~]# diskinfo -v da20 da20 512 # sectorsize 8001563222016 # mediasize in bytes (7.3T) 15628053168 # mediasize in sectors 4096 # stripesize 0 # stripeoffset 972801 # Cylinders according to firmware. 255 # Heads according to firmware. 63 # Sectors according to firmware. Z8405938 # Disk ident. enc@5003048001f311fd/elmtype@array_device/slot@21 # Physical path Drive Managed # Zone Mode Get information on a non-zoned drive: [root@sm4u-1 ~]# diskinfo -v da4 da4 512 # sectorsize 100030242816 # mediasize in bytes (93G) 195371568 # mediasize in sectors 0 # stripesize 0 # stripeoffset 12161 # Cylinders according to firmware. 255 # Heads according to firmware. 63 # Sectors according to firmware. 124903574F36 # Disk ident. enc@5003048001f311fd/elmtype@array_device/slot@5 # Physical path Not Zoned # Zone Mode Testing and comments are welcome. Thanks, Ken -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-scsi@freebsd.org Wed Nov 18 19:14:51 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 378A3A32914 for ; Wed, 18 Nov 2015 19:14:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2391B1580 for ; Wed, 18 Nov 2015 19:14:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id tAIJEpXx044041 for ; Wed, 18 Nov 2015 19:14:51 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-scsi@FreeBSD.org Subject: [Bug 204646] 10.2 iSCSI backed zpool shows imporper warnings about non-native block sizes that 10.1 doesn't show Date: Wed, 18 Nov 2015 19:14:51 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: chris@acsi.ca X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-scsi@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2015 19:14:51 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204646 --- Comment #1 from Christopher Forgeron --- Continuing to dig on this, I see that ctl is intentionally reporting volblocksize as the physical sector size. The 'blocksize' in ctl.conf is the logical sector size. I'm not sure I agree with this setup - However, perhaps I misunderstand volblocksize. With recordsize, the size of the write to disk will be a multiple of ashift, up to a max of recordsize. With volblocksize, is the write _always_ volblocksize in size? ie it really is acting like a sector size? If so, this behaviour may be correct - If not, then ctl should be advertising the translated ashift value. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-scsi@freebsd.org Thu Nov 19 17:04:22 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AC145A3244D; Thu, 19 Nov 2015 17:04:22 +0000 (UTC) (envelope-from jonathon.reinhart@gmail.com) Received: from mail-io0-x22b.google.com (mail-io0-x22b.google.com [IPv6:2607:f8b0:4001:c06::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7EFF513CF; Thu, 19 Nov 2015 17:04:22 +0000 (UTC) (envelope-from jonathon.reinhart@gmail.com) Received: by ioir85 with SMTP id r85so95776916ioi.1; Thu, 19 Nov 2015 09:04:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=eVc1ojBPPC4FgZ84Vwfo64FTyM1R6dmdVezB30dbm5w=; b=o8krYjU6zTghEll8ZHqxHfWbdeFBt5x+JGIB7RkSRkvi3J6pe1jnUZDZ8lPeJMa6+F nywyKnUyPDmGWWl1vPS/QicwuDA1wx+kQvm9Hx1HZn1IItKmTT7d7wEmkxevnCOJHMn9 vFxghHO3HC+vsB3k2qnM2DuFggPghhujOYz+9+eYSqgX24PEXhC5SX/Ctho6xwlZFyy1 J0cGCwEUhlUopsH5dw2X2W1qQfHUGYOHCRTNAvTNZ9jtSofm2ZA1sLS2Vs6IPKusGtCM Cp4NMwtf3hAsS1gIizlwI2QgcDg1zF6v1TJiRjWA0kIIxYn+qOWM4IZnxPQ+RhTw6MFZ rSrQ== X-Received: by 10.107.3.101 with SMTP id 98mr10489595iod.182.1447952661826; Thu, 19 Nov 2015 09:04:21 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.10.67 with HTTP; Thu, 19 Nov 2015 09:03:52 -0800 (PST) From: Jonathon Reinhart Date: Thu, 19 Nov 2015 12:03:52 -0500 Message-ID: Subject: ciss(4) HP Smart Array P840 To: freebsd-scsi@freebsd.org, freebsd-drivers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2015 17:04:22 -0000 Hello all, TL;DR: Why isn't the P840 supported by ciss(4)? I tried to setup FreeNAS 9.3 (running 9.3-RELEASE-p25) on an HP DL380 Gen9, with the P840 RAID controller. I didn't realize that the P840 is not officially supported by the ciss(4) driver. In my troubleshooting, I ended up in the same place as documented on this forums.freenas.org post : > It's interesting that camcontrol devlist shows the drives, but there are no da* or ada* designations for them. I don't know what that means, but I'd speculate it's something driver-related. Indeed: # camcontrol devlist -v scbus0 on ciss0 bus 0: scbus1 on ciss0 bus 32: at scbus1 target 4 lun 0 (pass0) at scbus1 target 5 lun 0 (pass1) scbus2 on ciss0 bus 33: ... But there are no device names listed there, after passX, or in /dev. I see in the ciss source (https://github.com/freebsd/freebsd/blame/master/sys/dev/ciss/ciss.c#L320) that the VID/DID for this card (0x103C, 0x3239) is listed. I'm new to FreeBSD, but I'm confused as to how the driver supports this card enough to read the drive model numbers, but not enough to actually use the drives. Is there something I'm missing, or is there some other technical reason this controller can't be used? Thank you for your time, Jonathon Reinhart From owner-freebsd-scsi@freebsd.org Thu Nov 19 17:24:57 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA388A32A9D for ; Thu, 19 Nov 2015 17:24:57 +0000 (UTC) (envelope-from sbruno@freebsd.org) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8A27312CE for ; Thu, 19 Nov 2015 17:24:57 +0000 (UTC) (envelope-from sbruno@freebsd.org) Received: from [192.168.200.208] (unknown [50.136.155.142]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 4EF981928D7; Thu, 19 Nov 2015 17:24:56 +0000 (UTC) Subject: Re: ciss(4) HP Smart Array P840 To: freebsd-scsi@freebsd.org References: From: Sean Bruno Message-ID: <564E05E7.8020307@freebsd.org> Date: Thu, 19 Nov 2015 09:24:55 -0800 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2015 17:24:57 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 11/19/15 09:03, Jonathon Reinhart wrote: > Hello all, > > TL;DR: Why isn't the P840 supported by ciss(4)? > I think I was the last person actively maintaining this driver for an employer. HP doesn't have staff to do this support any longer and I no longer work/have this hardware. That's pretty much it as far as I can tell with this product line. Someone has to put in the effort to get farther along with the hardware. > > I tried to setup FreeNAS 9.3 (running 9.3-RELEASE-p25) on an HP > DL380 Gen9, with the P840 RAID controller. I didn't realize that > the P840 is not officially supported by the ciss(4) driver. In my > troubleshooting, I ended up in the same place as documented on this > forums.freenas.org post > : > > >> It's interesting that camcontrol devlist shows the drives, but >> there are no da* or ada* designations for them. I don't know what >> that means, but I'd speculate it's something driver-related. > > Indeed: > > # camcontrol devlist -v scbus0 on ciss0 bus 0: scbus1 on ciss0 bus > 32: at scbus1 target 4 lun 0 > (pass0) at scbus1 target 5 lun 0 > (pass1) scbus2 on ciss0 bus 33: ... > > But there are no device names listed there, after passX, or in > /dev. > ciss(4) has "scsi-like" features which mean that behavior like this is kind of expected. > I see in the ciss source > (https://github.com/freebsd/freebsd/blame/master/sys/dev/ciss/ciss.c#L320) > > that the VID/DID for this card (0x103C, 0x3239) is listed. > Yup. I was asked to add that almost 3 years ago by HP before their team and my employment ended. > I'm new to FreeBSD, but I'm confused as to how the driver supports > this card enough to read the drive model numbers, but not enough > to actually use the drives. Is there something I'm missing, or is > there some other technical reason this controller can't be used? > > Thank you for your time, > > Jonathon Reinhart I'm more than willing to help out here, you might want to push on HP if possible to get someone to reach out to me with details on what needs to be modified to support this controller. I no longer have valid contacts to get this information. :-( sean bcc jonathon -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAEBCgBmBQJWTgXkXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRCQUFENDYzMkU3MTIxREU4RDIwOTk3REQx MjAxRUZDQTFFNzI3RTY0AAoJEBIB78oecn5kYgwH/RAP12sPmAKopLwVWcLFCV1L 29EA6P/8HGasN9Cu3PZ1jL/GV78wJz7KtOSE80jmmJ+f6Dp/T8rMJCUyvA1kw+wU cqziX4KebIBmXUYrvL3jFnIIDdZG0FpmBKmF7OftSFFcrodV51gxwl/j5mUdHY6g KgsZHMnsYv/K+qnlB53EQp01iIZzHEYLWOLhyHNZE6Ve2iUkPbI3vyMpb/pC7vgr q9bUVvgxDZNcjKRPbyJlgVbCsfOPz/i6IsqPWgrhZYoXE3AgEAuvRI+ko/iA3cz9 6e855pfihdu8/YV4yqXuKJmMVEdDUtC4XN4n9n1wgdPFXeVY6DfLolLJeXKmdt0= =Noeq -----END PGP SIGNATURE----- From owner-freebsd-scsi@freebsd.org Thu Nov 19 18:57:25 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C3E9DA33B0E for ; Thu, 19 Nov 2015 18:57:25 +0000 (UTC) (envelope-from fullermd@over-yonder.net) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id AA2BE1D44 for ; Thu, 19 Nov 2015 18:57:25 +0000 (UTC) (envelope-from fullermd@over-yonder.net) Received: by mailman.ysv.freebsd.org (Postfix) id A7F04A33B0B; Thu, 19 Nov 2015 18:57:25 +0000 (UTC) Delivered-To: scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A71D3A33B09; Thu, 19 Nov 2015 18:57:25 +0000 (UTC) (envelope-from fullermd@over-yonder.net) Received: from mail.infocus-llc.com (mail.infocus-llc.com [199.15.120.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 844041D41; Thu, 19 Nov 2015 18:57:25 +0000 (UTC) (envelope-from fullermd@over-yonder.net) Received: from draco.over-yonder.net (c-75-65-60-66.hsd1.ms.comcast.net [75.65.60.66]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.tarragon.infocus-llc.com (Postfix) with ESMTPSA id 3p1qlt0342zJC; Thu, 19 Nov 2015 12:48:42 -0600 (CST) Received: by draco.over-yonder.net (Postfix, from userid 100) id 3p1qls2gcLz2LF; Thu, 19 Nov 2015 12:48:41 -0600 (CST) Date: Thu, 19 Nov 2015 12:48:41 -0600 From: "Matthew D. Fuller" To: "Kenneth D. Merry" Cc: scsi@freebsd.org, current@freebsd.org Subject: Re: CAM Shingled Disk support patches available Message-ID: <20151119184841.GM30248@over-yonder.net> References: <20151118171309.GA3564@mithlond.kdm.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151118171309.GA3564@mithlond.kdm.org> X-Editor: vi X-OS: FreeBSD X-Virus-Scanned: clamav-milter 0.98.7 at mail.tarragon.infocus-llc.com X-Virus-Status: Clean User-Agent: Mutt/1.5.24-fullermd.4 (2015-08-30) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2015 18:57:25 -0000 On Wed, Nov 18, 2015 at 12:13:09PM -0500 I heard the voice of Kenneth D. Merry, and lo! it spake thus: > > Testing and comments are welcome. GELI does explicit handling of each BIO type, so will need to be updated to pass it through (possibly in the form of inverting the default handling?) or it'll just EOPNOTSUPP it, whether the underlying layer does or not. I wouldn't be surprised if there were other geom layers that did similar things. Not meant to be read as some kind of "you need to"; just a comment on a possible [lack of] impact. -- Matthew Fuller (MF4839) | fullermd@over-yonder.net Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream. From owner-freebsd-scsi@freebsd.org Thu Nov 19 19:45:03 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C6E76A3377C for ; Thu, 19 Nov 2015 19:45:03 +0000 (UTC) (envelope-from jonathon.reinhart@gmail.com) Received: from mail-io0-x236.google.com (mail-io0-x236.google.com [IPv6:2607:f8b0:4001:c06::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 98E161D71; Thu, 19 Nov 2015 19:45:03 +0000 (UTC) (envelope-from jonathon.reinhart@gmail.com) Received: by iouu10 with SMTP id u10so102317850iou.0; Thu, 19 Nov 2015 11:45:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=evT6ORqsGoxvAELwqJ+WXnzUXfY0wGFKtoSQsNbSu+o=; b=SUhbviBBy65sYf3b228eywWtsBVp7pgM9rZ5u9BygPqF7/DS8ffqwpAupOMITl3S7R sh7nRy/GnQxDi4LIBoHYxwJYBIQKv05FVDMefKw54LrxVMJDSo8AUOnuaWT5e0n9KL5q RcvLJ/fMTzfJpsFaq/35gVJwKipZgllEeIEy4omJR6904B3S2M6URlX/dtX6+LH9GxPu ZZ+/eXaeX5+LdVeu3UOq+H+yUGLD/5pGEfSgtUo3JvlVpoo9GIyjGUM41OT3x1ZZjxi6 c/4u9DW00NLUPLy7J4NEy61eRFAoKV4HgLmMufRO8oABcHchFDrWutcJNgYjH4mHpbOF EE/w== X-Received: by 10.107.9.18 with SMTP id j18mr9682123ioi.58.1447962302823; Thu, 19 Nov 2015 11:45:02 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.10.67 with HTTP; Thu, 19 Nov 2015 11:44:33 -0800 (PST) In-Reply-To: <564E05E7.8020307@freebsd.org> References: <564E05E7.8020307@freebsd.org> From: Jonathon Reinhart Date: Thu, 19 Nov 2015 14:44:33 -0500 Message-ID: Subject: Re: ciss(4) HP Smart Array P840 To: Sean Bruno Cc: freebsd-scsi@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2015 19:45:04 -0000 On Thu, Nov 19, 2015 at 12:24 PM, Sean Bruno wrote: > On 11/19/15 09:03, Jonathon Reinhart wrote: >> Hello all, >> >> TL;DR: Why isn't the P840 supported by ciss(4)? >> > I think I was the last person actively maintaining this driver for an > employer. HP doesn't have staff to do this support any longer and I > no longer work/have this hardware. > > That's pretty much it as far as I can tell with this product line. > Someone has to put in the effort to get farther along with the hardware. > >> >> I tried to setup FreeNAS 9.3 (running 9.3-RELEASE-p25) on an HP >> DL380 Gen9, with the P840 RAID controller. I didn't realize that >> the P840 is not officially supported by the ciss(4) driver. In my >> troubleshooting, I ended up in the same place as documented on this >> forums.freenas.org post >> : >> >> >>> It's interesting that camcontrol devlist shows the drives, but >>> there are no da* or ada* designations for them. I don't know what >>> that means, but I'd speculate it's something driver-related. >> >> Indeed: >> >> # camcontrol devlist -v scbus0 on ciss0 bus 0: scbus1 on ciss0 bus >> 32: at scbus1 target 4 lun 0 >> (pass0) at scbus1 target 5 lun 0 >> (pass1) scbus2 on ciss0 bus 33: ... >> >> But there are no device names listed there, after passX, or in >> /dev. >> > > ciss(4) has "scsi-like" features which mean that behavior like this is > kind of expected. > >> I see in the ciss source >> (https://github.com/freebsd/freebsd/blame/master/sys/dev/ciss/ciss.c#L320) >> >> > that the VID/DID for this card (0x103C, 0x3239) is listed. >> > > Yup. I was asked to add that almost 3 years ago by HP before their > team and my employment ended. > >> I'm new to FreeBSD, but I'm confused as to how the driver supports >> this card enough to read the drive model numbers, but not enough >> to actually use the drives. Is there something I'm missing, or is >> there some other technical reason this controller can't be used? >> >> Thank you for your time, >> >> Jonathon Reinhart > > > I'm more than willing to help out here, you might want to push on HP > if possible to get someone to reach out to me with details on what > needs to be modified to support this controller. I no longer have > valid contacts to get this information. :-( > > sean > > bcc jonathon Thanks for the quick reply, Sean. I took a quick look at the Linux hpsa driver (which obsoletes cciss) to see if they treat these new cards any differently, and they don't appear to do so, at all. AFAICT every supported board uses the same "SA5_access" method. Which makes me again wonder why, given the FreeBSD driver has the right VID/DID, it doesn't *just work*. I'd probably have to turn on CISS debugging to see if there's any useful info. Unfortunately I don't have the time to do this right now; I just thought I'd see if there was some simple fix I was missing. I don't really have any contacts at HP to push on. You may be able to look at linux/drivers/scsi/hpsa.c for more info. I'm not sure how FreeBSD / Linux sharing works. The code says: Copyright 2014-2015 PMC-Sierra, Inc. Copyright 2000,2009-2015 Hewlett-Packard Development Company, L.P. Questions/Comments/Bugfixes to storagedev@pmcs.com Perhaps you could contact that email address. Jonathon From owner-freebsd-scsi@freebsd.org Thu Nov 19 20:29:51 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EB3E7A33F60 for ; Thu, 19 Nov 2015 20:29:51 +0000 (UTC) (envelope-from ken@kdm.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id CF7501231 for ; Thu, 19 Nov 2015 20:29:51 +0000 (UTC) (envelope-from ken@kdm.org) Received: by mailman.ysv.freebsd.org (Postfix) id CADADA33F5E; Thu, 19 Nov 2015 20:29:51 +0000 (UTC) Delivered-To: scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C90F9A33F5C; Thu, 19 Nov 2015 20:29:51 +0000 (UTC) (envelope-from ken@kdm.org) Received: from mithlond.kdm.org (mithlond.kdm.org [96.89.93.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 86D98122F; Thu, 19 Nov 2015 20:29:50 +0000 (UTC) (envelope-from ken@kdm.org) Received: from mithlond.kdm.org (localhost [127.0.0.1]) by mithlond.kdm.org (8.15.2/8.14.9) with ESMTPS id tAJKTgtl025275 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 19 Nov 2015 15:29:43 -0500 (EST) (envelope-from ken@mithlond.kdm.org) Received: (from ken@localhost) by mithlond.kdm.org (8.15.2/8.14.9/Submit) id tAJKTgHD025274; Thu, 19 Nov 2015 15:29:42 -0500 (EST) (envelope-from ken) Date: Thu, 19 Nov 2015 15:29:42 -0500 From: "Kenneth D. Merry" To: "Matthew D. Fuller" Cc: scsi@freebsd.org, current@freebsd.org Subject: Re: CAM Shingled Disk support patches available Message-ID: <20151119202942.GA25213@mithlond.kdm.org> References: <20151118171309.GA3564@mithlond.kdm.org> <20151119184841.GM30248@over-yonder.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151119184841.GM30248@over-yonder.net> User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [127.0.0.1]); Thu, 19 Nov 2015 15:29:43 -0500 (EST) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS autolearn=ham autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mithlond.kdm.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2015 20:29:52 -0000 On Thu, Nov 19, 2015 at 12:48:41 -0600, Matthew D. Fuller wrote: > On Wed, Nov 18, 2015 at 12:13:09PM -0500 I heard the voice of > Kenneth D. Merry, and lo! it spake thus: > > > > Testing and comments are welcome. > > GELI does explicit handling of each BIO type, so will need to be > updated to pass it through (possibly in the form of inverting the > default handling?) or it'll just EOPNOTSUPP it, whether the underlying > layer does or not. I wouldn't be surprised if there were other geom > layers that did similar things. > > Not meant to be read as some kind of "you need to"; just a comment on > a possible [lack of] impact. You're correct. For GEOM classes like GELI that don't change the layout on disk, passing the BIO_ZONE bio through would be the right thing to do. For those that change the layout (i.e. the lba you write on the virtual disk doesn't match what goes down to the physical disk), like graid or gstripe, I think all we really need to do is just make sure they return EOPNOTSUPP. If someone wants to modify that code to handle shingled disks, they can certainly do that. Ken -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-scsi@freebsd.org Thu Nov 19 23:27:17 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4A9ABA3298B for ; Thu, 19 Nov 2015 23:27:17 +0000 (UTC) (envelope-from scott4long@yahoo.com) Received: from nm24-vm3.bullet.mail.gq1.yahoo.com (nm24-vm3.bullet.mail.gq1.yahoo.com [98.136.217.98]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 179B81A35 for ; Thu, 19 Nov 2015 23:27:16 +0000 (UTC) (envelope-from scott4long@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1447975258; bh=4V0t7A3SA6Pc2J7CHHP7TazKotOia3Ieg1e03Za0HCw=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject; b=oulBqINUMkznmLYAsXQXRYj0TMzaF33GCZ6jrOtIU68pKZ80KFNlRjZ1nedZetsIKfQ5FEYluEM3SoEEbF/lgwLxBDN7IswallBD09mJk8JSmbpype7YGh9puo5iZn1dGr+f9TzNRlGIWTojN0yfJlVCGMd3U9PC62RSPlVedkUnuddCMPUYdH0BDeVN+5QEHdcS1CIS0yXPkxCbofx9A4qQUYlr7lc2XM37EvshAmWd9ZBTQCDeEq48i9lk43yCf82CDX7FTdRV44T2Iy/HwGvmRLKQDoFxj8dSDPCirImFkkrHZWamEfAKY8LvWXwYCzcnKo3JOfS6sMpQy6kIlA== Received: from [98.137.12.56] by nm24.bullet.mail.gq1.yahoo.com with NNFMP; 19 Nov 2015 23:20:58 -0000 Received: from [98.136.164.76] by tm1.bullet.mail.gq1.yahoo.com with NNFMP; 19 Nov 2015 23:20:58 -0000 Received: from [127.0.0.1] by smtp238.mail.gq1.yahoo.com with NNFMP; 19 Nov 2015 23:20:58 -0000 X-Yahoo-Newman-Id: 889640.8541.bm@smtp238.mail.gq1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: T2V8cFIVM1lyvn9p4ReLqFEvACP.huAVQCzgxmJjXWNSaxQ 3N3Rh71hRsdn4OyDuwcVAhsBlfUuvgId_Ps2BmCLFsd7.dUu7SLIL9ESMXTg IWUKC.Up1q..MzhQooZd2dZSJdk9hZEbQmgFUEI2OFjJmEc1Vm8GWpiBpPk2 PqM.lnTRxGNx1TZHKjKj4wSiNwYSE0CuOryFA5kUW8TzIAv5.OstGnYXWrAR ww1nLsocYmk.YeT.7hny3TONT9P87nkbkgO75mNz8cPsummZ60bcrlUzRzk8 J5a.CDcWFj_qvLYwPwpy9wnUoMH11c18FN8p7QhxU3af4FCCX0X.iv.77nyk JHz1UH9d..B1gQ0H29.3aUAzE6u56gCq_SP7aosp20uJzHtJS9Hnwl5AgCtj 3gY1QjJtE9cJtQjhlw04r7QfpUdiMbsjchiJcoU9fk1il81cFLMeZUn6Q6d_ d9zCqg3FqNNEb9ZcqVFRrjrLhDZ1MnbUMw9G2jFJHEccDLw64aYEx6d8_Vlm J0G3YMELdeiYVuz44crM9Hyi81fSqJe.fKEeXljgA X-Yahoo-SMTP: clhABp.swBB7fs.LwIJpv3jkWgo2NU8- Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.1 \(3096.5\)) Subject: Re: ciss(4) HP Smart Array P840 From: Scott Long In-Reply-To: Date: Thu, 19 Nov 2015 16:20:57 -0700 Cc: Sean Bruno , freebsd-scsi@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <78A8F47E-211F-4DE1-8B39-8978A5D9B7F5@yahoo.com> References: <564E05E7.8020307@freebsd.org> To: Jonathon Reinhart X-Mailer: Apple Mail (2.3096.5) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2015 23:27:17 -0000 > On Nov 19, 2015, at 12:44 PM, Jonathon Reinhart = wrote: >=20 > On Thu, Nov 19, 2015 at 12:24 PM, Sean Bruno = wrote: >> On 11/19/15 09:03, Jonathon Reinhart wrote: >>> Hello all, >>>=20 >>> TL;DR: Why isn't the P840 supported by ciss(4)? >>>=20 >> I think I was the last person actively maintaining this driver for an >> employer. HP doesn't have staff to do this support any longer and I >> no longer work/have this hardware. >>=20 >> That's pretty much it as far as I can tell with this product line. >> Someone has to put in the effort to get farther along with the = hardware. >>=20 >>>=20 >>> I tried to setup FreeNAS 9.3 (running 9.3-RELEASE-p25) on an HP >>> DL380 Gen9, with the P840 RAID controller. I didn't realize that >>> the P840 is not officially supported by the ciss(4) driver. In my >>> troubleshooting, I ended up in the same place as documented on this >>> forums.freenas.org post >>> = : >>>=20 >>>=20 >>>> It's interesting that camcontrol devlist shows the drives, but >>>> there are no da* or ada* designations for them. I don't know what >>>> that means, but I'd speculate it's something driver-related. >>>=20 >>> Indeed: >>>=20 >>> # camcontrol devlist -v scbus0 on ciss0 bus 0: scbus1 on ciss0 bus >>> 32: at scbus1 target 4 lun 0 >>> (pass0) at scbus1 target 5 lun 0 >>> (pass1) scbus2 on ciss0 bus 33: ... >>>=20 >>> But there are no device names listed there, after passX, or in >>> /dev. >>>=20 >>=20 >> ciss(4) has "scsi-like" features which mean that behavior like this = is >> kind of expected. >>=20 >>> I see in the ciss source >>> = (https://github.com/freebsd/freebsd/blame/master/sys/dev/ciss/ciss.c#L320)= >>>=20 >>>=20 >> that the VID/DID for this card (0x103C, 0x3239) is listed. >>>=20 >>=20 >> Yup. I was asked to add that almost 3 years ago by HP before their >> team and my employment ended. >>=20 >>> I'm new to FreeBSD, but I'm confused as to how the driver supports >>> this card enough to read the drive model numbers, but not enough >>> to actually use the drives. Is there something I'm missing, or is >>> there some other technical reason this controller can't be used? >>>=20 >>> Thank you for your time, >>>=20 >>> Jonathon Reinhart >>=20 >>=20 >> I'm more than willing to help out here, you might want to push on HP >> if possible to get someone to reach out to me with details on what >> needs to be modified to support this controller. I no longer have >> valid contacts to get this information. :-( >>=20 >> sean >>=20 >> bcc jonathon >=20 > Thanks for the quick reply, Sean. >=20 > I took a quick look at the Linux hpsa driver (which obsoletes cciss) > to see if they treat these new cards any differently, and they don't > appear to do so, at all. AFAICT every supported board uses the same > "SA5_access" method. Which makes me again wonder why, given the > FreeBSD driver has the right VID/DID, it doesn't *just work*. I'd > probably have to turn on CISS debugging to see if there's any useful > info. Since you can see the volumes in camcontrol, the low-level driver and = communications protocol is working. What=E2=80=99s odd at the moment is = that the volumes are returning an inquiry response that CAM doesn=E2=80=99= t fully understand; that=E2=80=99s why you get the pass devices but no = da devices. Can you use camcontrol to do an inquiry request and send me = the results? Something like the following: sudo camcontrol cmd -n pass -u 0 -v -c "12 00 00 00 ff 00" -i 255 - | hd Thanks, Scott From owner-freebsd-scsi@freebsd.org Fri Nov 20 07:44:20 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7F212A339BD for ; Fri, 20 Nov 2015 07:44:20 +0000 (UTC) (envelope-from sthaug@nethelp.no) Received: from bizet.nethelp.no (bizet.nethelp.no [195.1.209.33]) by mx1.freebsd.org (Postfix) with SMTP id BB6801919 for ; Fri, 20 Nov 2015 07:44:19 +0000 (UTC) (envelope-from sthaug@nethelp.no) Received: (qmail 42847 invoked from network); 20 Nov 2015 07:44:10 -0000 Received: from bizet.nethelp.no (HELO localhost) (195.1.209.33) by bizet.nethelp.no with SMTP; 20 Nov 2015 07:44:10 -0000 Date: Fri, 20 Nov 2015 08:44:10 +0100 (CET) Message-Id: <20151120.084410.74714437.sthaug@nethelp.no> To: jonathon.reinhart@gmail.com Cc: freebsd-scsi@freebsd.org, freebsd-drivers@freebsd.org Subject: Re: ciss(4) HP Smart Array P840 From: sthaug@nethelp.no In-Reply-To: References: X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2015 07:44:20 -0000 > TL;DR: Why isn't the P840 supported by ciss(4)? > > > I tried to setup FreeNAS 9.3 (running 9.3-RELEASE-p25) on an HP DL380 > Gen9, with the P840 RAID controller. I didn't realize that the P840 is > not officially supported by the ciss(4) driver. In my troubleshooting, > I ended up in the same place as documented on this forums.freenas.org > post : > > > It's interesting that camcontrol devlist shows the drives, but there are no da* or ada* designations for them. I don't know what that means, but I'd speculate it's something driver-related. > > Indeed: > > # camcontrol devlist -v > scbus0 on ciss0 bus 0: > scbus1 on ciss0 bus 32: > at scbus1 target 4 lun 0 (pass0) > at scbus1 target 5 lun 0 (pass1) > scbus2 on ciss0 bus 33: > ... > > But there are no device names listed there, after passX, or in /dev. This is somewhat interesting to me since I actually have 10.2-STABLE up and running on a HP DL360 Gen9 server with what appears to be the same controller: # uname -a FreeBSD nova.noc.as2116.net 10.2-STABLE FreeBSD 10.2-STABLE #0 r290663M: Tue Nov 10 18:24:51 CET 2015 sthaug@x.y.z:/usr/src/sys/amd64/compile/DNS amd64 # pciconf -lv ... ciss0@pci0:3:0:0: class=0x010400 card=0x21c0103c chip=0x3239103c rev=0x01 hdr=0x00 vendor = 'Hewlett-Packard Company' device = 'Smart Array Gen9 Controllers' class = mass storage subclass = RAID /var/run/dmesg.boot: ... ciss0: port 0x2000-0x20ff mem 0x92c00000-0x92cfffff,0x92d00000-0x92d003ff irq 16 at device 0.0 on pci3 ciss0: PERFORMANT Transport # camcontrol devlist -v scbus0 on ciss0 bus 0: at scbus0 target 0 lun 0 (pass0,da0) scbus1 on ciss0 bus 32: scbus2 on ahcich5 bus 0: at scbus2 target 0 lun 0 (pass1) <> at scbus2 target -1 lun ffffffff () scbus3 on ahciem0 bus 0: at scbus3 target 0 lun 0 (ses0,pass2) <> at scbus3 target -1 lun ffffffff () scbus-1 on xpt0 bus 0: <> at scbus-1 target -1 lun ffffffff (xpt0) # df Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/da0s1a 10143484 1006628 8325380 11% / devfs 1 1 0 100% /dev /dev/da0s1d 25389052 6525188 16832740 28% /usr /dev/da0s1e 10143484 203340 9128668 2% /var /dev/da0s1g 77088700 16245472 54676132 23% /usr/local BIOS seetings that I changed: Disable UEFI, disable x2APIC. Steinar Haug, Nethelp consulting, sthaug@nethelp.no From owner-freebsd-scsi@freebsd.org Fri Nov 20 08:00:26 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DA5B8A33DA7 for ; Fri, 20 Nov 2015 08:00:26 +0000 (UTC) (envelope-from sthaug@nethelp.no) Received: from bizet.nethelp.no (bizet.nethelp.no [195.1.209.33]) by mx1.freebsd.org (Postfix) with SMTP id 2DE4A1E16 for ; Fri, 20 Nov 2015 08:00:25 +0000 (UTC) (envelope-from sthaug@nethelp.no) Received: (qmail 43634 invoked from network); 20 Nov 2015 08:00:24 -0000 Received: from bizet.nethelp.no (HELO localhost) (195.1.209.33) by bizet.nethelp.no with SMTP; 20 Nov 2015 08:00:24 -0000 Date: Fri, 20 Nov 2015 09:00:24 +0100 (CET) Message-Id: <20151120.090024.41664706.sthaug@nethelp.no> To: jonathon.reinhart@gmail.com Cc: freebsd-scsi@freebsd.org, freebsd-drivers@freebsd.org Subject: Re: ciss(4) HP Smart Array P840 From: sthaug@nethelp.no In-Reply-To: <20151120.084410.74714437.sthaug@nethelp.no> References: <20151120.084410.74714437.sthaug@nethelp.no> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2015 08:00:26 -0000 > > > It's interesting that camcontrol devlist shows the drives, but there are no da* or ada* designations for them. I don't know what that means, but I'd speculate it's something driver-related. > > > > Indeed: > > > > # camcontrol devlist -v > > scbus0 on ciss0 bus 0: > > scbus1 on ciss0 bus 32: > > at scbus1 target 4 lun 0 (pass0) > > at scbus1 target 5 lun 0 (pass1) > > scbus2 on ciss0 bus 33: > > ... > > > > But there are no device names listed there, after passX, or in /dev. > > This is somewhat interesting to me since I actually have 10.2-STABLE > up and running on a HP DL360 Gen9 server with what appears to be the > same controller: > > # uname -a > FreeBSD nova.noc.as2116.net 10.2-STABLE FreeBSD 10.2-STABLE #0 r290663M: Tue Nov 10 18:24:51 CET 2015 sthaug@x.y.z:/usr/src/sys/amd64/compile/DNS amd64 > > # pciconf -lv > ... > ciss0@pci0:3:0:0: class=0x010400 card=0x21c0103c chip=0x3239103c rev=0x01 hdr=0x00 > vendor = 'Hewlett-Packard Company' > device = 'Smart Array Gen9 Controllers' > class = mass storage > subclass = RAID Followup: According to HP specifications, the controllers available for DL360 Gen9 are: Either H240ar Smart Host Bus Adapter or P440ar Flexible Smart Array. So this may be a different storage controller, despite having the same PCI ID. Steinar Haug, Nethelp consulting, sthaug@nethelp.no From owner-freebsd-scsi@freebsd.org Fri Nov 20 14:23:45 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CD981A340AA for ; Fri, 20 Nov 2015 14:23:45 +0000 (UTC) (envelope-from papowell@astart.com) Received: from astart2.astart.com (wsip-72-214-30-30.sd.sd.cox.net [72.214.30.30]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9A5EA1FE8 for ; Fri, 20 Nov 2015 14:23:44 +0000 (UTC) (envelope-from papowell@astart.com) Received: from laptop_93.private (localhost [127.0.0.1]) by astart2.astart.com (8.14.9/8.14.9) with ESMTP id tAKENhVe038818 for ; Fri, 20 Nov 2015 06:23:43 -0800 (PST) (envelope-from papowell@astart.com) Message-ID: <564F2CEF.2010001@astart.com> Date: Fri, 20 Nov 2015 06:23:43 -0800 From: Patrick Powell Reply-To: papowell@astart.com Organization: Astart Technologies User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-scsi@freebsd.org Subject: Re: FibreChannel HBAs in target mode tested with isp driver References: <20121205213657.GA6316@nargothrond.kdm.org> In-Reply-To: <20121205213657.GA6316@nargothrond.kdm.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2015 14:23:45 -0000 I have been asked to set up a system which has a fibre channel HBA connecting to an external NAS/SAN/ God knows what Box. Is there a quick read/tutorial/How To on setting this stuff up? Do you have any suggestions on how to get started? -- Patrick Powell Astart Technologies papowell@astart.com 1530 Jamacha Rd, Suite X Network and System San Diego, CA 92019 Consulting 858-874-6543 FAX 858-751-2435 Web: www.astart.com From owner-freebsd-scsi@freebsd.org Fri Nov 20 16:20:32 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 08389A31774; Fri, 20 Nov 2015 16:20:32 +0000 (UTC) (envelope-from jonathon.reinhart@gmail.com) Received: from mail-ig0-x22f.google.com (mail-ig0-x22f.google.com [IPv6:2607:f8b0:4001:c05::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C9369126E; Fri, 20 Nov 2015 16:20:31 +0000 (UTC) (envelope-from jonathon.reinhart@gmail.com) Received: by igcto18 with SMTP id to18so15339580igc.0; Fri, 20 Nov 2015 08:20:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=uiLiDTO88NKRxSB1bvhvxc8PO/exs1DxJH6+Ra8Ki0g=; b=EHVboHxBBFLiBguuN9uHNm84IXnOclNI52U5+aaS9bhS0s5emMWrPFiveAOyw+MOhZ jY9vBV5zAdvSUH2srYc8dzmWUaLMrRVeA2jXeo4bgNPrDliSY5L+s1jEDgupxyrNTo7w 4f7vaoijjO/kUY/tk7DKnOioWgu38AYKvROjOjY7J+N8bwLsGKSyPv7Wn4mejyD5HLqD mPlGizL94EnizZgnMxrylZRZJ5IFbSjjNgmwjmvvUg8HOxtjuKFV123GflOLWWB6Qdo8 pFN8sGoVyZh2a5BP5TIaDkLCJIP3zp0/r9WNGCyRn+kahH3/OUcMU2H9HFXyR5rrsqgC F9Tg== X-Received: by 10.50.70.102 with SMTP id l6mr774975igu.90.1448036431206; Fri, 20 Nov 2015 08:20:31 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.10.67 with HTTP; Fri, 20 Nov 2015 08:20:01 -0800 (PST) In-Reply-To: <20151120.084410.74714437.sthaug@nethelp.no> References: <20151120.084410.74714437.sthaug@nethelp.no> From: Jonathon Reinhart Date: Fri, 20 Nov 2015 11:20:01 -0500 Message-ID: Subject: Re: ciss(4) HP Smart Array P840 To: sthaug@nethelp.no Cc: freebsd-scsi@freebsd.org, freebsd-drivers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2015 16:20:32 -0000 On Fri, Nov 20, 2015 at 2:44 AM, wrote: > This is somewhat interesting to me since I actually have 10.2-STABLE > up and running on a HP DL360 Gen9 server with what appears to be the > same controller: I forgot to mention that, because I was trying to use ZFS (on FreeNAS), I had put the controller into HBA mode. Sorry about that. > # uname -a > FreeBSD nova.noc.as2116.net 10.2-STABLE FreeBSD 10.2-STABLE #0 r290663M: Tue Nov 10 18:24:51 CET 2015 sthaug@x.y.z:/usr/src/sys/amd64/compile/DNS amd64 Again, I'm running 9.3-RELEASE-p25, but the release notes indicate no difference in hardware support for the ciss driver: https://www.freebsd.org/releases/9.3R/hardware.html#support https://www.freebsd.org/releases/10.2R/hardware.html#disk > BIOS seetings that I changed: Disable UEFI, disable x2APIC. I had to change these same settings as well. On Fri, Nov 20, 2015 at 3:00 AM, wrote: > Followup: According to HP specifications, the controllers available for > DL360 Gen9 are: Either H240ar Smart Host Bus Adapter or P440ar Flexible > Smart Array. So this may be a different storage controller, despite > having the same PCI ID. Mine is a DL380 Gen9, not a DL360. Specifically, it is a 779559-S01, which comes with the P840ar/4GB: http://h20195.www2.hp.com/v2/getpdf.aspx/c04375627.pdf From owner-freebsd-scsi@freebsd.org Fri Nov 20 16:50:16 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CA64EA31F08 for ; Fri, 20 Nov 2015 16:50:16 +0000 (UTC) (envelope-from jonathon.reinhart@gmail.com) Received: from mail-io0-x244.google.com (mail-io0-x244.google.com [IPv6:2607:f8b0:4001:c06::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 950381297; Fri, 20 Nov 2015 16:50:16 +0000 (UTC) (envelope-from jonathon.reinhart@gmail.com) Received: by ioek191 with SMTP id k191so11604726ioe.3; Fri, 20 Nov 2015 08:50:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; bh=YOmcETfIRs0KcohfM1HsL5lDPgVpK98+3rvN+bN8z1o=; b=xotsoElHi5eY8ZYFDzLN96keA51kY5gLxIkFpljn2mYBviRWR6VRFIE0HUigdSc7Ba aECLNG5ywWs+SPzU/YsZxgB3fkzwCkzoJdQfWe+RcJjzoB3/i099Q2++hpvHuyBF6Z09 cVuzeUr+P0H2ri3abXJeKQ1I3Yllbee9OzR+/hhIY6P/JuOerw6rXHghLaa0zZu/XTQy XqfQ2hb/1cvFQyAGNdSDv5LWC8RsBaME7IN7XtBhYpFmqGRRu5ZS4FXnZ/Oo9xyNciyI CiDeJ5bIx6hu6G91QIoT05f6msPOg7NTbrXqEyc6NaxqSyFaZ77VgYlYrBA3eyYB1Q/I 511A== X-Received: by 10.107.3.101 with SMTP id 98mr16475292iod.182.1448038215866; Fri, 20 Nov 2015 08:50:15 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.10.67 with HTTP; Fri, 20 Nov 2015 08:49:46 -0800 (PST) In-Reply-To: <78A8F47E-211F-4DE1-8B39-8978A5D9B7F5@yahoo.com> References: <564E05E7.8020307@freebsd.org> <78A8F47E-211F-4DE1-8B39-8978A5D9B7F5@yahoo.com> From: Jonathon Reinhart Date: Fri, 20 Nov 2015 11:49:46 -0500 Message-ID: Subject: Re: ciss(4) HP Smart Array P840 To: Scott Long Cc: Sean Bruno , freebsd-scsi@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2015 16:50:16 -0000 On Thu, Nov 19, 2015 at 6:20 PM, Scott Long wrote: > Since you can see the volumes in camcontrol, the low-level driver and com= munications protocol is working. What=E2=80=99s odd at the moment is that = the volumes are returning an inquiry response that CAM doesn=E2=80=99t full= y understand; that=E2=80=99s why you get the pass devices but no da devices= . Can you use camcontrol to do an inquiry request and send me the results?= Something like the following: > > sudo camcontrol cmd -n pass -u 0 -v -c "12 00 00 00 ff 00" -i 255 - | hd FYI: The controller is in HBA mode. [root@freenas] ~# uname -a FreeBSD foo.example.com 9.3-RELEASE-p25 FreeBSD 9.3-RELEASE-p25 #0 r281084+d3a5bf7: Wed Sep 2 15:00:10 PDT 2015 root@build3.ixsystems.com:/tank/home/jkh/build/FN/objs/os-base/amd64/tank/h= ome/jkh/build/FN/FreeBSD/src/sys/FREENAS.amd64 amd64 [root@freenas] ~# camcontrol devlist -v scbus0 on ciss0 bus 0: scbus1 on ciss0 bus 32: at scbus1 target 4 lun 0 (pass0) at scbus1 target 5 lun 0 (pass1) scbus2 on ciss0 bus 33: scbus3 on camsim0 bus 0: <> at scbus3 target -1 lun -1 () scbus4 on umass-sim0 bus 0: at scbus4 target 0 lun 0 (pass2,da0) at scbus4 target 0 lun 1 (pass3,da1) scbus-1 on xpt0 bus 0: <> at scbus-1 target -1 lun -1 (xpt0) [root@freenas] ~# camcontrol cmd -n pass -u 0 -v -c "12 00 00 00 ff 00" -i 255 - | hd 00000000 1f 00 06 02 5b 00 00 02 41 54 41 20 20 20 20 20 |....[...ATA = | 00000010 53 54 31 30 30 30 44 4d 30 30 33 2d 31 45 52 31 |ST1000DM003-1E= R1| 00000020 43 43 34 36 00 00 00 00 00 00 00 00 00 00 00 00 |CC46..........= ..| 00000030 00 00 00 00 00 00 00 00 00 00 00 a2 1e e0 04 66 |..............= .f| 00000040 04 c0 20 e0 17 61 00 00 00 00 00 00 00 00 00 00 |.. ..a........= ..| 00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |..............= ..| * 000000f0 [root@freenas] ~# camcontrol cmd -n pass -u 1 -v -c "12 00 00 00 ff 00" -i 255 - | hd 00000000 1f 00 06 02 5b 00 00 02 41 54 41 20 20 20 20 20 |....[...ATA = | 00000010 53 54 31 30 30 30 44 4d 30 30 33 2d 31 45 52 31 |ST1000DM003-1E= R1| 00000020 43 43 34 36 00 00 00 00 00 00 00 00 00 00 00 00 |CC46..........= ..| 00000030 00 00 00 00 00 00 00 00 00 00 00 a2 1e e0 04 66 |..............= .f| 00000040 04 c0 20 e0 17 61 00 00 00 00 00 00 00 00 00 00 |.. ..a........= ..| 00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |..............= ..| * 000000f0 [root@freenas] ~# lspci -nnv ... 08:00.0 Serial Attached SCSI controller [0107]: Hewlett-Packard Company Smart Array Gen9 Controllers [103c:3239] (rev 01) Subsystem: Hewlett-Packard Company P840 [103c:21cb] Flags: bus master, fast devsel, latency 0, IRQ 16 Memory at 92c00000 (64-bit, non-prefetchable) Memory at 92d00000 (64-bit, non-prefetchable) I/O ports at 2000 Capabilities: [80] Power Management version 3 Capabilities: [90] MSI: Enable- Count=3D1/32 Maskable- 64bit+ Capabilities: [b0] MSI-X: Enable+ Count=3D64 Masked- Capabilities: [c0] Express Endpoint, MSI 00 [root@freenas] ~# pciconf -lv ... ciss0@pci0:8:0:0: class=3D0x010700 card=3D0x21cb103c chip=3D0x3239103c rev=3D0x01 hdr=3D0x00 vendor =3D 'Hewlett-Packard Company' class =3D mass storage subclass =3D SAS ... From owner-freebsd-scsi@freebsd.org Fri Nov 20 17:24:56 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 133A4A33638; Fri, 20 Nov 2015 17:24:56 +0000 (UTC) (envelope-from jonathon.reinhart@gmail.com) Received: from mail-io0-x234.google.com (mail-io0-x234.google.com [IPv6:2607:f8b0:4001:c06::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C4F1F11E7; Fri, 20 Nov 2015 17:24:55 +0000 (UTC) (envelope-from jonathon.reinhart@gmail.com) Received: by ioc74 with SMTP id 74so130136558ioc.2; Fri, 20 Nov 2015 09:24:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=vIIVJp6wILeBwfes5EPvsb36BOrxxY5QKoXg38T9H7k=; b=dnIiCOtn2Dpu68ywdE7ORkTQUeSkduMVobAPNg3Y71U3r0h7Vws7kUWAjmABPOmA9G BWUeb9AuVyRS++KXj9KRRIN5JmcvL9KWjqjBd91jFJkKbCp6ozeZ2rpgXv7ksJl/eSbm s5T0SmAErKtnrHlGJJVALSfmfdxsGCgaIrWox2rYScdDIgyictmnlQFU53jKpcppukN2 rDRl2JRdCROLMHWEfIW+VR7i9LJQ5Dy+6GaqqOOWSgg81/Mz19irtf7YhTFR8lRcagCV C3fSfB/kgEfYmmOXODWwvL6ln8jqA3eqDatICsF9qncaW8R0oH5Ay2tMtUw7tCYjKQRC CnfQ== X-Received: by 10.107.3.101 with SMTP id 98mr16658984iod.182.1448040295027; Fri, 20 Nov 2015 09:24:55 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.10.67 with HTTP; Fri, 20 Nov 2015 09:24:25 -0800 (PST) In-Reply-To: <20151120.084410.74714437.sthaug@nethelp.no> References: <20151120.084410.74714437.sthaug@nethelp.no> From: Jonathon Reinhart Date: Fri, 20 Nov 2015 12:24:25 -0500 Message-ID: Subject: Re: ciss(4) HP Smart Array P840 To: sthaug@nethelp.no, Sean Bruno , Scott Long Cc: freebsd-scsi@freebsd.org, freebsd-drivers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2015 17:24:56 -0000 On Fri, Nov 20, 2015 at 2:44 AM, wrote: > This is somewhat interesting to me since I actually have 10.2-STABLE > up and running on a HP DL360 Gen9 server with what appears to be the > same controller: For giggles, I booted up FreeBSD 10.2-STABLE (r290274), and it successfully sees the drives (P840, HBA mode). # camcontrol devlist at scbus1 target 4 lun 0 (pass0, da0) at scbus1 target 5 lun 0 (pass0, da0) ... So I guess this is just a FreeBSD 9.3 vs 10.2 issue, and unfortunately FreeNAS uses the former. I'm still unsure why the difference exists, as it seems like ciss is largely unchanged from 9.3 to 10.2. From owner-freebsd-scsi@freebsd.org Fri Nov 20 18:17:21 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5D765A34565 for ; Fri, 20 Nov 2015 18:17:21 +0000 (UTC) (envelope-from scott4long@yahoo.com) Received: from nm25-vm8.bullet.mail.gq1.yahoo.com (nm25-vm8.bullet.mail.gq1.yahoo.com [98.136.217.119]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 298B51305 for ; Fri, 20 Nov 2015 18:17:20 +0000 (UTC) (envelope-from scott4long@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1448043039; bh=mtD9QqSIv5L3hP7kveEEBnaI5PS5C0CmLi3zyl7Ao6I=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject; b=jyfAQQL+UQ1XZSZqcONwCTDb4Q99g+X+h4n8ttF4VvVNBHx1zaYWfFu42xvh2jn9mk5Y7LsvYTIjvbmroroeTbtoFx4t3AXKBKuK9zyuutUYNLjqV1lzqZi0FKGR4nne2EzaHPCowgS/FhIst1rtW53pXVBvC8dM4dnKkQN0Yj8oLEdKV1dV6hB3Qz1WmbxoUTSEwd2zFZDpNHHdQDppKjYoAkc8FHcTYNaaIiQ45jr5IvSWhqX4EdUsi4zx1InGPxqIm+woA2tsVRykSPaX63zrGGOd4fjGxJkpIR6o9/xuhP3AoPN6x42McO4Ut7lBBHv4z7omikpuqDragLszrA== Received: from [98.137.12.190] by nm25.bullet.mail.gq1.yahoo.com with NNFMP; 20 Nov 2015 18:10:39 -0000 Received: from [208.71.42.197] by tm11.bullet.mail.gq1.yahoo.com with NNFMP; 20 Nov 2015 18:10:39 -0000 Received: from [127.0.0.1] by smtp208.mail.gq1.yahoo.com with NNFMP; 20 Nov 2015 18:10:39 -0000 X-Yahoo-Newman-Id: 136727.43299.bm@smtp208.mail.gq1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: ts0lR1cVM1kn_B7wESfwNKiVy4U_DErM9XOD7b6PHSN1wn4 C9kAVZt_vZ.LNhZRxQ2JY0GX_tFZvRE5ixpu5xhwB0PIErwpnjZMK4E6imPe 3p0M7qET0_6IckAOgifvQUZyPv.20S0XpWPRDve1ONcbZxngbHD2_eNYSEdQ sznSZZoRFz0pdpZmFEVPwGwBZxv.b5COUUg0YK00stZdqU9rKDp14NMnfQ2O 8xpfKYJCmQ1pYlxvE.tp9acnlLg7nzopyOsumv9es1.xIkH3ahAvJCPmc3kF bVaBEPQ6JJKzd7AAXOVROq1aLWEDvZbckOaw9bFxEI.OweQT6chX1Kyl.ITv X9ZpCeevJzJjdMOqm4PvpsJcw_KsM5f3GWlasrhBAmI0vi6QeljCyBtCEdVr CDOAJllSCJgRylBlej6D6WpzJ92GOEFAfGraqQdHfwO0Je0rqxKrxEYpaJY0 s_FbymfpelTIZVNn.N0Vb1OmiXg43uZ4yWiyzmGbY.tuu3eP9p8LfqdqzHPI PYqdDhKtBtRVZ2WMVlCkvg.4vrfl3Wuq3NVujZpzoFA-- X-Yahoo-SMTP: clhABp.swBB7fs.LwIJpv3jkWgo2NU8- Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.1 \(3096.5\)) Subject: Re: ciss(4) HP Smart Array P840 From: Scott Long In-Reply-To: Date: Fri, 20 Nov 2015 11:10:38 -0700 Cc: sthaug@nethelp.no, Sean Bruno , freebsd-scsi@freebsd.org, freebsd-drivers@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <30640027-5C49-4AC0-9BAF-7E6208F7DD13@yahoo.com> References: <20151120.084410.74714437.sthaug@nethelp.no> To: Jonathon Reinhart X-Mailer: Apple Mail (2.3096.5) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2015 18:17:21 -0000 Very strange. Thanks for testing FreeBSD 10. The problem seems to be = that the driver thinks that the controller is NOT in JBOD mode. = There=E2=80=99s special code to hide component members in RAID mode; it = changes the first byte of the inquiry response to 0x1f. That change = indeed shows up in the results that you gave me. I don=E2=80=99t see = any significant changes in the ciss driver between 10.x and 11 that = would explain why one driver detects the card mode and the other does = not. Scott > On Nov 20, 2015, at 10:24 AM, Jonathon Reinhart = wrote: >=20 > On Fri, Nov 20, 2015 at 2:44 AM, wrote: >> This is somewhat interesting to me since I actually have 10.2-STABLE >> up and running on a HP DL360 Gen9 server with what appears to be the >> same controller: >=20 > For giggles, I booted up FreeBSD 10.2-STABLE (r290274), and it > successfully sees the drives (P840, HBA mode). >=20 > # camcontrol devlist > at scbus1 target 4 lun 0 (pass0, da0) > at scbus1 target 5 lun 0 (pass0, da0) > ... >=20 > So I guess this is just a FreeBSD 9.3 vs 10.2 issue, and unfortunately > FreeNAS uses the former. > I'm still unsure why the difference exists, as it seems like ciss is > largely unchanged from 9.3 to 10.2. From owner-freebsd-scsi@freebsd.org Fri Nov 20 18:31:48 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8BDF0A34978; Fri, 20 Nov 2015 18:31:48 +0000 (UTC) (envelope-from jonathon.reinhart@gmail.com) Received: from mail-io0-x22c.google.com (mail-io0-x22c.google.com [IPv6:2607:f8b0:4001:c06::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 530D41E64; Fri, 20 Nov 2015 18:31:48 +0000 (UTC) (envelope-from jonathon.reinhart@gmail.com) Received: by iofh3 with SMTP id h3so132531031iof.3; Fri, 20 Nov 2015 10:31:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; bh=/UIIbRwDFAd1kiO7m9Pnc8iqRwihRHCzxnBYALL8olg=; b=q8r4e3apKj8P8k+Zem9GRus1Crpp2qw2iZ+vBm8iSXSF3MTO9axmIufCcTzJdp6wVz cggnxl+7fX7Va1h8L6cdpDJC/WxYMZ1202Jy83MCM9LASznOL3N7qYgwSTsDXFKPKlKY 7vYr+ysEK02bK3b44tcNQpRc2e5BQ+gsOnkAxZvW7XvrvpHZtWU4jqNqeXG+vpibM9f+ 2Fbgbgrv6c7q3NzjE/+XWEq/ej7RySyXKluR/NBn0b/oenpscUkFTbpCo3gn4xBuzKlt O1CCANxfFsrfXRvrtvLojv4R1be2V3Rv/B30nCFhI/PouK9nQ6aLJSV7mZMPf+LprrqI TiZA== X-Received: by 10.107.9.18 with SMTP id j18mr14454701ioi.58.1448044307726; Fri, 20 Nov 2015 10:31:47 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.10.67 with HTTP; Fri, 20 Nov 2015 10:31:18 -0800 (PST) In-Reply-To: <30640027-5C49-4AC0-9BAF-7E6208F7DD13@yahoo.com> References: <20151120.084410.74714437.sthaug@nethelp.no> <30640027-5C49-4AC0-9BAF-7E6208F7DD13@yahoo.com> From: Jonathon Reinhart Date: Fri, 20 Nov 2015 13:31:18 -0500 Message-ID: Subject: Re: ciss(4) HP Smart Array P840 To: Scott Long Cc: sthaug@nethelp.no, Sean Bruno , freebsd-scsi@freebsd.org, freebsd-drivers@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2015 18:31:48 -0000 I've got a little bit of time, and the system opened up, if there are any further tests that you'd like me to try. On Fri, Nov 20, 2015 at 1:10 PM, Scott Long wrote: > Very strange. Thanks for testing FreeBSD 10. The problem seems to be th= at the driver thinks that the controller is NOT in JBOD mode. There=E2=80= =99s special code to hide component members in RAID mode; it changes the fi= rst byte of the inquiry response to 0x1f. That change indeed shows up in t= he results that you gave me. I don=E2=80=99t see any significant changes i= n the ciss driver between 10.x and 11 that would explain why one driver det= ects the card mode and the other does not. From owner-freebsd-scsi@freebsd.org Fri Nov 20 19:16:13 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 664A3A32097 for ; Fri, 20 Nov 2015 19:16:13 +0000 (UTC) (envelope-from scott4long@yahoo.com) Received: from nm1-vm1.bullet.mail.gq1.yahoo.com (nm1-vm1.bullet.mail.gq1.yahoo.com [98.136.218.80]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 31C0A11CF for ; Fri, 20 Nov 2015 19:16:13 +0000 (UTC) (envelope-from scott4long@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1448046966; bh=ysVcGDfNbBAHYXHqnnsZp3pKELZlYV3+xiYTK/nRrdw=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject; b=jGMpo3b8m+Ybz3/rcY+8vy7QdqSMMMSVnjn+u2BKIk51coZayjmLv4ZnSvJatnA4kaFw/v5Cixogvbav9z9HrVkjFEigm0h+OT/kpOUztTpOyhz5i5g/pKRPg9WUbMFUsMdseHDPjzGZuaTIP4A6tx2ZnyQX+2znqgZIzLgoZhR3j+Fhpqb1Opf9ks4TRgTV4AiGje2cDMCd33eH2k/QRgilxGSBVA2WT9L+hagHEYRDLTxER9Ew/WRN9Q8QfrSTs842kjmeeNY7QQCyfNPHZxHdnx81JMCNYpz+f0CzfCGUJVDc5VVtA9kztiRHEXx3oUjv5MLIt1IijAtEqsFVmg== Received: from [98.137.12.59] by nm1.bullet.mail.gq1.yahoo.com with NNFMP; 20 Nov 2015 19:16:06 -0000 Received: from [208.71.42.190] by tm4.bullet.mail.gq1.yahoo.com with NNFMP; 20 Nov 2015 19:16:06 -0000 Received: from [127.0.0.1] by smtp201.mail.gq1.yahoo.com with NNFMP; 20 Nov 2015 19:16:06 -0000 X-Yahoo-Newman-Id: 276527.46281.bm@smtp201.mail.gq1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: HB1wOQcVM1mY8d_A1yVVYMHK1xLpy_zEHX5w8lqZ2mJ7UEF yxbEc2.cymhgmwE75hrLWZF6U2cbpTNHzyEAMlzP993LgYGzy86zM6SQuWV6 N9B9j8iisxFEgBr5G5DLwVtsiD32LLzeto38L4yjg1mcb.ATN4wlTNJrTeqJ NPtavkwjXbmevqeR8DGhlujx07HLbCCg4vqhs90BL1K0FuSnd1wyzoEc1ZHb sNvLFV1ZehpJOj3go2k_EXE_aXGCG86y3E3TPmJNX9UVcVvnQRdTKUVwXDR1 A3q4hMREh5exPv0r1TOdcrq14hrvp9DLqjOQRvl96PCXurMCmbvjSrUH.9t6 HHdGh20TOYw7ltVvOQL7Wz4S.Z6602PuZxBb6M7D6P9bk0o1DO4rzUJdvYLe oGq_UKEUTQuitRdI7PLwdzsxvp8vgLpYV2yIrkhovHcIL5W6_DB3evrJeMDl EBXEK.zDYMnq2DrnQqaa.o2FYns4GI7UDXGN0JyY7BBkOETNqj.P2LczJikj YzR9oSoXAt7D4kVNwxscbDCQpmtUzEhsdiE67KJNHSQgAvpt4ETFXLWxV1oN tgWEYKJ.h02Hk X-Yahoo-SMTP: clhABp.swBB7fs.LwIJpv3jkWgo2NU8- Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (1.0) Subject: Re: ciss(4) HP Smart Array P840 From: Scott Long X-Mailer: iPhone Mail (13B143) In-Reply-To: Date: Fri, 20 Nov 2015 12:16:04 -0700 Cc: sthaug@nethelp.no, Sean Bruno , freebsd-scsi@freebsd.org, freebsd-drivers@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <20151120.084410.74714437.sthaug@nethelp.no> <30640027-5C49-4AC0-9BAF-7E6208F7DD13@yahoo.com> To: Jonathon Reinhart X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2015 19:16:13 -0000 Hi jonathon, Like Sean, I haven't worked on the driver in years, so my knowledge is rusty= . If you want to dig around in the code, you could try disabling the 'fixup= ' function in sys/deb/ciss/ciss.c that gets called on command completion. Scott Sent from my iPhone > On Nov 20, 2015, at 11:31 AM, Jonathon Reinhart wrote: >=20 > I've got a little bit of time, and the system opened up, if there are > any further tests that you'd like me to try. >=20 >> On Fri, Nov 20, 2015 at 1:10 PM, Scott Long wrote:= >> Very strange. Thanks for testing FreeBSD 10. The problem seems to be th= at the driver thinks that the controller is NOT in JBOD mode. There=E2=80=99= s special code to hide component members in RAID mode; it changes the first b= yte of the inquiry response to 0x1f. That change indeed shows up in the res= ults that you gave me. I don=E2=80=99t see any significant changes in the c= iss driver between 10.x and 11 that would explain why one driver detects the= card mode and the other does not. From owner-freebsd-scsi@freebsd.org Fri Nov 20 19:33:20 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 77FF4A32581; Fri, 20 Nov 2015 19:33:20 +0000 (UTC) (envelope-from jonathon.reinhart@gmail.com) Received: from mail-io0-x22d.google.com (mail-io0-x22d.google.com [IPv6:2607:f8b0:4001:c06::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4394B1BFA; Fri, 20 Nov 2015 19:33:20 +0000 (UTC) (envelope-from jonathon.reinhart@gmail.com) Received: by iouu10 with SMTP id u10so136037781iou.0; Fri, 20 Nov 2015 11:33:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=Z5sUCLee41escSAjow/7sy2CZpwuIwYxOljEVKmdst0=; b=pwsPFuVKjTDErofMebUSddlcc0XK0PbZeKmvDOajSmtI2ZW12l/Pm/JDJWW5DSzpni RZm6/9fMa5iciGkIu+h40TOBkzvzmA61DkM/aMCUV69af7Upj8PTgaa2v+dC2KmdcYMQ //+IenBW06mdfxfYIpCoJjByWowkuDbSCSE0obrQz914qrzEVRgPzTaEaCuiT2LtreC9 5+AbTzcSa9ShXImephpu4n9YNeDiZmYouXWBBo4XS0CLC5pkCyhWMHJloY42jxYhartp RK2+46wFnundSzBoHqNPefznyJNSpjDhf5uzzJA0Z9KFDK7k4nxj6WCLaHmCefvE8YKW XogA== X-Received: by 10.107.3.101 with SMTP id 98mr17263653iod.182.1448047999695; Fri, 20 Nov 2015 11:33:19 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.10.67 with HTTP; Fri, 20 Nov 2015 11:32:50 -0800 (PST) In-Reply-To: References: <20151120.084410.74714437.sthaug@nethelp.no> <30640027-5C49-4AC0-9BAF-7E6208F7DD13@yahoo.com> From: Jonathon Reinhart Date: Fri, 20 Nov 2015 14:32:50 -0500 Message-ID: Subject: Re: ciss(4) HP Smart Array P840 To: Scott Long Cc: sthaug@nethelp.no, Sean Bruno , freebsd-scsi@freebsd.org, freebsd-drivers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2015 19:33:20 -0000 Since FreeNAS 10 will presumably be based on FreeBSD 10 (I haven't seen any solid information about how they track versions), I will probably let this issue go for now. Thanks everyone, for your time, and for the education. From owner-freebsd-scsi@freebsd.org Fri Nov 20 22:41:11 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 73FD1A34C24 for ; Fri, 20 Nov 2015 22:41:11 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 60B3914CF for ; Fri, 20 Nov 2015 22:41:11 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id 5BAD5A34C22; Fri, 20 Nov 2015 22:41:11 +0000 (UTC) Delivered-To: scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5AE0AA34C20; Fri, 20 Nov 2015 22:41:11 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from gw.catspoiler.org (unknown [IPv6:2602:304:b010:ef20::f2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gw.catspoiler.org", Issuer "gw.catspoiler.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 38BF514CA; Fri, 20 Nov 2015 22:41:11 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.15.2/8.15.2) with ESMTP id tAKMf1KS086433; Fri, 20 Nov 2015 14:41:05 -0800 (PST) (envelope-from truckman@FreeBSD.org) Message-Id: <201511202241.tAKMf1KS086433@gw.catspoiler.org> Date: Fri, 20 Nov 2015 14:41:01 -0800 (PST) From: Don Lewis Subject: Re: CAM Shingled Disk support patches available To: ken@FreeBSD.ORG cc: fullermd@over-yonder.net, current@freebsd.org, scsi@freebsd.org In-Reply-To: <20151119202942.GA25213@mithlond.kdm.org> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2015 22:41:11 -0000 On 19 Nov, Kenneth D. Merry wrote: > On Thu, Nov 19, 2015 at 12:48:41 -0600, Matthew D. Fuller wrote: >> On Wed, Nov 18, 2015 at 12:13:09PM -0500 I heard the voice of >> Kenneth D. Merry, and lo! it spake thus: >> > >> > Testing and comments are welcome. >> >> GELI does explicit handling of each BIO type, so will need to be >> updated to pass it through (possibly in the form of inverting the >> default handling?) or it'll just EOPNOTSUPP it, whether the underlying >> layer does or not. I wouldn't be surprised if there were other geom >> layers that did similar things. >> >> Not meant to be read as some kind of "you need to"; just a comment on >> a possible [lack of] impact. > > You're correct. For GEOM classes like GELI that don't change the layout on > disk, passing the BIO_ZONE bio through would be the right thing to do. > > For those that change the layout (i.e. the lba you write on the virtual > disk doesn't match what goes down to the physical disk), like graid or partitioning or > gstripe, I think all we really need to do is just make sure they return > EOPNOTSUPP. If someone wants to modify that code to handle shingled disks, > they can certainly do that. > > Ken