From owner-freebsd-scsi@FreeBSD.ORG Mon Dec 9 07:44:46 2013 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B1DF9F55 for ; Mon, 9 Dec 2013 07:44:46 +0000 (UTC) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5DE431DAA for ; Mon, 9 Dec 2013 07:44:45 +0000 (UTC) Received: by people.fsn.hu (Postfix, from userid 1001) id A8506124A2C6; Mon, 9 Dec 2013 08:38:43 +0100 (CET) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.3 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MF-ACE0E1EA [pR: 5.6070] X-CRM114-CacheID: sfid-20131209_08384_71489A2F X-CRM114-Status: Good ( pR: 5.6070 ) X-DSPAM-Result: Whitelisted X-DSPAM-Processed: Mon Dec 9 08:38:43 2013 X-DSPAM-Confidence: 0.6954 X-DSPAM-Probability: 0.0000 X-DSPAM-Signature: 52a57383286641441810667 X-DSPAM-Factors: 27, From*Attila Nagy , 0.00010, SCST, 0.00436, From*Attila, 0.00475, Received*online.co.hu+[195.228.243.99]), 0.01000, Received*japan.t, 0.01000, Received*japan.t+online.private, 0.01000, Received*[195.228.243.99]), 0.01000, Received*9+Dec, 0.99000, Received*online.private+(japan.t, 0.01000, Date*09+Dec, 0.99000, Received*online.co.hu, 0.01000, From*Attila+Nagy, 0.01000, Received*from+japan.t, 0.01000, Received*(japan.t, 0.01000, From*Nagy+; Mon, 9 Dec 2013 08:38:42 +0100 (CET) Message-ID: <52A57381.7090501@fsn.hu> Date: Mon, 09 Dec 2013 08:38:41 +0100 From: Attila Nagy MIME-Version: 1.0 To: FreeBSD-SCSI Subject: CTL LUN masking (FC target)? Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Dec 2013 07:44:46 -0000 Hi, Should we expect LUN masking to appear in CTL anytime soon? It would be nice to convert from SCST. Thanks, From owner-freebsd-scsi@FreeBSD.ORG Mon Dec 9 10:03:48 2013 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D67AAE9E for ; Mon, 9 Dec 2013 10:03:48 +0000 (UTC) Received: from mail-la0-x242.google.com (mail-la0-x242.google.com [IPv6:2a00:1450:4010:c03::242]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 611131894 for ; Mon, 9 Dec 2013 10:03:48 +0000 (UTC) Received: by mail-la0-f66.google.com with SMTP id er20so574321lab.1 for ; Mon, 09 Dec 2013 02:03:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=P9p8Tymfg1f1VH92F+TMNktE1puo/obhHy11iYGzLBc=; b=V0AY9ARrgqJ2MZw5vUCjPuZ+tElM4VDzmqVFgIMNemcer5/QKERe9TXhAF0Jf338gZ m4FT0cT5mnQbCwN+37moT38XaItCFd99d9AoWx+pe0xoI7Vy5V/qAZWgCUNuTBPBsoun sdlyrCNJtc43mdUcGMVb4vJh6u8XJmQuZ7m1qOylREmpm5+Z/wl21BVli8WLUCDz3P63 BRPDM8wOQazO8r0QNObqNLEkkD7Jk3ATr35TbgeHXL9BE3zyKwtLJlxctXhC7OtkswaT zC+oKdUapgGWN5qcG3Qf9NfzQ82MFH/7bzUZVOTFgR5mCp2DS0oJsDi3DUou8oSbb84X WKVw== MIME-Version: 1.0 X-Received: by 10.152.180.228 with SMTP id dr4mr4768407lac.32.1386583426331; Mon, 09 Dec 2013 02:03:46 -0800 (PST) Received: by 10.152.224.162 with HTTP; Mon, 9 Dec 2013 02:03:46 -0800 (PST) Date: Mon, 9 Dec 2013 15:33:46 +0530 Message-ID: Subject: NPIV support in freebsd From: Bharat Singh To: freebsd-scsi@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Dec 2013 10:03:48 -0000 Hi, Is there a support for node port_id virtualization (NPIV) in freebsd. I see a lot of other implementations for IBM/Vmware/Solaris, but couldn't find for freebsd. So is it supported in CTL, if not any plans to for the same. Thanks, Bharat From owner-freebsd-scsi@FreeBSD.ORG Mon Dec 9 11:06:53 2013 Return-Path: Delivered-To: freebsd-scsi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 972F0226 for ; Mon, 9 Dec 2013 11:06:53 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 825171E89 for ; Mon, 9 Dec 2013 11:06:53 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id rB9B6r2p071143 for ; Mon, 9 Dec 2013 11:06:53 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id rB9B6rRp071141 for freebsd-scsi@FreeBSD.org; Mon, 9 Dec 2013 11:06:53 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 9 Dec 2013 11:06:53 GMT Message-Id: <201312091106.rB9B6rRp071141@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-scsi@FreeBSD.org Subject: Current problem reports assigned to freebsd-scsi@FreeBSD.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Dec 2013 11:06:53 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/184059 scsi [mps] mps SCSI driver causes FreeBSD to hang during bo o kern/179932 scsi [ciss] ciss i/o stall problem with HP Bl Gen8 (and HP o kern/178795 scsi [mps] MSI for mps driver doesn't work under vmware o kern/165982 scsi [mpt] mpt instability, drive resets, and losses on Fre o kern/165740 scsi [cam] SCSI code must drain callbacks before free f kern/162256 scsi [mpt] QUEUE FULL EVENT and 'mpt_cam_event: 0x0' o docs/151336 scsi Missing documentation of scsi_ and ata_ functions in c o kern/148083 scsi [aac] Strange device reporting o kern/144648 scsi [aac] Strange values of speed and bus width in dmesg o kern/142351 scsi [mpt] LSILogic driver performance problems o kern/134488 scsi [mpt] MPT SCSI driver probes max. 8 LUNs per device o kern/130621 scsi [mpt] tranfer rate is inscrutable slow when use lsi213 f kern/129602 scsi [ahd] ahd(4) gets confused and wedges SCSI bus f kern/123674 scsi [ahc] ahc driver dumping o sparc/121676 scsi [iscsi] iscontrol do not connect iscsi-target on sparc 15 problems total. From owner-freebsd-scsi@FreeBSD.ORG Mon Dec 9 05:50:46 2013 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 905448BA; Mon, 9 Dec 2013 05:50:46 +0000 (UTC) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1lp0150.outbound.protection.outlook.com [207.46.163.150]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 17CF415D4; Mon, 9 Dec 2013 05:50:45 +0000 (UTC) Received: from BN1PR07MB247.namprd07.prod.outlook.com (10.141.64.146) by BN1PR07MB358.namprd07.prod.outlook.com (10.141.60.144) with Microsoft SMTP Server (TLS) id 15.0.837.10; Mon, 9 Dec 2013 05:50:36 +0000 Received: from BN1PR07MB247.namprd07.prod.outlook.com ([169.254.7.70]) by BN1PR07MB247.namprd07.prod.outlook.com ([169.254.7.17]) with mapi id 15.00.0837.004; Mon, 9 Dec 2013 05:50:36 +0000 From: "Desai, Kashyap" To: Doug Ambrisko Subject: RE: LSI - MR-Fusion controller driver patch and man page Thread-Topic: LSI - MR-Fusion controller driver patch and man page Thread-Index: Ac7ykKZzGPABTpK8R3+D5UAv3TD3WQADqpeAAICPhOA= Date: Mon, 9 Dec 2013 05:50:35 +0000 Message-ID: <4ebf0c915b784464827c30858a42d54f@BN1PR07MB247.namprd07.prod.outlook.com> References: <08ba2a262fba45f687cdd3225f325110@BN1PR07MB247.namprd07.prod.outlook.com> <20131206162211.GB14280@cisco.com> In-Reply-To: <20131206162211.GB14280@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.19.239.250] x-forefront-prvs: 00550ABE1F x-forefront-antispam-report: SFV:NSPM; SFS:(199002)(189002)(24454002)(13464003)(164054003)(51704005)(377454003)(81686001)(90146001)(56816005)(87936001)(2656002)(85306002)(80976001)(47976001)(74706001)(76482001)(74316001)(87266001)(74366001)(85852002)(19580405001)(83322001)(81342001)(74876001)(49866001)(19580395003)(81816001)(74662001)(81542001)(59766001)(77982001)(54356001)(66066001)(51856001)(4396001)(80022001)(46102001)(65816001)(79102001)(76576001)(47446002)(54316002)(56776001)(50986001)(47736001)(74502001)(53806001)(83072001)(76786001)(33646001)(63696002)(77096001)(69226001)(31966008)(76796001)(24736002); DIR:OUT; SFP:; SCL:1; SRVR:BN1PR07MB358; H:BN1PR07MB247.namprd07.prod.outlook.com; CLIP:192.19.239.250; FPR:; RD:InfoNoRecords; A:1; MX:1; LANG:en; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: lsi.com X-Mailman-Approved-At: Mon, 09 Dec 2013 16:23:14 +0000 Cc: "scottl@netflix.com" , "Radford, Adam" , "Kenneth D. Merry" , "sean_bruno@yahoo.com" , "Mankani, Krishnaraddi" , "dwhite@ixsystems.com" , "Maloy, Joe" , "jpaetzel@freebsd.org" , "freebsd-scsi@freebsd.org" , "McConnell, Stephen" X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Dec 2013 05:50:46 -0000 > -----Original Message----- > From: Doug Ambrisko [mailto:ambrisko@cisco.com] > Sent: Friday, December 06, 2013 9:52 PM > To: Desai, Kashyap > Cc: freebsd-scsi@freebsd.org; scottl@netflix.com; sean_bruno@yahoo.com; > dwhite@ixsystems.com; jpaetzel@freebsd.org; Maloy, Joe; Mankani, > Krishnaraddi; McConnell, Stephen; Saxena, Sumit; Radford, Adam; Kenneth > D. Merry > Subject: Re: LSI - MR-Fusion controller driver patch and man page >=20 > Sounds like good progress. I'll need to play with it a bit. One questio= n I have > with fast path, is how does LSI determine if they should use that method > versus the RAID firmware? The problem I've seen with fast path is that i= t > skips the NVRAM cache which is a huge performance boost for us. Is there= a > sysctl to control it?=20 Fast Path will not be enabled for Cached VDs. Driver choose Fast Path provi= ded information from firmware. You can find out those bits (fpCapable) in Raid Map. > I could probably read through the code to figure it out but > thought it would be good to get an idea of how LSI plans to use it since = you > guys are the experts on this feature. I understand from Adam that LSI us= es > this to increase the IOPS. I can see it potentially used if an SSD is in= the > system so then LD or PD access could be improved to that type of device. = In > past experience fast path was slower and created a bunch of SCSI sense > errors reported by the RAID firmware. That is one reason why I removed > support of fastpath in mfi. I also didn't want to introduce potential bu= gs if > the RAID firmware and driver got out of sync. due to bugs. Recently we > found with SW RAID that trying to load balance IOs across drives can defe= at > read ahead caches of disks. LSI have never heard anything like above in linux Driver or = any other OS driver for MR controllers. If there is really such a case, we can correct it or provide user option to= bypass load balancing, but as I mentioned, we have never hear anything lik= e that from customer. Thanks, Kashyap >=20 > Thanks, >=20 > Doug A. >=20 > On Fri, Dec 06, 2013 at 02:37:21PM +0000, Desai, Kashyap wrote: > | Hi, > | > | Please consider attached patch for FreeBSD upstream check-in. > | > | Please find attached patch for driver for LSI's MegaRaid > Controllers. This driver supports Thunderbolt onwards Device IDs of MR > controllers. > | Currently it supports 0x005B and 0x005D Device IDs. > | > | NOTE : This driver will not eliminate or by pass any functionality of <= mfi> > driver which already support above to Device IDs to keep existing user > experience unchanged. > | > | Driver will be always given priority over driver and > | only if customer/user wants to use/migrate from to , it > | will hook up into kernel via device.hint rules. (Attached is mrsas man > | page for more info.) > | > | LSI will continue to update driver in future in timely bases. > | We have another set of patch in pipeline to be submitted for , > | but need first go ahead for attached patch and later we will continue > | to keep up-to-date (In sync with LSI released driver which is > | available at lsi's external site) > | > | Apply patch with "patch -p0 < patchname.patch" from head directory. > | > | -- Few notes for user-- > | LSI recommends using fusion_force bit In hint settings at start of the = day, if > they want to use . ( will be a default choice for MR-Fusion > HBA), if will be changed only with fusion_force hint settings. (See mrsas= man > page) Changing any default behavior is well tested for most of the condit= ion. > | Switching from to for MR-Fusion options is designed to > allow user as one time choice, though multiple time switch from to > is possible, it is not recommended. So, user needs to decide from > start of the day, which driver they want to use for MR-Fusion card. > | > | -- Implementation details -- > | To support this feature, we have modify code to change default > return type from probe. Currently driver return > "BUS_PROBE_DEFAULT". driver has been be changed to return > "BUS_PROBE_LOW_PRIORITY" if fusion_force hint from device.hints is set. > | Please notice, above mentioned implementation in driver is only > applicable in case of MR-Fusion controller detection. For any other > controllers, supported by driver, the behavior of probe return will > remain same as before. > | > | > | -- High level feature list of -- 1. Supports Fast Path feature > | of LSI controllers. > | 2. Supports 4K sector Drives. > | 3. CAM layer based interface. All VDs will be attached to CAM layer > | (Expected storage will be visible in "camcontroll") 4. Complete > | support of Online Controller Reset. (OCR) 5. OCR on Fimrware fault and= IO > timeout case. > | 6. Work well with management application which is generic > application provided by LSI for all other Operating system. > | 7. Supporst DIF enabled VDs (Same support as provided in Linux and > | other OSes in FreeBSD) 8. Fast Path Load balance support. > | > | - In summary, this driver is in part with Linux based MR drivers and > | all other features will be available to as planned activity > | from LSI > | > | This code is well tested by LSI Q/A team on 32 bit and 64 bit FreeBSD > Released OSes. > | > | > | Thanks, Kashyap > | >=20 >=20 >=20 From owner-freebsd-scsi@FreeBSD.ORG Mon Dec 9 20:23:08 2013 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1C6D0141 for ; Mon, 9 Dec 2013 20:23:08 +0000 (UTC) Received: from nargothrond.kdm.org (nargothrond.kdm.org [70.56.43.81]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D78DD1DD8 for ; Mon, 9 Dec 2013 20:23:07 +0000 (UTC) Received: from nargothrond.kdm.org (localhost [127.0.0.1]) by nargothrond.kdm.org (8.14.2/8.14.2) with ESMTP id rB9KN6YK079525; Mon, 9 Dec 2013 13:23:06 -0700 (MST) (envelope-from ken@nargothrond.kdm.org) Received: (from ken@localhost) by nargothrond.kdm.org (8.14.2/8.14.2/Submit) id rB9KN6b3079524; Mon, 9 Dec 2013 13:23:06 -0700 (MST) (envelope-from ken) Date: Mon, 9 Dec 2013 13:23:06 -0700 From: "Kenneth D. Merry" To: Bharat Singh Subject: Re: NPIV support in freebsd Message-ID: <20131209202306.GA78605@nargothrond.kdm.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2i Cc: freebsd-scsi@freebsd.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Dec 2013 20:23:08 -0000 On Mon, Dec 09, 2013 at 15:33:46 +0530, Bharat Singh wrote: > Hi, > > Is there a support for node port_id virtualization (NPIV) in freebsd. > I see a lot of other implementations for IBM/Vmware/Solaris, but couldn't > find for freebsd. > So is it supported in CTL, if not any plans to for the same. The Qlogic driver (isp(4)) supports NPIV to some extent, but the firmware currently in the tree for the 8Gb cards at least doesn't seem to support multiple IDs: isp0: port 0xa400-0xa4ff mem 0xfb9b8000-0xfb9bbfff irq 16 at device 0.0 on pci3 isp0: non-MULTIID f/w loaded, only can enable 1 of 5 channels isp1: port 0xa800-0xa8ff mem 0xfb9bc000-0xfb9bffff irq 17 at device 0.1 on pci3 isp1: setting role to 0x1 isp1: setting role to 0x1 isp1: setting role to 0x1 isp1: setting role to 0x1 isp1: setting role to 0x1 isp1: non-MULTIID f/w loaded, only can enable 1 of 5 channels It is pretty much transparent to CTL when it is turned on, because it just looks like there are more frontend ports to CTL. The way you turn it on is setting the number of virtual ports like this in /boot/loader.conf: hint.isp.0.vports=4 hint.isp.1.vports=4 I did some testing with NPIV successfully a few years ago (2009), but I haven't done anything with it since. Matt Jacob probably knows more about the current state. Ken -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-scsi@FreeBSD.ORG Tue Dec 10 02:06:51 2013 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5457FBF9; Tue, 10 Dec 2013 02:06:51 +0000 (UTC) Received: from mail-ve0-x233.google.com (mail-ve0-x233.google.com [IPv6:2607:f8b0:400c:c01::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id F0DE816FF; Tue, 10 Dec 2013 02:06:50 +0000 (UTC) Received: by mail-ve0-f179.google.com with SMTP id jw12so4404994veb.10 for ; Mon, 09 Dec 2013 18:06:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=eK5njQsveflU1z8PNrgDgcMJBZLakiP08AZY01Mdmxc=; b=UrIU/NTuQfKf9PygIlXf1eKQj27o28CD2wdnoSP2oeokXD43KAKdJRWWionOmHkiBZ g/0fmPyuv9uP4tEouC4O8XLl/krDsD4IZ6ClJ8gxN1jIRirlcXx1tLKZFZHl0J/aG2Xd OXVWVSfNdWYDev/Etj3kWdovixsOzg07MTXymgM3wMPDAlWPzW1zPMqbyfnE9OKidkws x3pkMFNflFNxI6w72tp/O05Xkvgu1S5IR6AcuaP42s4cXGGNod+Bfnd3j16g1/X8WYqI ul+Ll3gbnyaguhlDyKeJTCP3L4JSAs8oKfMGCHzt8Z/Obj/I76WycpWzkR/ntLJN5uYK mMlw== X-Received: by 10.220.79.136 with SMTP id p8mr277442vck.54.1386641210167; Mon, 09 Dec 2013 18:06:50 -0800 (PST) MIME-Version: 1.0 Received: by 10.58.200.35 with HTTP; Mon, 9 Dec 2013 18:06:30 -0800 (PST) In-Reply-To: <20131209202306.GA78605@nargothrond.kdm.org> References: <20131209202306.GA78605@nargothrond.kdm.org> From: bharat singh Date: Tue, 10 Dec 2013 07:36:30 +0530 Message-ID: Subject: Re: NPIV support in freebsd To: "Kenneth D. Merry" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-scsi@freebsd.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Dec 2013 02:06:51 -0000 Hi Ken, root@FCNode1:~ # tail -1 /boot/loader.conf hint.isp.0.vports=4 root@FCNode1:~ # ctladm port -l Port Online Type Name pp vp WWNN WWPN Speed Vendor 0 YES IOCTL CTL ioctl 0 0 0 0 0 1 YES INTERNAL ctl2cam 0 0 0x5000000aa3c77b00 0x5000000aa3c77b02 0 2 YES INTERNAL CTL internal 0 0 0 0 0 3 YES FC isp0 0 0 0x2000001b32118e22 0x2100001b32118e22 4000000 Qlogic <<<<< no vports enabled after reboot I tried to add the hint to /boot/device.hints also but still no success. @Matt any comments ? On Tue, Dec 10, 2013 at 1:53 AM, Kenneth D. Merry wrote: > On Mon, Dec 09, 2013 at 15:33:46 +0530, Bharat Singh wrote: > > Hi, > > > > Is there a support for node port_id virtualization (NPIV) in freebsd. > > I see a lot of other implementations for IBM/Vmware/Solaris, but couldn't > > find for freebsd. > > So is it supported in CTL, if not any plans to for the same. > > The Qlogic driver (isp(4)) supports NPIV to some extent, but the > firmware currently in the tree for the 8Gb cards at least doesn't > seem to support multiple IDs: > > isp0: port 0xa400-0xa4ff mem > 0xfb9b8000-0xfb9bbfff irq 16 at device 0.0 on pci3 > isp0: non-MULTIID f/w loaded, only can enable 1 of 5 channels > isp1: port 0xa800-0xa8ff mem > 0xfb9bc000-0xfb9bffff irq 17 at device 0.1 on pci3 > isp1: setting role to 0x1 > isp1: setting role to 0x1 > isp1: setting role to 0x1 > isp1: setting role to 0x1 > isp1: setting role to 0x1 > isp1: non-MULTIID f/w loaded, only can enable 1 of 5 channels > > It is pretty much transparent to CTL when it is turned on, because it just > looks like there are more frontend ports to CTL. > > The way you turn it on is setting the number of virtual ports like this > in /boot/loader.conf: > > hint.isp.0.vports=4 > hint.isp.1.vports=4 > > I did some testing with NPIV successfully a few years ago (2009), but I > haven't done anything with it since. > > Matt Jacob probably knows more about the current state. > > Ken > -- > Kenneth Merry > ken@FreeBSD.ORG > -- Bharat Singh Member Technical Staff, NetApp From owner-freebsd-scsi@FreeBSD.ORG Tue Dec 10 04:41:40 2013 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7B0497C4 for ; Tue, 10 Dec 2013 04:41:40 +0000 (UTC) Received: from nargothrond.kdm.org (nargothrond.kdm.org [70.56.43.81]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 344B51158 for ; Tue, 10 Dec 2013 04:41:39 +0000 (UTC) Received: from nargothrond.kdm.org (localhost [127.0.0.1]) by nargothrond.kdm.org (8.14.2/8.14.2) with ESMTP id rBA4fcax024746; Mon, 9 Dec 2013 21:41:38 -0700 (MST) (envelope-from ken@nargothrond.kdm.org) Received: (from ken@localhost) by nargothrond.kdm.org (8.14.2/8.14.2/Submit) id rBA4fcDH024745; Mon, 9 Dec 2013 21:41:38 -0700 (MST) (envelope-from ken) Date: Mon, 9 Dec 2013 21:41:38 -0700 From: "Kenneth D. Merry" To: bharat singh Subject: Re: NPIV support in freebsd Message-ID: <20131210044138.GA23864@nargothrond.kdm.org> References: <20131209202306.GA78605@nargothrond.kdm.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2i Cc: freebsd-scsi@freebsd.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Dec 2013 04:41:40 -0000 On Tue, Dec 10, 2013 at 07:36:30 +0530, bharat singh wrote: > Hi Ken, > > root@FCNode1:~ # tail -1 /boot/loader.conf > hint.isp.0.vports=4 > > root@FCNode1:~ # ctladm port -l > Port Online Type Name pp vp WWNN WWPN > Speed Vendor > 0 YES IOCTL CTL ioctl 0 0 0 0 > 0 > 1 YES INTERNAL ctl2cam 0 0 0x5000000aa3c77b00 > 0x5000000aa3c77b02 0 > 2 YES INTERNAL CTL internal 0 0 0 0 > 0 > 3 YES FC isp0 0 0 0x2000001b32118e22 > 0x2100001b32118e22 4000000 Qlogic <<<<< no vports enabled after reboot > > I tried to add the hint to /boot/device.hints also but still no success. > @Matt any comments ? It doesn't work for the reason I mentioned below -- the driver says that the firmware included in the driver does not support multiple ports. ("non-MULTIID f/w loaded") I did a quick experiment, and commented out the check, and was apparently able to enable multiple ports in CTL. CTL just saw more target-capable ports. But my FreeBSD initiator had some definite problems connecting to the target with multiple ports enabled. > On Tue, Dec 10, 2013 at 1:53 AM, Kenneth D. Merry wrote: > > > On Mon, Dec 09, 2013 at 15:33:46 +0530, Bharat Singh wrote: > > > Hi, > > > > > > Is there a support for node port_id virtualization (NPIV) in freebsd. > > > I see a lot of other implementations for IBM/Vmware/Solaris, but couldn't > > > find for freebsd. > > > So is it supported in CTL, if not any plans to for the same. > > > > The Qlogic driver (isp(4)) supports NPIV to some extent, but the > > firmware currently in the tree for the 8Gb cards at least doesn't > > seem to support multiple IDs: > > > > isp0: port 0xa400-0xa4ff mem > > 0xfb9b8000-0xfb9bbfff irq 16 at device 0.0 on pci3 > > isp0: non-MULTIID f/w loaded, only can enable 1 of 5 channels > > isp1: port 0xa800-0xa8ff mem > > 0xfb9bc000-0xfb9bffff irq 17 at device 0.1 on pci3 > > isp1: setting role to 0x1 > > isp1: setting role to 0x1 > > isp1: setting role to 0x1 > > isp1: setting role to 0x1 > > isp1: setting role to 0x1 > > isp1: non-MULTIID f/w loaded, only can enable 1 of 5 channels > > > > It is pretty much transparent to CTL when it is turned on, because it just > > looks like there are more frontend ports to CTL. > > > > The way you turn it on is setting the number of virtual ports like this > > in /boot/loader.conf: > > > > hint.isp.0.vports=4 > > hint.isp.1.vports=4 > > > > I did some testing with NPIV successfully a few years ago (2009), but I > > haven't done anything with it since. > > > > Matt Jacob probably knows more about the current state. > > > > Ken > > -- > > Kenneth Merry > > ken@FreeBSD.ORG > > > > > > -- > Bharat Singh > Member Technical Staff, NetApp Ken -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-scsi@FreeBSD.ORG Tue Dec 10 21:09:33 2013 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 540843E7; Tue, 10 Dec 2013 21:09:33 +0000 (UTC) Received: from i3mail.icecube.wisc.edu (i3mail.icecube.wisc.edu [128.104.255.23]) by mx1.freebsd.org (Postfix) with ESMTP id 2751910B7; Tue, 10 Dec 2013 21:09:33 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by i3mail.icecube.wisc.edu (Postfix) with ESMTP id D980638240; Tue, 10 Dec 2013 15:09:31 -0600 (CST) X-Virus-Scanned: amavisd-new at icecube.wisc.edu Received: from i3mail.icecube.wisc.edu ([127.0.0.1]) by localhost (i3mail.icecube.wisc.edu [127.0.0.1]) (amavisd-new, port 10030) with ESMTP id gN79nNLMPvt3; Tue, 10 Dec 2013 15:09:31 -0600 (CST) Received: from terminus.icecube.wisc.edu (terminus.icecube.wisc.edu [172.16.223.97]) by i3mail.icecube.wisc.edu (Postfix) with ESMTPSA id C943038228; Tue, 10 Dec 2013 15:09:31 -0600 (CST) Message-ID: <52A7830B.2090803@freebsd.org> Date: Tue, 10 Dec 2013 15:09:31 -0600 From: Nathan Whitehorn User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: "freebsd-current@freebsd.org" , FreeBSD SCSI Subject: [CAM] Widening lun_id_t to 64-bits Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Dec 2013 21:09:33 -0000 Modern SCSI hardware often uses 64-bit logical units (LUNs). The patch found at http://people.freebsd.org/~nwhitehorn/lun64.diff widens the type of lun_id_t to 64 bits, bumps CAM_VERSION, and begins exposing these to drivers that are marked as supporting extended LUNs. No behavior is changed except that peripheral with very long LUNs that didn't work before will start working. Binary compatibility with old code is also kept. There is, however, a chance that some 3rd party software might be unhappy about the type widening, so I'd appreciate any testing results. Barring any issues, I will commit this on Friday. -Nathan From owner-freebsd-scsi@FreeBSD.ORG Tue Dec 10 23:50:37 2013 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BED31487; Tue, 10 Dec 2013 23:50:37 +0000 (UTC) Received: from smtp.infotech.no (smtp.infotech.no [82.134.31.41]) by mx1.freebsd.org (Postfix) with ESMTP id 7650F1CAE; Tue, 10 Dec 2013 23:50:37 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp.infotech.no (Postfix) with ESMTP id 6A3502041AF; Wed, 11 Dec 2013 00:41:24 +0100 (CET) X-Virus-Scanned: by amavisd-new-2.6.6 (20110518) (Debian) at infotech.no Received: from smtp.infotech.no ([127.0.0.1]) by localhost (smtp.infotech.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5wowa618CACn; Wed, 11 Dec 2013 00:41:22 +0100 (CET) Received: from [10.0.0.132] (142.87.202.84.customer.cdi.no [84.202.87.142]) by smtp.infotech.no (Postfix) with ESMTPA id 3FCA1204079; Wed, 11 Dec 2013 00:41:22 +0100 (CET) Message-ID: <52A7A69E.3030703@interlog.com> Date: Wed, 11 Dec 2013 00:41:18 +0100 From: Douglas Gilbert User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.1.1 MIME-Version: 1.0 To: Nathan Whitehorn , "freebsd-current@freebsd.org" , FreeBSD SCSI Subject: Re: [CAM] Widening lun_id_t to 64-bits References: <52A7830B.2090803@freebsd.org> In-Reply-To: <52A7830B.2090803@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Hannes Reinecke X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: dgilbert@interlog.com List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Dec 2013 23:50:37 -0000 On 13-12-10 10:09 PM, Nathan Whitehorn wrote: > Modern SCSI hardware often uses 64-bit logical units (LUNs). The patch found at > http://people.freebsd.org/~nwhitehorn/lun64.diff widens the type of lun_id_t to > 64 bits, bumps CAM_VERSION, and begins exposing these to drivers that are marked > as supporting extended LUNs. No behavior is changed except that peripheral with > very long LUNs that didn't work before will start working. Binary compatibility > with old code is also kept. There is, however, a chance that some 3rd party > software might be unhappy about the type widening, so I'd appreciate any testing > results. Barring any issues, I will commit this on Friday. Interesting, Hannes Reinecke is trying to do something very similar in the Linux SCSI subsystem. His patch set today will be the third attempt in a year (by my count) and he might just get over the top this time. There is some support in my sg3_utils package for the way Linux is implementing "64 bit LUNs". The sg3_utils package also supports FreeBSD so I'm interested in what your mapping will be. Now, as you are no doubt aware, SCSI (www.t10.org and specifically sam5r15.pdf) does not have 64 bit LUNs, it has 8 byte LUNs in SCSI order (i.e. big endian). Given that major architectures like i386 and x86_64 are little endian, the mapping between a 64 bit integer in native form and an 8 byte SCSI LUN is a bit of a puzzle. That becomes a little harder when you try for low numbered integers representing the T10 3 bit LUNs (showing my age), 8 bit LUNs and 16 bit LUNs. Down to brass tacks: what exactly will a SCSI REPORT LUNS WELL KNOWN LOGICAL UNIT number [T10 (in hex): c1 01 00 00 00 00 00 00] be in one of your 64 bits LUNs? Will that be the same in little endian and big endian architectures? There is also the representation of that LUN in logs; for example lun=13907397124296802304 is not very intuitive. More examples would be great, perhaps from the 4, 6 and 8 byte "extended logical unit addressing format". Robert Elliott who has been a T10 technical editor has written a paper on this subject but google fails me, perhaps someone else can supply the url. His advice was too late for Linux and perhaps it is already too late for FreeBSD. Doug Gilbert P.S. I know Linux has stupid typedefs in its kernel and was hoping FreeBSD would be better. That was until I saw u_int64_t rather than the standard (and shorter) uint64_t From owner-freebsd-scsi@FreeBSD.ORG Tue Dec 10 23:57:23 2013 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 832C85D7; Tue, 10 Dec 2013 23:57:23 +0000 (UTC) Received: from i3mail.icecube.wisc.edu (i3mail.icecube.wisc.edu [128.104.255.23]) by mx1.freebsd.org (Postfix) with ESMTP id 383041D13; Tue, 10 Dec 2013 23:57:22 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by i3mail.icecube.wisc.edu (Postfix) with ESMTP id 9973638249; Tue, 10 Dec 2013 17:57:22 -0600 (CST) X-Virus-Scanned: amavisd-new at icecube.wisc.edu Received: from i3mail.icecube.wisc.edu ([127.0.0.1]) by localhost (i3mail.icecube.wisc.edu [127.0.0.1]) (amavisd-new, port 10030) with ESMTP id qXUK5PLCMgf5; Tue, 10 Dec 2013 17:57:22 -0600 (CST) Received: from terminus.icecube.wisc.edu (terminus.icecube.wisc.edu [172.16.223.97]) by i3mail.icecube.wisc.edu (Postfix) with ESMTPSA id 7FA5238246; Tue, 10 Dec 2013 17:57:22 -0600 (CST) Message-ID: <52A7AA62.8020500@freebsd.org> Date: Tue, 10 Dec 2013 17:57:22 -0600 From: Nathan Whitehorn User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: dgilbert@interlog.com, "freebsd-current@freebsd.org" , FreeBSD SCSI Subject: Re: [CAM] Widening lun_id_t to 64-bits References: <52A7830B.2090803@freebsd.org> <52A7A69E.3030703@interlog.com> In-Reply-To: <52A7A69E.3030703@interlog.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Hannes Reinecke X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Dec 2013 23:57:23 -0000 On 12/10/13 17:41, Douglas Gilbert wrote: > On 13-12-10 10:09 PM, Nathan Whitehorn wrote: >> Modern SCSI hardware often uses 64-bit logical units (LUNs). The >> patch found at >> http://people.freebsd.org/~nwhitehorn/lun64.diff widens the type of >> lun_id_t to >> 64 bits, bumps CAM_VERSION, and begins exposing these to drivers that >> are marked >> as supporting extended LUNs. No behavior is changed except that >> peripheral with >> very long LUNs that didn't work before will start working. Binary >> compatibility >> with old code is also kept. There is, however, a chance that some 3rd >> party >> software might be unhappy about the type widening, so I'd appreciate >> any testing >> results. Barring any issues, I will commit this on Friday. > > Interesting, Hannes Reinecke is trying to do something > very similar in the Linux SCSI subsystem. His patch set > today will be the third attempt in a year (by my count) > and he might just get over the top this time. There is > some support in my sg3_utils package for the way Linux > is implementing "64 bit LUNs". The sg3_utils package > also supports FreeBSD so I'm interested in what your > mapping will be. > > Now, as you are no doubt aware, SCSI (www.t10.org and specifically > sam5r15.pdf) does not have 64 bit LUNs, it has 8 byte LUNs in > SCSI order (i.e. big endian). Given that major architectures > like i386 and x86_64 are little endian, the mapping between > a 64 bit integer in native form and an 8 byte SCSI LUN is > a bit of a puzzle. That becomes a little harder when you try > for low numbered integers representing the T10 3 bit LUNs > (showing my age), 8 bit LUNs and 16 bit LUNs. > > Down to brass tacks: what exactly will a SCSI REPORT LUNS > WELL KNOWN LOGICAL UNIT number [T10 (in hex): c1 01 00 00 > 00 00 00 00] be in one of your 64 bits LUNs? Will that be > the same in little endian and big endian architectures? > There is also the representation of that LUN in logs; for > example lun=13907397124296802304 is not very intuitive. > > More examples would be great, perhaps from the 4, 6 and 8 byte > "extended logical unit addressing format". We're following the path-of-least-resistance from Solaris. I've momentarily forgotten how this works in the Linux patches, but the approach is as follows (this has actually been in HEAD for a couple months now). Extended LUNs are stored in host byte order with swizzled 16-bit word order so that, for devices implementing LUN addressing (like SCSI-2), the numerical representation of the LUN is identical before and after the change. Thus this keeps most behavior, and user-facing LUN IDs, unchanged. A macro (CAM_EXTLUN_BYTE_SWIZZLE) is provided to transform a lun_id_t into a uint64_t ordered for the wire. Most of the kernel prints these in hex as per SAM5. camcontrol prints them out in various ways if it knows the addressing component to which they correspond and otherwise prints them in hex. This seemed like by far the least painful approach: it has a (fairly) simple direct mapping onto the wire format, nothing changes for users, and almost nothing changes for code. > Robert Elliott who has been a T10 technical editor has written > a paper on this subject but google fails me, perhaps someone > else can supply the url. His advice was too late for Linux > and perhaps it is already too late for FreeBSD. Hopefully this corresponds to that advice, whatever it was :) Any suggestions for changes would be appreciated if you have them. -Nathan > Doug Gilbert > > > P.S. I know Linux has stupid typedefs in its kernel and > was hoping FreeBSD would be better. That was until > I saw u_int64_t rather than the standard (and > shorter) uint64_t > CAM is old still, so I've tried to keep the existing style. Updates are probably a good idea. -Nathan From owner-freebsd-scsi@FreeBSD.ORG Tue Dec 10 23:57:49 2013 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BD12A712; Tue, 10 Dec 2013 23:57:49 +0000 (UTC) Received: from smtp.infotech.no (smtp.infotech.no [82.134.31.41]) by mx1.freebsd.org (Postfix) with ESMTP id 77B0E1D1E; Tue, 10 Dec 2013 23:57:49 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp.infotech.no (Postfix) with ESMTP id 5B22D2041AF; Wed, 11 Dec 2013 00:57:48 +0100 (CET) X-Virus-Scanned: by amavisd-new-2.6.6 (20110518) (Debian) at infotech.no Received: from smtp.infotech.no ([127.0.0.1]) by localhost (smtp.infotech.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uUpqQStU5KZT; Wed, 11 Dec 2013 00:57:46 +0100 (CET) Received: from [10.0.0.132] (142.87.202.84.customer.cdi.no [84.202.87.142]) by smtp.infotech.no (Postfix) with ESMTPA id 39795204079; Wed, 11 Dec 2013 00:57:46 +0100 (CET) Message-ID: <52A7AA76.3060208@interlog.com> Date: Wed, 11 Dec 2013 00:57:42 +0100 From: Douglas Gilbert User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.1.1 MIME-Version: 1.0 To: Nathan Whitehorn , FreeBSD SCSI Subject: Re: [CAM] Widening lun_id_t to 64-bits References: <52A7830B.2090803@freebsd.org> <52A7A69E.3030703@interlog.com> In-Reply-To: <52A7A69E.3030703@interlog.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Hannes Reinecke X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: dgilbert@interlog.com List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Dec 2013 23:57:49 -0000 On 13-12-11 12:41 AM, Douglas Gilbert wrote: > On 13-12-10 10:09 PM, Nathan Whitehorn wrote: >> Modern SCSI hardware often uses 64-bit logical units (LUNs). The patch found at >> http://people.freebsd.org/~nwhitehorn/lun64.diff widens the type of lun_id_t to >> 64 bits, bumps CAM_VERSION, and begins exposing these to drivers that are marked >> as supporting extended LUNs. No behavior is changed except that peripheral with >> very long LUNs that didn't work before will start working. Binary compatibility >> with old code is also kept. There is, however, a chance that some 3rd party >> software might be unhappy about the type widening, so I'd appreciate any testing >> results. Barring any issues, I will commit this on Friday. > > Interesting, Hannes Reinecke is trying to do something > very similar in the Linux SCSI subsystem. His patch set > today will be the third attempt in a year (by my count) > and he might just get over the top this time. There is > some support in my sg3_utils package for the way Linux > is implementing "64 bit LUNs". The sg3_utils package > also supports FreeBSD so I'm interested in what your > mapping will be. > > Now, as you are no doubt aware, SCSI (www.t10.org and specifically > sam5r15.pdf) does not have 64 bit LUNs, it has 8 byte LUNs in > SCSI order (i.e. big endian). Given that major architectures > like i386 and x86_64 are little endian, the mapping between > a 64 bit integer in native form and an 8 byte SCSI LUN is > a bit of a puzzle. That becomes a little harder when you try > for low numbered integers representing the T10 3 bit LUNs > (showing my age), 8 bit LUNs and 16 bit LUNs. > > Down to brass tacks: what exactly will a SCSI REPORT LUNS > WELL KNOWN LOGICAL UNIT number [T10 (in hex): c1 01 00 00 > 00 00 00 00] be in one of your 64 bits LUNs? Will that be > the same in little endian and big endian architectures? > There is also the representation of that LUN in logs; for > example lun=13907397124296802304 is not very intuitive. > > More examples would be great, perhaps from the 4, 6 and 8 byte > "extended logical unit addressing format". > > > Robert Elliott who has been a T10 technical editor has written > a paper on this subject but google fails me, perhaps someone > else can supply the url. His advice was too late for Linux > and perhaps it is already too late for FreeBSD. The document I was trying to find was a www.t10.org proposal: 06-003r1, see its overview for a rationale. The Logical Unit Representation Format section was accepted and can be found in sam5r15.pdf section 4.7.2 Doug Gilbert From owner-freebsd-scsi@FreeBSD.ORG Tue Dec 10 23:58:28 2013 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7C865750 for ; Tue, 10 Dec 2013 23:58:28 +0000 (UTC) Received: from i3mail.icecube.wisc.edu (i3mail.icecube.wisc.edu [128.104.255.23]) by mx1.freebsd.org (Postfix) with ESMTP id 4C3091D26 for ; Tue, 10 Dec 2013 23:58:28 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by i3mail.icecube.wisc.edu (Postfix) with ESMTP id CF2B138249; Tue, 10 Dec 2013 17:58:27 -0600 (CST) X-Virus-Scanned: amavisd-new at icecube.wisc.edu Received: from i3mail.icecube.wisc.edu ([127.0.0.1]) by localhost (i3mail.icecube.wisc.edu [127.0.0.1]) (amavisd-new, port 10030) with ESMTP id JIBEeEnzReWK; Tue, 10 Dec 2013 17:58:27 -0600 (CST) Received: from terminus.icecube.wisc.edu (terminus.icecube.wisc.edu [172.16.223.97]) by i3mail.icecube.wisc.edu (Postfix) with ESMTPSA id BACF438246; Tue, 10 Dec 2013 17:58:27 -0600 (CST) Message-ID: <52A7AAA3.5050404@freebsd.org> Date: Tue, 10 Dec 2013 17:58:27 -0600 From: Nathan Whitehorn User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: dgilbert@interlog.com, FreeBSD SCSI Subject: Re: [CAM] Widening lun_id_t to 64-bits References: <52A7830B.2090803@freebsd.org> <52A7A69E.3030703@interlog.com> <52A7AA76.3060208@interlog.com> In-Reply-To: <52A7AA76.3060208@interlog.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Hannes Reinecke X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Dec 2013 23:58:28 -0000 On 12/10/13 17:57, Douglas Gilbert wrote: > On 13-12-11 12:41 AM, Douglas Gilbert wrote: >> On 13-12-10 10:09 PM, Nathan Whitehorn wrote: >>> Modern SCSI hardware often uses 64-bit logical units (LUNs). The >>> patch found at >>> http://people.freebsd.org/~nwhitehorn/lun64.diff widens the type of >>> lun_id_t to >>> 64 bits, bumps CAM_VERSION, and begins exposing these to drivers >>> that are marked >>> as supporting extended LUNs. No behavior is changed except that >>> peripheral with >>> very long LUNs that didn't work before will start working. Binary >>> compatibility >>> with old code is also kept. There is, however, a chance that some >>> 3rd party >>> software might be unhappy about the type widening, so I'd appreciate >>> any testing >>> results. Barring any issues, I will commit this on Friday. >> >> Interesting, Hannes Reinecke is trying to do something >> very similar in the Linux SCSI subsystem. His patch set >> today will be the third attempt in a year (by my count) >> and he might just get over the top this time. There is >> some support in my sg3_utils package for the way Linux >> is implementing "64 bit LUNs". The sg3_utils package >> also supports FreeBSD so I'm interested in what your >> mapping will be. >> >> Now, as you are no doubt aware, SCSI (www.t10.org and specifically >> sam5r15.pdf) does not have 64 bit LUNs, it has 8 byte LUNs in >> SCSI order (i.e. big endian). Given that major architectures >> like i386 and x86_64 are little endian, the mapping between >> a 64 bit integer in native form and an 8 byte SCSI LUN is >> a bit of a puzzle. That becomes a little harder when you try >> for low numbered integers representing the T10 3 bit LUNs >> (showing my age), 8 bit LUNs and 16 bit LUNs. >> >> Down to brass tacks: what exactly will a SCSI REPORT LUNS >> WELL KNOWN LOGICAL UNIT number [T10 (in hex): c1 01 00 00 >> 00 00 00 00] be in one of your 64 bits LUNs? Will that be >> the same in little endian and big endian architectures? >> There is also the representation of that LUN in logs; for >> example lun=13907397124296802304 is not very intuitive. >> >> More examples would be great, perhaps from the 4, 6 and 8 byte >> "extended logical unit addressing format". >> >> >> Robert Elliott who has been a T10 technical editor has written >> a paper on this subject but google fails me, perhaps someone >> else can supply the url. His advice was too late for Linux >> and perhaps it is already too late for FreeBSD. > > The document I was trying to find was a www.t10.org proposal: > 06-003r1, see its overview for a rationale. The Logical Unit > Representation Format section was accepted and can be found > in sam5r15.pdf section 4.7.2 > > Doug Gilbert > > Ah, OK. That we are following. -Nathan From owner-freebsd-scsi@FreeBSD.ORG Wed Dec 11 00:15:49 2013 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4BED3A91; Wed, 11 Dec 2013 00:15:49 +0000 (UTC) Received: from smtp.infotech.no (smtp.infotech.no [82.134.31.41]) by mx1.freebsd.org (Postfix) with ESMTP id C3DDE1E52; Wed, 11 Dec 2013 00:15:48 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp.infotech.no (Postfix) with ESMTP id B6A7D2041AF; Wed, 11 Dec 2013 01:15:47 +0100 (CET) X-Virus-Scanned: by amavisd-new-2.6.6 (20110518) (Debian) at infotech.no Received: from smtp.infotech.no ([127.0.0.1]) by localhost (smtp.infotech.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VfYLsp6jtK1n; Wed, 11 Dec 2013 01:15:45 +0100 (CET) Received: from [10.0.0.132] (142.87.202.84.customer.cdi.no [84.202.87.142]) by smtp.infotech.no (Postfix) with ESMTPA id B1170204079; Wed, 11 Dec 2013 01:15:45 +0100 (CET) Message-ID: <52A7AEAE.1030307@interlog.com> Date: Wed, 11 Dec 2013 01:15:42 +0100 From: Douglas Gilbert User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.1.1 MIME-Version: 1.0 To: Nathan Whitehorn , FreeBSD SCSI Subject: Re: [CAM] Widening lun_id_t to 64-bits References: <52A7830B.2090803@freebsd.org> <52A7A69E.3030703@interlog.com> <52A7AA76.3060208@interlog.com> <52A7AAA3.5050404@freebsd.org> In-Reply-To: <52A7AAA3.5050404@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Hannes Reinecke X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: dgilbert@interlog.com List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Dec 2013 00:15:49 -0000 On 13-12-11 12:58 AM, Nathan Whitehorn wrote: > On 12/10/13 17:57, Douglas Gilbert wrote: >> On 13-12-11 12:41 AM, Douglas Gilbert wrote: >>> On 13-12-10 10:09 PM, Nathan Whitehorn wrote: >>>> Modern SCSI hardware often uses 64-bit logical units (LUNs). The patch found at >>>> http://people.freebsd.org/~nwhitehorn/lun64.diff widens the type of lun_id_t to >>>> 64 bits, bumps CAM_VERSION, and begins exposing these to drivers that are >>>> marked >>>> as supporting extended LUNs. No behavior is changed except that peripheral with >>>> very long LUNs that didn't work before will start working. Binary compatibility >>>> with old code is also kept. There is, however, a chance that some 3rd party >>>> software might be unhappy about the type widening, so I'd appreciate any >>>> testing >>>> results. Barring any issues, I will commit this on Friday. >>> >>> Interesting, Hannes Reinecke is trying to do something >>> very similar in the Linux SCSI subsystem. His patch set >>> today will be the third attempt in a year (by my count) >>> and he might just get over the top this time. There is >>> some support in my sg3_utils package for the way Linux >>> is implementing "64 bit LUNs". The sg3_utils package >>> also supports FreeBSD so I'm interested in what your >>> mapping will be. >>> >>> Now, as you are no doubt aware, SCSI (www.t10.org and specifically >>> sam5r15.pdf) does not have 64 bit LUNs, it has 8 byte LUNs in >>> SCSI order (i.e. big endian). Given that major architectures >>> like i386 and x86_64 are little endian, the mapping between >>> a 64 bit integer in native form and an 8 byte SCSI LUN is >>> a bit of a puzzle. That becomes a little harder when you try >>> for low numbered integers representing the T10 3 bit LUNs >>> (showing my age), 8 bit LUNs and 16 bit LUNs. >>> >>> Down to brass tacks: what exactly will a SCSI REPORT LUNS >>> WELL KNOWN LOGICAL UNIT number [T10 (in hex): c1 01 00 00 >>> 00 00 00 00] be in one of your 64 bits LUNs? Will that be >>> the same in little endian and big endian architectures? >>> There is also the representation of that LUN in logs; for >>> example lun=13907397124296802304 is not very intuitive. >>> >>> More examples would be great, perhaps from the 4, 6 and 8 byte >>> "extended logical unit addressing format". >>> >>> >>> Robert Elliott who has been a T10 technical editor has written >>> a paper on this subject but google fails me, perhaps someone >>> else can supply the url. His advice was too late for Linux >>> and perhaps it is already too late for FreeBSD. >> >> The document I was trying to find was a www.t10.org proposal: >> 06-003r1, see its overview for a rationale. The Logical Unit >> Representation Format section was accepted and can be found >> in sam5r15.pdf section 4.7.2 >> >> Doug Gilbert >> >> > > Ah, OK. That we are following. > -Nathan BTW A T10 troublemaker tried to extend 8 byte LUNs to something longer in order to hold a proposed conglomerate LUN format. He did succeed in getting conglomerate LUN format in but not extending LUNs beyond 8 bytes. However he did get a 65th bit in, sort of. See the new LU_CONG bit in a standard INQUIRY response (spc4r36n.pdf). Doug Gilbert From owner-freebsd-scsi@FreeBSD.ORG Wed Dec 11 07:13:50 2013 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7DE4F77B; Wed, 11 Dec 2013 07:13:50 +0000 (UTC) Received: from mx2.suse.de (cantor2.suse.de [195.135.220.15]) by mx1.freebsd.org (Postfix) with ESMTP id 410821183; Wed, 11 Dec 2013 07:13:49 +0000 (UTC) Received: from relay1.suse.de (unknown [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 85CFFA7D9E; Wed, 11 Dec 2013 07:59:16 +0100 (CET) Message-ID: <52A829F1.2010206@suse.de> Date: Wed, 11 Dec 2013 10:01:37 +0100 From: Hannes Reinecke User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: dgilbert@interlog.com Subject: Re: [CAM] Widening lun_id_t to 64-bits References: <52A7830B.2090803@freebsd.org> <52A7A69E.3030703@interlog.com> <52A7AA76.3060208@interlog.com> <52A7AAA3.5050404@freebsd.org> <52A7AEAE.1030307@interlog.com> In-Reply-To: <52A7AEAE.1030307@interlog.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: FreeBSD SCSI , Nathan Whitehorn X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Dec 2013 07:13:50 -0000 On 12/11/2013 01:15 AM, Douglas Gilbert wrote: [ .. ] > > BTW A T10 troublemaker tried to extend 8 byte LUNs to something > longer in order to hold a proposed conglomerate LUN format. > He did succeed in getting conglomerate LUN format in but not > extending LUNs beyond 8 bytes. However he did get a 65th bit in, > sort of. See the new LU_CONG bit in a standard INQUIRY > response (spc4r36n.pdf). > Gnaa. And I know that troublemaker. His company has been approaching me for conglomerate LUN support. _This_ will get fun. And I guess I need to update the patchset to handle this ... Cheers, Hannes -- Dr. Hannes Reinecke zSeries & Storage hare@suse.de +49 911 74053 688 SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg)