From owner-freebsd-hackers Sat Jun 5 13:42:42 1999 Delivered-To: freebsd-hackers@freebsd.org Received: from nomis.simon-shapiro.org (nomis.simon-shapiro.org [209.86.126.163]) by hub.freebsd.org (Postfix) with SMTP id BA94314D75 for ; Sat, 5 Jun 1999 13:42:38 -0700 (PDT) (envelope-from shimon@simon-shapiro.org) Received: (qmail 55845 invoked by uid 1000); 5 Jun 1999 20:46:52 -0000 Message-ID: X-Mailer: XFMail 1.3 [p0] on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 Date: Sat, 05 Jun 1999 16:46:52 -0400 (EDT) X-Face: --Q&x/1^%>&*}<-P_Nc`3TWi's/BcC?8]O1b,;r$#7wzaSo-\/:3NE{Wm=?;i8{R'h.gsd8 %Hz|Y~=sSW"`VLLNW/{>Ap1%:OB*:KP|LD>" =)I@7r|wU}a To: freebsd-hackers@freebsd.org Subject: Major/Minor devices for i2o Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Hi Y'll, Have not been hiding, just working on i2o stuff. Question: How to best map i2o devices to major and minor devices? Background: Currently, all i2o subsystem is handled by one driver. The driver collects i2o HRT and LCT data from all i2o IOPs and builds the appropriate tables. Translated to non-i2o English: The driver build a list of all i2o devices that are visible to the host (IOPs). Each controller builds a table of the controllers it controls (HRT) and of all the logical devices attached to these controllers (LCT). Problem: How do we map all these devices into major/minor? (It has to work on 2.2.8 as well as 3.x and 4.x, preferably with the same solution). Complications: * We need to be able to address individual IOPs (For example, each DPT controller is thre PCI devices and one IOP). * Each IOP has many physical devices (say 126 disks on each FCAL channel). * Each IOP has multiple levels of logical devices. * At most times the physical devices are not interesting, but at other times they are very interesting. Let me illustrate with a complex example: 2 DPT IOPs: 0 - 3 SCSI UW2 busses, each bus with 12 disks and two environmental controllers. 1 - Two FCAL channels, each with 112 drives and 8 environmental controllers. Disks are arranges as follows: On IOP-0 we have 17 RAID-1 arrays. These arrays are arranged as a RAID-0 with 18 strips in it. Two drives are hot spares. On IOP-0 we have 4 RAID-5 arrays (28 drives per array) arranged as a 4 stripe RAID-0. Looking at the physical devices attached we have: i0b0t0 through i0b2t15 (with t13, and t14 empty) i1bot0 through i1b1t113 (to make thing simple. This is already a problem as we have only 8 bits in the minor number. But wait, we also have logical views; one descrives the first level arrays (the RAID-1 and RAID-5, and a second that describes the second level array. Graphically, we have: disk0 disk1 disk2 disk3 ... diskN | | | | +--+--+ +--+--+ | | Array0 Array1 ... ArrayN | | +-----+-----+ | SuperArray0 ... SuperArrayN The i2o driver builds a series of tables and trees to represent this data. Ownership and availability are covered both as part of the i2o spec and as exportable data in the driver. The question is how to best express all this as major/minor numbers. Here are my proposals: Proposal 1: * Major numbers are converted to 16 bit quantities, and masked assignements are permitted. For example, the current major number continues to occupy the lower 8 bits, and a sub-major is allowed in the upper 8 bits. When assigning major numbers, only the lower 8 bits are considred, but the devswitch passes the entire 16 bits to the driver, after masking the upper bits for switching purposes. The driver uses the high bit (or some other bit) to differentiate character (raw) devices from block devices. Example; Major number 0x01fe means second IOP in the (old) major number 126. This will give us up to 16 IOPs. * Minor numbers also become 16 bit wide, 12 bits are used for the TID (Target ID, unique number identifying the I2O device on a given IOP). the upper 4 bits are reserved. This mapping allows for 16 IOPs, and a direct mapping of an i2o TID to a minor number. Proposal 2: * Major numbers are left alone. * The minor number is increased to 16 bits; 12 lower bits to map the TID, 4 upper bits to map the IOP. Again, we will have support for 16 IOPs and streight mapping of TID numbers. When moving to 3.x/4.x, I would recommend supporting the same plus, by default, using devfs to map the devices. For devfs, I suggest the following structure: /dev/i2o - The primary directory which contains all i2o entities. /dev/i2o/0-N - Each IOP gets its own directory. /dev/i2o/N/ctl - A file fo/from which control messages can be written, or IOCTL syscalls be made. /dev/i2o/N/data - If the IOP supports a single device (such as a simple network card), then this is the file representing that device normal data stream. /dev/i2o/N/xxT - For each found TID, a directory which name is the TID number. xx is a string representing the device class. See below. /dev/i2o/N/T/xxctl - Write/read to/from this file sends control messages and receives replies. IOCTL to this file does what one expects IOCTL to do. Control Files (ctl): Allow non-ioctl applications (such as shells) to interact, in a portable manner, with devices. It also bypasses the data length limitation of the IOCTL system call. Device Classes: I2O recognizes the following device classes. On the left is the xx code, on the right a definition. To be more accurate, perhaps we should consider a longer name, that will include a sub-class. ex - Executive, the i2o driver itself. dm - I2o DDM (low level device driver (mostly invisible to us) rb - Random Block Storage; abstracts floppies, IDE, SCSI, RAID disks and other such devices. Proposed sub-devices: da - Direct Access Read/Write (disk, RAID, etc.) ww - Write Once (WORM) cd - CD-ROM op - Optical Storage sq - Sequential Storage (such as tape, etc. Proposed Sub-devices: 12 - i/2inch reel-to-reel qc - QIC Cartridge 34 - 3480 Cartridge 4m - 4mm Digital Storage 8m - 8mm Digital Storage dt - DLT Cartridge la - LAN Controller abstraction wa - WAN Controller abstraction ft - FiberChannel Port abstraction fp - FiberChannel Peripheral abstraction sc - SCSI Peripheral (for SCSI passthrough, etc.) at - ATE Port management ap - ATE Peripheral abstraction fc - Floppy Controller fd - Floppy device ba - Bus Adapter Port pp - Portal to Peer to Peer connections So, with DEVFS, one could have a /dev/i2o/0/0/rbda meaning, the first IOP, TID 0, is a random access device, while /dev/i2o/1/1251/ft may be a fiber channel connection on TID 1251 on the second IOP. Well, this is a lot to chew on, but the time by which FreeBSd will have a preliminary i2o support is nearing rapidly. Any suggestions will be more than welcome. Simon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message