Date: Wed, 19 Sep 2012 20:56:46 +0530 From: Jack <jacks.1785@gmail.com> To: freebsd-drivers@freebsd.org Subject: Re: Exclusive access of SCSI/ATA devices from user space Message-ID: <CACmXQA0CfGwgk%2BdJuhmvwdGXkvsN4WcSRxg2ehT2gw09QibJTQ@mail.gmail.com> In-Reply-To: <CACmXQA3Rg3QAjLHQOd1_mY=8nXEonTuGnUCpZ=WutwpLNRBLmg@mail.gmail.com> References: <CACmXQA09W56rSvVrprD8cuAbZ3T7hFmFKmfREiyXGOZwfpH5=g@mail.gmail.com> <201209190825.07384.jhb@freebsd.org> <5059CC56.8040705@FreeBSD.org> <CACmXQA3Rg3QAjLHQOd1_mY=8nXEonTuGnUCpZ=WutwpLNRBLmg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Sep 19, 2012 at 8:53 PM, Jack <jacks.1785@gmail.com> wrote: > On Wed, Sep 19, 2012 at 7:14 PM, Alexander Motin <mav@freebsd.org> wrote: >> On 19.09.2012 15:25, John Baldwin wrote: >>> >>> On Wednesday, September 19, 2012 8:08:41 am jack sparrow wrote: >>>> >>>> Hi all, >>>> >>>> I'm developing a userland/usermode utility for FreeBSD. The utility >>>> description follows. >>>> >>>> The utility have options to select the hard drives and tape drives >>>> attached to the system. >>>> Once the user selects a tape drive or hard drive, he/she can read, >>>> write, reset, standby(in case of ATA), idle(in case of ATA), start, >>>> stop, issue identify device(in case of ATA), and inquiry in case of >>>> SCSI device. >>>> >>>> Now the point is, I need to make sure that this device must not be >>>> mounted at all, so that user can't directly read/write to the device. >>>> >>>> Another constraint is that there must not be sorting/scheduling of >>>> disk/drive I/O requests, not even at the kernel level. >>>> e.g. If user read 10 LBAs beginning from LBA 5000 say, and the LBA >>>> 1000, and then LBA 6000, then the i/o requests must be passed in this >>>> same order to the disk/drive - ie there must not be reordering of disk >>>> i/o requests, and the kernel must not cache/buffer the read/write data >>>> . >>>> >>>> Another constraint is that only the utility's process/thread must be >>>> able to access that drive, and no other userland process. >>>> >>>> >>>> I do not need filesystem access, just raw sector/byte read/write, for >>>> the selected drive. >>>> >>>> >>>> Till now after reading docs and books related to FreeBSD, it seems >>>> that I need to write a kernel mode driver(in form of kernel module) >>>> for the purpose, that will communicate with userland utility via a >>>> custom protocol. This driver will attach itself to the drive that is >>>> selected by the user from utility. This driver need not concerned for >>>> other devices except the selected one, as kernel has already >>>> drivers/modules for this. >>>> >>>> This driver will issue appropriate commands to the device selected, >>>> and read/write the LBAs directly from/to userland utility buffer. >>>> >>>> It seems I may need to write a custom system call too. Am I right? >>>> >>>> The point is, I need direct control of the device selected. The other >>>> devices can be read/write normally as they would be. >>>> For e.g. say the utility reads LBA 5000 from the selected device, and >>>> somehow device failed to respond after a timeout, then the drive will >>>> issue reset command to the device. >>>> The LBA read will be passed to userland process, and no disk i/o >>>> scheduling or sorting must be done on the i/o requests made by >>>> userland process. >>>> >>>> In case of ATA disk drives, I also need to control that whether the >>>> data transfer will be via PIO or DMA transfers. >>>> >>>> Can someone enlighten me how could I begin with? I don't know at which >>>> I/O layer such a driver should sit - just above CAM peripheral layer, >>>> or at CAM peripheral layer or CAM transport layer, or, at just above >>>> raw device layer, or at raw device layer itself, or something else. >>>> >>>> Also I wanna know is there other way that goes w/o writing such a >>>> driver, so that only userland code will do the work effectively. >>>> >>>> I'm targetting FreeBSD 9.0-RELEASE and onwards. >>> >>> >>> My first take would be to use a custom GEOM that claims exclusive access >>> to >>> the disk (that prevents mounting, etc.). However, that still sits above >>> the driver layer, and some of the things you want to do really depend on >>> the controller's driver (e.g. I/O sorting is typically done in the >>> controller >>> driver via bioq_disksort(), as are decisions like PIO vs DMA). >>> >>> I don't think you need a custom system call btw. The easiest thing to do >>> is to export a new file in /dev/ for each disk and userland applications >>> can >>> just use read/write against that directly. This should be easy to do with >>> a GEOM module. >>> >>> It may be that you can even get the desired semantics you want if you hack >>> on each controller driver to accept custom GEOM control requests (that >>> would >>> come from your module) to do things like disable any sorting and toggle >>> PIO vs DMA. You might have to hack on the CAM da/ada drivers to pass >>> those >>> requests down to the underlying sim as well as the controller drivers. >>> I'm >>> not sure if we have any similar control requests like that in place >>> already >>> that you could use as a reference. >> >> >> Jack. You've started your mail speaking about SCSI/ATA layer access, but >> finished closer to block device layer. Each device can be accessed both >> ways, depending on your needs. >> >> SCSI/ATA access you may get through the CAM pass driver. It allows you to >> run any SCSI or ATA commands, it does not modify them and does not reorder >> them during normal operation. But CAM does not restrict simultaneous access >> to the devices, so the same device can be accessed through both pass and >> da/ada drivers (or any other if they happen) at the same time. Usually it is >> not a problem because pass interface is used only by short list of tools, >> like ones for CD/DVD recording. >> >> Block access you may get through da/ada CAM drivers and respectfully through >> GEOM. GEOM does controls concurrent device access. If you open some device >> from user-level for writing, nothing else will be able to access it at the >> same time on a block layer (raw access through the pass driver is still >> possible). But before device can be opened, GEOM will do its usual job, >> tasting device for some metadata, partitions, file systems, etc. I don't >> know whether it is a problem, but the only way to prevent that is hack CAM >> transport or da/ada drivers to not attach to specific devices at all, >> leaving only pass interface. Also, as John told, there is a request sorter >> on a top side of ada and da drivers. So if several request sent >> simultaneously, they may be reordered. There is special BIO request flag >> BIO_ORDERED to force original request order, but I don't know way to send it >> from user-level. If you need absolute guarantee of order, you should wait >> for request completion. Neither CAM nor GEOM are caching requests, so that >> will help with ordering, but limit performance. >> >> CAM provides control over ATA PIO/DMA modes via camcontrol tool or >> respective calls via the pass interface. It works for all supported ATA/SATA >> controllers. For SAS controllers it can be problematic, as these controllers >> implement some translation and may not provide such interfaces. >> >> -- >> Alexander Motin > Thank you both, for your responses, I'll need to grasp the approaches you discussed. I'll post the update. Thanks.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CACmXQA0CfGwgk%2BdJuhmvwdGXkvsN4WcSRxg2ehT2gw09QibJTQ>