From owner-freebsd-drivers@FreeBSD.ORG Wed Sep 19 15:26:47 2012 Return-Path: Delivered-To: freebsd-drivers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5BDB5106566C for ; Wed, 19 Sep 2012 15:26:47 +0000 (UTC) (envelope-from jacks.1785@gmail.com) Received: from mail-ie0-f182.google.com (mail-ie0-f182.google.com [209.85.223.182]) by mx1.freebsd.org (Postfix) with ESMTP id 206998FC0C for ; Wed, 19 Sep 2012 15:26:47 +0000 (UTC) Received: by iea17 with SMTP id 17so2281392iea.13 for ; Wed, 19 Sep 2012 08:26:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=i16vvXF28hSTiWO0D15SII5cbQW8+iJxGG7DqUMsOwI=; b=AEiQQy0vEGT/pRr0V7NdjFRCyGVg3ijUcrbld7YfClxJEOUDh2n2Jxflo715IyZ9hE xFJhjRWM9vFHrlTLYQVfIdd2uQlX7voNCK950x5wsQGr4tLGmC+c8IlIZaK5JPdobmOA TYwO29kUtqPRG/Q73M4QtV3LpLCNhHvguj21AkAedUMDbmbyF6+8/D14AS87C9G1ri9h iCl4FNttSvjv2OG2yZkINKh4vJKENAQLCJjBFN23kaxZlbolf+05zJFIdZQOYLUgZ+OF wCsXF9I5b9K0yYapahiXLLn1JZCnbbyyU8cozDPyfCr8TL6N4M2mojcTXse44eXdCrN0 we9w== MIME-Version: 1.0 Received: by 10.50.181.168 with SMTP id dx8mr3161533igc.8.1348068406510; Wed, 19 Sep 2012 08:26:46 -0700 (PDT) Received: by 10.64.44.105 with HTTP; Wed, 19 Sep 2012 08:26:46 -0700 (PDT) In-Reply-To: References: <201209190825.07384.jhb@freebsd.org> <5059CC56.8040705@FreeBSD.org> Date: Wed, 19 Sep 2012 20:56:46 +0530 Message-ID: From: Jack To: freebsd-drivers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Re: Exclusive access of SCSI/ATA devices from user space X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Sep 2012 15:26:47 -0000 On Wed, Sep 19, 2012 at 8:53 PM, Jack wrote: > On Wed, Sep 19, 2012 at 7:14 PM, Alexander Motin wrote: >> On 19.09.2012 15:25, John Baldwin wrote: >>> >>> On Wednesday, September 19, 2012 8:08:41 am jack sparrow wrote: >>>> >>>> Hi all, >>>> >>>> I'm developing a userland/usermode utility for FreeBSD. The utility >>>> description follows. >>>> >>>> The utility have options to select the hard drives and tape drives >>>> attached to the system. >>>> Once the user selects a tape drive or hard drive, he/she can read, >>>> write, reset, standby(in case of ATA), idle(in case of ATA), start, >>>> stop, issue identify device(in case of ATA), and inquiry in case of >>>> SCSI device. >>>> >>>> Now the point is, I need to make sure that this device must not be >>>> mounted at all, so that user can't directly read/write to the device. >>>> >>>> Another constraint is that there must not be sorting/scheduling of >>>> disk/drive I/O requests, not even at the kernel level. >>>> e.g. If user read 10 LBAs beginning from LBA 5000 say, and the LBA >>>> 1000, and then LBA 6000, then the i/o requests must be passed in this >>>> same order to the disk/drive - ie there must not be reordering of disk >>>> i/o requests, and the kernel must not cache/buffer the read/write data >>>> . >>>> >>>> Another constraint is that only the utility's process/thread must be >>>> able to access that drive, and no other userland process. >>>> >>>> >>>> I do not need filesystem access, just raw sector/byte read/write, for >>>> the selected drive. >>>> >>>> >>>> Till now after reading docs and books related to FreeBSD, it seems >>>> that I need to write a kernel mode driver(in form of kernel module) >>>> for the purpose, that will communicate with userland utility via a >>>> custom protocol. This driver will attach itself to the drive that is >>>> selected by the user from utility. This driver need not concerned for >>>> other devices except the selected one, as kernel has already >>>> drivers/modules for this. >>>> >>>> This driver will issue appropriate commands to the device selected, >>>> and read/write the LBAs directly from/to userland utility buffer. >>>> >>>> It seems I may need to write a custom system call too. Am I right? >>>> >>>> The point is, I need direct control of the device selected. The other >>>> devices can be read/write normally as they would be. >>>> For e.g. say the utility reads LBA 5000 from the selected device, and >>>> somehow device failed to respond after a timeout, then the drive will >>>> issue reset command to the device. >>>> The LBA read will be passed to userland process, and no disk i/o >>>> scheduling or sorting must be done on the i/o requests made by >>>> userland process. >>>> >>>> In case of ATA disk drives, I also need to control that whether the >>>> data transfer will be via PIO or DMA transfers. >>>> >>>> Can someone enlighten me how could I begin with? I don't know at which >>>> I/O layer such a driver should sit - just above CAM peripheral layer, >>>> or at CAM peripheral layer or CAM transport layer, or, at just above >>>> raw device layer, or at raw device layer itself, or something else. >>>> >>>> Also I wanna know is there other way that goes w/o writing such a >>>> driver, so that only userland code will do the work effectively. >>>> >>>> I'm targetting FreeBSD 9.0-RELEASE and onwards. >>> >>> >>> My first take would be to use a custom GEOM that claims exclusive access >>> to >>> the disk (that prevents mounting, etc.). However, that still sits above >>> the driver layer, and some of the things you want to do really depend on >>> the controller's driver (e.g. I/O sorting is typically done in the >>> controller >>> driver via bioq_disksort(), as are decisions like PIO vs DMA). >>> >>> I don't think you need a custom system call btw. The easiest thing to do >>> is to export a new file in /dev/ for each disk and userland applications >>> can >>> just use read/write against that directly. This should be easy to do with >>> a GEOM module. >>> >>> It may be that you can even get the desired semantics you want if you hack >>> on each controller driver to accept custom GEOM control requests (that >>> would >>> come from your module) to do things like disable any sorting and toggle >>> PIO vs DMA. You might have to hack on the CAM da/ada drivers to pass >>> those >>> requests down to the underlying sim as well as the controller drivers. >>> I'm >>> not sure if we have any similar control requests like that in place >>> already >>> that you could use as a reference. >> >> >> Jack. You've started your mail speaking about SCSI/ATA layer access, but >> finished closer to block device layer. Each device can be accessed both >> ways, depending on your needs. >> >> SCSI/ATA access you may get through the CAM pass driver. It allows you to >> run any SCSI or ATA commands, it does not modify them and does not reorder >> them during normal operation. But CAM does not restrict simultaneous access >> to the devices, so the same device can be accessed through both pass and >> da/ada drivers (or any other if they happen) at the same time. Usually it is >> not a problem because pass interface is used only by short list of tools, >> like ones for CD/DVD recording. >> >> Block access you may get through da/ada CAM drivers and respectfully through >> GEOM. GEOM does controls concurrent device access. If you open some device >> from user-level for writing, nothing else will be able to access it at the >> same time on a block layer (raw access through the pass driver is still >> possible). But before device can be opened, GEOM will do its usual job, >> tasting device for some metadata, partitions, file systems, etc. I don't >> know whether it is a problem, but the only way to prevent that is hack CAM >> transport or da/ada drivers to not attach to specific devices at all, >> leaving only pass interface. Also, as John told, there is a request sorter >> on a top side of ada and da drivers. So if several request sent >> simultaneously, they may be reordered. There is special BIO request flag >> BIO_ORDERED to force original request order, but I don't know way to send it >> from user-level. If you need absolute guarantee of order, you should wait >> for request completion. Neither CAM nor GEOM are caching requests, so that >> will help with ordering, but limit performance. >> >> CAM provides control over ATA PIO/DMA modes via camcontrol tool or >> respective calls via the pass interface. It works for all supported ATA/SATA >> controllers. For SAS controllers it can be problematic, as these controllers >> implement some translation and may not provide such interfaces. >> >> -- >> Alexander Motin > Thank you both, for your responses, I'll need to grasp the approaches you discussed. I'll post the update. Thanks.