Date: Wed, 07 Dec 2005 21:35:56 -0700 From: Scott Long <scottl@samsco.org> To: Eric Anderson <anderson@centtech.com> Cc: hackers@freebsd.org, Nate Lawson <nate@root.org> Subject: Re: scsi-target and the buffer cache Message-ID: <4397B82C.5020004@samsco.org> In-Reply-To: <4397B731.6010308@centtech.com> References: <4395BF04.50101@centtech.com> <43960F55.3010508@root.org> <43975926.1010302@centtech.com> <43975F5F.5080901@samsco.org> <439782AA.6000408@root.org> <4397B731.6010308@centtech.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Eric Anderson wrote: > Nate Lawson wrote: > >> Scott Long wrote: >> >>> Eric Anderson wrote: >>> >>>> Nate Lawson wrote: >>>> >>>>> Eric Anderson wrote: >>>>> >>>>>> I'm curious about whether a target mode device would use the >>>>>> buffer cache or not. Here's a scenario: >>>>>> >>>>>> Host A: has fibre channel host adapter, in target mode, large >>>>>> memory pool, and another fiber channel host adapter connecting to >>>>>> fibre channel block device. >>>>>> Host B: Fibre channel host adapter, connecting to Host A. 'sees' >>>>>> the target mode block device created by Host A. >>>>>> >>>>>> Will Host A use the buffer cache to cache blocks between the real >>>>>> block device, and the shared target mode device? >>>>>> What about if Host A put a filesystem on the block device, created >>>>>> a single file the size of the filesystem, and shared that >>>>>> filesystem via a target mode device to Host B? >>>>>> What I'm wanting is a box (FreeBSD?) that can be placed between a >>>>>> fibre channel block device (like a RAID array), and a fibre >>>>>> channel host using that block device, and act as a block cache for >>>>>> that device, using the FreeBSD's memory. If it had a significant >>>>>> amount of memory, this could be very useful. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> If you use the example scsi_target usermode >>>>> (usr/share/examples/scsi_target), then the buffer cache will be >>>>> used since its reads/writes are from usermode like normal. If you >>>>> don't want that behavior, you can set O_DIRECT in the open() call >>>>> of the backing store file. >>>>> >>>>> If you chose to modify the kernel side, you'd have to make sure >>>>> your accesses were through the VOP layer and then it would be cached. >>>>> >>>>> You should check to be sure the target mode performance meets your >>>>> expectations also. >>>>> >>>> >>>> I guess I would be using the user mode tool, unless there's another >>>> way? Your comment on performance also makes me a little worried >>>> about that now - do you think I would see a large performance hit? >>>> Thanks! >>>> Eric >>>> >>>> >>> >>> The way the target mode stack works in FreeBSD is that the kernel >>> provides some of the basic services, but the actual target emulator >>> is meant to live in userland. The userland program responds to >>> events from the kernel via the select interface. This generally >>> works pretty well. However, it does mean that control has to >>> cross the kernel-userland boundary at least once for every event. >>> >>> What I'd suggest doing is prototyping your target emulator in userland >>> and evaluating the performance there, and then moving it to the kernel >>> if you _really_ need more performance. >> >> >> >> Agree 100%. While having it in usermode means there are boundary >> crossings that increase per-transaction latency, the actual bulk data >> transfer is via zero-copy IO and you should be able to exceed the data >> transfer rates of several 10K RPM drives on decent hardware. >> > > > Ok, great.. Now, will scsi_target work ok with raw devices, or only > files? (although I'm not sure theres all that much difference really). > > Thanks!! > Eric > > You can write your userland code to use whatever files or devices you want. Are you talking about the scs_target.c code in /usr/share/examples? That's just a skeletal example that you can use as a starting point for your own work. Scott Scott
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4397B82C.5020004>