Date: Wed, 07 Dec 2005 22:31:45 -0600 From: Eric Anderson <anderson@centtech.com> To: Nate Lawson <nate@root.org> Cc: Scott Long <scottl@samsco.org>, hackers@freebsd.org Subject: Re: scsi-target and the buffer cache Message-ID: <4397B731.6010308@centtech.com> In-Reply-To: <439782AA.6000408@root.org> References: <4395BF04.50101@centtech.com> <43960F55.3010508@root.org> <43975926.1010302@centtech.com> <43975F5F.5080901@samsco.org> <439782AA.6000408@root.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Nate Lawson wrote: > Scott Long wrote: > >> Eric Anderson wrote: >> >>> Nate Lawson wrote: >>> >>>> Eric Anderson wrote: >>>> >>>>> I'm curious about whether a target mode device would use the >>>>> buffer cache or not. Here's a scenario: >>>>> >>>>> Host A: has fibre channel host adapter, in target mode, large >>>>> memory pool, and another fiber channel host adapter connecting to >>>>> fibre channel block device. >>>>> Host B: Fibre channel host adapter, connecting to Host A. 'sees' >>>>> the target mode block device created by Host A. >>>>> >>>>> Will Host A use the buffer cache to cache blocks between the real >>>>> block device, and the shared target mode device? >>>>> What about if Host A put a filesystem on the block device, created >>>>> a single file the size of the filesystem, and shared that >>>>> filesystem via a target mode device to Host B? >>>>> What I'm wanting is a box (FreeBSD?) that can be placed between a >>>>> fibre channel block device (like a RAID array), and a fibre >>>>> channel host using that block device, and act as a block cache for >>>>> that device, using the FreeBSD's memory. If it had a significant >>>>> amount of memory, this could be very useful. >>>> >>>> >>>> >>>> >>>> >>>> If you use the example scsi_target usermode >>>> (usr/share/examples/scsi_target), then the buffer cache will be >>>> used since its reads/writes are from usermode like normal. If you >>>> don't want that behavior, you can set O_DIRECT in the open() call >>>> of the backing store file. >>>> >>>> If you chose to modify the kernel side, you'd have to make sure >>>> your accesses were through the VOP layer and then it would be cached. >>>> >>>> You should check to be sure the target mode performance meets your >>>> expectations also. >>>> >>> >>> I guess I would be using the user mode tool, unless there's another >>> way? Your comment on performance also makes me a little worried >>> about that now - do you think I would see a large performance hit? >>> Thanks! >>> Eric >>> >>> >> >> The way the target mode stack works in FreeBSD is that the kernel >> provides some of the basic services, but the actual target emulator >> is meant to live in userland. The userland program responds to >> events from the kernel via the select interface. This generally >> works pretty well. However, it does mean that control has to >> cross the kernel-userland boundary at least once for every event. >> >> What I'd suggest doing is prototyping your target emulator in userland >> and evaluating the performance there, and then moving it to the kernel >> if you _really_ need more performance. > > > Agree 100%. While having it in usermode means there are boundary > crossings that increase per-transaction latency, the actual bulk data > transfer is via zero-copy IO and you should be able to exceed the data > transfer rates of several 10K RPM drives on decent hardware. > Ok, great.. Now, will scsi_target work ok with raw devices, or only files? (although I'm not sure theres all that much difference really). Thanks!! Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4397B731.6010308>