Date: Thu, 08 Dec 2005 09:47:38 +0900 From: Nate Lawson <nate@root.org> To: Scott Long <scottl@samsco.org> Cc: hackers@freebsd.org, Eric Anderson <anderson@centtech.com> Subject: Re: scsi-target and the buffer cache Message-ID: <439782AA.6000408@root.org> In-Reply-To: <43975F5F.5080901@samsco.org> References: <4395BF04.50101@centtech.com> <43960F55.3010508@root.org> <43975926.1010302@centtech.com> <43975F5F.5080901@samsco.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Scott Long wrote: > Eric Anderson wrote: > >> Nate Lawson wrote: >> >>> Eric Anderson wrote: >>> >>>> I'm curious about whether a target mode device would use the buffer >>>> cache or not. Here's a scenario: >>>> >>>> Host A: has fibre channel host adapter, in target mode, large memory >>>> pool, and another fiber channel host adapter connecting to fibre >>>> channel block device. >>>> Host B: Fibre channel host adapter, connecting to Host A. 'sees' >>>> the target mode block device created by Host A. >>>> >>>> Will Host A use the buffer cache to cache blocks between the real >>>> block device, and the shared target mode device? >>>> What about if Host A put a filesystem on the block device, created a >>>> single file the size of the filesystem, and shared that filesystem >>>> via a target mode device to Host B? >>>> What I'm wanting is a box (FreeBSD?) that can be placed between a >>>> fibre channel block device (like a RAID array), and a fibre channel >>>> host using that block device, and act as a block cache for that >>>> device, using the FreeBSD's memory. If it had a significant amount >>>> of memory, this could be very useful. >>> >>> >>> >>> >>> If you use the example scsi_target usermode >>> (usr/share/examples/scsi_target), then the buffer cache will be used >>> since its reads/writes are from usermode like normal. If you don't >>> want that behavior, you can set O_DIRECT in the open() call of the >>> backing store file. >>> >>> If you chose to modify the kernel side, you'd have to make sure your >>> accesses were through the VOP layer and then it would be cached. >>> >>> You should check to be sure the target mode performance meets your >>> expectations also. >>> >> >> I guess I would be using the user mode tool, unless there's another >> way? Your comment on performance also makes me a little worried about >> that now - do you think I would see a large performance hit? >> Thanks! >> Eric >> >> > > The way the target mode stack works in FreeBSD is that the kernel > provides some of the basic services, but the actual target emulator > is meant to live in userland. The userland program responds to > events from the kernel via the select interface. This generally > works pretty well. However, it does mean that control has to > cross the kernel-userland boundary at least once for every event. > > What I'd suggest doing is prototyping your target emulator in userland > and evaluating the performance there, and then moving it to the kernel > if you _really_ need more performance. Agree 100%. While having it in usermode means there are boundary crossings that increase per-transaction latency, the actual bulk data transfer is via zero-copy IO and you should be able to exceed the data transfer rates of several 10K RPM drives on decent hardware. -- Nate
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?439782AA.6000408>