From owner-freebsd-fs@FreeBSD.ORG Thu Jan 12 01:46:29 2012 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2F8351065676 for ; Thu, 12 Jan 2012 01:46:29 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id B15DC8FC0C for ; Thu, 12 Jan 2012 01:46:28 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ap4EAJw6Dk+DaFvO/2dsb2JhbABDhQ6pAoFyAQEEASNWBRYYAgINGQJZGYd6pjKRVoEvh1KCBoEWBIg6jFKSWg X-IronPort-AV: E=Sophos;i="4.71,495,1320642000"; d="scan'208";a="151713084" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 11 Jan 2012 20:45:51 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 50F6BB3F36; Wed, 11 Jan 2012 20:45:51 -0500 (EST) Date: Wed, 11 Jan 2012 20:45:51 -0500 (EST) From: Rick Macklem To: mdf@FreeBSD.org Message-ID: <450250386.94820.1326332751322.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: "Mikhail T." , fs@freebsd.org Subject: Re: How to best send files over network? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jan 2012 01:46:29 -0000 Matthew Fleming wrote: > On Wed, Jan 11, 2012 at 8:07 AM, Rick Macklem > wrote: > > Mikhail T. wrote: > >> On 10.01.2012 18:49, Bob Friesenhahn wrote: > >> > Don't use mmap on zfs since doing so wastes memory (zfs ARC is > >> > not > >> > coherent with mmap page cache). Instead do normal file I/O (e.g. > >> > write, fwrite) using the filesystem blocksize (e.g. 128K) or a > >> > small > >> > multiple thereof. > >> Well, that was the reason cited for not using sendfile over ZFS. > >> But > >> mmap/write, it was claimed, was efficient. > >> > >> What's the general opinion of using mmap/write, when the file is on > >> UFS? > >> Is it just as good as sendfile, or can sendfile be better under > >> some > >> circumstances? > >> > >> > It is useful to cache several blocks on the receiving end and use > >> > a > >> > thread to receive data from the network in case zfs temporarily > >> > stalls > >> > during write (which it periodically does). > >> No, thanks. I'm certainly not doing a read/write loop -- that's > >> just > >> too > >> disgusting in the age of better interfaces (even if those aren't > >> well > >> implemented yet) :-) > > > > I think Bob was referring to the receive end and not the send end, > > which > > might be why the answer didn't make sense to you? (For the receive > > end, > > it sounds like a good suggestion to me.) > > Some day, when there's time, I'd like to roll up Isilon's recvfile(2) > into a proper patch. It's the obvious analogue to sendfile(2). I > think it also requires two new VOPs, for efficiency, VOP_READ_MBUF > (for sendfile) and VOP_WRITE_MBUF, which use an mbuf chain rather than > uio to do their work. > The NFS code has worked around this by allocating a bunch of mbufs and creating an iovec that refers to their data areas, pretty well forever. (Just guessing this would avoid adding the VOP_xxx_MBUF()s?) However, don't mistakenly interpret this as me volunteering to do this:-) rick > Perhaps at BSDCan, if this is my priority at the time. Or, if someone > else wants to work on it I can provide patches that will require work. > > Thanks, > matthew