Date: Mon, 21 Dec 2009 01:27:44 -0600 From: Dan Nelson <dnelson@allantgroup.com> To: Zaphod Beeblebrox <zbeeble@gmail.com> Cc: freebsd-hackers@freebsd.org Subject: Re: scp more perfectly fills the pipe than NFS/TCP Message-ID: <20091221072743.GD98917@dan.emsphone.com> In-Reply-To: <5f67a8c40912202147t9d9b64al88060bd8a73c28b0@mail.gmail.com> References: <5f67a8c40912182147t1adc158ew9fd3d94c4c4c955f@mail.gmail.com> <20091220052703.GA98917@dan.emsphone.com> <5f67a8c40912202147t9d9b64al88060bd8a73c28b0@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
In the last episode (Dec 21), Zaphod Beeblebrox said: > On Sun, Dec 20, 2009 at 12:27 AM, Dan Nelson <dnelson@allantgroup.com> wrote: > > In the last episode (Dec 19), Zaphod Beeblebrox said: > >> Here's an interesting conundrum. I don't know what's different between > >> the TCP that scp uses from the TCP that NFS uses, but given the same > >> two FreeBSD machines, SCP fills the pipe with packets better. > >> > >> Examine the following graphic: http://www.eicat.ca/~dgilbert/example-mrtg.png > >> > >> The system doing the scp and the NFS server is FreeBSD-7.2-p1. The > >> system receiving the scp and the NFS client is FreeBSD-8.0-p1 > >> > >> The scp transfer is the left hand side of the graph and the NFS > >> transfer is on the right. > >> > >> The NFS is mounted with "-3 -T -b -l -i" and no other options. Files > >> are being moved over NFS with the system "mv" command. The files in > >> each case are large (50 to 500 meg files). > > > > If you increase the NFS blocksize (-r 32768 for example) you will get > > slightly better performance, but you will likely never match the scp > > results. They're doing two different things under the hood: scp is > > streaming the entire file in one operation, while NFS is performing many > > "read 8k at offset 0", "read 8k at offset 8k", etc requests one after > > another, so a high-latency connection will take a performance hit due to > > the latency in issuing each command. According to the mount_nfs > > manpage, it looks like there is some prefetching that can be enabled > > with the "-a ##" option. It doesn't say what the default is, though. > > While the link is slow, it is really directly connected with a latency > of 10ms or so. Isn't mv mmap()'ing large enough regions to cause > there to be a reasonable queue to transfer? I've never been impressed with FreeBSD's ability to detect sequential read patterns and prefetch blocks ahead of time, even on local ufs filesystems. -- Dan Nelson dnelson@allantgroup.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20091221072743.GD98917>