From owner-freebsd-fs@freebsd.org Mon Jan 4 17:19:14 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 94D7CA61480 for ; Mon, 4 Jan 2016 17:19:14 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 734AE12CC for ; Mon, 4 Jan 2016 17:19:14 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (ppp121-45-250-125.lns20.per4.internode.on.net [121.45.250.125]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id u04HJAqU064511 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Mon, 4 Jan 2016 09:19:13 -0800 (PST) (envelope-from julian@freebsd.org) Subject: Re: NFS reads vs. writes To: freebsd-fs@freebsd.org References: <568880D3.3010402@aldan.algebra.com> From: Julian Elischer Message-ID: <568AA988.50903@freebsd.org> Date: Tue, 5 Jan 2016 01:19:04 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 In-Reply-To: <568880D3.3010402@aldan.algebra.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jan 2016 17:19:14 -0000 On 3/01/2016 10:00 AM, Mikhail T. wrote: > I have two similar 10.2-STABLE machines mounting each other's > filesystems via NFS: a:/a and b:/b > > When moving some large files around today, I found a huge discrepancy > between writing and reading over NFS. That is, machine a copying files > from its own /a to NFS-mounted b:/b was pushing measly 2-3Mb/s over the > Ethernet. When I cancelled that, logged-in to machine b and proceed to > copy from a:/a to the now-local /b, I got 56Mb/s. I tried changing the > wsize and rsize options, but it did not seem to make a difference... > > Any ideas, what is happening here? Why are NFS-writes some 25 times > slower here, than reads? Both filesystems are zfs-backed (though with > different options), both systems are plugged into the same switch, both > use mtu of 9000. look at the work from the point of view of each machine: If A is writing to B's filesystem, it needs to wait synchronously for each write to complete before doing the next. (that's what NFS does by default). If B is reading from A's filesystem it can read the next bloc as soon as it has read the previous, and in fact given that there is read-ahead logic at several layers, it's probably not even waiting that long, but queuing up (at one end or the other) requests to disk faster than they can be satisfied. Basically the reads are parallelised and the writes are async in the B case. I see Rick has replied too. > > Thanks! Yours, > > -mi > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >