From owner-freebsd-stable@FreeBSD.ORG Sun Jun 25 06:41:23 2006 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9525A16A49E for ; Sun, 25 Jun 2006 06:41:23 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from mail13.syd.optusnet.com.au (mail13.syd.optusnet.com.au [211.29.132.194]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4FF8443D48 for ; Sun, 25 Jun 2006 06:41:16 +0000 (GMT) (envelope-from peterjeremy@optushome.com.au) Received: from turion.vk2pj.dyndns.org (c220-239-19-236.belrs4.nsw.optusnet.com.au [220.239.19.236]) by mail13.syd.optusnet.com.au (8.12.11/8.12.11) with ESMTP id k5P6fEKI029080 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Sun, 25 Jun 2006 16:41:14 +1000 Received: from turion.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by turion.vk2pj.dyndns.org (8.13.6/8.13.6) with ESMTP id k5P6fERA008573; Sun, 25 Jun 2006 16:41:14 +1000 (EST) (envelope-from peter@turion.vk2pj.dyndns.org) Received: (from peter@localhost) by turion.vk2pj.dyndns.org (8.13.6/8.13.6/Submit) id k5P6fE1N008572; Sun, 25 Jun 2006 16:41:14 +1000 (EST) (envelope-from peter) Date: Sun, 25 Jun 2006 16:41:14 +1000 From: Peter Jeremy To: Nikolas Britton Message-ID: <20060625064114.GF747@turion.vk2pj.dyndns.org> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="jTMWTj4UTAEmbWeb" Content-Disposition: inline In-Reply-To: X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.11 Cc: freebsd-stable@freebsd.org Subject: Re: Effects of changing tar's -b option. X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Jun 2006 06:41:23 -0000 --jTMWTj4UTAEmbWeb Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, 2006-Jun-24 20:12:56 -0700, Nikolas Britton wrote: >Test Setup: >250 50MB files (13068252KB) >dd if=3D/dev/random of=3Dtestfile bs=3D1m count=3D50 >Ethernet mtu=3D6500 >Transferred files were wiped after every test with 'rm -r *'. > >Test: >hostB: nc -4l port | tar xpbf n - >hostA: date; tar cbf n - . | nc hostB port; date > >Test Results: >seconds =3D n >645sec. =3D 1024 >670sec. =3D 512 >546sec. =3D 256 >503sec. =3D 128 >500sec. =3D 128 (control) >515sec. =3D 96 >508sec. =3D 64 >501sec. =3D 20 (default) > >Conclusions: Make your own. I don't think that's so unexpected. tar doesn't use multiple buffers so filling and emptying the buffer is done serially. Once the buffer exceeds the space in the pipe buffer and the local TCP send buffer, then the write from the hostA tar is delayed until the TCP buffer can drain. At the same time, the read from the hostB tar is blocked waiting for data from the network. Optimal throughput will depend on maximally overlapping the file reads on hostB with the network traffic and file writes on hostB. This, in turn, means you want to be able to hold at least a full buffer of data in the intervening processes and kernel buffers. Assuming that you aren't network bandwidth limited, you should look at increasing net.inet.tcp.sendspace and maybe net.inet.tcp.recvspace, or using an intervening program on hostA that does its own re-buffering. --=20 Peter Jeremy --jTMWTj4UTAEmbWeb Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (FreeBSD) iD8DBQFEnjAI/opHv/APuIcRAsuBAJ4+byjnPI45lBohjlYEGFNEzPV3TQCfchD1 W1GahgtbkahJJiObj4OkSIg= =ABUZ -----END PGP SIGNATURE----- --jTMWTj4UTAEmbWeb--