From owner-freebsd-questions@FreeBSD.ORG Fri Jan 23 07:54:06 2004 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D4C6316A4CE for ; Fri, 23 Jan 2004 07:54:06 -0800 (PST) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 15A5F43D5E for ; Fri, 23 Jan 2004 07:53:49 -0800 (PST) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.10/8.12.10) with ESMTP id i0NFphUd013611; Fri, 23 Jan 2004 10:51:43 -0500 (EST) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i0NFpX2W013515; Fri, 23 Jan 2004 10:51:43 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Fri, 23 Jan 2004 10:51:33 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Karl Pielorz In-Reply-To: <12844453.1074872903@raptor> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-questions@freebsd.org Subject: Re: FreeBSD tunnels / performance et'al (gif/tun etc.) X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jan 2004 15:54:06 -0000 On Fri, 23 Jan 2004, Karl Pielorz wrote: > > On Tue, 20 Jan 2004, Karl Pielorz wrote: > > > >> I've just setup a FreeBSD tunnel (we've tried both gif and tun [via > >> nos-tun]) now between two fairly large networks of machines... > > > > What version of FreeBSD are you using? If using FreeBSD 5.x, you may well > > want to switch to 4.x for at least one more minor version, as interrupt > > latency hasn't been optimized in 5.x yet since the move to interrupt > > threads, and the network stack also runs with Giant in 5.2 out of the > > box. I wouldn't think this would hurt you as much as seen below, but > > it's worth keeping in mind. > > > > Also, I would generally expect gif, gre, et al, to be faster than > > tun-based tunneling, as they avoid the trip through userspace, which > > involves a number of packet copies. > > We're already using 4.9. I also take your point about gif being quicker > than switching to user space and back (And, in testing - tun was indeed > even slower than gif). > > In the end we fixed this problem by putting stupidly fast machines at > each end (i.e. P4 2.6Ghz) - we also made some tweaks to the tcp sysctls > (such as disabling delayed acks, and closing the window size down) - > which also seemed to help. > > I'm just wondering if it was something 'weird' such as the delay over > the tunnel being on average 'just the right delay time' to cause > problems that you wouldn't get on a LAN or something? :) I agree that something sounds weird -- I've had no problem tunneling hundreds of megabits using similar hardware to what you're using, and what sounds like a similar configuration. So it seems like someting is going on. Do you have any load information available on the systems -- i.e., interrupt rate as measured by vmstat, cpu usage, etc? Are you using natd or other address space translation? Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Senior Research Scientist, McAfee Research