From owner-freebsd-net@FreeBSD.ORG Fri Jul 29 03:05:35 2005 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A73A116A41F for ; Fri, 29 Jul 2005 03:05:35 +0000 (GMT) (envelope-from julian@elischer.org) Received: from postoffice.vicor-nb.com (postoffice.vicor.com [69.26.56.52]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4B14943D45 for ; Fri, 29 Jul 2005 03:05:35 +0000 (GMT) (envelope-from julian@elischer.org) Received: from localhost (localhost [127.0.0.1]) by postoffice.vicor-nb.com (Postfix) with ESMTP id D818C4CE988; Thu, 28 Jul 2005 20:05:34 -0700 (PDT) Received: from postoffice.vicor-nb.com ([127.0.0.1]) by localhost (postoffice.vicor-nb.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 65390-09; Thu, 28 Jul 2005 20:05:34 -0700 (PDT) Received: from [208.206.78.97] (julian.vicor-nb.com [208.206.78.97]) by postoffice.vicor-nb.com (Postfix) with ESMTP id 1D6214CE7B7; Thu, 28 Jul 2005 20:05:34 -0700 (PDT) Message-ID: <42E99CFD.6070803@elischer.org> Date: Thu, 28 Jul 2005 20:05:33 -0700 From: Julian Elischer User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.8) Gecko/20050629 X-Accept-Language: en, hu MIME-Version: 1.0 To: Chuck Swiger References: <200507290834.10268.doconnor@gsoft.com.au> <200507291035.46770.doconnor@gsoft.com.au> <42E98725.1020600@mac.com> <200507291115.06612.doconnor@gsoft.com.au> <42E99868.1080306@mac.com> In-Reply-To: <42E99868.1080306@mac.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at postoffice.vicor.com Cc: Daniel O'Connor , freebsd-net@freebsd.org Subject: Re: AltQ + ng_iface X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jul 2005 03:05:35 -0000 Chuck Swiger wrote: > Daniel O'Connor wrote: > >> On Friday 29 July 2005 11:02, Chuck Swiger wrote: >> >>> Either the "established" or the "tcpflags !syn,ack" keywords in a rule >>> adding matching packets to a high-priority queue ought to do it...? Or >>> perhaps you meant something more specific than just "TCP packets with >>> TH_ACK" set? :-) >> >> >> Hmm, I guess you could make those skip the pipe.. >> >>> Anyway, I'm not convinced that trying to classify packets within an >>> established TCP connection in order to place them on different >>> queues is a >>> really good idea, since you're quite likely to reorder the packets >>> by doing >>> so. I'd expect both latency and bandwidth of a TCP connection to >>> suffer >>> very noticably if more than 10% or so of the packets arrive out of >>> order... >> >> >> The theory is that by prioritising outgoing ACKs you don't cause >> downstream delays when your upstream is full. eg >> http://www.benzedrine.cx/ackpri.html > > > Ah. OK, it makes sense that delaying outgoing ACKs too much would > slow things down. So you want to send dataless ACKs at a higher > priority than generic big packets full of data, maybe via the "iplen" > keyword with "established", look for packets smaller than ~100 bytes? I do this to great effect.. consider: two sites connected by links in which teh bottleneck is 200KB/sec (1 E1?) when a lot of data is flowing from 1 to 2 then data from 2 to 1 is also slowed down because the acks have to go through the queues on ingress side of the bottleneck router. I add a dummynet entry on 1, limiting output to 190KB/sec, so that the queue is in dummynet and not the intermediate router, and then allow small ack packets to bypass that queue. As a result the data from 2 to 1 also flows at near capacity, and with a much lower latency. SInce data flows tend to be large packets, I sometimes actually prioitise ALL small packets allowing interactive stuff to bypass ftps etc. and sometimes I do it on both ends. > > My other thought on this is to wonder about window size and whether > that was scaling properly up to a reasonable value, and whether both > sides implement a sane network stack, or whether the other side was a > windows box looking for quick responses and usage of SACK, rather than > BSD (new-reno?) delayed ACKs... > >>> [ Hmm. I suppose that one could make an exception to the above >>> generalization if URG was set, but the TCP stack already makes an >>> effort to >>> prioritize and deliver out-of-band urgent stuff as quickly as possible, >>> anyway, right? ] >> >> >> Maybe, but it doesn't appear to do a particularly good job for a lot >> of situations :) > > > I guess. :-) Getting 25% of the hoped-for max performance under a > problematic case isn't so horrible, either, but I suspect other > factors were involved, too. > > A tcpdump would've been informative.... >