From owner-freebsd-questions@FreeBSD.ORG Fri Mar 31 13:15:20 2006 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 845C816A400 for ; Fri, 31 Mar 2006 13:15:20 +0000 (UTC) (envelope-from wmoran@collaborativefusion.com) Received: from mx00.pub.collaborativefusion.com (mx00.pub.collaborativefusion.com [206.210.89.199]) by mx1.FreeBSD.org (Postfix) with ESMTP id 31B8143D6B for ; Fri, 31 Mar 2006 13:15:13 +0000 (GMT) (envelope-from wmoran@collaborativefusion.com) Received: from localhost (monrovll-cuda1-24-53-251-44.pittpa.adelphia.net [24.53.251.44]) (AUTH: LOGIN wmoran, TLS: TLSv1/SSLv3,256bits,AES256-SHA) by wingspan with esmtp; Fri, 31 Mar 2006 08:15:12 -0500 id 00056430.442D2B60.0000D697 Date: Fri, 31 Mar 2006 08:15:11 -0500 From: Bill Moran To: "Ashish Awasthi" Message-Id: <20060331081511.1079dffe.wmoran@collaborativefusion.com> In-Reply-To: <5289d17a0603302327o69228ac0l611c434176d0a31c@mail.gmail.com> References: <5289d17a0603291735u194807cbu853b438edc80049b@mail.gmail.com> <5289d17a0603291736s2f05bb55m293e2ec32af8a3af@mail.gmail.com> <20060330092327.3f4fc954.wmoran@collaborativefusion.com> <5289d17a0603302327o69228ac0l611c434176d0a31c@mail.gmail.com> Organization: Collaborative Fusion X-Mailer: Sylpheed version 1.0.6 (GTK+ 1.2.10; i386-portbld-freebsd5.4) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-questions@freebsd.org Subject: Re: Packet drops and queue length upon bandwidth limiting in PF X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 31 Mar 2006 13:15:20 -0000 "Ashish Awasthi" wrote: > On 3/30/06, Bill Moran wrote: > > > > "Ashish Awasthi" wrote: > > > > > I am a relative newbie, so please don't flame me if my question doesn't > > make > > > sense. > > > > > > In a network experiment to determine appropriate length of router > > buffers, I > > > am using pfctl on FreeBSD 5.3 to limit the bandwidth to 100 Mbps on a 1 > > Gig > > > link and limit the queue to 240 packets, and I use iperf for sending out > > > data. Connection is maintained between two routers running FreeBSD 5.3, > > > connected by a 1 Gig link. I monitor on sender the pfctl and iperf > > > statisitcs. > > > > > > As I see the iperf throughput go down from 94 Mbps to 50 Mbps and then > > rise > > > again in accordance with the classic sawtooth curve of TCP, it is clear > > that > > > there must have been a packet drop, but "pfctl -s -queue -v -v" at the > > > sender shows 0 losses and 0 drops. Moreover, the queue length as > > reported > > > never overflows. Even netstat shows 0 retransmissions! > > > > > > I tried this with queue lengths of 50, 100, 240, 10 and 5. Only when > > queue > > > length is on the order of 5 or 10 do I see packet drops in pfctl report > > (and > > > also retransmissions in the netstat report); however, since I have > > limited > > > the bandwidth and the outgoing traffic is shaped by this limitation, it > > is > > > clear that there must be some packet losses in other cases as well. > > > > > > So, I tend to think that some other queueing is occuring apart from the > > > ALTQ, and drops are occuring there. If so, how can I obtain those > > > statistics? > > > > You're making a lot of assumptions about how things work, so I'll follow > > in kind. > > > > I would assume that pf is sending ICMP source quench messages to the > > sending machine to avoid overflowing its queues. If it's proactive > > in doing this, it would never overflow, except in the case where the > > queue is so short that it can't reply with a source quench fast enough. > > To me, this would be expected behaviour. A little packet sniffing should > > show whether this is what is actually happening or not. > > > > As a side note, this is why arbitrarily blocking all ICMP messages is a > > bad > > idea. > > Hi, > > Thanks for your response. However, the problem still remains. I did check > for ICMP packets at both the source and the router, but there are NO source > quench packets at all in the tcpdump traces. > > Where should I be looking? Any suggestions? Look at the network traffic. If you're getting dropped packets, those will be obvious from the retransmits. If not, you'll be able to see what is actually controlling the speed. I suppose the kernel could be limiting how it sends ACKs. I suggest Ethereal for this kind of thing. It has a lot of nifty features that make it easy (i.e. it automagically flags retransmitted packets). -- Bill Moran Potential Technologies http://www.potentialtech.com