From owner-freebsd-questions@FreeBSD.ORG Thu Mar 30 14:23:35 2006 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B5BCD16A420 for ; Thu, 30 Mar 2006 14:23:35 +0000 (UTC) (envelope-from wmoran@collaborativefusion.com) Received: from mx00.pub.collaborativefusion.com (mx00.pub.collaborativefusion.com [206.210.89.199]) by mx1.FreeBSD.org (Postfix) with ESMTP id A36E743D75 for ; Thu, 30 Mar 2006 14:23:30 +0000 (GMT) (envelope-from wmoran@collaborativefusion.com) Received: from localhost (monrovll-cuda1-24-53-250-148.pittpa.adelphia.net [24.53.250.148]) (AUTH: LOGIN wmoran, TLS: TLSv1/SSLv3,256bits,AES256-SHA) by wingspan with esmtp; Thu, 30 Mar 2006 09:23:29 -0500 id 00056422.442BE9E1.0000815E Date: Thu, 30 Mar 2006 09:23:27 -0500 From: Bill Moran To: "Ashish Awasthi" Message-Id: <20060330092327.3f4fc954.wmoran@collaborativefusion.com> In-Reply-To: <5289d17a0603291736s2f05bb55m293e2ec32af8a3af@mail.gmail.com> References: <5289d17a0603291735u194807cbu853b438edc80049b@mail.gmail.com> <5289d17a0603291736s2f05bb55m293e2ec32af8a3af@mail.gmail.com> Organization: Collaborative Fusion X-Mailer: Sylpheed version 1.0.6 (GTK+ 1.2.10; i386-portbld-freebsd5.4) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-questions@freebsd.org Subject: Re: Packet drops and queue length upon bandwidth limiting in PF X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Mar 2006 14:23:35 -0000 "Ashish Awasthi" wrote: > I am a relative newbie, so please don't flame me if my question doesn't make > sense. > > In a network experiment to determine appropriate length of router buffers, I > am using pfctl on FreeBSD 5.3 to limit the bandwidth to 100 Mbps on a 1 Gig > link and limit the queue to 240 packets, and I use iperf for sending out > data. Connection is maintained between two routers running FreeBSD 5.3, > connected by a 1 Gig link. I monitor on sender the pfctl and iperf > statisitcs. > > As I see the iperf throughput go down from 94 Mbps to 50 Mbps and then rise > again in accordance with the classic sawtooth curve of TCP, it is clear that > there must have been a packet drop, but "pfctl -s -queue -v -v" at the > sender shows 0 losses and 0 drops. Moreover, the queue length as reported > never overflows. Even netstat shows 0 retransmissions! > > I tried this with queue lengths of 50, 100, 240, 10 and 5. Only when queue > length is on the order of 5 or 10 do I see packet drops in pfctl report (and > also retransmissions in the netstat report); however, since I have limited > the bandwidth and the outgoing traffic is shaped by this limitation, it is > clear that there must be some packet losses in other cases as well. > > So, I tend to think that some other queueing is occuring apart from the > ALTQ, and drops are occuring there. If so, how can I obtain those > statistics? You're making a lot of assumptions about how things work, so I'll follow in kind. I would assume that pf is sending ICMP source quench messages to the sending machine to avoid overflowing its queues. If it's proactive in doing this, it would never overflow, except in the case where the queue is so short that it can't reply with a source quench fast enough. To me, this would be expected behaviour. A little packet sniffing should show whether this is what is actually happening or not. As a side note, this is why arbitrarily blocking all ICMP messages is a bad idea. -- Bill Moran