From owner-freebsd-net@FreeBSD.ORG Sun Oct 4 13:47:26 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0148F1065676 for ; Sun, 4 Oct 2009 13:47:26 +0000 (UTC) (envelope-from rihad@mail.ru) Received: from mx33.mail.ru (mx33.mail.ru [94.100.176.47]) by mx1.freebsd.org (Postfix) with ESMTP id B4A928FC13 for ; Sun, 4 Oct 2009 13:47:25 +0000 (UTC) Received: from [217.25.27.27] (port=38280 helo=[217.25.27.27]) by mx33.mail.ru with asmtp id 1MuRQq-000OxJ-00 for freebsd-net@freebsd.org; Sun, 04 Oct 2009 17:47:24 +0400 Message-ID: <4AC8A76B.3050502@mail.ru> Date: Sun, 04 Oct 2009 18:47:23 +0500 From: rihad User-Agent: Mozilla-Thunderbird 2.0.0.22 (X11/20090706) MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam: Not detected X-Mras: Ok Subject: dummynet dropping too many packets X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rihad List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Oct 2009 13:47:26 -0000 Hi, we have around 500-600 mbit/s traffic flowing through a 7.1R Dell PowerEdge w/ 2 GigE bce cards. There are currently around 4 thousand ISP users online limited by dummynet pipes of various speeds. According to netstat -s output around 500-1000 packets are being dropped every second (this accounts for wasting around 7-12 mbit/s worth of traffic according to systat -ifstat): # while :; do netstat -z -s 2>/dev/null | fgrep -w "output packets dropped"; sleep 1; done 16824 output packets dropped due to no bufs, etc. 548 output packets dropped due to no bufs, etc. 842 output packets dropped due to no bufs, etc. 709 output packets dropped due to no bufs, etc. 652 output packets dropped due to no bufs, etc. ^C Pipes have been created like this: ipfw pipe 1024 config bw 1024kbit/s mask dst-ip 0xffffffff queue 350KBytes etc., and then assigned to users by application (ipfw tablearg). I've tried playing with the queue setting, from as little as 1 slot to as much as 4096KBytes - packets are still being dropped, more or less. Should I somehow calculate the proper queue value for the given pipe width? The manpage says 50 slots is typical for Ethernet devices (not mentioning whether it's 10, 100 or 1000 mbit/s), and that's it. sysctls: kern.ipc.nmbclusters=50000 net.inet.ip.dummynet.io_fast=1 Polling can't be enabled with bce. Any hints? Should I provide any further info? Thanks.