From owner-freebsd-net@FreeBSD.ORG Sat Feb 20 11:10:41 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 39D9E106568D for ; Sat, 20 Feb 2010 11:10:41 +0000 (UTC) (envelope-from rihad@mail.ru) Received: from mx76.mail.ru (mx76.mail.ru [94.100.176.91]) by mx1.freebsd.org (Postfix) with ESMTP id EA3418FC0A for ; Sat, 20 Feb 2010 11:10:40 +0000 (UTC) Received: from [217.25.27.27] (port=16521 helo=[217.25.27.27]) by mx76.mail.ru with asmtp id 1NinEN-0001np-00; Sat, 20 Feb 2010 14:10:39 +0300 Message-ID: <4B7FC334.7000108@mail.ru> Date: Sat, 20 Feb 2010 15:10:44 +0400 From: rihad User-Agent: Thunderbird 2.0.0.23 (X11/20090817) MIME-Version: 1.0 To: Luigi Rizzo References: <4B7EDD00.5080308@mail.ru> <20100220095941.GA82976@onelab2.iet.unipi.it> In-Reply-To: <20100220095941.GA82976@onelab2.iet.unipi.it> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam: Not detected X-Mras: Ok Cc: freebsd-net@freebsd.org Subject: Re: Slow speeds experienced with Dummynet X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 11:10:41 -0000 Luigi Rizzo wrote: > On Fri, Feb 19, 2010 at 10:48:32PM +0400, rihad wrote: >> Hi, all, >> >> Recalling my old posting "dummynet dropping too many packets" dated >> October 4, 2009, the problem isn't over just yet. This time, there are >> no interface i/o drops (just a reminder: we have 2 bce(4) GigE cards >> connected to a Cisco router, one for input, and one for output. The box >> itself does some traffic accounting and enforces speed limits w/ >> ipfw/dummynet. There are normally around 5-6k users online). > > If i remember well, the previous discussion ended when you > raised the intr_queue_maxlen (and perhaps increased HZ) to > avoid that the bursts produced by periodic invocation of > dummynet_io() could overflow that queue. > I've never seen intr_queue_drops rise. It was HZ=2000 and changing if_bce.c to force ifp->if_snd.ifq_drv_maxlen = 4096 (8192 now) stopped the output drops back then. Lately we've been experiencing interface-level input drops as well (as showed by netstat -i Ierrs column), setting HZ=4000 solved the issue. >>From the rest of your post it is not completely clear if you have > not found any working configuration, or there is some setting (e.g. > with "queue 1000" or larger) which do produce a smooth experience > for your customers. It's pretty hard to set slots in terms of seconds, as slots may vary in size. I'll try setting queue sizes to Kbytes for the duration of 1-2 seconds as you suggested, and use the taildrop queuing algorithm with it. > > Another thing i'd like to understand is whether all of your pipes > have a /32 mask, or there are some which cover multiple hosts. > Typical TCP connections have around 50 packets in flight when the > connection is fully open (which in turn is hard to happen on a 512k > pipe) so a queue of 100-200 is unlikely to overflow. > Yes, all masks are currently /32. 512 isn't used that much, speeds of several mbps are. > In fact, long queues are very detrimental for customers because > they increase the delay of the congestion control loop -- as a rule > of thumb, you should try to limit the queue size to at most 1-2s > of data. >