From owner-freebsd-current@FreeBSD.ORG Wed Mar 10 09:05:32 2010 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5CF7F1065670 for ; Wed, 10 Mar 2010 09:05:32 +0000 (UTC) (envelope-from ianf@clue.co.za) Received: from inbound01.jnb1.gp-online.net (inbound01.jnb1.gp-online.net [41.161.16.135]) by mx1.freebsd.org (Postfix) with ESMTP id CFD5B8FC08 for ; Wed, 10 Mar 2010 09:05:30 +0000 (UTC) Received: from [41.154.88.19] (helo=clue.co.za) by inbound01.jnb1.gp-online.net with esmtpsa (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.63) (envelope-from ) id 1NpHr5-0005nO-Ps; Wed, 10 Mar 2010 11:05:27 +0200 Received: from localhost ([127.0.0.1] helo=clue.co.za) by clue.co.za with esmtp (Exim 4.69 (FreeBSD)) (envelope-from ) id 1NpHr0-000HWT-It; Wed, 10 Mar 2010 11:05:22 +0200 To: "David Christensen" From: Ian FREISLICH In-Reply-To: <5D267A3F22FD854F8F48B3D2B52381933AF90EEDA7@IRVEXCHCCR01.corp.ad.broadcom.com> References: <5D267A3F22FD854F8F48B3D2B52381933AF90EEDA7@IRVEXCHCCR01.corp.ad.broadcom.com> <20100305210435.GF14818@michelle.cdnetworks.com> <20100305184046.GD14818@michelle.cdnetworks.com> <20100305175639.GB14818@michelle.cdnetworks.com> <20100309212139.GO1311@michelle.cdnetworks.com> <5D267A3F22FD854F8F48B3D2B52381933AF90EED69@IRVEXCHCCR01.corp.ad.broadcom.com> <20100309214012.GQ1311@michelle.cdnetworks.com> X-Attribution: BOFH Date: Wed, 10 Mar 2010 11:05:22 +0200 Message-Id: Cc: "pyunyh@gmail.com" , current@freebsd.org Subject: Re: dev.bce.X.com_no_buffers increasing and packet loss X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 09:05:32 -0000 "David Christensen" wrote: > > Yeah, but the question is why bce(4) has no available RX buffers. > > The system has a lot of available mbufs so I don't see the=20 > > root cause here. > > What's the traffic look like? Jumbo, standard, short frames? Any=20 > good ideas on profiling the code? I haven't figured out how to use > the CPU TSC but there is a free running timer on the device that > might be usable to calculate where the driver's time is spent. It looks like the traffic that provoked it was this: 10:18:42.319370 IP X.4569 > X.4569: UDP, length 12 10:18:42.319402 IP X.4569 > X.4569: UDP, length 12 10:18:42.319438 IP X.4569 > X.4569: UDP, length 12 10:18:42.319484 IP X.4569 > X.4569: UDP, length 12 10:18:42.319517 IP X.4569 > X.4569: UDP, length 12 A flurry of UDP tinygrams on an IAX2 trunk. The packet rate isn't spectacular at about 30kpps which on top of the base load of 60kpps still isn't a fantastic packet rate. The interesting thing is that while this storm was inprogress, it almost entirely excluded other traffic on the network. There have been reports of backplane congestion on the switches we use when UDP packets smaller than 400 bytes arrive within 40us of eachother. But that still doesn't explain the counter increases and high interrupt CPU usage, unless the switch was producing garbage output in response. Ian -- Ian Freislich