From owner-freebsd-current@FreeBSD.ORG Tue Mar 9 21:46:19 2010 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E79AE1065675; Tue, 9 Mar 2010 21:46:19 +0000 (UTC) (envelope-from davidch@broadcom.com) Received: from mms1.broadcom.com (mms1.broadcom.com [216.31.210.17]) by mx1.freebsd.org (Postfix) with ESMTP id B8B2B8FC13; Tue, 9 Mar 2010 21:46:19 +0000 (UTC) Received: from [10.9.200.131] by mms1.broadcom.com with ESMTP (Broadcom SMTP Relay (Email Firewall v6.3.2)); Tue, 09 Mar 2010 13:31:56 -0800 X-Server-Uuid: 02CED230-5797-4B57-9875-D5D2FEE4708A Received: from IRVEXCHCCR01.corp.ad.broadcom.com ([10.252.49.30]) by IRVEXCHHUB01.corp.ad.broadcom.com ([10.9.200.131]) with mapi; Tue, 9 Mar 2010 13:31:56 -0800 From: "David Christensen" To: "pyunyh@gmail.com" , "Ian FREISLICH" Date: Tue, 9 Mar 2010 13:31:55 -0800 Thread-Topic: dev.bce.X.com_no_buffers increasing and packet loss Thread-Index: Acq/zpF0ZHFVHAVzSZyaqLAYjZ2tvgAANqMQ Message-ID: <5D267A3F22FD854F8F48B3D2B52381933AF90EED69@IRVEXCHCCR01.corp.ad.broadcom.com> References: <20100305210435.GF14818@michelle.cdnetworks.com> <20100305184046.GD14818@michelle.cdnetworks.com> <20100305175639.GB14818@michelle.cdnetworks.com> <20100309212139.GO1311@michelle.cdnetworks.com> In-Reply-To: <20100309212139.GO1311@michelle.cdnetworks.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US MIME-Version: 1.0 X-WSS-ID: 678861C620S53417274-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Cc: "current@freebsd.org" , David Christensen Subject: RE: dev.bce.X.com_no_buffers increasing and packet loss X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 21:46:20 -0000 > > > patch can fix the RX issue you're suffering from. Anyway,=20 > would you=20 > > > give it try the patch at the following URL? > > > http://people.freebsd.org/~yongari/bce/bce.20100305.diff > > > The patch was generated against CURRENT and you may see a message=20 > > > like "Disabling COAL_NOW timedout!" during interface up. You can=20 > > > ignore that message. > >=20 > > It's been running for about 1:23 on the patched driver. I'm still=20 > > seeing the com_no_buffers increase: > >=20 > > [firewall2.jnb1] ~ # sysctl dev.bce |grep com_no_buffers > > dev.bce.0.com_no_buffers: 5642 > > dev.bce.1.com_no_buffers: 497 > > dev.bce.2.com_no_buffers: 6260612 > > dev.bce.3.com_no_buffers: 4871338 > >=20 >=20 > Still have no idea why these counters are increasing here. > Actually the counter is read from scratch pad of completion=20 > processor. The datasheet does not even document the counter. > Maybe david know better what's happening here(CCed). >=20 > > Interupt rate is down now, at about 3500 per second per interface. > >=20 > > Interestingly setting net.inet.ip.fastforwarding=3D0 reduces CPU=20 > > consumption from 25% to 9% and less packet loss. The com_no_buffers statistic comes from firmware and indicates how many times a valid frame was received but could not be placed into the receive chain because there were no available RX buffers. The firmware will then drop the frame but that dropped frame won't be reflected in any of the hardware based statistics. Dave=