From owner-freebsd-current@FreeBSD.ORG Fri Mar 12 23:58:48 2010 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C0181106564A for ; Fri, 12 Mar 2010 23:58:48 +0000 (UTC) (envelope-from davidch@broadcom.com) Received: from MMS3.broadcom.com (mms3.broadcom.com [216.31.210.19]) by mx1.freebsd.org (Postfix) with ESMTP id 936A88FC16 for ; Fri, 12 Mar 2010 23:58:48 +0000 (UTC) Received: from [10.9.200.133] by MMS3.broadcom.com with ESMTP (Broadcom SMTP Relay (Email Firewall v6.3.2)); Fri, 12 Mar 2010 15:58:33 -0800 X-Server-Uuid: B55A25B1-5D7D-41F8-BC53-C57E7AD3C201 Received: from IRVEXCHCCR01.corp.ad.broadcom.com ([10.252.49.31]) by IRVEXCHHUB02.corp.ad.broadcom.com ([10.9.200.133]) with mapi; Fri, 12 Mar 2010 15:59:57 -0800 From: "David Christensen" To: "Ian FREISLICH" , "pyunyh@gmail.com" Date: Fri, 12 Mar 2010 15:58:31 -0800 Thread-Topic: dev.bce.X.com_no_buffers increasing and packet loss Thread-Index: AcrA6Vwt0ZDaUZGKRN2wvtRBpJh3wgBViJAg Message-ID: <5D267A3F22FD854F8F48B3D2B52381933B0A21DB97@IRVEXCHCCR01.corp.ad.broadcom.com> References: <20100310230220.GI10657@michelle.cdnetworks.com> <20100309212139.GO1311@michelle.cdnetworks.com> <5D267A3F22FD854F8F48B3D2B52381933AF90EED69@IRVEXCHCCR01.corp.ad.broadcom.com> <20100309214012.GQ1311@michelle.cdnetworks.com> <5D267A3F22FD854F8F48B3D2B52381933AF90EF16F@IRVEXCHCCR01.corp.ad.broadcom.com> <20100310195206.GB10657@michelle.cdnetworks.com> <5D267A3F22FD854F8F48B3D2B52381933AF90EF25A@IRVEXCHCCR01.corp.ad.broadcom.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US MIME-Version: 1.0 X-WSS-ID: 67840AA331G58121710-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Cc: "current@freebsd.org" Subject: RE: dev.bce.X.com_no_buffers increasing and packet loss X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 23:58:48 -0000 > Pyun YongHyeon wrote: > > On Wed, Mar 10, 2010 at 02:45:47PM -0800, David Christensen wrote: > > > The bce(4) hardware supports a linked list of pages for RX buffer=20 > > > descriptors. The stock build supports 2 pages (RX_PAGES) with a=20 > > > total of 511 BD's per page. The hardware can support a=20 > maximum of=20 > > > 64K BD's but that would be an unnecessarily large amount of mbufs=20 > > > for an infrequent problem. >=20 > I think that depends on how you define infrequent. Our use=20 > case is a largish core router. It's highly likely that we'll=20 > see this again and again in various packet storms on our network. >=20 Are the packet storms always always from the same host or do they come from multiple hosts? The hardware supports RSS which can spread the network load across multiple receive queues and multiple CPU cores, but only when the traffic is spread across several hosts. (The current bce(4) driver doesn't include support for RSS.) If a storm of small frames comes from a single host then almost all adapters will be challenged to handle the flow. Dave=