From owner-freebsd-stable@FreeBSD.ORG Tue Aug 8 18:28:20 2006 Return-Path: X-Original-To: stable@freebsd.org Delivered-To: freebsd-stable@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0D85A16A4E0; Tue, 8 Aug 2006 18:28:20 +0000 (UTC) (envelope-from davidch@broadcom.com) Received: from mms2.broadcom.com (mms2.broadcom.com [216.31.210.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8CFEC43D45; Tue, 8 Aug 2006 18:28:17 +0000 (GMT) (envelope-from davidch@broadcom.com) Received: from 10.10.64.154 by mms2.broadcom.com with ESMTP (Broadcom SMTP Relay (Email Firewall v6.2.0)); Tue, 08 Aug 2006 11:28:13 -0700 X-Server-Uuid: D9EB6F12-1469-4C1C-87A2-5E4C0D6F9D06 Received: by mail-irva-10.broadcom.com (Postfix, from userid 47) id 046D82B1; Tue, 8 Aug 2006 11:28:12 -0700 (PDT) Received: from mail-irva-8.broadcom.com (mail-irva-8 [10.10.64.221]) by mail-irva-10.broadcom.com (Postfix) with ESMTP id C9D2C2AF; Tue, 8 Aug 2006 11:28:12 -0700 (PDT) Received: from mail-irva-12.broadcom.com (mail-irva-12.broadcom.com [10.10.64.146]) by mail-irva-8.broadcom.com (MOS 3.7.5a-GA) with ESMTP id ECC43915; Tue, 8 Aug 2006 11:28:06 -0700 (PDT) Received: from NT-IRVA-0750.brcm.ad.broadcom.com (nt-irva-0750 [10.8.194.64]) by mail-irva-12.broadcom.com (Postfix) with ESMTP id 13CD269CA3; Tue, 8 Aug 2006 11:28:06 -0700 (PDT) X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Date: Tue, 8 Aug 2006 11:28:05 -0700 Message-ID: <09BFF2FA5EAB4A45B6655E151BBDD90301AB72BF@NT-IRVA-0750.brcm.ad.broadcom.com> In-Reply-To: <20060808003404.GA5411@cdnetworks.co.kr> Thread-Topic: Re: Re: bce0: Error mapping mbuf into TX chain! Thread-Index: Aca6gl8lmGjvoDZvTzuOjH6QQdtP7QAkoejA From: "David (Controller AE) Christensen" To: pyunyh@gmail.com X-TMWD-Spam-Summary: SEV=1.1; DFV=A2006080806; IFV=2.0.6,4.0-7; RPD=4.00.0004; RPDID=303030312E30413031303230352E34344438443731442E303032462D412D; ENG=IBF; TS=20060808182814; CAT=NONE; CON=NONE; X-MMS-Spam-Filter-ID: A2006080806_4.00.0004_2.0.6,4.0-7 X-WSS-ID: 68C608370X82129696-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Cc: stable@freebsd.org, Scott Wilson , davidch@freebsd.org, Eric Hodel Subject: RE: Re: Re: bce0: Error mapping mbuf into TX chain! X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2006 18:28:20 -0000 > On Mon, Aug 07, 2006 at 01:48:09PM -0700, David (Controller=20 > AE) Christensen wrote: > > Scott, > >=20 > > What are you doing when this problem occurs? Is it something I can > > easily duplicate here? When I tested the fix on -CURRENT=20 > I used the > > following command suggested by Doug to bring out the=20 > failure quickly: > >=20 > > ssh "dd if=3D/dev/zero bs=3D1" > /dev/null > >=20 > > Does this same command fail for you too? > >=20 >=20 > Since BCE_MAX_SEGMENTS is too small I guess it will happen on highly > fragmented packets under heavy loads. To simulate the situation > you can use m_fragment(9) to fragment the frame in bce_tx_encap(). > With m_fragment(9), "ping -f -s 65507 x.x.x.x" may trigger it. >=20 I didn't know about m_fragment before. I'll write a note to myself and look at how to add it to the debug path for a future driver revision. > Btw, I've never seen this small number of Tx DMA segments support(=20 > BCE_MAX_SEGMENTS =3D=3D 8) on GigE. Is this hardware limitation? >=20 The real value for BCE_MAX_SEGMENTS should be 16, not 8. I chose 8 as a reasonable value to start with. If the number of fragments exceeds 16=20 then we would expect to see performance drop and it is probably faster to have the OS defragment the packet rather than try to perform so many DMAs. > > > > Hmm... I can see several bus_dma(9) related bugs in bce(4). > > > > For architectures that have IOMMU hardware it may have=20 > corrupted DMA > > > > mapping and I'm pretty sure it wouldn't work on sparc64. > > > > When it has to handle many fragmented frame or has insufficient > > > > number of free Tx descriptors it would show unexpected results. > > > > Unfortunately I don't have hardwares supported by bce(4) and > > > > fixing requiries a working hardware. :-( > > > > > > >=20 If I were able to get you a 5706 (PCI-X) or 5708 (PCI-Express) NIC would you be willing to put some time into helping sort out any bus_dma related issues? I have the opposite problem and don't have any Sparc64 system to use for testing, plenty of NICs though ;-)