From owner-freebsd-net@FreeBSD.ORG Tue Jul 10 13:56:54 2007 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D55E816A41F for ; Tue, 10 Jul 2007 13:56:54 +0000 (UTC) (envelope-from tom@tomjudge.com) Received: from s200aog11.obsmtp.com (s200aog11.obsmtp.com [207.126.144.125]) by mx1.freebsd.org (Postfix) with SMTP id D67F713C44B for ; Tue, 10 Jul 2007 13:56:51 +0000 (UTC) (envelope-from tom@tomjudge.com) Received: from source ([217.206.187.80]) by eu1sys200aob011.postini.com ([207.126.147.11]) with SMTP; Tue, 10 Jul 2007 13:56:43 UTC Received: from [10.0.0.89] (bill.mintel.co.uk [10.0.0.89]) by rodney.mintel.co.uk (Postfix) with ESMTP id 80CC6181422; Tue, 10 Jul 2007 14:56:43 +0100 (BST) Message-ID: <46938E75.80900@tomjudge.com> Date: Tue, 10 Jul 2007 14:49:41 +0100 From: Tom Judge User-Agent: Thunderbird 1.5.0.12 (X11/20070604) MIME-Version: 1.0 To: Tom Judge References: <46680DB1.9050905@tomjudge.com> <09BFF2FA5EAB4A45B6655E151BBDD9030414B1EC@NT-IRVA-0750.brcm.ad.broadcom.com> <466873FA.9030800@tomjudge.com> <09BFF2FA5EAB4A45B6655E151BBDD9030423EE13@NT-IRVA-0750.brcm.ad.broadcom.com> <46823A78.7020501@tomjudge.com> <4683C578.6070009@tomjudge.com> <09BFF2FA5EAB4A45B6655E151BBDD90304571430@NT-IRVA-0750.brcm.ad.broadcom.com> <09BFF2FA5EAB4A45B6655E151BBDD903045714EF@NT-IRVA-0750.brcm.ad.broadcom.com> <4684D5C0.3040709@tomjudge.com> <09BFF2FA5EAB4A45B6655E151BBDD90304571BF1@NT-IRVA-0750.brcm.ad.broadcom.com> <468A2D55.60301@tomjudge.com> <46937D97.7030507@tomjudge.com> In-Reply-To: <46937D97.7030507@tomjudge.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Sepherosa Ziehau , freebsd-net , David Christensen Subject: Re: Problems with BCE network adapter (Dell PE2950) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jul 2007 13:56:54 -0000 Tom Judge wrote: > Tom Judge wrote: >> David Christensen wrote: >>>> Sorry for the top post, please try following patch: >>>> http://people.freebsd.org/~sephe/if_bce.c.diff >>>> >>>> This is probably the cause; I noticed it when bce(4) was ported to >>>> DragonFly. >>>> >>> >>> Thanks Sephe, I think you're on to something. I have some >>> debug code in the driver to simulate mbuf allocation >>> failures and when I enable that I start receiving the same >>> error messages Tom reported (along with various kernel >>> panics), but when I include your change the system seems >>> to keep humming along. I'll certainly add your code into an update >>> shortly. >>> >>> Dave >>> >> >> I'm not going to have a chance to test this patch until next week but >> I will let you know what the results are. >> >> Tom > > > So here goes, after 2 days testing we have come up with the following > data. > > The configuration > > [PE[12]950] ----> [PowerConnect 5324] > > The system is running 8192 byte Jumbo Frames. > > sultan# ifconfig bce0 > bce0: flags=8847 mtu 8192 > options=3b > inet 172.31.0.28 netmask 0xffffff00 broadcast 172.31.0.255 > inet 172.31.0.163 netmask 0xffffffff broadcast 172.31.0.163 > ether 00:19:b9:e4:4d:cc > media: Ethernet autoselect (1000baseTX ) > status: active > > > After applying both David and Sephe's patches I have yet to get a system > in a state where it is stable with jumbo frames enabled, the systems > crash almost immediately after the switch changes the port state > (Spanning tree) from LEARNING to FORWARDING. The output from this crash > can be found attached as crash-1.txt.gz. > > If the frame size is left at 1500 then the interface seems stable, > however I can't fully test this as the interface is connected to a GigE > only network with an mtu of 8192. > > If BCE_DEBUG is remove from if_bcereg.h then the system just exhibits > the original problem and may or may not crash. > > The next test was to try the kernel with BCE_DEBUG and with the > following extra patch (so that the driver does not jump to the > breakpoint when an unexpected mbuf is found in the rx buffer). > > --- if_bce.c (revision 62) > +++ if_bce.c (revision 66) > @@ -4050,7 +4050,8 @@ > DBRUNIF((!(rxbd->rx_bd_flags & RX_BD_FLAGS_END)), > BCE_PRINTF("%s(%d): Unexpected mbuf > found in rx_bd[0x%04X]!\n", > __FILE__, __LINE__, sw_chain_cons); > - bce_breakpoint(sc)); > + bce_dump_mbuf(sc, m)); > +// bce_breakpoint(sc)); > > /* > * ToDo: If the received packet is small enough > > > With this patch the system boots and does not crash straight away, > however it is almost completely unusable. The output with this kernel > can be found attached as crash-2.txt.gz. Also this causes the following > new error message: > > fgrep -n leak crash-2.txt > 3194:bce0: /usr/src/sys/dev/bce/if_bce.c(3842): Memory leak! Lost 114 > mbufs from rx chain! > > Has no one else come across this problem, or are Jumbo frames not widely > used? > > Tom > It would seem that the crash can be simulated just by increasing the MTU above 1500 (tested in single user mode). Tom