From owner-freebsd-net@FreeBSD.ORG Wed Sep 13 17:03:27 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3116216A47E for ; Wed, 13 Sep 2006 17:03:27 +0000 (UTC) (envelope-from davidch@broadcom.com) Received: from MMS3.broadcom.com (mms3.broadcom.com [216.31.210.19]) by mx1.FreeBSD.org (Postfix) with ESMTP id CE46C43D95 for ; Wed, 13 Sep 2006 17:03:00 +0000 (GMT) (envelope-from davidch@broadcom.com) Received: from 10.10.64.154 by MMS3.broadcom.com with ESMTP (Broadcom SMTP Relay (Email Firewall v6.2.2)); Wed, 13 Sep 2006 10:02:47 -0700 X-Server-Uuid: 450F6D01-B290-425C-84F8-E170B39A25C9 Received: by mail-irva-10.broadcom.com (Postfix, from userid 47) id 613E12AF; Wed, 13 Sep 2006 10:02:47 -0700 (PDT) Received: from mail-irva-8.broadcom.com (mail-irva-8 [10.10.64.221]) by mail-irva-10.broadcom.com (Postfix) with ESMTP id 3E3142AE; Wed, 13 Sep 2006 10:02:47 -0700 (PDT) Received: from mail-irva-12.broadcom.com (mail-irva-12.broadcom.com [10.10.64.146]) by mail-irva-8.broadcom.com (MOS 3.7.5a-GA) with ESMTP id EEZ39426; Wed, 13 Sep 2006 10:02:46 -0700 (PDT) Received: from NT-IRVA-0750.brcm.ad.broadcom.com (nt-irva-0750 [10.8.194.64]) by mail-irva-12.broadcom.com (Postfix) with ESMTP id 5F26869CA3; Wed, 13 Sep 2006 10:02:46 -0700 (PDT) X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Date: Wed, 13 Sep 2006 10:02:45 -0700 Message-ID: <09BFF2FA5EAB4A45B6655E151BBDD90301F133F0@NT-IRVA-0750.brcm.ad.broadcom.com> In-Reply-To: <787bbe1c0609130609l33fb29dawc465b7bcfb2f430e@mail.gmail.com> Thread-Topic: Rapid link state changes on bge(4) interface Thread-Index: AcbXNf+rMuLpmU8hToqxrVEnck8boAAH871A From: "David Christensen" To: "Slawek Zak" , freebsd-net@freebsd.org X-TMWD-Spam-Summary: TS=20060913170250; SEV=2.0.2; DFV=A2006091306; IFV=2.0.4,4.0-8; RPD=4.00.0004; ENG=IBF; RPDID=303030312E30413031303230342E34353038333830342E303035382D452D4A4456364D43776A676C427A6D4741774B73695447673D3D; CAT=NONE; CON=NONE X-MMS-Spam-Filter-ID: A2006091306_4.00.0004_4.0-8 X-WSS-ID: 6916E63D230999763-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Cc: Subject: RE: Rapid link state changes on bge(4) interface X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Sep 2006 17:03:27 -0000 > I'm testing network failover on IBM BladeCenter running FreeBSD 6.1 > STABLE for Sep 6th. >=20 > I suspect a problem with link state change detection in bge code. When > I disable internal port on chassis built-in ethernet switch, kernel > floods syslog with messages about link state changes and coalescing > them. Log snippet follows: >=20 > Sep 13 14:58:28 w3-6 kernel: bge1: link state changed to DOWN > Sep 13 14:58:28 w3-6 kernel: bge1: link state changed to UP > Sep 13 14:58:29 w3-6 kernel: bge1: link state changed to DOWN > Sep 13 14:58:29 w3-6 kernel: bge1: link state changed to UP > Sep 13 14:58:29 w3-6 kernel: bge1: link state changed to DOWN > Sep 13 14:58:29 w3-6 kernel: bge1: 4 link states coalesced > Sep 13 14:58:29 w3-6 kernel: bge1: link state changed to DOWN > Sep 13 14:58:29 w3-6 kernel: bge1: 11 link states coalesced > Sep 13 14:58:29 w3-6 kernel: bge1: link state changed to UP > Sep 13 14:58:30 w3-6 kernel: bge1: link state changed to DOWN > Sep 13 14:58:30 w3-6 kernel: bge1: 3 link states coalesced > Sep 13 14:58:30 w3-6 kernel: bge1: link state changed to UP > Sep 13 14:58:30 w3-6 kernel: bge1: 7 link states coalesced > Sep 13 14:58:30 w3-6 kernel: bge1: link state changed to DOWN > Sep 13 14:58:30 w3-6 kernel: bge1: 4 link states coalesced > Sep 13 14:58:30 w3-6 kernel: bge1: link state changed to DOWN > Sep 13 14:58:30 w3-6 kernel: bge1: 2 link states coalesced >=20 > As you can see, messages are generated in rapid succession and > therefore any probing of link state change by ng_one2many for > interface failover is meaningless. Ethernet switch doesn't register > and log any interface state changes after disabling this port. LS20 > blades use chipset 8850. My firmware is 3.38, full changelog, if it is > of any help, is here: >=20 > http://www-307.ibm.com/pc/support/site.wss/license.do?filename =3Dpc_servers/brcm_fw_nic_12021_anyos_anycpu.chg >=20 > Any ideas what might be wrong? I can't access the information on this web site through Mozilla after clicking "I Accept". Is this a 5704 controller using a SerDes link? I'm familiar with some Blade Center problems in the past (which I think were related to Sigdet) though I'm not in the office and can't=20 look into it right away. A comparison between the serdes code in the Linux driver vs. the FreeBSD driver is probably the first area to investigate. Dave