From owner-freebsd-net@FreeBSD.ORG Tue Mar 28 21:15:59 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A014B16A422; Tue, 28 Mar 2006 21:15:59 +0000 (UTC) (envelope-from oleg@lath.rinet.ru) Received: from lath.rinet.ru (lath.rinet.ru [195.54.192.90]) by mx1.FreeBSD.org (Postfix) with ESMTP id A722A443D2; Tue, 28 Mar 2006 20:55:30 +0000 (GMT) (envelope-from oleg@lath.rinet.ru) Received: from lath.rinet.ru (localhost [127.0.0.1]) by lath.rinet.ru (8.13.4/8.13.4) with ESMTP id k2SKtF0P061339 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 29 Mar 2006 00:55:15 +0400 (MSD) (envelope-from oleg@lath.rinet.ru) Received: (from oleg@localhost) by lath.rinet.ru (8.13.4/8.13.4/Submit) id k2SKtEgo061338; Wed, 29 Mar 2006 00:55:14 +0400 (MSD) (envelope-from oleg) Date: Wed, 29 Mar 2006 00:55:14 +0400 From: Oleg Bulyzhin To: Doug Ambrisko Message-ID: <20060328205514.GB60718@lath.rinet.ru> References: <20060328202525.GA60718@lath.rinet.ru> <200603282038.k2SKcguE014226@ambrisko.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="m51xatjYGsM+13rf" Content-Disposition: inline In-Reply-To: <200603282038.k2SKcguE014226@ambrisko.com> User-Agent: Mutt/1.5.11 Cc: "Stephen P. Cravey" , ambrisko@freebsd.org, freebsd-net@freebsd.org Subject: Re: IPMI and bge (again) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Mar 2006 21:15:59 -0000 --m51xatjYGsM+13rf Content-Type: multipart/mixed; boundary="O5XBE6gyVG5Rl6Rj" Content-Disposition: inline --O5XBE6gyVG5Rl6Rj Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Mar 28, 2006 at 12:38:42PM -0800, Doug Ambrisko wrote: > Oleg Bulyzhin writes: > | On Tue, Mar 28, 2006 at 11:24:06AM -0600, Stephen P. Cravey wrote: > | > Does anyone have chip documentation on the broadcom BGE chips? I'm=20 > having an ongoing issue with IPMI that I'd really like to get resolved. T= he issue seems to be that during the driver start sequence, a flag is getti= ng set in the chip that's disabling the IPMI passthrough that I need in ord= er for data destined for a second mac address on the interface to recieve p= ackets. Or, a flag that this process needs isn't getting set. Not sure whic= h, but I could really use some help here.=20 > | >=20 > | > Or should I ask the frequent committers to the driver directly? > | >=20 > | > Thank you. > | >=20 > | > PR: > | > kern/79143 > | > kern/88741 > | > _______________________________________________ > |=20 > | Could you please test attached patch (RELENG_6 version)? It's slightly= =20 > | modified version of Doug Ambrisko > | (ambrisko@FreeBSD.org) patch (original version didnt work for me: > | bcm5721 @ hp proliant dl145g2 server). >=20 > I don't see it attached. I'd like to see your changes and I'd like know > of your plans about getting it in. >=20 > Thanks, >=20 > Doug A. Oops... my fault. Should be attached now. --=20 Oleg. --O5XBE6gyVG5Rl6Rj Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="bge_asf.patch" Content-Transfer-Encoding: quoted-printable Index: if_bgereg.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/sys/dev/bge/if_bgereg.h,v retrieving revision 1.36.2.4 diff -u -r1.36.2.4 if_bgereg.h --- if_bgereg.h 5 Feb 2006 18:07:15 -0000 1.36.2.4 +++ if_bgereg.h 28 Mar 2006 20:18:15 -0000 @@ -74,6 +74,11 @@ #define BGE_SOFTWARE_GENCOMM 0x00000B50 #define BGE_SOFTWARE_GENCOMM_SIG 0x00000B54 #define BGE_SOFTWARE_GENCOMM_NICCFG 0x00000B58 +#define BGE_SOFTWARE_GENCOMM_FW 0x00000B78 +#define BGE_FW_DRV_ALIVE 0x00000001 +#define BGE_FW_PAUSE 0x00000002 +#define BGE_SOFTWARE_GENNCOMM_FW_LEN 0x00000B7C +#define BGE_SOFTWARE_GENNCOMM_FW_DATA 0x00000B80 #define BGE_SOFTWARE_GENCOMM_END 0x00000FFF #define BGE_UNMAPPED 0x00001000 #define BGE_UNMAPPED_END 0x00001FFF @@ -1627,6 +1632,7 @@ #define BGE_MODE_CTL 0x6800 #define BGE_MISC_CFG 0x6804 #define BGE_MISC_LOCAL_CTL 0x6808 +#define BGE_CPU_EVENT 0x6810 #define BGE_EE_ADDR 0x6838 #define BGE_EE_DATA 0x683C #define BGE_EE_CTL 0x6840 @@ -2009,6 +2015,7 @@ #define BGE_HWCFG_VOLTAGE 0x00000003 #define BGE_HWCFG_PHYLED_MODE 0x0000000C #define BGE_HWCFG_MEDIA 0x00000030 +#define BGE_HWCFG_ASF 0x00000080 =20 #define BGE_VOLTAGE_1POINT3 0x00000000 #define BGE_VOLTAGE_1POINT8 0x00000001 @@ -2385,6 +2392,10 @@ int val; }; =20 +#define ASF_ENABLE 1 +#define ASF_NEW_HANDSHAKE 2 +#define ASF_STACKUP 4 + struct bge_softc { struct ifnet *bge_ifp; /* interface info */ device_t bge_dev; @@ -2403,6 +2414,8 @@ u_int8_t bge_asicrev; u_int8_t bge_chiprev; u_int8_t bge_no_3_led; + u_int8_t bge_asf_mode; + u_int8_t bge_asf_count; u_int8_t bge_pcie; struct bge_ring_data bge_ldata; /* rings */ struct bge_chain_data bge_cdata; /* mbufs */ Index: if_bge.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/sys/dev/bge/if_bge.c,v retrieving revision 1.91.2.13 diff -u -r1.91.2.13 if_bge.c --- if_bge.c 4 Mar 2006 09:34:48 -0000 1.91.2.13 +++ if_bge.c 28 Mar 2006 20:18:21 -0000 @@ -209,6 +209,7 @@ static void bge_txeof (struct bge_softc *); static void bge_rxeof (struct bge_softc *); =20 +static void bge_asf_driver_up (struct bge_softc *); static void bge_tick_locked (struct bge_softc *); static void bge_tick (void *); static void bge_stats_update (struct bge_softc *); @@ -271,7 +272,12 @@ int count); #endif =20 -static void bge_reset (struct bge_softc *); +#define BGE_RESET_START 1 +#define BGE_RESET_STOP 2 +static void bge_sig_post_reset(struct bge_softc *, int); +static void bge_sig_legacy(struct bge_softc *, int); +static void bge_sig_pre_reset(struct bge_softc *, int); +static int bge_reset (struct bge_softc *); static void bge_link_upd (struct bge_softc *); =20 static device_method_t bge_methods[] =3D { @@ -609,6 +615,18 @@ DELAY(40); } =20 + if (sc->bge_asf_mode & ASF_STACKUP + && pci_get_device(sc->bge_dev) =3D=3D BCOM_DEVICEID_BCM5721) { + switch (reg) { + case MII_BMSR: + val |=3D BMSR_100TXFDX | BMSR_100TXHDX | BMSR_10TFDX + | BMSR_10THDX | BMSR_EXTSTAT + | BMSR_MFPS + | BMSR_ANEG | BMSR_EXTCAP; + break; + } + } + if (val & BGE_MICOMM_READFAIL) return(0); =20 @@ -660,10 +678,10 @@ { struct bge_softc *sc; struct mii_data *mii; - sc =3D device_get_softc(dev); mii =3D device_get_softc(sc->bge_miibus); =20 + BGE_CLRBIT(sc, BGE_MAC_MODE, BGE_MACMODE_PORTMODE); if (IFM_SUBTYPE(mii->mii_media_active) =3D=3D IFM_1000_T) { BGE_SETBIT(sc, BGE_MAC_MODE, BGE_PORTMODE_GMII); @@ -1007,6 +1025,80 @@ return; } =20 +static void +bge_sig_pre_reset(sc, type) + struct bge_softc *sc; + int type; +{ + bge_writemem_ind(sc, BGE_SOFTWARE_GENCOMM, BGE_MAGIC_NUMBER); + + if (sc->bge_asf_mode & ASF_NEW_HANDSHAKE) { + switch (type) { + case BGE_RESET_START: + bge_writemem_ind(sc, BGE_SDI_STATUS, 0x1); /* START */ + break; + case BGE_RESET_STOP: + bge_writemem_ind(sc, BGE_SDI_STATUS, 0x2); /* UNLOAD */ + break; + } + } +} + +static void +bge_sig_post_reset(sc, type) + struct bge_softc *sc; + int type; +{ + if (sc->bge_asf_mode & ASF_NEW_HANDSHAKE) { + switch (type) { + case BGE_RESET_START: + bge_writemem_ind(sc, BGE_SDI_STATUS, 0x80000001);=20 + /* START DONE */ + break; + case BGE_RESET_STOP: + bge_writemem_ind(sc, BGE_SDI_STATUS, 0x80000002);=20 + break; + } + } +} + +static void +bge_sig_legacy(sc, type) + struct bge_softc *sc; + int type; +{ + if (sc->bge_asf_mode) { + switch (type) { + case BGE_RESET_START: + bge_writemem_ind(sc, BGE_SDI_STATUS, 0x1); /* START */ + break; + case BGE_RESET_STOP: + bge_writemem_ind(sc, BGE_SDI_STATUS, 0x2); /* UNLOAD */ + break; + } + } +} + +void bge_stop_fw(struct bge_softc *); +void +bge_stop_fw(sc) + struct bge_softc *sc; +{ + int i; + + if (sc->bge_asf_mode) { + bge_writemem_ind(sc, BGE_SOFTWARE_GENCOMM_FW, BGE_FW_PAUSE); + CSR_WRITE_4(sc, BGE_CPU_EVENT, + CSR_READ_4(sc, BGE_CPU_EVENT) !=3D (1 << 14)); + + for (i =3D 0; i < 100; i++ ) { + if (!(CSR_READ_4(sc, BGE_CPU_EVENT) & (1 << 14))) + break; + DELAY(10); + } + } +} + /* * Do endian, PCI and DMA initialization. Also check the on-board ROM * self-test results. @@ -1018,9 +1110,10 @@ int i; u_int32_t dma_rw_ctl; =20 - /* Set endian type before we access any non-PCI registers. */ + /* Set endianness before we access any non-PCI registers. */ pci_write_config(sc->bge_dev, BGE_PCI_MISC_CTL, BGE_INIT, 4); =20 +#ifdef DJA /* * Check the 'ROM failed' bit on the RX CPU to see if * self-tests passed. @@ -1029,7 +1122,7 @@ device_printf(sc->bge_dev, "RX CPU self-diagnostics failed!\n"); return(ENODEV); } - +#endif /* Clear the MAC control register */ CSR_WRITE_4(sc, BGE_MAC_MODE, 0); =20 @@ -1101,6 +1194,9 @@ BGE_MODECTL_MAC_ATTN_INTR|BGE_MODECTL_HOST_SEND_BDS| BGE_MODECTL_TX_NO_PHDR_CSUM); =20 + if (sc->bge_asf_mode & ASF_STACKUP) + BGE_SETBIT(sc, BGE_MODE_CTL, BGE_MODECTL_STACKUP); + /* * Disable memory write invalidate. Apparently it is not supported * properly by these devices. @@ -2054,6 +2150,7 @@ u_int32_t mac_tmp =3D 0; u_char eaddr[6]; int error =3D 0, rid; + int trys; =20 sc =3D device_get_softc(dev); sc->bge_dev =3D dev; @@ -2121,8 +2218,35 @@ } } =20 + sc->bge_asf_mode =3D 0; + /* Try to reset the chip. */ - bge_reset(sc); + bge_stop_fw(sc); + bge_sig_pre_reset(sc, BGE_RESET_STOP); + if (bge_reset(sc)) { + device_printf(sc->bge_dev, "chip reset failed\n"); + bge_release_resources(sc); + error =3D ENXIO; + goto fail; + } + + if (bge_readmem_ind(sc, BGE_SOFTWARE_GENCOMM_SIG) + =3D=3D BGE_MAGIC_NUMBER) { + if (bge_readmem_ind(sc, BGE_SOFTWARE_GENCOMM_NICCFG) + & BGE_HWCFG_ASF) { + sc->bge_asf_mode |=3D ASF_ENABLE; + if (CSR_READ_4(sc, BGE_MODE_CTL) + & BGE_MODECTL_STACKUP ) { + sc->bge_asf_mode |=3D ASF_STACKUP; + } + if (sc->bge_asicrev =3D=3D BGE_ASICREV_BCM5750) { + sc->bge_asf_mode |=3D ASF_NEW_HANDSHAKE; + } + } + } + + bge_sig_legacy(sc, BGE_RESET_STOP); + bge_sig_post_reset(sc, BGE_RESET_STOP); =20 if (bge_chipinit(sc)) { device_printf(sc->bge_dev, "chip initialization failed\n"); @@ -2252,13 +2376,26 @@ /* * Do transceiver setup. */ + BGE_CLRBIT(sc, BGE_MODE_CTL, BGE_MODECTL_STACKUP); +again: + bge_asf_driver_up(sc); + + trys =3D 0; if (mii_phy_probe(dev, &sc->bge_miibus, bge_ifmedia_upd, bge_ifmedia_sts)) { + if (trys++ < 4) { + device_printf(sc->bge_dev, "Try again\n"); + bge_miibus_writereg(sc->bge_dev, 1, MII_BMCR, BMCR_RESET); + goto again; + } + device_printf(sc->bge_dev, "MII without any PHY!\n"); bge_release_resources(sc); error =3D ENXIO; goto fail; } + if (sc->bge_asf_mode & ASF_STACKUP) + BGE_SETBIT(sc, BGE_MODE_CTL, BGE_MODECTL_STACKUP); } =20 /* @@ -2372,7 +2509,7 @@ return; } =20 -static void +static int bge_reset(sc) struct bge_softc *sc; { @@ -2435,11 +2572,18 @@ sc->bge_asicrev !=3D BGE_ASICREV_BCM5750) CSR_WRITE_4(sc, BGE_MARB_MODE, BGE_MARBMODE_ENABLE); =20 +#if DJA + DELAY(1000000); /* - * Prevent PXE restart: write a magic number to the - * general communications memory at 0xB50. + * Check the 'ROM failed' bit on the RX CPU to see if + * self-tests passed. */ - bge_writemem_ind(sc, BGE_SOFTWARE_GENCOMM, BGE_MAGIC_NUMBER); + if (CSR_READ_4(sc, BGE_RXCPU_MODE) & BGE_RXCPUMODE_ROMFAIL) { + device_printf(sc->bge_dev, "RX CPU self-diagnostics failed!\n"); + return(ENODEV); + } +#endif + /* * Poll the value location we just wrote until * we see the 1's complement of the magic number. @@ -2455,7 +2599,7 @@ =20 if (i =3D=3D BGE_TIMEOUT) { device_printf(sc->bge_dev, "firmware handshake timed out\n"); - return; + return(0); } =20 /* @@ -2475,6 +2619,8 @@ /* Fix up byte swapping */ CSR_WRITE_4(sc, BGE_MODE_CTL, BGE_DMA_SWAP_OPTIONS| BGE_MODECTL_BYTESWAP_DATA); + if (sc->bge_asf_mode & ASF_STACKUP) + BGE_SETBIT(sc, BGE_MODE_CTL, BGE_MODECTL_STACKUP); =20 CSR_WRITE_4(sc, BGE_MAC_MODE, 0); =20 @@ -2499,7 +2645,7 @@ } DELAY(10000); =20 - return; + return(0); } =20 /* @@ -2680,6 +2826,7 @@ { struct bge_tx_bd *cur_tx =3D NULL; struct ifnet *ifp; + int acked =3D 0; =20 BGE_LOCK_ASSERT(sc); =20 @@ -2716,11 +2863,36 @@ } sc->bge_txcnt--; BGE_INC(sc->bge_tx_saved_considx, BGE_TX_RING_CNT); - ifp->if_timer =3D 0; + ifp->if_timer =3D (sc->bge_txcnt =3D=3D 0) ? 0 : 5; + + acked =3D 1; } =20 if (cur_tx !=3D NULL) ifp->if_drv_flags &=3D ~IFF_DRV_OACTIVE; + + switch (sc->bge_chipid) { + case BGE_CHIPID_BCM5701_A0: + case BGE_CHIPID_BCM5701_B0: + case BGE_CHIPID_BCM5701_B2: + case BGE_CHIPID_BCM5701_B5: + /* + * Sometimes the RX engine never gets started. We detect + * this by checking to see if we have sent some packets and + * never got a packet. If we haven't got a packet reset. + * this is only triggered if we sent packets. + */ +=20 + if (acked && ifp->if_opackets > 5 && !ifp->if_ipackets) { + device_printf(sc->bge_dev, "reset the card packets out %ld in %ld\n" + , ifp->if_opackets, + ifp->if_ipackets); + ifp->if_flags &=3D~ IFF_DRV_RUNNING; + bge_stop(sc); + bge_init(sc); + } + break; + } } =20 #ifdef DEVICE_POLLING @@ -2728,7 +2900,7 @@ bge_poll(struct ifnet *ifp, enum poll_cmd cmd, int count) { struct bge_softc *sc =3D ifp->if_softc; -=09 + BGE_LOCK(sc); if (ifp->if_drv_flags & IFF_DRV_RUNNING) bge_poll_locked(ifp, cmd, count); @@ -2834,6 +3006,26 @@ } =20 static void +bge_asf_driver_up(sc) + struct bge_softc *sc; +{ + if (sc->bge_asf_mode & ASF_STACKUP) { + /* Send ASF heartbeat aprox. every 2s */ + if (sc->bge_asf_count) + sc->bge_asf_count --; + else { + sc->bge_asf_count =3D 5; + bge_writemem_ind(sc, BGE_SOFTWARE_GENCOMM_FW, + BGE_FW_DRV_ALIVE); + bge_writemem_ind(sc, BGE_SOFTWARE_GENNCOMM_FW_LEN, 4); + bge_writemem_ind(sc, BGE_SOFTWARE_GENNCOMM_FW_DATA, 3); + CSR_WRITE_4(sc, BGE_CPU_EVENT, + CSR_READ_4(sc, BGE_CPU_EVENT) !=3D (1 << 14)); + } + } + } + +static void bge_tick_locked(sc) struct bge_softc *sc; { @@ -2849,7 +3041,9 @@ =20 if (!sc->bge_tbi) { mii =3D device_get_softc(sc->bge_miibus); - mii_tick(mii); + /* Don't mess with the PHY in IPMI/ASF mode */ + if (!((sc->bge_asf_mode & ASF_STACKUP) && (sc->bge_link))) + mii_tick(mii); } else { /* * Since in TBI mode auto-polling can't be used we should poll @@ -2866,6 +3060,8 @@ } } =20 + bge_asf_driver_up(sc); + callout_reset(&sc->bge_stat_ch, hz, bge_tick, sc); } =20 @@ -3215,7 +3411,13 @@ =20 /* Cancel pending I/O and flush buffers. */ bge_stop(sc); + + bge_stop_fw(sc); + bge_sig_pre_reset(sc, BGE_RESET_START); bge_reset(sc); + bge_sig_legacy(sc, BGE_RESET_START); + bge_sig_post_reset(sc, BGE_RESET_START); + bge_chipinit(sc); =20 /* @@ -3298,14 +3500,14 @@ CSR_WRITE_4(sc, BGE_HCC_TX_MAX_COAL_BDS_INT, 1); } else #endif -=09 + /* Enable host interrupts. */ { BGE_SETBIT(sc, BGE_PCI_MISC_CTL, BGE_PCIMISCCTL_CLEAR_INTA); BGE_CLRBIT(sc, BGE_PCI_MISC_CTL, BGE_PCIMISCCTL_MASK_PCI_INTR); CSR_WRITE_4(sc, BGE_MBX_IRQ0_LO, 0); } -=09 + bge_ifmedia_upd(ifp); =20 ifp->if_drv_flags |=3D IFF_DRV_RUNNING; @@ -3646,7 +3848,16 @@ /* * Tell firmware we're shutting down. */ - BGE_CLRBIT(sc, BGE_MODE_CTL, BGE_MODECTL_STACKUP); + + bge_stop_fw(sc); + bge_sig_pre_reset(sc, BGE_RESET_STOP); + bge_reset(sc); + bge_sig_legacy(sc, BGE_RESET_STOP); + bge_sig_post_reset(sc, BGE_RESET_STOP); + if (sc->bge_asf_mode & ASF_STACKUP) + BGE_SETBIT(sc, BGE_MODE_CTL, BGE_MODECTL_STACKUP); + else + BGE_CLRBIT(sc, BGE_MODE_CTL, BGE_MODECTL_STACKUP); =20 /* Free the RX lists. */ bge_free_rx_ring_std(sc); @@ -3670,6 +3881,7 @@ /* * If we are called from bge_detach(), mii is already NULL. */ + //if (mii !=3D NULL && !(sc->bge_asf_mode & ASF_ENABLE)) { if (mii !=3D NULL) { ifm =3D mii->mii_media.ifm_cur; mtmp =3D ifm->ifm_media; --O5XBE6gyVG5Rl6Rj-- --m51xatjYGsM+13rf Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFEKaKyryLc73jOEF8RAmlhAJoDwsxBwbGnyU6XYNVX7RNIJ3PV2wCglMni nLSqN0jC3Vj3vB5dT08uid4= =xdGQ -----END PGP SIGNATURE----- --m51xatjYGsM+13rf--