From owner-freebsd-net@FreeBSD.ORG Thu Aug 24 17:36:38 2006 Return-Path: X-Original-To: net@FreeBSD.org Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4413A16A4DF; Thu, 24 Aug 2006 17:36:38 +0000 (UTC) (envelope-from davidch@broadcom.com) Received: from mms2.broadcom.com (mms2.broadcom.com [216.31.210.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id B013543D49; Thu, 24 Aug 2006 17:36:32 +0000 (GMT) (envelope-from davidch@broadcom.com) Received: from 10.10.64.154 by mms2.broadcom.com with ESMTP (Broadcom SMTP Relay (Email Firewall v6.2.0)); Thu, 24 Aug 2006 10:36:17 -0700 X-Server-Uuid: D9EB6F12-1469-4C1C-87A2-5E4C0D6F9D06 Received: by mail-irva-10.broadcom.com (Postfix, from userid 47) id 6A43B2AF; Thu, 24 Aug 2006 10:36:17 -0700 (PDT) Received: from mail-irva-8.broadcom.com (mail-irva-8 [10.10.64.221]) by mail-irva-10.broadcom.com (Postfix) with ESMTP id 444B52AE; Thu, 24 Aug 2006 10:36:17 -0700 (PDT) Received: from mail-irva-12.broadcom.com (mail-irva-12.broadcom.com [10.10.64.146]) by mail-irva-8.broadcom.com (MOS 3.7.5a-GA) with ESMTP id EDN93093; Thu, 24 Aug 2006 10:36:17 -0700 (PDT) Received: from NT-IRVA-0750.brcm.ad.broadcom.com (nt-irva-0750 [10.8.194.64]) by mail-irva-12.broadcom.com (Postfix) with ESMTP id EEB3A69CA3; Thu, 24 Aug 2006 10:36:16 -0700 (PDT) X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Date: Thu, 24 Aug 2006 10:36:17 -0700 Message-ID: <09BFF2FA5EAB4A45B6655E151BBDD90301D43265@NT-IRVA-0750.brcm.ad.broadcom.com> In-Reply-To: <20060824075253.GH76666@cell.sick.ru> Thread-Topic: bge(4) one packet wedge Thread-Index: AcbHUlhNeGOZxoDJR+iqP9PqN7O7KQAUJPCA From: "David Christensen" To: "Gleb Smirnoff" X-TMWD-Spam-Summary: SEV=1.1; DFV=A2006082406; IFV=2.0.6,4.0-7; RPD=4.00.0004; RPDID=303030312E30413031303230352E34344544453237372E303030342D422D466E61662B777136512B6F5752675750386F344B49513D3D; ENG=IBF; TS=20060824173619; CAT=NONE; CON=NONE; X-MMS-Spam-Filter-ID: A2006082406_4.00.0004_2.0.6,4.0-7 X-WSS-ID: 68F33C1B3881746646-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Cc: brad@openbsd.org, oleg@FreeBSD.org, net@FreeBSD.org Subject: RE: bge(4) one packet wedge X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Aug 2006 17:36:38 -0000 > On Wed, Aug 23, 2006 at 12:53:49PM -0700, David Christensen wrote: > D> This "lost interrupt" type of problem is addressed by the=20 > use of the > D> status_tag=20 > D> field in the status block. (Listed as bge_rsvd0 in the=20 > bge_status_block > D> structure).=20 > D> Everytime the status block is updated a new tag value is=20 > written to the > D> status block. =20 > D> When the ISR starts the driver should record the=20 > status_tag value. At > D> the end > D> of the ISR, the driver should compare the current=20 > status_tag value is > D> the status > D> block with the value recorded on entry to the ISR. If the=20 > values are > D> the same > D> then no additional status block updates have occurred so=20 > there shouldn't > D> be > D> any packets hanging around. If the values are different=20 > then additional > D> packets > D> or completions are waiting around so the ISR should loop=20 > around again. > D> At the=20 > D> end of the ISR the driver will write the status_tag value it last > D> handled to a > D> mailbox register, letting the hardware know the last=20 > status block update > D> handled. > D> If necessary the hardware will generate a new interrupt=20 > and start the > D> process over > D> again. > D>=20 > D> This entire process should be included in the Linux=20 > driver, I don't see > D> it being > D> used in the bge driver (bge_intr()). >=20 > In the Linux driver for the not tag capable controllers there=20 > is a following > comment and code: >=20 > /* In INTx mode, it is possible for the interrupt to arrive at > * the CPU before the status block posted prior to=20 > the interrupt. > * Reading the PCI State register will confirm whether the > * interrupt is ours and will flush the status block. > */ > if ((sblk->status & SD_STATUS_UPDATED) || > !(tr32(TG3PCI_PCISTATE) & PCISTATE_INT_NOT_ACTIVE)) { >=20 > So, I tried to add (void)pci_read_config(sc->bge_dev,=20 > BGE_PCI_PCISTATE, 4) > to the bge_intr(). Unfortunately, this didn't help. >=20 This fix is really designed to handle bridges that haven't posted the status block memory write ahead of the PCI interrupt and I wouldn't expect it to=20 resolve the reported problem. A better solution would be to try and minimize the time between the last status block check and the time the interrupt is re-enabled. Unfortunately I don't think the problem can be completely eliminated with even the most creative coding solutions but I'd be happy to be proven wrong. Dave