From owner-freebsd-current@FreeBSD.ORG Mon Jul 9 03:54:14 2007 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 16B6616A468 for ; Mon, 9 Jul 2007 03:54:14 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.177]) by mx1.freebsd.org (Postfix) with ESMTP id DD01413C489 for ; Mon, 9 Jul 2007 03:54:13 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: by wa-out-1112.google.com with SMTP id j37so1200451waf for ; Sun, 08 Jul 2007 20:54:13 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:received:received:date:from:to:subject:message-id:reply-to:mime-version:content-type:content-disposition:user-agent; b=X8sdDG0YIT+p8hT3c29AceJQvyJtS3ODjH++72xysBT0KnwpIuufHEQL/AD6k2z5M1p3eNh79RuFKwxD4qyfKuuSgjLwCp+89O3V9I5ViW7dWVEIARePrf7dWY4oa4D48xJHeea5H/v5dfQuw6g9/dH+NtBovMhFT6YxnTJp2kE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:date:from:to:subject:message-id:reply-to:mime-version:content-type:content-disposition:user-agent; b=nn4XTdVVrxDwhOHw6SOQlymp6lF9rJ/6x6Mj4Z/9QKoOwEV0J0YH5A3S6yUJalo9QBn0uzE5h4O7tGQYsjnvnWoEkWs2Uc3SzjaT8cFWr5zUxVmLM9vJTKVGTfrU3jDHpYXp7z5Vy0n8eZPOAMwR2NzMkUf2vcp6WvCY70Ra8pg= Received: by 10.115.32.1 with SMTP id k1mr2730607waj.1183953253278; Sun, 08 Jul 2007 20:54:13 -0700 (PDT) Received: from michelle.cdnetworks.co.kr ( [211.53.35.84]) by mx.google.com with ESMTP id j29sm24007916waf.2007.07.08.20.54.10 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 08 Jul 2007 20:54:11 -0700 (PDT) Received: from michelle.cdnetworks.co.kr (localhost.cdnetworks.co.kr [127.0.0.1]) by michelle.cdnetworks.co.kr (8.13.5/8.13.5) with ESMTP id l693s5P2002138 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 9 Jul 2007 12:54:05 +0900 (KST) (envelope-from pyunyh@gmail.com) Received: (from yongari@localhost) by michelle.cdnetworks.co.kr (8.13.5/8.13.5/Submit) id l693s4B4002137 for freebsd-current@FreeBSD.org; Mon, 9 Jul 2007 12:54:04 +0900 (KST) (envelope-from pyunyh@gmail.com) Date: Mon, 9 Jul 2007 12:54:04 +0900 From: Pyun YongHyeon To: freebsd-current@FreeBSD.org Message-ID: <20070709035404.GF1062@cdnetworks.co.kr> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="ALfTUftag+2gvp1h" Content-Disposition: inline User-Agent: Mutt/1.4.2.1i Cc: Subject: CFT: re(4) MSI support X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jul 2007 03:54:14 -0000 --ALfTUftag+2gvp1h Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, I've added MSI capability to re(4) and it would fix occasional watchdog timeout erros seen on re(4) when its interrupt is shared with other devices. The fix only affects PCIe based hardwares. In these days, more and more hardwares use PCIe based hardwares and these hardwares surly share interrupts with other devices. If you ever encountered watchdog timeouts on re(4) with PCIe hardwares please try attached patch and report back the result. Thanks. -- Regards, Pyun YongHyeon --ALfTUftag+2gvp1h Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="re.msi.patch" Index: if_re.c =================================================================== RCS file: /home/ncvs/src/sys/dev/re/if_re.c,v retrieving revision 1.91 diff -u -r1.91 if_re.c --- if_re.c 6 Jul 2007 00:05:11 -0000 1.91 +++ if_re.c 9 Jul 2007 01:52:23 -0000 @@ -160,6 +160,10 @@ #include +/* Tunables. */ +static int msi_disable = 0; +TUNABLE_INT("hw.re.msi_disable", &msi_disable); + #define RE_CSUM_FEATURES (CSUM_IP | CSUM_TCP | CSUM_UDP) /* @@ -1159,6 +1163,7 @@ int hwrev; u_int16_t re_did = 0; int error = 0, rid, i; + int msic, reg; sc = device_get_softc(dev); sc->rl_dev = dev; @@ -1185,15 +1190,45 @@ sc->rl_btag = rman_get_bustag(sc->rl_res); sc->rl_bhandle = rman_get_bushandle(sc->rl_res); - /* Allocate interrupt */ - rid = 0; - sc->rl_irq = bus_alloc_resource_any(dev, SYS_RES_IRQ, &rid, - RF_SHAREABLE | RF_ACTIVE); + msic = 0; + if (pci_find_extcap(dev, PCIY_EXPRESS, ®) == 0) { + msic = pci_msi_count(dev); + if (bootverbose) + device_printf(dev, "MSI count : %d\n", msic); + } + if (msic == RL_MSI_MESSAGES && msi_disable == 0) { + if (pci_alloc_msi(dev, &msic) == 0) { + if (msic == RL_MSI_MESSAGES) { + device_printf(dev, "Using %d MSI messages\n", + msic); + sc->rl_msi = 1; + } else + pci_release_msi(dev); + } + } - if (sc->rl_irq == NULL) { - device_printf(dev, "couldn't map interrupt\n"); - error = ENXIO; - goto fail; + /* Allocate interrupt */ + if (sc->rl_msi == 0) { + rid = 0; + sc->rl_irq[0] = bus_alloc_resource_any(dev, SYS_RES_IRQ, &rid, + RF_SHAREABLE | RF_ACTIVE); + if (sc->rl_irq[0] == NULL) { + device_printf(dev, "couldn't allocate IRQ resources\n"); + error = ENXIO; + goto fail; + } + } else { + for (i = 0, rid = 1; i < RL_MSI_MESSAGES; i++, rid++) { + sc->rl_irq[i] = bus_alloc_resource_any(dev, + SYS_RES_IRQ, &rid, RF_ACTIVE); + if (sc->rl_irq[i] == NULL) { + device_printf(dev, + "couldn't llocate IRQ resources for " + "message %d\n", rid); + error = ENXIO; + goto fail; + } + } } /* Reset the adapter. */ @@ -1320,8 +1355,19 @@ #endif /* Hook interrupt last to avoid having to lock softc */ - error = bus_setup_intr(dev, sc->rl_irq, INTR_TYPE_NET | INTR_MPSAFE, - re_intr, NULL, sc, &sc->rl_intrhand); + if (sc->rl_msi == 0) + error = bus_setup_intr(dev, sc->rl_irq[0], + INTR_TYPE_NET | INTR_MPSAFE, re_intr, NULL, sc, + &sc->rl_intrhand[0]); + else { + for (i = 0; i < RL_MSI_MESSAGES; i++) { + error = bus_setup_intr(dev, sc->rl_irq[i], + INTR_TYPE_NET | INTR_MPSAFE, re_intr, NULL, sc, + &sc->rl_intrhand[i]); + if (error != 0) + break; + } + } if (error) { device_printf(dev, "couldn't set up irq\n"); ether_ifdetach(ifp); @@ -1348,7 +1394,7 @@ { struct rl_softc *sc; struct ifnet *ifp; - int i; + int i, rid; sc = device_get_softc(dev); ifp = sc->rl_ifp; @@ -1393,12 +1439,31 @@ * stopped here. */ - if (sc->rl_intrhand) - bus_teardown_intr(dev, sc->rl_irq, sc->rl_intrhand); + for (i = 0; i < RL_MSI_MESSAGES; i++) { + if (sc->rl_intrhand[i] != NULL) { + bus_teardown_intr(dev, sc->rl_irq[i], + sc->rl_intrhand[i]); + sc->rl_intrhand[i] = NULL; + } + } if (ifp != NULL) if_free(ifp); - if (sc->rl_irq) - bus_release_resource(dev, SYS_RES_IRQ, 0, sc->rl_irq); + if (sc->rl_msi == 0) { + if (sc->rl_irq[0] != NULL) { + bus_release_resource(dev, SYS_RES_IRQ, 0, + sc->rl_irq[0]); + sc->rl_irq[0] = NULL; + } + } else { + for (i = 0, rid = 1; i < RL_MSI_MESSAGES; i++, rid++) { + if (sc->rl_irq[i] != NULL) { + bus_release_resource(dev, SYS_RES_IRQ, rid, + sc->rl_irq[i]); + sc->rl_irq[i] = NULL; + } + } + pci_release_msi(dev); + } if (sc->rl_res) bus_release_resource(dev, RL_RES, RL_RID, sc->rl_res); Index: ../../pci/if_rl.c =================================================================== RCS file: /home/ncvs/src/sys/pci/if_rl.c,v retrieving revision 1.169 diff -u -r1.169 if_rl.c --- ../../pci/if_rl.c 23 Feb 2007 12:19:03 -0000 1.169 +++ ../../pci/if_rl.c 9 Jul 2007 01:52:24 -0000 @@ -828,10 +828,10 @@ /* Allocate interrupt */ rid = 0; - sc->rl_irq = bus_alloc_resource_any(dev, SYS_RES_IRQ, &rid, + sc->rl_irq[0] = bus_alloc_resource_any(dev, SYS_RES_IRQ, &rid, RF_SHAREABLE | RF_ACTIVE); - if (sc->rl_irq == NULL) { + if (sc->rl_irq[0] == NULL) { device_printf(dev, "couldn't map interrupt\n"); error = ENXIO; goto fail; @@ -971,8 +971,8 @@ ether_ifattach(ifp, eaddr); /* Hook interrupt last to avoid having to lock softc */ - error = bus_setup_intr(dev, sc->rl_irq, INTR_TYPE_NET | INTR_MPSAFE, - NULL, rl_intr, sc, &sc->rl_intrhand); + error = bus_setup_intr(dev, sc->rl_irq[0], INTR_TYPE_NET | INTR_MPSAFE, + NULL, rl_intr, sc, &sc->rl_intrhand[0]); if (error) { device_printf(sc->rl_dev, "couldn't set up irq\n"); ether_ifdetach(ifp); @@ -1022,10 +1022,10 @@ device_delete_child(dev, sc->rl_miibus); bus_generic_detach(dev); - if (sc->rl_intrhand) - bus_teardown_intr(dev, sc->rl_irq, sc->rl_intrhand); - if (sc->rl_irq) - bus_release_resource(dev, SYS_RES_IRQ, 0, sc->rl_irq); + if (sc->rl_intrhand[0]) + bus_teardown_intr(dev, sc->rl_irq[0], sc->rl_intrhand[0]); + if (sc->rl_irq[0]) + bus_release_resource(dev, SYS_RES_IRQ, 0, sc->rl_irq[0]); if (sc->rl_res) bus_release_resource(dev, RL_RES, RL_RID, sc->rl_res); Index: ../../pci/if_rlreg.h =================================================================== RCS file: /home/ncvs/src/sys/pci/if_rlreg.h,v retrieving revision 1.66 diff -u -r1.66 if_rlreg.h --- ../../pci/if_rlreg.h 6 Jul 2007 00:05:12 -0000 1.66 +++ ../../pci/if_rlreg.h 9 Jul 2007 01:52:24 -0000 @@ -664,6 +664,8 @@ #define RE_RX_DESC_BUFLEN MCLBYTES #endif +#define RL_MSI_MESSAGES 2 + #define RL_ADDR_LO(y) ((uint64_t) (y) & 0xFFFFFFFF) #define RL_ADDR_HI(y) ((uint64_t) (y) >> 32) @@ -710,8 +712,8 @@ bus_space_tag_t rl_btag; /* bus space tag */ device_t rl_dev; struct resource *rl_res; - struct resource *rl_irq; - void *rl_intrhand; + struct resource *rl_irq[RL_MSI_MESSAGES]; + void *rl_intrhand[RL_MSI_MESSAGES]; device_t rl_miibus; bus_dma_tag_t rl_parent_tag; bus_dma_tag_t rl_tag; @@ -742,6 +744,7 @@ struct mtx rl_intlock; int rl_txstart; int rl_link; + int rl_msi; }; #define RL_LOCK(_sc) mtx_lock(&(_sc)->rl_mtx) --ALfTUftag+2gvp1h--