From owner-freebsd-net@FreeBSD.ORG Wed Jul 18 06:39:55 2007 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 33C7716A405 for ; Wed, 18 Jul 2007 06:39:55 +0000 (UTC) (envelope-from paketix@bluewin.ch) Received: from mx27.bluewin.ch (mx27.bluewin.ch [195.186.18.40]) by mx1.freebsd.org (Postfix) with ESMTP id E835313C4B5 for ; Wed, 18 Jul 2007 06:39:54 +0000 (UTC) (envelope-from paketix@bluewin.ch) Received: from hbnnbsddev03.sharedtcs.net (213.3.8.216) by mx27.bluewin.ch (Bluewin 7.3.122) id 46641D9A049BDC32 for freebsd-net@freebsd.org; Wed, 18 Jul 2007 06:39:53 +0000 Received: from hbnnbsddev03.sharedtcs.net (localhost.sharedtcs.net [127.0.0.1]) by hbnnbsddev03.sharedtcs.net (8.13.8/8.13.8) with ESMTP id l6I6hPFr040897 for ; Wed, 18 Jul 2007 08:43:25 +0200 (CEST) (envelope-from tbeoepa1@hbnnbsddev03.sharedtcs.net) Received: (from tbeoepa1@localhost) by hbnnbsddev03.sharedtcs.net (8.13.8/8.13.8/Submit) id l6I6c13j040870 for freebsd-net@freebsd.org; Wed, 18 Jul 2007 08:38:01 +0200 (CEST) (envelope-from tbeoepa1) Date: Wed, 18 Jul 2007 08:38:01 +0200 From: Patrick Oeschger To: freebsd-net@freebsd.org Message-ID: <20070718063801.GA40653@hbnnbsddev03.sharedtcs.net> Mail-Followup-To: freebsd-net@freebsd.org References: <20070717110156.GA37722@hbnnbsddev03.sharedtcs.net> <2a41acea0707170917n56e35cfbyc6e0bca094bfa729@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2a41acea0707170917n56e35cfbyc6e0bca094bfa729@mail.gmail.com> User-Agent: Mutt/1.4.2.1i Subject: Re: em(4)/82571eb: fifo not dma'ed to host memory X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jul 2007 06:39:55 -0000 thanks for your feedback jack - i tried the MSI stuff as first task this morning the interface init'ed without a problem and did not complain about MSI what i like most is that no IRQ is assigned now, everything seems to get handled in a TASKQ (IRQs were quite weird sometimes when having a bunch of interfaces...) but now the bad news: even with MSI it is not possible to recv any frames what makes me curious is, that the head pointer for receive fifo is *not* counting up after receiving a frame (good_packets counter incrementing) as far as i understand the 82571EB chipset manual, recv head counter inc is done in hardware after receiving a good_packet and having dma'ed it to host memory from if_em.c/em_process_receive_interrupts: * This routine executes in interrupt context. It replenishes * the mbufs in the descriptor and sends data which has been * dma'ed into host memory to upper layer. i did a quick test to check this on 6.1-RELENG by inserting a return at the top of em_process_receive_interrupts (see diff) register RDH0 is still incrementing for each good_packet received - even without receive interrupts being processed at the driver level freebsd61# sysctl dev.em.0.debug_info=1 dev.em.0.debug_iemnfo: 0:-1 Adapter hardware address = 0xc4b7c928 em0: CTRL = 0x81c0241 RCTL = 0x8002 em0: RDBAH0 = 0x0 RDBAL0 = 0x3e0ff000 em0: RDLEN0 = 0x1000 RDH0 = 0x3 em0: RDT0 = 0xff RXDCTL = 0x10000 ...so IMHO this increment (and also dma transfer to host memory) is done in hardware - RDH0 is not counting up when running under 7.0-CURRENT... erroneous init of chipset? anyone else running 7.0-CURRENT on a 82571EB pci express chipset? TIA for any help on this pat details for dmesg/if_stats/diff: pcib9: irq 19 at device 11.0 on pci2 pcib9: secondary bus 9 pcib9: subordinate bus 9 pcib9: I/O decode 0x6000-0x6fff pcib9: memory decode 0xfd700000-0xfd7fffff pcib9: prefetched decode 0xfce00000-0xfcefffff pci9: on pcib9 pci9: physical bus=9 found-> vendor=0x8086, dev=0x105e, revid=0x06 bus=9, slot=0, func=0 class=02-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 2 supports D0 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 32, base 0, size 17, memory disabled map[14]: type Memory, range 32, base 0, size 17, memory disabled map[18]: type I/O Port, range 32, base 0, size 5, port disabled found-> vendor=0x8086, dev=0x105e, revid=0x06 bus=9, slot=0, func=1 class=02-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=255 powerspec 2 supports D0 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 32, base 0, size 17, memory disabled map[14]: type Memory, range 32, base 0, size 17, memory disabled map[18]: type I/O Port, range 32, base 0, size 5, port disabled em0: at device 0.0 on pci9 pcib9: em0 requested memory range 0xfd700000-0xfd7fffff: good pcib2: em0 requested memory range 0xfd700000-0xfd7fffff: good pcib1: em0 requested memory range 0xfd700000-0xfd7fffff: good em0: Lazy allocation of 0x20000 bytes rid 0x10 type 3 at 0xfd700000 em0: attempting to allocate 1 MSI vectors (1 supported) msi: routing MSI IRQ 256 to vector 49 em0: using IRQ 256 for MSI em0: bpf attached em0: Ethernet address: 00:10:f3:0c:5b:2a em0: [FILTER] pci9: at device 0.1 (no driver attached) freebsd61# diff -u sys/dev/em/if_em.c.orig sys/dev/em/if_em.c --- sys/dev/em/if_em.c.orig Wed Jul 18 07:32:53 2007 +++ sys/dev/em/if_em.c Wed Jul 18 07:48:42 2007 @@ -277,6 +277,11 @@ INIT_DEBUGOUT("em_probe: begin"); + // prevent init of all em(4) devices except pci9/0/0 + if(pci_get_bus(dev) != 0x09 || pci_get_slot(dev) != 0x00 || + pci_get_function(dev) != 0x00) + return (ENXIO); + pci_vendor_id = pci_get_vendor(dev); if (pci_vendor_id != EM_VENDOR_ID) return(ENXIO); @@ -2906,6 +2911,8 @@ u_int8_t eop = 0; u_int16_t len, desc_len, prev_len_adj; int i; + + return; /* Pointer to the receive descriptor being examined. */ struct em_rx_desc *current_desc; On Tue, Jul 17, 2007 at 09:17:36AM -0700, Jack Vogel wrote: > As an experiment, search in if_em.c for 'msix' look at the logic there. > If the adapter is 82575 it will use msix, but the else clause has it > use msi ONLY if its '>82571', change that to >=82571 and let's > see if maybe using MSI will help you with your problems as I'm > pretty confident its interrupt related. > > I had intended on making this change shortly anyway, my tests > have shown the 571 to work fine this way. > > Let me know of the results. > > Jack