From owner-freebsd-stable@FreeBSD.ORG Sun Sep 26 22:19:12 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 02465106564A for ; Sun, 26 Sep 2010 22:19:12 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 69DD88FC12 for ; Sun, 26 Sep 2010 22:19:11 +0000 (UTC) Received: by wwc33 with SMTP id 33so5438866wwc.31 for ; Sun, 26 Sep 2010 15:19:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=Miore+x1PodUlnqGcdMoq2tC2GZVcssmHMkrtN7Z5QI=; b=u8iXjP474mCqi3KFKhBoqxASBOa0KOz/p+DtZ6iNfpYj3qaw7k+lBgBg9Ed2ODhc7p dU4JOqrFvTySMUpmN1rX9XwMYT92xDmALT9RY7lAtqQrKHEcAsDFUvZrISgv0BD7S/2L K3J5tLRVSP+T+p1VKMM8x1wNjPFWi+2T4/LqA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=lqYpCm0xudYd1hNcPfMNedsuI9c1H1nXFl0hIl7pIv7fqF4DZgcevcv0t67dQ/4lHo eeBR8r0VaptW60YFzEQp/OHN5vCAKehJRT+q6eTYme1zgP+w4y0dpIkU4s3CiCXOYtL9 BaWRzeNq7t2w6YGBfnPDaklb3Ly39PMpQ1Ih0= MIME-Version: 1.0 Received: by 10.216.2.141 with SMTP id 13mr12049498wef.84.1285539550196; Sun, 26 Sep 2010 15:19:10 -0700 (PDT) Received: by 10.216.48.20 with HTTP; Sun, 26 Sep 2010 15:19:10 -0700 (PDT) In-Reply-To: <201009262157.o8QLvR0L012171@lava.sentex.ca> References: <201006102031.o5AKVCH2016467@lava.sentex.ca> <201007021739.o62HdMOU092319@lava.sentex.ca> <20100702193654.GD10862@michelle.cdnetworks.com> <201008162107.o7GL76pA080191@lava.sentex.ca> <20100817185208.GA6482@michelle.cdnetworks.com> <201008171955.o7HJt67T087902@lava.sentex.ca> <20100817200020.GE6482@michelle.cdnetworks.com> <201009141759.o8EHxcZ0013539@lava.sentex.ca> <201009262157.o8QLvR0L012171@lava.sentex.ca> Date: Sun, 26 Sep 2010 15:19:10 -0700 Message-ID: From: Jack Vogel To: Mike Tancsa Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: pyunyh@gmail.com, freebsd-stable@freebsd.org Subject: Re: RELENG_7 em problems (and RELENG_8) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Sep 2010 22:19:12 -0000 The NIC has 5 MSIX vectors and can have 2 queues, I have been trying to release code with both queues active, but its been unstable, I finally concluded its not worth the aggrevation :) Your em1 is using MSI not MSIX and thus can't have multiple queues. I'm not sure whats broken from what you show here. I will try to get the new driver out shortly for you to try. Jack On Sun, Sep 26, 2010 at 2:57 PM, Mike Tancsa wrote: > At 06:36 PM 9/24/2010, Jack Vogel wrote: > >> There is a new revision of the em driver coming next week, its going thru >> some >> stress pounding over the weekend, if no issues show up I'll put it into >> HEAD. >> >> Yongari's changes in TX context handling which effects checksum and tso >> are added. I've also decided that multiple queues in 82574 just are a >> source >> of problems without a lot of benefit, so it still uses MSIX but with only >> 3 vectors, >> meaning it seperates TX and RX but has a single queue. >> > > Thanks, looking forward to trying it out! With respect to the multiple > queues, I thought the driver already used just the one on RELENG_8 ? If > not, is there a way to force the existing driver to use just the one queue ? > > On the box that has the NIC locking up, it shows > > em1@pci0:9:0:0: class=0x020000 card=0x34ec8086 chip=0x10d38086 rev=0x00 > hdr=0x00 > > vendor = 'Intel Corporation' > device = 'Intel 82574L Gigabit Ethernet Controller (82574L)' > class = network > subclass = ethernet > cap 01[c8] = powerspec 2 supports D0 D3 current D0 > cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) > > and > > vmstat -i shows > > irq256: em0 5129063 353 > irq257: em1 531251 36 > > in a wedged state, stats look like > > dev.em.1.%desc: Intel(R) PRO/1000 Network Connection 7.0.5 > dev.em.1.%driver: em > dev.em.1.%location: slot=0 function=0 handle=\_SB_.PCI0.PEX4.HART > dev.em.1.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 > subdevice=0x34ec class=0x020000 > dev.em.1.%parent: pci9 > dev.em.1.nvm: -1 > dev.em.1.rx_int_delay: 0 > dev.em.1.tx_int_delay: 66 > dev.em.1.rx_abs_int_delay: 66 > dev.em.1.tx_abs_int_delay: 66 > dev.em.1.rx_processing_limit: 100 > dev.em.1.link_irq: 0 > dev.em.1.mbuf_alloc_fail: 0 > dev.em.1.cluster_alloc_fail: 0 > dev.em.1.dropped: 0 > dev.em.1.tx_dma_fail: 0 > dev.em.1.fc_high_water: 18432 > dev.em.1.fc_low_water: 16932 > dev.em.1.mac_stats.excess_coll: 0 > dev.em.1.mac_stats.symbol_errors: 0 > dev.em.1.mac_stats.sequence_errors: 0 > dev.em.1.mac_stats.defer_count: 0 > dev.em.1.mac_stats.missed_packets: 41522 > dev.em.1.mac_stats.recv_no_buff: 19 > dev.em.1.mac_stats.recv_errs: 0 > dev.em.1.mac_stats.crc_errs: 0 > dev.em.1.mac_stats.alignment_errs: 0 > dev.em.1.mac_stats.coll_ext_errs: 0 > dev.em.1.mac_stats.rx_overruns: 41398 > dev.em.1.mac_stats.watchdog_timeouts: 0 > dev.em.1.mac_stats.xon_recvd: 0 > dev.em.1.mac_stats.xon_txd: 0 > dev.em.1.mac_stats.xoff_recvd: 0 > dev.em.1.mac_stats.xoff_txd: 0 > dev.em.1.mac_stats.total_pkts_recvd: 95229129 > dev.em.1.mac_stats.good_pkts_recvd: 95187607 > dev.em.1.mac_stats.bcast_pkts_recvd: 79244 > dev.em.1.mac_stats.mcast_pkts_recvd: 0 > dev.em.1.mac_stats.rx_frames_64: 93680 > dev.em.1.mac_stats.rx_frames_65_127: 1516349 > dev.em.1.mac_stats.rx_frames_128_255: 4464941 > dev.em.1.mac_stats.rx_frames_256_511: 4024 > dev.em.1.mac_stats.rx_frames_512_1023: 2096067 > dev.em.1.mac_stats.rx_frames_1024_1522: 87012546 > dev.em.1.mac_stats.good_octets_recvd: 0 > dev.em.1.mac_stats.good_octest_txd: 0 > dev.em.1.mac_stats.total_pkts_txd: 66775098 > dev.em.1.mac_stats.good_pkts_txd: 66775098 > dev.em.1.mac_stats.bcast_pkts_txd: 509 > dev.em.1.mac_stats.mcast_pkts_txd: 7 > dev.em.1.mac_stats.tx_frames_64: 48038472 > dev.em.1.mac_stats.tx_frames_65_127: 13402833 > dev.em.1.mac_stats.tx_frames_128_255: 5324413 > dev.em.1.mac_stats.tx_frames_256_511: 957 > dev.em.1.mac_stats.tx_frames_512_1023: 319 > dev.em.1.mac_stats.tx_frames_1024_1522: 8104 > dev.em.1.mac_stats.tso_txd: 1069 > dev.em.1.mac_stats.tso_ctx_fail: 0 > dev.em.1.interrupts.asserts: 0 > dev.em.1.interrupts.rx_pkt_timer: 0 > dev.em.1.interrupts.rx_abs_timer: 0 > dev.em.1.interrupts.tx_pkt_timer: 0 > dev.em.1.interrupts.tx_abs_timer: 0 > dev.em.1.interrupts.tx_queue_empty: 0 > dev.em.1.interrupts.tx_queue_min_thresh: 0 > dev.em.1.interrupts.rx_desc_min_thresh: 0 > dev.em.1.interrupts.rx_overrun: 0 > dev.em.1.host.breaker_tx_pkt: 0 > dev.em.1.host.host_tx_pkt_discard: 0 > dev.em.1.host.rx_pkt: 0 > dev.em.1.host.breaker_rx_pkts: 0 > dev.em.1.host.breaker_rx_pkt_drop: 0 > dev.em.1.host.tx_good_pkt: 0 > dev.em.1.host.breaker_tx_pkt_drop: 0 > dev.em.1.host.rx_good_bytes: 0 > dev.em.1.host.tx_good_bytes: 0 > dev.em.1.host.length_errors: 0 > dev.em.1.host.serdes_violation_pkt: 0 > dev.em.1.host.header_redir_missed: 0 > > ifconfig down/up just panics or locks up the box when its in this state. I > also have IPMI enabled on this nic, but it shows the same issue with it > disabled. > > ---Mike > > > > -------------------------------------------------------------------- > Mike Tancsa, tel +1 519 651 3400 > Sentex Communications, mike@sentex.net > Providing Internet since 1994 www.sentex.net > Cambridge, Ontario Canada www.sentex.net/mike > >