From owner-freebsd-net@FreeBSD.ORG Mon Apr 4 16:11:18 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E3B551065678 for ; Mon, 4 Apr 2011 16:11:18 +0000 (UTC) (envelope-from lacombar@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 948038FC15 for ; Mon, 4 Apr 2011 16:11:18 +0000 (UTC) Received: by qwc9 with SMTP id 9so3990303qwc.13 for ; Mon, 04 Apr 2011 09:11:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=oppJywNubEDHG2rlUolri0v7Kgb0kFfOIN+HlrUjKzc=; b=s3y2s9mv3AN/p4agRhW7+hRNov/vjfgbhSra7F6mVyIA+NtB0MY6SH/IoNqyoQk6WB z23U40bvM5JKvplPzwhyyj11z7Up9xeO7CQNmJkojJsbA/dX0rekgEolU5hg6a2LpPOA kZKdHntud9FDr6jgIw05aD1lkVK0QP7FXUrH0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=NhpikB0DZQUk5tM9PjZCpWNOu7CqKmQR1m5L83Nv5iseFC3MDk692gqME0ZZtpO3yK frfbdBsVbrHlhpXP/v9TVN/fYkO8fClb2/mB9fqWCQYimT2SuCPf8NJjmifoQJ1npQxn T2m18+wwY3nCLqpmsLz8H5qhqcOQnGZyezP1k= MIME-Version: 1.0 Received: by 10.229.28.68 with SMTP id l4mr495289qcc.93.1301933477416; Mon, 04 Apr 2011 09:11:17 -0700 (PDT) Received: by 10.229.33.74 with HTTP; Mon, 4 Apr 2011 09:11:17 -0700 (PDT) In-Reply-To: References: Date: Mon, 4 Apr 2011 12:11:17 -0400 Message-ID: From: Arnaud Lacombe To: Jack Vogel Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net@freebsd.org Subject: Re: em(4) hang [Was: Re: igb(4) won't start with "igb0: Could not setup receive structures"] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Apr 2011 16:11:19 -0000 Hi Jack, On Fri, Apr 1, 2011 at 2:55 PM, Jack Vogel wrote: > Arnaud, > > Please try the code change I just checked into HEAD, it should finally > resolve any hang that is due to mbufs not being refreshed. That's not > to say there may not be other reasons out there but I'm keeping my > fingers crossed that this is behind at least some of the hangs. > Thanks! I backported these latest changes to 7-stable on Friday and re-ran tests. The box, and em0, has been fine for 3 days with the same test pattern that made it hang after a few hour previously. Some stats: # sysctl dev.em.0 dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.2.3 dev.em.0.%driver: em dev.em.0.%location: slot=0 function=0 dev.em.0.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000 dev.em.0.%parent: pci1 dev.em.0.nvm: -1 dev.em.0.debug: -1 dev.em.0.rx_int_delay: 0 dev.em.0.tx_int_delay: 66 dev.em.0.rx_abs_int_delay: 66 dev.em.0.tx_abs_int_delay: 66 dev.em.0.rx_processing_limit: 100 dev.em.0.flow_control: 3 dev.em.0.eee_control: 0 dev.em.0.link_irq: 20516 dev.em.0.mbuf_alloc_fail: 0 dev.em.0.cluster_alloc_fail: 0 dev.em.0.dropped: 0 dev.em.0.tx_dma_fail: 0 dev.em.0.rx_overruns: 0 dev.em.0.watchdog_timeouts: 0 dev.em.0.device_control: 1477444168 dev.em.0.rx_control: 67141634 dev.em.0.fc_high_water: 18432 dev.em.0.fc_low_water: 16932 dev.em.0.queue0.txd_head: 175 dev.em.0.queue0.txd_tail: 175 dev.em.0.queue0.tx_irq: 433666694 dev.em.0.queue0.no_desc_avail: 0 dev.em.0.queue0.rxd_head: 654 dev.em.0.queue0.rxd_tail: 653 dev.em.0.queue0.rx_irq: 280606824 dev.em.0.mac_stats.excess_coll: 0 dev.em.0.mac_stats.single_coll: 0 dev.em.0.mac_stats.multiple_coll: 0 dev.em.0.mac_stats.late_coll: 0 dev.em.0.mac_stats.collision_count: 0 dev.em.0.mac_stats.symbol_errors: 0 dev.em.0.mac_stats.sequence_errors: 0 dev.em.0.mac_stats.defer_count: 0 dev.em.0.mac_stats.missed_packets: 61034 dev.em.0.mac_stats.recv_no_buff: 1974 dev.em.0.mac_stats.recv_undersize: 0 dev.em.0.mac_stats.recv_fragmented: 0 dev.em.0.mac_stats.recv_oversize: 0 dev.em.0.mac_stats.recv_jabber: 0 dev.em.0.mac_stats.recv_errs: 0 dev.em.0.mac_stats.crc_errs: 0 dev.em.0.mac_stats.alignment_errs: 0 dev.em.0.mac_stats.coll_ext_errs: 0 dev.em.0.mac_stats.xon_recvd: 0 dev.em.0.mac_stats.xon_txd: 21 dev.em.0.mac_stats.xoff_recvd: 0 dev.em.0.mac_stats.xoff_txd: 57286 dev.em.0.mac_stats.total_pkts_recvd: 541037465 dev.em.0.mac_stats.good_pkts_recvd: 540976431 dev.em.0.mac_stats.bcast_pkts_recvd: 216 dev.em.0.mac_stats.mcast_pkts_recvd: 0 dev.em.0.mac_stats.rx_frames_64: 527887692 dev.em.0.mac_stats.rx_frames_65_127: 13088739 dev.em.0.mac_stats.rx_frames_128_255: 0 dev.em.0.mac_stats.rx_frames_256_511: 0 dev.em.0.mac_stats.rx_frames_512_1023: 0 dev.em.0.mac_stats.rx_frames_1024_1522: 0 dev.em.0.mac_stats.good_octets_recvd: 34785499386 dev.em.0.mac_stats.good_octets_txd: 909161087533 dev.em.0.mac_stats.total_pkts_txd: 761197293 dev.em.0.mac_stats.good_pkts_txd: 761139986 dev.em.0.mac_stats.bcast_pkts_txd: 1 dev.em.0.mac_stats.mcast_pkts_txd: 0 dev.em.0.mac_stats.tx_frames_64: 42029962 dev.em.0.mac_stats.tx_frames_65_127: 40597084 dev.em.0.mac_stats.tx_frames_128_255: 19591094 dev.em.0.mac_stats.tx_frames_256_511: 31040789 dev.em.0.mac_stats.tx_frames_512_1023: 69157381 dev.em.0.mac_stats.tx_frames_1024_1522: 558723676 dev.em.0.mac_stats.tso_txd: 0 dev.em.0.mac_stats.tso_ctx_fail: 0 dev.em.0.interrupts.asserts: 16073 dev.em.0.interrupts.rx_pkt_timer: 0 dev.em.0.interrupts.rx_abs_timer: 0 dev.em.0.interrupts.tx_pkt_timer: 0 dev.em.0.interrupts.tx_abs_timer: 1 dev.em.0.interrupts.tx_queue_empty: 0 dev.em.0.interrupts.tx_queue_min_thresh: 0 dev.em.0.interrupts.rx_desc_min_thresh: 0 dev.em.0.interrupts.rx_overrun: 3 # netstat -m 33905/48040/81945 mbufs in use (current/cache/total) 30335/2433/32768/32768 mbuf clusters in use (current/cache/total/max) 2781/348 mbuf+clusters out of packet secondary zone in use (current/cache) 3194/9607/12801/12800 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 83770K/55304K/139074K bytes allocated to network (current/cache/total) 0/31564/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/7/6656 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines - Arnaud