From owner-freebsd-embedded@FreeBSD.ORG Mon Jul 14 08:07:13 2014 Return-Path: Delivered-To: freebsd-embedded@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8D3DB1FB; Mon, 14 Jul 2014 08:07:13 +0000 (UTC) Received: from zibbi.meraka.csir.co.za (zibbi.meraka.csir.co.za [IPv6:2001:4200:7000:2::1]) by mx1.freebsd.org (Postfix) with ESMTP id 7A0202B46; Mon, 14 Jul 2014 08:07:12 +0000 (UTC) Received: by zibbi.meraka.csir.co.za (Postfix, from userid 3973) id 7C057B842; Mon, 14 Jul 2014 10:07:08 +0200 (SAST) Date: Mon, 14 Jul 2014 10:07:08 +0200 From: John Hay To: Adrian Chadd Subject: Re: CAMBRIA and more than one atheros card Message-ID: <20140714080708.GA88464@zibbi.meraka.csir.co.za> References: <1404753166.65432.10.camel@revolution.hippie.lan> <20140707182814.GA75629@zibbi.meraka.csir.co.za> <20140707193252.GA79553@zibbi.meraka.csir.co.za> <20140708165438.GA83704@zibbi.meraka.csir.co.za> <20140709044612.GA51378@zibbi.meraka.csir.co.za> <20140710040227.GA73872@zibbi.meraka.csir.co.za> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140710040227.GA73872@zibbi.meraka.csir.co.za> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-wireless@freebsd.org" , "freebsd-embedded@freebsd.org" , Ian Lepore X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Jul 2014 08:07:13 -0000 On Thu, Jul 10, 2014 at 06:02:27AM +0200, John Hay wrote: > On Wed, Jul 09, 2014 at 06:46:12AM +0200, John Hay wrote: > > Hi Guys, > > > > The problem is back / still there. I initially saw the problem during > > boot, with the interface configs in rc.conf, but because it is so > > mixed with the rest, I took it out and put it in the script, then > > after multi-user boot was finished, I did a login and ran the script, > > with the output I showed in the initial post. > > > > So I put the interface configs back into rc.conf and I'm seeing the > > same problem, here is a cut during boot: > > > > ############### > > Starting file system checks: > > /dev/ad0s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS > > /dev/ad0s1a: clean, 93385 free (33 frags, 11669 blocks, 0.0% fragmentation) > > Mounting local file systems:. > > Writing entropy file:. > > Setting hostname: tst-cambria-11. > > wlan0: Ethernet address: 00:21:a4:35:70:42 > > wlan1: Ethernet address: 00:21:a4:35:6c:96 > > ath1: unable to start recv logic > > wlan2: Ethernet address: 00:21:a4:32:38:c2 > > ath2: unable to start recv logic > > ############### > > > > Looking at the vmstat -z output the 256 Bucket fail is much higher than > > if I let it boot to multiuser and then configured the interfaces. It > > was in the 6000, while now it is much higher: > > > > Without wlan configs in rc.conf, but configured afterwards: > > 16 Bucket: 64, 0, 15, 300, 3139, 16, 0 > > 256 Bucket: 1024, 0, 31, 1, 592,6062, 0 > > vmem btag: 28, 0, 4496, 256, 4496, 32, 0 > > > > With wlan configs in rc.conf: > > 16 Bucket: 64, 0, 16, 299, 8611, 16, 0 > > 256 Bucket: 1024, 0, 26, 6, 773,16928, 0 > > vmem btag: 28, 0, 4405, 59, 4405, 30, 0 > > > > Both of them boot from a ro compact-flash with md /etc and /var, but > > they are small 4.3M and 2.1M. I have hacked in the use of tmpfs, but > > that did not make a difference. > > Friday I power cycled the board and it came up without an error. All 3 atheros cards configured in rc.conf. So I left it on for the weekend and by this morning there was still no errors, so I rebooted it and again saw the "ath1: unable to start recv logic" message for ath1 and ath2. I power cycled it, but still get the error, so Friday was just lucky with some timing thing, maybe if the card receive while it is still configuring? Also if I leave the board booted in this state, ath0 also start to give problems: ################# ath0: ath_rx_proc: no mbuf! ath0: ath_rx_proc: no mbuf! ... ath0: device timeout ath0: ath_reset: unable to start recv logic ... ################# In anycase it seems that memory allocation problem. How do I figure out where it is? "netstat -m" does not seem to point to an error. The mbuf info in vmstat -z also look ok. It is only "256 Bucket" in vmstat -z that point to an alloc failure. How do I figure out where that is and how do I fix it or work around it? ################### tst-11-arm:~ # netstat -m 128/382/510 mbufs in use (current/cache/total) 127/129/256/7654 mbuf clusters in use (current/cache/total/max) 127/126 mbuf+clusters out of packet secondary zone in use (current/cache) 0/0/0/3827 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/1134 9k jumbo clusters in use (current/cache/total/max) 0/0/0/637 16k jumbo clusters in use (current/cache/total/max) 286K/353K/639K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/3/1488 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile tst-11-arm:~ # vmstat -z | grep mbuf mbuf_packet: 256, 48990, 127, 126, 2294, 0, 0 mbuf: 256, 48990, 1, 256, 920, 0, 0 mbuf_cluster: 2048, 7654, 253, 3, 253, 0, 0 mbuf_jumbo_page: 4096, 3827, 0, 0, 0, 0, 0 mbuf_jumbo_9k: 9216, 1134, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 637, 0, 0, 0, 0, 0 mbuf_ext_refcnt: 4, 0, 0, 504, 1, 0, 0 tst-11-arm:~ # vmstat -z | grep Bucket 4 Bucket: 16, 0, 7, 497, 1565, 0, 0 6 Bucket: 24, 0, 0, 0, 0, 0, 0 8 Bucket: 32, 0, 3, 375, 83, 0, 0 12 Bucket: 48, 0, 2, 334, 5, 0, 0 16 Bucket: 64, 0, 14, 301, 43687, 16, 0 32 Bucket: 128, 0, 7, 148, 196, 0, 0 64 Bucket: 256, 0, 19, 56, 77, 0, 0 128 Bucket: 512, 0, 16, 16, 48, 0, 0 256 Bucket: 1024, 0, 29, 3, 1521,86738, 0 tst-11-arm:~ # ################### Regards John -- John Hay -- jhay@meraka.csir.co.za / jhay@meraka.org.za / jhay@FreeBSD.org