From owner-freebsd-wireless@FreeBSD.ORG Mon Jul 14 16:48:58 2014 Return-Path: Delivered-To: freebsd-wireless@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4E85E536; Mon, 14 Jul 2014 16:48:58 +0000 (UTC) Received: from mail-qa0-x22c.google.com (mail-qa0-x22c.google.com [IPv6:2607:f8b0:400d:c00::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D9FFC2B42; Mon, 14 Jul 2014 16:48:57 +0000 (UTC) Received: by mail-qa0-f44.google.com with SMTP id f12so3340137qad.17 for ; Mon, 14 Jul 2014 09:48:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Q4qqEWyB1SqUeUDPNnhWvHGrSccGQya7Le6zhkO/dvw=; b=da1mnTEvS8IzkYp/RmBgTX7f93VBUcmcxhPs4YgLUnqXnvaB1rZdK2QdPufpOWThUu wSMqdnM35rj53dkprVHqxoXRNx/nLP6lw2LcBi+iXm2kwyB0KBeH/HYpN5b98d+qlOTl 4GT9QVmy1+cRDVolPqSHpzuvrl1UKX/YL806fbY7GVNpGiugd7FbyXaZiJV6suX3Qzuu KmBU2HpSXxnbw5jSY8IizLqHItpJ8wxm4F+pJcq3sDwny/YDNrCEb3ZrGGjgK2Nb+mRr GQM2h6dqbL7PSbnpl3EYMGoGUfjIuQ3N3j2lzPhmAQtEgYUgBAdL/qpIbRY5NCxN9pms m5/A== MIME-Version: 1.0 X-Received: by 10.140.88.243 with SMTP id t106mr10457799qgd.12.1405356536961; Mon, 14 Jul 2014 09:48:56 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Mon, 14 Jul 2014 09:48:56 -0700 (PDT) In-Reply-To: <20140714080708.GA88464@zibbi.meraka.csir.co.za> References: <1404753166.65432.10.camel@revolution.hippie.lan> <20140707182814.GA75629@zibbi.meraka.csir.co.za> <20140707193252.GA79553@zibbi.meraka.csir.co.za> <20140708165438.GA83704@zibbi.meraka.csir.co.za> <20140709044612.GA51378@zibbi.meraka.csir.co.za> <20140710040227.GA73872@zibbi.meraka.csir.co.za> <20140714080708.GA88464@zibbi.meraka.csir.co.za> Date: Mon, 14 Jul 2014 09:48:56 -0700 X-Google-Sender-Auth: oPmwykdbe53Z8kaZtKSrkB2pc4c Message-ID: Subject: Re: CAMBRIA and more than one atheros card From: Adrian Chadd To: John Hay Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-wireless@freebsd.org" , "freebsd-embedded@freebsd.org" , Ian Lepore X-BeenThere: freebsd-wireless@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussions of 802.11 stack, tools device driver development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Jul 2014 16:48:58 -0000 Hm, it could be some bus related stupidity. It's allocating 2k or 4k mbufs for the receive path because wifi frames are bigger than ethernet frames. If you're not seeing failures in 4k mbuf allocations then I'm not sure what it could be. I'll see about firing it up locally and checking. -a On 14 July 2014 01:07, John Hay wrote: > On Thu, Jul 10, 2014 at 06:02:27AM +0200, John Hay wrote: >> On Wed, Jul 09, 2014 at 06:46:12AM +0200, John Hay wrote: >> > Hi Guys, >> > >> > The problem is back / still there. I initially saw the problem during >> > boot, with the interface configs in rc.conf, but because it is so >> > mixed with the rest, I took it out and put it in the script, then >> > after multi-user boot was finished, I did a login and ran the script, >> > with the output I showed in the initial post. >> > >> > So I put the interface configs back into rc.conf and I'm seeing the >> > same problem, here is a cut during boot: >> > >> > ############### >> > Starting file system checks: >> > /dev/ad0s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS >> > /dev/ad0s1a: clean, 93385 free (33 frags, 11669 blocks, 0.0% fragmentation) >> > Mounting local file systems:. >> > Writing entropy file:. >> > Setting hostname: tst-cambria-11. >> > wlan0: Ethernet address: 00:21:a4:35:70:42 >> > wlan1: Ethernet address: 00:21:a4:35:6c:96 >> > ath1: unable to start recv logic >> > wlan2: Ethernet address: 00:21:a4:32:38:c2 >> > ath2: unable to start recv logic >> > ############### >> > >> > Looking at the vmstat -z output the 256 Bucket fail is much higher than >> > if I let it boot to multiuser and then configured the interfaces. It >> > was in the 6000, while now it is much higher: >> > >> > Without wlan configs in rc.conf, but configured afterwards: >> > 16 Bucket: 64, 0, 15, 300, 3139, 16, 0 >> > 256 Bucket: 1024, 0, 31, 1, 592,6062, 0 >> > vmem btag: 28, 0, 4496, 256, 4496, 32, 0 >> > >> > With wlan configs in rc.conf: >> > 16 Bucket: 64, 0, 16, 299, 8611, 16, 0 >> > 256 Bucket: 1024, 0, 26, 6, 773,16928, 0 >> > vmem btag: 28, 0, 4405, 59, 4405, 30, 0 >> > >> > Both of them boot from a ro compact-flash with md /etc and /var, but >> > they are small 4.3M and 2.1M. I have hacked in the use of tmpfs, but >> > that did not make a difference. >> > > > Friday I power cycled the board and it came up without an error. All 3 > atheros cards configured in rc.conf. So I left it on for the weekend > and by this morning there was still no errors, so I rebooted it and > again saw the "ath1: unable to start recv logic" message for ath1 and > ath2. I power cycled it, but still get the error, so Friday was just > lucky with some timing thing, maybe if the card receive while it is > still configuring? Also if I leave the board booted in this state, > ath0 also start to give problems: > > ################# > ath0: ath_rx_proc: no mbuf! > ath0: ath_rx_proc: no mbuf! > ... > ath0: device timeout > ath0: ath_reset: unable to start recv logic > ... > ################# > > In anycase it seems that memory allocation problem. How do I figure out > where it is? "netstat -m" does not seem to point to an error. The mbuf > info in vmstat -z also look ok. It is only "256 Bucket" in vmstat -z > that point to an alloc failure. How do I figure out where that is and > how do I fix it or work around it? > > ################### > tst-11-arm:~ # netstat -m > 128/382/510 mbufs in use (current/cache/total) > 127/129/256/7654 mbuf clusters in use (current/cache/total/max) > 127/126 mbuf+clusters out of packet secondary zone in use (current/cache) > 0/0/0/3827 4k (page size) jumbo clusters in use (current/cache/total/max) > 0/0/0/1134 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/637 16k jumbo clusters in use (current/cache/total/max) > 286K/353K/639K bytes allocated to network (current/cache/total) > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0/3/1488 sfbufs in use (current/peak/max) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > tst-11-arm:~ # vmstat -z | grep mbuf > mbuf_packet: 256, 48990, 127, 126, 2294, 0, 0 > mbuf: 256, 48990, 1, 256, 920, 0, 0 > mbuf_cluster: 2048, 7654, 253, 3, 253, 0, 0 > mbuf_jumbo_page: 4096, 3827, 0, 0, 0, 0, 0 > mbuf_jumbo_9k: 9216, 1134, 0, 0, 0, 0, 0 > mbuf_jumbo_16k: 16384, 637, 0, 0, 0, 0, 0 > mbuf_ext_refcnt: 4, 0, 0, 504, 1, 0, 0 > tst-11-arm:~ # vmstat -z | grep Bucket > 4 Bucket: 16, 0, 7, 497, 1565, 0, 0 > 6 Bucket: 24, 0, 0, 0, 0, 0, 0 > 8 Bucket: 32, 0, 3, 375, 83, 0, 0 > 12 Bucket: 48, 0, 2, 334, 5, 0, 0 > 16 Bucket: 64, 0, 14, 301, 43687, 16, 0 > 32 Bucket: 128, 0, 7, 148, 196, 0, 0 > 64 Bucket: 256, 0, 19, 56, 77, 0, 0 > 128 Bucket: 512, 0, 16, 16, 48, 0, 0 > 256 Bucket: 1024, 0, 29, 3, 1521,86738, 0 > tst-11-arm:~ # > ################### > > Regards > > John > -- > John Hay -- jhay@meraka.csir.co.za / jhay@meraka.org.za / jhay@FreeBSD.org