From owner-freebsd-wireless@FreeBSD.ORG Thu Jul 17 10:34:42 2014 Return-Path: Delivered-To: freebsd-wireless@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6451FDCB; Thu, 17 Jul 2014 10:34:42 +0000 (UTC) Received: from zibbi.meraka.csir.co.za (zibbi.meraka.csir.co.za [146.64.24.58]) by mx1.freebsd.org (Postfix) with ESMTP id 55FF4276F; Thu, 17 Jul 2014 10:34:40 +0000 (UTC) Received: by zibbi.meraka.csir.co.za (Postfix, from userid 3973) id A8C0CB843; Thu, 17 Jul 2014 12:34:30 +0200 (SAST) Date: Thu, 17 Jul 2014 12:34:30 +0200 From: John Hay To: Adrian Chadd Subject: Re: CAMBRIA and more than one atheros card Message-ID: <20140717103430.GA88108@zibbi.meraka.csir.co.za> References: <20140714184209.GA21922@zibbi.meraka.csir.co.za> <20140714190946.GA49930@zibbi.meraka.csir.co.za> <20140714191416.GB49930@zibbi.meraka.csir.co.za> <20140714194932.GA51947@zibbi.meraka.csir.co.za> <20140714200015.GA54311@zibbi.meraka.csir.co.za> <20140714203707.GB54311@zibbi.meraka.csir.co.za> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140714203707.GB54311@zibbi.meraka.csir.co.za> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-wireless@freebsd.org" , "freebsd-embedded@freebsd.org" , Ian Lepore X-BeenThere: freebsd-wireless@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussions of 802.11 stack, tools device driver development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Jul 2014 10:34:42 -0000 On Mon, Jul 14, 2014 at 10:37:07PM +0200, John Hay wrote: > On Mon, Jul 14, 2014 at 01:02:03PM -0700, Adrian Chadd wrote: > > > > Ah, you're not even using it in 11n mode. Ok. > > > > Well, that's 72 mbufs allocated, and up to 200 more being allocated > > during transmit. Then there's the handful of pages being used for DMA > > descriptors. So yeah, over time that may become a problem. > > > > Try doubling the number of bounce buffers available and see what happens. > > The first boot looks good, all 3 aths came up. Bounce buffers just > after login: > > tst-11-arm:~ # sysctl hw.busdma > hw.busdma.total_bpages: 128 > hw.busdma.zone0.total_bpages: 128 > hw.busdma.zone0.free_bpages: 128 > hw.busdma.zone0.reserved_bpages: 0 > hw.busdma.zone0.active_bpages: 0 > hw.busdma.zone0.total_bounced: 0 > hw.busdma.zone0.total_deferred: 0 > hw.busdma.zone0.lowaddr: 0x4000fff > hw.busdma.zone0.alignment: 4096 ... > Great! At last it seems like we are making progress. I'll leave it > till the morning and then do a few reboots and then some traffic. > > One question, should the active bounce pages keep growing or stabilize > somewhere? How do I know if I have to few or if there is a leak? > > Thanks Adrian for your help. I still ran out sometimes during a reboot, so I upped it to 512 and added a sysctl to show the max that active_bpages has been. Sometimes just after a reboot active_bpages is low and other times higher, I have seen 140. A few times during a reboot I see timeouts, which only a reboot seem to recover from. Can it be a left over config from the previous boot that does not get cleaned out? Or maybe traffic being received while the config is happening? Here is an example during the boot: #################### ... ath0: irq 28 at device 1.0 on pci0 alloc_bounce_pages: numpages 63 [ath] enabling AN_TOP2_FIXUP alloc_bounce_pages: numpages 63 alloc_bounce_pages: numpages 63 alloc_bounce_pages: numpages 63 alloc_bounce_pages: numpages 63 alloc_bounce_pages: numpages 63 alloc_bounce_pages: numpages 63 alloc_bounce_pages: numpages 63 alloc_bounce_pages: numpages 8 ath0: AR9220 mac 128.2 RF5133 phy 13.0 ath0: 2GHz radio: 0x0000; 5GHz radio: 0x00c0 ath1: irq 27 at device 2.0 on pci0 [ath] enabling AN_TOP2_FIXUP ath1: AR9220 mac 128.2 RF5133 phy 13.0 ath1: 2GHz radio: 0x0000; 5GHz radio: 0x00c0 ath2: irq 26 at device 3.0 on pci0 ath2: AR5413 mac 10.5 RF5413 phy 6.1 ath2: 2GHz radio: 0x0000; 5GHz radio: 0x0063 ixpclk0: on ixp0 ixpiic0: on ixp0 ... /sbin/ifconfig.bin wlan0 create wlandev ath0 wlanmode adhoc country ZA wlan0: Ethernet address: 00:21:a4:35:70:42 /sbin/ifconfig.bin wlan0 inet6 ifdisabled /sbin/ifconfig.bin wlan0 -bgscan channel 140 ssid mesh bssid 02:8c:ca:fe:ca:00 /sbin/ifconfig.bin wlan0 inet6 -ifdisabled /sbin/ifconfig.bin wlan0 inet6 auto_linklocal /sbin/ifconfig.bin wlan0 inet6 -ifdisabled dev.ath.0.debug: 0 -> 8 /sbin/ifconfig.bin wlan0 up ath0: ath_init: if_flags 0x8803 ath0: ath_stop_locked: invalid 0 if_flags 0x8803 dev.ath.0.debug: 8 -> 0 ath0: hardware error; resetting ath0: 0x00000000 0x00000020 0x00000000, 0x00000000 0x00000000 0x00000000 ar5416StopDmaReceive: dma failed to stop in 10ms AR_CR=0x00000024 AR_DIAG_SW=0x42000020 /sbin/ifconfig.bin wlan0 inet6 fd35:ac5b:b116:10:: prefixlen 64 eui64 alias /sbin/ifconfig.bin wlan0 inet6 fd35:ac5b:b116:10:: prefixlen 64 alias anycast /sbin/ifconfig.bin wlan1 create wlandev ath1 wlanmode adhoc country ZA wlan1: Ethernet address: 00:21:a4:35:6c:96 /sbin/ifconfig.bin wlan1 inet6 ifdisabled /sbin/ifconfig.bin wlan1 -bgscan channel 136 ssid mesh bssid 02:88:ca:fe:ca:00 /sbin/ifconfig.bin wlan1 inet6 -ifdisabled /sbin/ifconfig.bin wlan1 inet6 auto_linklocal /sbin/ifconfig.bin wlan1 inet6 -ifdisabled dev.ath.1.debug: 0 -> 8 /sbin/ifconfig.bin wlan1 up ath1: ath_init: if_flags 0x8803 ath1: ath_stop_locked: invalid 0 if_flags 0x8803 R[ 0] (DS.V:0xcd027000 DS.P:0x1227000) L:01227060 D:00089000 * 00000000 00000800 0b802c2c 000000b2 0000175f 00000100 2f808080 00000000 00000000 00000000 00000003 R[ 0] (DS.V:0xcd027060 DS.P:0x1227060) L:012270c0 D:0008a000 00000000 00000800 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 dev.ath.1.debug: 8 -> 0 /sbin/ifconfig.bin wlan1 inet6 fd35:ac5b:b116:10:: prefixlen 64 eui64 alias /sbin/ifconfig.bin wlan1 inet6 fd35:ac5b:b116:10:: prefixlen 64 alias anycast ifa_add_loopback_route: insertion failed: 17 ifconfig.bin: ioctl (SIOCAIFADDR): File exists /sbin/ifconfig.bin wlan2 create wlandev ath2 wlanmode adhoc country ZA wlan2: Ethernet address: 00:21:a4:32:38:c2 /sbin/ifconfig.bin wlan2 inet6 ifdisabled ath0: device timeout ar5416StopDmaReceive: dma failed to stop in 10ms AR_CR=0x00000024 AR_DIAG_SW=0x42000020 /sbin/ifconfig.bin wlan2 -bgscan channel 13 ssid hotspot bssid 02:0d:ca:fe:ca:00 /sbin/ifconfig.bin wlan2 inet6 -ifdisabled /sbin/ifconfig.bin wlan2 inet6 auto_linklocal /sbin/ifconfig.bin wlan2 inet6 -ifdisabled dev.ath.2.debug: 0 -> 8 /sbin/ifconfig.bin wlan2 up ath2: ath_init: if_flags 0x8803 ath2: ath_stop_locked: invalid 0 if_flags 0x8803 dev.ath.2.debug: 8 -> 0 /sbin/ifconfig.bin wlan2 inet 10.56.244.1/24 alias /sbin/ifconfig.bin wlan2 inet6 fd35:ac5b:b116:38f4:: prefixlen 64 eui64 alias /sbin/ifconfig.bin wlan2 inet6 fd35:ac5b:b116:38f4:: prefixlen 64 alias anycast ... ath0: device timeout ar5416StopDmaReceive: dma failed to stop in 10ms AR_CR=0x00000024 AR_DIAG_SW=0x42000020 ... tst-11-arm:~ # uptime 9:18AM up 2 mins, 1 user, load averages: 1.63, 0.70, 0.28 tst-11-arm:~ # sysctl hw.busdma hw.busdma.total_bpages: 512 hw.busdma.zone0.total_bpages: 512 hw.busdma.zone0.free_bpages: 375 hw.busdma.zone0.reserved_bpages: 0 hw.busdma.zone0.active_bpages: 137 hw.busdma.zone0.max_active_bpages: 137 hw.busdma.zone0.total_bounced: 1739 hw.busdma.zone0.total_deferred: 0 hw.busdma.zone0.lowaddr: 0x4000fff hw.busdma.zone0.alignment: 4096 tst-11-arm:~ # ath0: device timeout ar5416StopDmaReceive: dma failed to stop in 10ms AR_CR=0x00000024 AR_DIAG_SW=0x42000020 tst-11-arm:~ # uptime 9:20AM up 4 mins, 1 user, load averages: 0.68, 0.68, 0.34 tst-11-arm:~ # sysctl hw.busdma hw.busdma.total_bpages: 512 hw.busdma.zone0.total_bpages: 512 hw.busdma.zone0.free_bpages: 326 hw.busdma.zone0.reserved_bpages: 0 hw.busdma.zone0.active_bpages: 186 hw.busdma.zone0.max_active_bpages: 190 hw.busdma.zone0.total_bounced: 6592 hw.busdma.zone0.total_deferred: 0 hw.busdma.zone0.lowaddr: 0x4000fff hw.busdma.zone0.alignment: 4096 tst-11-arm:~ # ath0: device timeout ar5416StopDmaReceive: dma failed to stop in 10ms AR_CR=0x00000024 AR_DIAG_SW=0x42000020 ath0: device timeout ar5416StopDmaReceive: dma failed to stop in 10ms AR_CR=0x00000024 AR_DIAG_SW=0x42000020 #################### The 2 other interfaces did not seem to have a problem and after a reboot ath0/wlan0 was working fine again. John -- John Hay -- jhay@meraka.csir.co.za / jhay@meraka.org.za