From owner-freebsd-embedded@FreeBSD.ORG Mon Jul 7 19:43:19 2014 Return-Path: Delivered-To: freebsd-embedded@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0E6BCF9D; Mon, 7 Jul 2014 19:43:19 +0000 (UTC) Received: from mail-qa0-x22d.google.com (mail-qa0-x22d.google.com [IPv6:2607:f8b0:400d:c00::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8D58029AE; Mon, 7 Jul 2014 19:43:18 +0000 (UTC) Received: by mail-qa0-f45.google.com with SMTP id s7so11151qap.32 for ; Mon, 07 Jul 2014 12:43:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Yf4etpaldGJyOg8kaS3wjl0sFACCt3/DCs6pYZYHH0Q=; b=WZFKxU6wfqcSse0DJahb5smr6rdVdoXhgiXxeHR4hA7l4+MtjXb/ub6F3fLrOcfQC8 g8WxQdJUvqzy3+95B+aeUPHj+dNhO28GWUQf5ChSAowYze+XjVqE+CM2iqv2fO/R04Pt hhjoZ6I1EOTVyAfs/aT31DBFJFQBKeFFQeNZ/fjDbAkc+r7PupaTujC8Th+FK83cizL8 2Oh1vTWBAV7SdNd2UmgrdgUEHCg6QHGjxJFY1Ybm5n+Qh/rP5LFL3OXktPJ4ysZWIe+3 rEO7cbBItPOWffMHhQZUrKJN6zYHHUrFFu/8zQjEufx5fxHFTjqXkLXuiv2ta/+7czuq DTOw== MIME-Version: 1.0 X-Received: by 10.140.47.48 with SMTP id l45mr48216555qga.24.1404762197753; Mon, 07 Jul 2014 12:43:17 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Mon, 7 Jul 2014 12:43:17 -0700 (PDT) In-Reply-To: <20140707193252.GA79553@zibbi.meraka.csir.co.za> References: <20140707142538.GA43661@zibbi.meraka.csir.co.za> <1404753166.65432.10.camel@revolution.hippie.lan> <20140707182814.GA75629@zibbi.meraka.csir.co.za> <20140707193252.GA79553@zibbi.meraka.csir.co.za> Date: Mon, 7 Jul 2014 12:43:17 -0700 X-Google-Sender-Auth: DIMKQaCgAkMLktJoTFB5qjIMUwo Message-ID: Subject: Re: CAMBRIA and more than one atheros card From: Adrian Chadd To: John Hay Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-wireless@freebsd.org" , Ian Lepore , "freebsd-embedded@freebsd.org" X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 19:43:19 -0000 Sweet! Can you run the same commands above on the Avila board? Would you mind filing a PR to ensure we get that option into the relevant kernel(s) ? (I'm kinda tempted to suggest we auto-select the scaling factor at startup depending upon physical RAM .. ) -a On 7 July 2014 12:32, John Hay wrote: > On Mon, Jul 07, 2014 at 11:59:03AM -0700, hiren panchasara wrote: >> On Mon, Jul 7, 2014 at 11:31 AM, Adrian Chadd wrote: >> > On 7 July 2014 11:28, John Hay wrote: >> >> On Mon, Jul 07, 2014 at 11:22:46AM -0700, Adrian Chadd wrote: >> >>> On 7 July 2014 10:12, Ian Lepore wrote: >> >>> > On Mon, 2014-07-07 at 09:25 -0700, Adrian Chadd wrote: >> >>> >> hi, >> >>> >> >> >>> >> That call is returning ENOMEM. I'm not sure why. It allocated an mbuf >> >>> >> fine, but it couldn't allocate the DMA map. >> >>> >> >> >>> >> What's the output of "vmstat -z" ? I wonder if it's failing an allocation. >> >>> >> >> >>> >> >> >>> >> >> >>> >> -a >> >>> > >> >>> > Lack of bounce buffers is a posibility that won't show up in vmstat >> >>> > output. >> >>> >> >>> right, but there's a bunch of already failing vmstat entries. >> >>> >> >>> >> >>> John - there's a vmscale parameter somewhere. Hiren had to drop it >> >>> down for his APs to work in 64MB of RAM. I think it's >> >>> vm.kmem_size_scale . What's it say for you? >> >>> >> >> >> >> :~ # sysctl vm.kmem_size_scale >> >> vm.kmem_size_scale: 3 >> > >> > Ok. Search the archives for an email from Hiren titled "mbuf autotuning effect". >> > >> > TL;DR - set it to 1 and recompile. There's a kernel option somewhere >> > to do exactly that. >> >> Yes. http://lists.freebsd.org/pipermail/freebsd-mips/2013-September/003081.html >> >> I went through this for my tplink. >> >> John, can you show o/p of: >> >> sysctl -a | grep hw | grep mem >> and >> sysctl -a | grep maxmbuf > > I already compiled a new kernel with "options VM_KMEM_SIZE_SCALE=1" and > the problems went away. :-) On the new kernel the output is: > > ########################## > :~ # sysctl -a | grep hw | grep mem > hw.physmem: 128196608 > hw.usermem: 103845888 > hw.realmem: 134213632 > :~ # sysctl -a | grep maxmbuf > kern.ipc.maxmbufmem: 62705664 > ########################## > > I rebooted with the old kernel and its output is: > > ########################## > :~ # sysctl -a | grep hw | grep mem > hw.physmem: 128196608 > hw.usermem: 104124416 > hw.realmem: 134213632 > :~ # sysctl -a | grep maxmbuf > kern.ipc.maxmbufmem: 20901888 > ########################## > > So for the heck of it, I ran my script again and this time it successfully > configured the 3 atheros interfaces. ? How can that be? One other thing I > also do not understand. I have an Avila (with 64M RAM, half of the CAMBRIA), > also with 3 atheros cards and there it always works. > > But if "options VM_KMEM_SIZE_SCALE=1" keeps the problem away, then I'll > leave that in my kernel. :-) > > Thanks for everyone that helped. > > John > -- > John Hay -- jhay@meraka.csir.co.za / jhay@meraka.org.za