From owner-freebsd-net@FreeBSD.ORG Sun Sep 8 00:25:19 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0257995A; Sun, 8 Sep 2013 00:25:18 +0000 (UTC) (envelope-from hiren.panchasara@gmail.com) Received: from mail-ee0-x22f.google.com (mail-ee0-x22f.google.com [IPv6:2a00:1450:4013:c00::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 30A182CCF; Sun, 8 Sep 2013 00:25:18 +0000 (UTC) Received: by mail-ee0-f47.google.com with SMTP id d49so2349414eek.6 for ; Sat, 07 Sep 2013 17:25:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=wzQX3A8xfDuNDhu+K/pgNbPEnXMcWTpC/kzjDnK6xGY=; b=v3wuF2zarKoOA9yF0X2p73kqhcKFr1nHa14OyUDdamx3uXRZL+AWXL3opciL/e1MWN +kvxAcSmxwmgHUt/gcYTzFmbGdO79DtWv5vIVkNQeIGQ5XZR1MkIFBTK1CHv4dMFN1Sl Dlh80LHVgZH9YVDmGnrEQpBkZ7JL+tzAxkWWu1ceNcARvXjOeg7H7FKXfEVUloywSnl0 K5G8znjfuRWsN1y00wXAgTZh7swFa/rYowz0EV+sdqLnvcR13egSUuiRzOMxEKGE+ZFY a3tqBsp6RmX0WJYigIRnP7QHH/2alD3TTtABR39X+5NNO8RxtRFcev1WZQq0wMWIr48o lsBQ== MIME-Version: 1.0 X-Received: by 10.15.64.1 with SMTP id n1mr17328564eex.15.1378599915944; Sat, 07 Sep 2013 17:25:15 -0700 (PDT) Received: by 10.14.105.137 with HTTP; Sat, 7 Sep 2013 17:25:15 -0700 (PDT) In-Reply-To: <1378583762.1111.512.camel@revolution.hippie.lan> References: <9CBFAD35-D651-4E28-BEBB-DC3717F38567@bsdimp.com> <1378583762.1111.512.camel@revolution.hippie.lan> Date: Sat, 7 Sep 2013 17:25:15 -0700 Message-ID: Subject: Re: mbuf autotuning effect From: hiren panchasara To: Ian Lepore Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-net@freebsd.org" , "freebsd-mips@freebsd.org" , Warner Losh X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Sep 2013 00:25:19 -0000 On Sat, Sep 7, 2013 at 12:56 PM, Ian Lepore wrote: > On Sat, 2013-09-07 at 12:21 -0700, hiren panchasara wrote: > > On Sep 6, 2013 8:26 PM, "Warner Losh" wrote: > > > > > > > > > On Sep 6, 2013, at 7:11 PM, Adrian Chadd wrote: > > > > > > > Yeah, why is VM_KMEM_SIZE only 12mbyte for MIPS? That's a little > > low > > for a > > > > platform that has a direct map that's slightly larger than 12mb :) > > > > > > > > Warner? Juli? > > > > > > All architectures have it at 12MB, except sparc64 where it is 16MB. > > This > > can be changed with the options VM_KMEM_SIZE=xxxxx in the config file. > > > > Right. Does that mean for any platform, if we do not have nmbclusters > > pre-set in kmeminit() than we will always have pretty low value of > > vm_kmem_size. And because of that, if maxmbufmem is not pre-set (via > > loader.conf) inside tunable_mbinit() , we will have very low value for > > maxmbufmem too. > > > > I hope (partially believe) that my understanding is not entirely > > correct. > > Because if its correct, we arw depending on loader.conf instead of > > actually > > auto tuning. > > > I think the part of this that strikes me as strange is calling 20% of > physical memory used for network buffers a "very low value". It seems > outrageously high to me. I'd be pissed if that much memory got wasted > on network buffers on one of our $work platforms with so little memory. > Interesting. So here how it looks on my laptop running amd64 GENERIC looks like: (without any special loader.conf settings) flymockour-l7% uname -a FreeBSD flymockour-l7.corp.yahoo.com 10.0-CURRENT FreeBSD 10.0-CURRENT #1 r253512M: Sat Jul 20 23:00:51 PDT 2013 hirenp@flymockour-l7.corp.yahoo.com:/usr/obj/usr/home/hirenp/head/sys/GENERIC amd64 flymockour-l7% sysctl -a | grep hw| grep mem hw.physmem: 8496877568 hw.usermem: 3538432000 hw.realmem: 9093251072 flymockour-l7% sysctl kern.ipc.maxmbufmem kern.ipc.maxmbufmem: 4132540416 flymockour-l7% sysctl -a | grep vm.kmem_ vm.kmem_size: 8265080832 vm.kmem_size_min: 0 vm.kmem_size_max: 329853485875 vm.kmem_size_scale: 1 vm.kmem_map_size: 1380515840 vm.kmem_map_free: 5796265984 VM_KMEM_SIZE_SCALE is 1 for amd64 while 3 for mips. Which might be one reason. > So the fact that you think it's crazy-low and I think it's crazy-high > may be a sign that it's auto-tuned to a reasonable compromise, and in > both our cases the right fix would be to use the available knobs to tune > things for our particular uses. > I am pretty ignorant on what the value _should_ be. I will try to find out more. cheers, Hiren