From owner-freebsd-net@FreeBSD.ORG Thu Jan 30 03:37:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 316ED84F for ; Thu, 30 Jan 2014 03:37:26 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id E73981B5E for ; Thu, 30 Jan 2014 03:37:25 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Aj8MACXI6VKDaFve/2dsb2JhbABZDoFkBoFMV4MBuUtPgRp0giUBAQEEAQEBICsgCxsYAgINGQIjBgEJJgYIBwQBHAEDh1ADEQ2qbpdHDYkcF4Epi0GBPgYBAQEaNAeCb4FJBIlJineBFWeDHosshUGCbl0eMXwIFyI X-IronPort-AV: E=Sophos;i="4.95,746,1384318800"; d="scan'208";a="91742974" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 29 Jan 2014 22:37:24 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id D2A4479283; Wed, 29 Jan 2014 22:37:24 -0500 (EST) Date: Wed, 29 Jan 2014 22:37:24 -0500 (EST) From: Rick Macklem To: Garrett Wollman Message-ID: <323566728.18752313.1391053044849.JavaMail.root@uoguelph.ca> In-Reply-To: <201401300322.s0U3Mt3s010029@hergotha.csail.mit.edu> Subject: Re: Big physically contiguous mbuf clusters MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: freebsd-net@freebsd.org, nparhar@gmail.com X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jan 2014 03:37:26 -0000 Garrett Wollman wrote: > In article <20140129231138$3db6@grapevine.csail.mit.edu>, > nparhar@gmail.com writes: > > >I think this would be very useful. For example, a zone_jumbo32 > >would > >hit a sweet spot -- enough to fit 3 jumbo frames and some loose > >change > >for metadata. I'd like to see us improve our allocators and VM > >system > >to work better with larger contiguous allocations, rather than > >deprecating the larger zones. It seems backwards to push towards > >smaller allocation units when installed physical memory in a typical > >system continues to rise. > > In order to resist fragmentation, you need to be willing to dedicate > some partition of physical memory to larger allocations. That's fine > for a special-purpose device like a switch, but is not so good for a > general-purpose operating system. But if you were willing to > reserve, > say, 1/64th of physical memory at boot time, make it all > direct-mapped > using superpages, and allocate it in fixed-power-of-two-sized chunks, > you would probably get a performance win. But the chunks *have* to > be > fixed-size, otherwise you are nearly guaranteed to get your arena > checkerboarded. I'd consider giving 2 GB on a 128-GB machine for > that. > > For NFS performance, you'd probably want to be able to take a whole > chunk, read the desired data into it in a single VOP, then pass the > whole thing to the socket layer wrapped in an mbuf. > Yep, 1 64K (or 128K soon) mbuf would be nice for read, readdir, write. (Assuming tcp_output knows how to split it up for net interfaces that can't handle TSO segments that large.) I'm not sure why, but most use 65535 (max IP datagram size) as if_hw_tsomax. (This guarantees the 64K NFS send gets split up. Doesn't TSO split it up into mtu sized segments? If so, I don't see why if_hw_tsomax would be a limit?) I'm not knowledgible w.r.t. TSO, so feel free to ignore or ocrrect this. rick > -GAWollman > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" >