From owner-freebsd-net@FreeBSD.ORG Fri Mar 8 20:13:29 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id DE245969; Fri, 8 Mar 2013 20:13:29 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-vb0-x22a.google.com (mail-vb0-x22a.google.com [IPv6:2607:f8b0:400c:c02::22a]) by mx1.freebsd.org (Postfix) with ESMTP id 7BEF4844; Fri, 8 Mar 2013 20:13:29 +0000 (UTC) Received: by mail-vb0-f42.google.com with SMTP id ff1so806973vbb.1 for ; Fri, 08 Mar 2013 12:13:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=WSoNqN97tyowqEdqZLWxrDvqQNDpD7DH6Yyp44TzIZo=; b=YKpA/eGUhk18+UYso/10hrMYdAzx6ZUj5x7P8mlFQC8Sr7KRgSB/xkMPhmG6EYC/EG xa6LbqI3Stls3gC6lboeXurnmx1rlcRFWi2xV/Tc+pk52B9Dzitj6zNYGNYVpRSWLtqb Ib7YphqtJdq2KwNPQmOiV6+w5zj2oz9bUR9GRENWmIIuNDnxM6gPAgP8NtB0NKrl4eI8 MigSZH2mSBPG4V0ni9tVRToLrn5WJ01qutU1woqSgWhRm4wBX1rida4iMMfLLBm1N6Iw 6RYTk7uUegtmickuNHP2Z8fN/TbP/HcHalfe2BTYE9Le9C9PhtV8M0gDP/PxeEi37TVi e01w== MIME-Version: 1.0 X-Received: by 10.52.19.239 with SMTP id i15mr1292070vde.47.1362773608886; Fri, 08 Mar 2013 12:13:28 -0800 (PST) Received: by 10.220.191.132 with HTTP; Fri, 8 Mar 2013 12:13:28 -0800 (PST) In-Reply-To: <513A2887.2010408@freebsd.org> References: <20793.36593.774795.720959@hergotha.csail.mit.edu> <51399926.6020201@freebsd.org> <20794.6692.191898.682241@hergotha.csail.mit.edu> <513A2887.2010408@freebsd.org> Date: Fri, 8 Mar 2013 12:13:28 -0800 Message-ID: Subject: Re: Limits on jumbo mbuf cluster allocation From: Jack Vogel To: Andre Oppermann Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-net@freebsd.org, Garrett Wollman X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Mar 2013 20:13:29 -0000 Yes, in the past the code was in this form, it should work fine Garrett, just make sure the 4K pool is large enough. I've actually been thinking about making the ring mbuf allocation sparse, and what type of strategy could be used. Right now I'm thinking of implementing a tunable threshold, and as long as I'm doing that, the 82599 hardware has an interrupt that can be set up for a low descriptor condition. This could have some performance benefits, I could decouple the mbuf refresh even further from rxeof, relying on the interrupt to initiate the refresh rather than a count as it is now. It needs some experimentation/testing first, but I will look into this. Jack On Fri, Mar 8, 2013 at 10:05 AM, Andre Oppermann wrote: > On 08.03.2013 18:04, Garrett Wollman wrote: > >> < said: >> >> I am not strongly opposed to trying the 4k mbuf pool for all larger >>> sizes, >>> Garrett maybe if you would try that on your system and see if that helps >>> you, I could envision making this a tunable at some point perhaps? >>> >> >> If you can provide a patch I can certainly build it in to our kernel >> and have it ready the next time the production server crashes. I'd >> like it to be at least a *little* tested by someone else beforehand, >> though. >> > > This should do the trick. > > -- > Andre > > Index: dev/ixgbe/ixgbe.c > ==============================**==============================**======= > --- dev/ixgbe/ixgbe.c (revision 247893) > +++ dev/ixgbe/ixgbe.c (working copy) > @@ -1120,12 +1120,8 @@ > */ > if (adapter->max_frame_size <= 2048) > adapter->rx_mbuf_sz = MCLBYTES; > - else if (adapter->max_frame_size <= 4096) > + else > adapter->rx_mbuf_sz = MJUMPAGESIZE; > - else if (adapter->max_frame_size <= 9216) > - adapter->rx_mbuf_sz = MJUM9BYTES; > - else > - adapter->rx_mbuf_sz = MJUM16BYTES; > > /* Prepare receive descriptors and buffers */ > if (ixgbe_setup_receive_**structures(adapter)) { >