From owner-cvs-all@FreeBSD.ORG Tue May 29 13:21:22 2007 Return-Path: X-Original-To: cvs-all@FreeBSD.org Delivered-To: cvs-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2592016A469; Tue, 29 May 2007 13:21:22 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.freebsd.org (Postfix) with ESMTP id DB7EC13C457; Tue, 29 May 2007 13:21:21 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.14.0/8.14.0) with ESMTP id l4TDLLZo002780 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 29 May 2007 09:21:21 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.12.9p2/8.12.9/Submit) id l4TDKrZs034750; Tue, 29 May 2007 09:20:53 -0400 (EDT) (envelope-from gallatin) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18012.10444.46382.757318@grasshopper.cs.duke.edu> Date: Tue, 29 May 2007 09:20:53 -0400 (EDT) To: Gleb Smirnoff In-Reply-To: <20070528112812.GO89017@FreeBSD.org> References: <200705251938.l4PJcWRY007635@repoman.freebsd.org> <20070525154540.A22931@grasshopper.cs.duke.edu> <46574DD2.7080200@freebsd.org> <20070528112812.GO89017@FreeBSD.org> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Cc: cvs-src@FreeBSD.org, src-committers@FreeBSD.org, Andre Oppermann , cvs-all@FreeBSD.org Subject: Re: cvs commit: src/sys/dev/mxge if_mxge.c X-BeenThere: cvs-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: CVS commit messages for the entire tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 May 2007 13:21:22 -0000 Gleb Smirnoff writes: > On Fri, May 25, 2007 at 10:57:54PM +0200, Andre Oppermann wrote: > A> Andrew Gallatin wrote: > A> > Andrew Gallatin [gallatin@FreeBSD.org] wrote: > A> >> gallatin 2007-05-25 19:38:32 UTC > A> >> > A> >> FreeBSD src repository > A> >> > A> >> Modified files: > A> >> sys/dev/mxge if_mxge.c Log: > A> >> - Use m_getcl() rather than m_getjcl() when we're allocating 2KB > A> >> clusters. This helps quite a bit on my low end machines (improves > A> >> performance by about 300Kpps when being blasted by a hardware > A> >> packet generator). > A> > BTW, thanks to bmilekic for reminding me of the packet zone. > A> > It would be nice if such a thing existed for the jumbo sized > A> > clusters :) > A> > A> Secondary zones are not perfectly integrated with UMA and there > A> are some nits that prevent doing just that for the moment. We're > A> working on it. It's not trivial though. I want to de-inline some > A> functions in mbuf.h to make backend changes transparent to consumers > A> of it. > > Just 1 penny: Andrews commit shows that packet zone shouldn't be deleted, > it really improves performance. It is not clear that it always improves performance. I think it may have at least a little to do with the platform (athlon64 rather than xeon), and the fact that I allocate my receive buffers in the interrupt thread, rather than having a separate thread to allocate them. Both scottl and kmacy have reported substantial gains from allocating clusters well in advance (in a taskq for cxbe, not sure what scottl did for em) and allocating the pkthdr mbufs after the interrupt comes in. I did not notice an improvement when I tried allocating pkhdrs late (and I never tried a taskq/thread to allocate buffers). Again, it might be the platform. I'll try more experiments when I get access to Xeons again.. Speaking of mbufs, Bosko pointed out that in revision 1.156 of kern/uipc_mbuf.c, code was added which clears M_EXT information at mb_free_ext() time. He was concerned that doing this at free time might be cache unfriendly, and that this clearing used to be done at ctor time. Drew