From owner-freebsd-net@FreeBSD.ORG Mon Apr 23 13:55:15 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 720F9106564A for ; Mon, 23 Apr 2012 13:55:15 +0000 (UTC) (envelope-from freebsd-net@m.gmane.org) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) by mx1.freebsd.org (Postfix) with ESMTP id 21D4D8FC15 for ; Mon, 23 Apr 2012 13:55:15 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1SMJjQ-0003Hh-3R for freebsd-net@freebsd.org; Mon, 23 Apr 2012 15:55:08 +0200 Received: from w4336.dip.tu-dresden.de ([141.76.185.80]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 23 Apr 2012 15:55:08 +0200 Received: from js by w4336.dip.tu-dresden.de with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 23 Apr 2012 15:55:08 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Julian Stecklina Date: Mon, 23 Apr 2012 15:54:58 +0200 Lines: 37 Message-ID: <87k4161rkd.fsf@alien8.de> References: <20120419133018.GA91364@onelab2.iet.unipi.it> <4F907011.9080602@freebsd.org> <20120419204622.GA94904@onelab2.iet.unipi.it> <20120419212224.GA95459@onelab2.iet.unipi.it> <20120420144410.GA3629@onelab2.iet.unipi.it> <20120421155638.E982@besplex.bde.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: w4336.dip.tu-dresden.de User-Agent: Gnus/5.13 (Gnus v5.13) Cancel-Lock: sha1:vEefavturl0Q5W87Op6KqtYaAak= Subject: Re: Some performance measurements on the FreeBSD network stack X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Apr 2012 13:55:15 -0000 Thus spake Bruce Evans : > On Fri, 20 Apr 2012, K. Macy wrote: > >> On Fri, Apr 20, 2012 at 4:44 PM, Luigi Rizzo wrote: > >>> The small penalty when flowtable is disabled but compiled in is >>> probably because the net.flowtable.enable flag is checked >>> a bit deep in the code. >>> >>> The advantage with non-connect()ed sockets is huge. I don't >>> quite understand why disabling the flowtable still helps there. >> >> Do you mean having it compiled in but disabled still helps >> performance? Yes, that is extremely strange. > > This reminds me that when I worked on this, I saw very large throughput > differences (in the 20-50% range) as a result of minor changes in > unrelated code. I could get these changes intentionally by adding or > removing padding in unrelated unused text space, so the differences were > apparently related to text alignment. I thought I had some significant > micro-optimizations, but it turned out that they were acting mainly by > changing the layout in related used text space where it is harder to > control. For short code paths, code layout can significantly influence performance. We have been puzzled (in a project unrelated to FreeBSD) by a 10% performance drop in some microbenchmark that was ultimately caused by having all our code hotspots linked at 8K aligned addresses, which caused them to evict each other from the L1 instruction cache, because its associativity was too small. A simple way to check for this would be to have the option to build a kernel with random linking order. I don't know how difficult it is to implement that in the current FreeBSD toolchain. Julian