From owner-freebsd-performance@FreeBSD.ORG Tue Jul 3 20:33:20 2012 Return-Path: Delivered-To: performance@freebsd.org Received: from mx2.freebsd.org (mx2.freebsd.org [69.147.83.53]) by hub.freebsd.org (Postfix) with ESMTP id F0D10106567E; Tue, 3 Jul 2012 20:33:20 +0000 (UTC) (envelope-from melifaro@FreeBSD.org) Received: from dhcp170-36-red.yandex.net (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx2.freebsd.org (Postfix) with ESMTP id 25AC9158FDC; Tue, 3 Jul 2012 20:33:04 +0000 (UTC) Message-ID: <4FF356BC.2060306@FreeBSD.org> Date: Wed, 04 Jul 2012 00:31:56 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:12.0) Gecko/20120511 Thunderbird/12.0.1 MIME-Version: 1.0 To: Luigi Rizzo References: <4FF319A2.6070905@FreeBSD.org> <20120703165506.GA90114@onelab2.iet.unipi.it> <4FF32DE2.2010606@FreeBSD.org> <20120703202757.GA90741@onelab2.iet.unipi.it> In-Reply-To: <20120703202757.GA90741@onelab2.iet.unipi.it> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: hackers@freebsd.org, performance@freebsd.org, net@freebsd.org Subject: Re: FreeBSD 10G forwarding performance @Intel X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jul 2012 20:33:21 -0000 On 04.07.2012 00:27, Luigi Rizzo wrote: > On Tue, Jul 03, 2012 at 09:37:38PM +0400, Alexander V. Chernikov wrote: > ... >> Thanks, another good point. I forgot to merge this option from andre's >> patch. >> >> Another 30-40-50kpps to win. > > not much gain though. > What about the other IPSTAT_INC counters ? Well, we should then remove all such counters (total, forwarded) and per-interface statistics (at least for forwarded packets). > I think the IPSTAT_INC macros were introduced (by rwatson ?) > following a discussion on how to make the counters per-cpu > and avoid the contention on cache lines. > But they are still implemented as a single instance, > and neither volatile nor atomic, so it is not even clear > that they can give reliable results, let alone the fact > that you are likely to get some cache misses. > > the relevant macro is in ip_var.h. Hm. This seems to be just per-vnet structure instance. We've got some more real DPCPU stuff (sys/pcpu.h && kern/subr_pcpu.c) which can be used for global ipstat structure, however since it is allocated from single area without possibility to free we can't use it for per-interface counters. I'll try to run tests without any possibly contested counters and report the results on Thursday. > > Cheers > luigi > >> >> +u_int rt_count = 1; >> +SYSCTL_INT(_net, OID_AUTO, rt_count, CTLFLAG_RW,&rt_count, 1, ""); >> >> @@ -601,17 +625,20 @@ passout: >> if (error != 0) >> IPSTAT_INC(ips_odropped); >> else { >> - ro.ro_rt->rt_rmx.rmx_pksent++; >> + if (rt_count) >> + ro.ro_rt->rt_rmx.rmx_pksent++; >> IPSTAT_INC(ips_forward); >> IPSTAT_INC(ips_fastforward); >> >> >>> >>> cheers >>> luigi >>> >> >> >> -- >> WBR, Alexander >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- WBR, Alexander