Date: Tue, 03 Jul 2012 21:37:38 +0400 From: "Alexander V. Chernikov" <melifaro@FreeBSD.org> To: Luigi Rizzo <rizzo@iet.unipi.it> Cc: hackers@freebsd.org, performance@freebsd.org, net@freebsd.org Subject: Re: FreeBSD 10G forwarding performance @Intel Message-ID: <4FF32DE2.2010606@FreeBSD.org> In-Reply-To: <20120703165506.GA90114@onelab2.iet.unipi.it> References: <4FF319A2.6070905@FreeBSD.org> <20120703165506.GA90114@onelab2.iet.unipi.it>
next in thread | previous in thread | raw e-mail | index | archive | help
On 03.07.2012 20:55, Luigi Rizzo wrote:
> On Tue, Jul 03, 2012 at 08:11:14PM +0400, Alexander V. Chernikov wrote:
>> Hello list!
>>
>> I'm quite stuck with bad forwarding performance on many FreeBSD boxes
>> doing firewalling.
> ...
>> In most cases system can forward no more than 700 (or 1400) kpps which
>> is quite a bad number (Linux does, say, 5MPPs on nearly the same hardware).
>
> among the many interesting tests you have run, i am curious
> if you have tried to remove the update of the counters on route
> entries. They might be another severe contention point.
21:47 [0] m@test15 netstat -I ix0 -w 1
input (ix0) output
packets errs idrops bytes packets errs bytes colls
1785514 52785 0 121318340 1784650 0 117874854 0
1773126 52437 0 120701470 1772977 0 117584736 0
1781948 52154 0 121060126 1778271 0 75029554 0
1786169 52982 0 121451160 1787312 0 160967392 0
21:47 [0] test15# sysctl net.rt_count=0
net.rt_count: 1 -> 0
1814465 22546 0 121302076 1814291 0 76860092 0
1817769 14272 0 120984922 1816254 0 163643534 0
1815311 13113 0 120831970 1815340 0 120159118 0
1814059 13698 0 120799132 1813738 0 120172092 0
1818030 13513 0 120960140 1814578 0 120332662 0
1814169 14351 0 120836182 1814003 0 120164310 0
Thanks, another good point. I forgot to merge this option from andre's
patch.
Another 30-40-50kpps to win.
+u_int rt_count = 1;
+SYSCTL_INT(_net, OID_AUTO, rt_count, CTLFLAG_RW, &rt_count, 1, "");
@@ -601,17 +625,20 @@ passout:
if (error != 0)
IPSTAT_INC(ips_odropped);
else {
- ro.ro_rt->rt_rmx.rmx_pksent++;
+ if (rt_count)
+ ro.ro_rt->rt_rmx.rmx_pksent++;
IPSTAT_INC(ips_forward);
IPSTAT_INC(ips_fastforward);
>
> cheers
> luigi
>
--
WBR, Alexander
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4FF32DE2.2010606>