Date: Tue, 17 Jul 2012 12:36:41 +0400 From: "Alexander V. Chernikov" <melifaro@FreeBSD.org> To: Luigi Rizzo <rizzo@iet.unipi.it> Cc: Doug Barton <dougb@freebsd.org>, net@freebsd.org Subject: Re: FreeBSD 10G forwarding performance @Intel Message-ID: <50052419.7010601@FreeBSD.org> In-Reply-To: <20120716212249.GA14607@onelab2.iet.unipi.it> References: <4FF36438.2030902@FreeBSD.org> <4FF3E2C4.7050701@FreeBSD.org> <4FF3FB14.8020006@FreeBSD.org> <4FF402D1.4000505@FreeBSD.org> <20120704091241.GA99164@onelab2.iet.unipi.it> <4FF412B9.3000406@FreeBSD.org> <20120704154856.GC3680@onelab2.iet.unipi.it> <4FF59955.5090406@FreeBSD.org> <20120706061126.GA65432@onelab2.iet.unipi.it> <500452A5.3070501@FreeBSD.org> <20120716212249.GA14607@onelab2.iet.unipi.it>
next in thread | previous in thread | raw e-mail | index | archive | help
On 17.07.2012 01:22, Luigi Rizzo wrote:
> On Mon, Jul 16, 2012 at 09:43:01PM +0400, Alexander V. Chernikov wrote:
>> On 06.07.2012 10:11, Luigi Rizzo wrote:
>>> On Thu, Jul 05, 2012 at 05:40:37PM +0400, Alexander V. Chernikov wrote:
>>>> On 04.07.2012 19:48, Luigi Rizzo wrote:
> well, it seems that the counters are costing some 10% which is
> not negligible (60ns per packet according to your test).
> Also i'd be curious if you get better savings if you
> have actual conflicts on the rulesets (e.g. what happens
> with a ruleset that has, say, ten "count ip from any to any" rules) ?
It is a bit difficult to get _exact_ performance numbers since 0.5% of
linerate is ~ 70kpps, however
1.98 MPPS
>> net.inet.ip.fw.update_counters=1
>> net.inet.ip.fw.enable=1
1.67 MPPS
.. And here it is time to check ipfw rmlock performance another time,
since we're acquiring recursive rmlock (pfil) and rwlock (ipfw) twice.
input (ix0) output
packets errs idrops bytes packets errs bytes colls
1664518 0 0 109910406 1664280 0 110055646 0
1664155 0 0 110018508 1664960 0 109921738 0
1663795 0 0 109839018 1664618 0 109965576 0
00100 count ip from any to any
1633118 22691 0 109539808 1621567 0 107402164 0
1625215 42836 0 110080554 1625638 0 107257950 0
1630848 34315 0 109932628 1631634 0 72449482 0
1613686 44167 0 109493942 1613811 0 142363996 0
1613387 53236 0 110075314 1614144 0 106479880 0
1611789 52348 0 109932904 1611600 0 106542318 0
1608327 56371 0 109947824 1608218 0 106229134 0
1615790 50527 0 110015368 1615528 0 106638914 0
1613453 50508 0 109872060 1614115 0 72118650 0
1614382 50955 0 109957958 1613808 0 141208126 0
1612053 54185 0 110002138 1611855 0 106490270 0
1538015 13138 0 102872260 1547403 0 102004436 0
1538084 0 0 101536936 1538034 0 66189600 0
1536305 0 0 101456028 1533714 0 101844506 0
1537533 0 0 101458596 1533775 0 101425338 0
00200 count ip from any to any
1529260 6840 0 101471016 1526825 0 100819632 0
1532496 5926 0 101540068 1534299 0 101292096 0
1532535 4412 0 101596090 1531906 0 101148828 0
1527551 9545 0 101488912 1527051 0 100957332 0
1538293 1523 0 101655604 1539942 0 101557938 0
1536673 0 0 84887698 1537473 0 69175150 0
1538330 0 0 118127042 1537831 0 134094698 0
..
00300 count ip from any to any
1481474 0 0 97944746 1481326 0 67132158 0
1489185 2409 0 98413408 1480661 0 128797966 0
1476976 9604 0 98221830 1478449 0 97810444 0
1476286 12574 0 98464756 1479253 0 97638972 0
1483221 10545 0 98451838 1479954 0 97677840 0
So (very rought),
First rule: 0.25-0.35MPPS (10-15% overall performance)
Second rule: 0.05MPPS
Third rule: 0.11MPPS (!)
>> Additionally, I've got (from my previous pcpu attempt) a small patch
>> permitting ipfw to re-use rule map allocation instead of reallocating
>> on every rule. This saves a bit of system time:
>>
>> loading 20k rules with ipfw binary gives us:
>> 5.1s system time before and 4.1s system time after.
>
> not bad but in this case i wonder if one wouldn't
> get much higher savings by support multiple rule loading with a
> single syscall. The format used in IPFW3 should help that.
This seems to be more reasonable in general. So, skipping this patch?
>
> cheers
> luigi
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?50052419.7010601>