Date: Mon, 13 Jul 2009 02:20:32 -0700 (PDT) From: Barney Cordoba <barney_cordoba@yahoo.com> To: freebsd-current@freebsd.org, Chris Buechler <cmb@pfsense.org> Subject: Re: Flowtables -- any tuning hints? Message-ID: <837660.83798.qm@web63903.mail.re1.yahoo.com>
index | next in thread | raw e-mail
--- On Sun, 7/12/09, Chris Buechler <cmb@pfsense.org> wrote: > From: Chris Buechler <cmb@pfsense.org> > Subject: Re: Flowtables -- any tuning hints? > To: freebsd-current@freebsd.org > Date: Sunday, July 12, 2009, 6:41 PM > Kip Macy wrote: > > On Sat, Jul 11, 2009 at 10:24 AM, Scott Ullrich<sullrich@gmail.com> > wrote: > > > >> Hello Freebsd-current@ folks, > >> > >> I see with the commit "svn commit: r191259 - > head/sys/netinet" > >> flowtables have been added.. Cool! > >> > >> Does anyone have any tuning hints for this > addition -- specifically > >> how much memory does the hash table > consume? Or better yet does any > >> documentation exist for this newly added feature? > >> > >> Looking for an easy way to calculate max flows for > the amount of > >> memory installed in a FreeBSD machine. > >> > > > > > > You want to avoid hash collisions. So, generally > speaking you want the > > hash table to be sized 2x larger than the number of > unique connection > > destinations. You want the maximum number of > flows to be as large as > > the maximum number of unique destinations x number of > cores. When you > > get to the case of hundreds of thousands of unique > destinations as in > > the case of a small ISP doing IP forwarding, you're > probably better > > off disabling the flowtable. For most other workloads > its likely to be > > a clear win. Running a process on an 8-core system > with 8 threads each > > calling sendto(...) with 10 bytes I can push 3.5 - > 4Mpps (with cxgb - > > you won't get this with most cards) with the flowtable > enabled. With > > the flowtable disabled lock contention causes > performance to degrade > > to 330kpps with the aforementioned workload. > > > This is interesting functionality, but I think we need to > look at it a bit closer for our use case. Is there any > benefit in running this in a firewall scenario? That's > primarily what Scott and I (pfsense) are interested in. In > our world, if you're pushing 50Kpps+, you're almost > certainly falling into the "small ISP doing IP forwarding" > scenario with hundreds of thousands of unique destinations. > Where we usually see these kinds of loads are small ISPs, > web hosting companies, or universities (which are > functionally not much diff from a small ISP), all of which > I'm familiar with falling into the "better off disabling" > category. I also suspect pf's locking negates some or all of > the benefits here. > > I suspect it's not applicable to the specific workload our > users normally have, where you're almost entirely doing IP > forwarding, and initiating very little if any traffic. bz@ > said it's not something you want on a router. Is that a fair > assessment? > best, > Chris > I don't see the need for flowtables or any sort of hashing with multi queue boards in a pass-through environment (such as a firewall). The flows are already hashed by the hardware so doing all of the gobbledygook in flowtables seems like a waste of cycles. Just matching your receive and transmit queues in a properly designed driver is all thats necessary for lock avoidance. Barneyhelp
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?837660.83798.qm>