Date: Sun, 21 Sep 2014 23:29:23 -0700 From: Adrian Chadd <adrian@freebsd.org> To: "K. Macy" <kmacy@freebsd.org> Cc: Rumen Telbizov <telbizov@gmail.com>, Tom Elite <qingli@freebsd.org>, "freebsd-stable@freebsd.org" <freebsd-stable@freebsd.org> Subject: Re: FreeBSD 10 network performance problems Message-ID: <CAJ-VmomJckhbJP4Ks_eiomNeSVaMH9g62%2BBimBZcWpMooSA%2BDQ@mail.gmail.com> In-Reply-To: <CAHM0Q_MXmn2P=vfFgaj5pZSqcTSm3h%2BKPAotc4K_8qpQUFh1dQ@mail.gmail.com> References: <CAENR%2B_VDVvnY0zWKVXHOjz2vWw27s%2ByVrz9ZFokZ=p6P6oFNvw@mail.gmail.com> <1411259605.674669006.339g4pd4@frv35.fwdcdn.com> <CAENR%2B_UwjMGoOqKkhCeL4zmLnSgTiABoeR-7x71MBvOnCF8z%2BA@mail.gmail.com> <CAHM0Q_MXmn2P=vfFgaj5pZSqcTSm3h%2BKPAotc4K_8qpQUFh1dQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi! On 21 September 2014 15:08, K. Macy <kmacy@freebsd.org> wrote: > What you're dealing with is hardly an edge case. Most people don't need to > push more than a couple of Gbps in production. > > Flowtable is hardly "untested." However, it has been a source of friction > at times because it can be somewhat brittle, having limits on the number of > cache entries that it can store that are frequently too low for people with > very large numbers of active flows. Without raising this limit > substantially these systems will fail in a rather spectacular fashion. > Additionally, flowtable was not written with the intent of being a routing > cache. It was developed to support stateful multipath routing for load > balancing. In its current incarnation, stripped of much of the code for its > initial purpose, it's really just a band-aid around locking problems in > routing. That said, the handful of commercial users of FreeBSD that do have > large amounts of traffic (10s of Gbps) per system that I personally know of > all have flowtable enabled. > > Unfortunately, at least in terms of what is in HEAD, little has been done > to fix the contention that flowtable works around. For your purposes the > response that Adrian gave you is the closest to "optimal." Gleb and I spent a bunch of time late last year and early this year finding and fixing a lot of the corner cases with Flowtable handling. I'm pretty sure that it'll behave predictably and reliably for people now. If it doesn't then please file PRs. It's no longer some corner of the codebase that isn't well understood. At least two people besides the author (Gleb and I) understand what it is, how it works and how it ties into things. What's missing is someone sitting down and adding flowtable support to the rest of the forwarding path(s). It shouldn't be too hard - as long as you have an mbuf that has the IPv4/IPv6 header setup as that's what flowtable currently expects when doing lookups - but it has to be done. So I thoroughly recommend someone who has the test setup and the desire to do so and post results. I have enough equipment now to test this out and develop it but I'm really busy doing work, wireless RSS stuff. So I just don't have the spare cycles to do it. I do think it'll be a pretty simple task. In theory - once you have the flowtable code working correctly in the forwarding path you shouldn't see any rtentry lock contention except during route changes (which will invalidate flowtable entries and cause normal routing table lookups until the flowtable has all the route entries in question.) -a
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-VmomJckhbJP4Ks_eiomNeSVaMH9g62%2BBimBZcWpMooSA%2BDQ>