Date: Thu, 18 Sep 2014 12:30:07 -0700 From: Freddie Cash <fjwcash@gmail.com> To: "freebsd-ipfw@freebsd.org" <freebsd-ipfw@freebsd.org> Subject: Re: High intr CPU % and slow throughput Message-ID: <CAOjFWZ79RN_Pr%2Bk6w6E0fW0QzWiYrSuaHVJnN3ft5Bzg4sXK6A@mail.gmail.com> In-Reply-To: <CAOjFWZ7DjjTUmk%2Ba9VdLuetwuTrZdQ9OkrrS3FX3c%2BWs18E-pQ@mail.gmail.com> References: <CAOjFWZ7DjjTUmk%2Ba9VdLuetwuTrZdQ9OkrrS3FX3c%2BWs18E-pQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
=E2=80=8BAha! I believe I've found the cause of our current issue. In an effort to allow reloading of the firewall rules during the day without disconnecting anyone (dropping TCP connections), I started playing with rule sets. And everything appeared to be working wonderfully, in that I could restart the rules multiple times without dropping any packets or disconnecting anyone. But, CPU usage skyrocketed on large downloads and =E2=80=8B =E2=80=8Bwe were capped at a little less than 40 Mbps. :( It seems that if you do the following (at least twice, to make sure rules are in both sets), your CPU will melt: - clear set 1 - disable set 1 - load 4000 rules into set 1 - enable set 1 - swap sets 1 and 0 - disable set 1 =E2=80=8BI thought that would leave only the rules in set 0 active, which w= ould be the equivalent of only having loaded rules into set 0. However, it seems that ipfw still checks rules in disabled sets! Or does some kind of processing with disabled sets. pmcstat was showing lots (200-2000) of unresolved samples and ipfw.ko sitting at 80-90% in the list, even when CPU usage was around 30%. I did the above, but added "ipfw -f set 1 flush" as the last step, and everything is back to normal. pmstat is now empty (0 unresolved). We can now push 75 Mbps through the firewall with CPU usage under 80%. More importantly, though, other traffic is not impacted by large downloads and speedtests and streaming video! And, CPU usage is sitting at under 10% for "normal" traffic. =E2=80=8BYes, I know 4000 rules is =E2=80=8Ba lot (doing NAT for 66 systems= and 2 local subnets). Until now I was focusing on getting things working (migrating from FreeBSD 7 using IPFW+natd with lots of private IP to private IP rules; to FreeBSD 10 using IPFW + in-kernel NAT and proper double-NAT across networks using public IPs only). Optimisation work is just now beginning. :) --=20 Freddie Cash fjwcash@gmail.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOjFWZ79RN_Pr%2Bk6w6E0fW0QzWiYrSuaHVJnN3ft5Bzg4sXK6A>