From owner-freebsd-net@FreeBSD.ORG Thu May 4 17:05:24 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CF24216A403 for ; Thu, 4 May 2006 17:05:24 +0000 (UTC) (envelope-from babolo@cicuta.babolo.ru) Received: from ints.mail.pike.ru (ints.mail.pike.ru [85.30.199.194]) by mx1.FreeBSD.org (Postfix) with ESMTP id CEBC743D46 for ; Thu, 4 May 2006 17:05:23 +0000 (GMT) (envelope-from babolo@cicuta.babolo.ru) Received: (qmail 54666 invoked from network); 4 May 2006 17:05:21 -0000 Received: from cicuta.babolo.ru (85.30.224.245) by ints.mail.pike.ru with SMTP; 4 May 2006 17:05:21 -0000 Received: (nullmailer pid 82829 invoked by uid 136); Thu, 04 May 2006 17:13:51 -0000 X-ELM-OSV: (Our standard violations) hdr-charset=KOI8-R; no-hdr-encoding=1 In-Reply-To: <44595B76.9010901@widesoft.com.br> To: tpeixoto@widesoft.com.br Date: Thu, 4 May 2006 21:13:51 +0400 (MSD) From: .@babolo.ru X-Mailer: ELM [version 2.4ME+ PL99b (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Message-Id: <1146762831.921056.82828.nullmailer@cicuta.babolo.ru> Cc: Lee Johnston , freebsd-net@freebsd.org, Julian Elischer , mihai@duras.ro Subject: Re: Packet loss with traffic shaper and routing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 May 2006 17:05:24 -0000 [ Charset ISO-8859-1 unsupported, converting... ] > Very good. You're right! > I inserted a rule to match all non-layer2 packets on the top of the > ruleset and interrupts dropped 10~20% immediately. > Given that, I went to apply Julian's idea of grouping 'in' and 'out' > pipe rules to reduce the searching on the firewall and that gave me a > little bit more of performance. > As interrupts were still hitting 60% mark, I did some more experiences: > > Test 1: I changed all 'pipe' rules to 'allow' rules, so all packets were > allowed and no shaping was done. The pipes were still there, but there > were no rules pointing packets to them. > Result: No difference. Interrupts are the same as before. > Conclusion: It's not the shaping itself that slows the system. > > Test 2: With the same ruleset of test 1, I just removed all pipes (ipfw > pipe flush). As far as I understand traffic stops after pipe flush, and this is reason for CPU goes down > Result: Interrupts were only 20%! > Conclusion: Lots of pipes bother the system. I didn't figure out why, > but it's not a coincidence. I tested several times to make sure. > > Test 3: I applied Michael's idea of using 'mask src-ip' and 'mask > dst-ip' in the pipes to use them as a template for dynamic generated pipes. > Result: Worked like a charm. Now I have only 18 pipes instead of 3200. > Interrupts are ~30%. > Conclusion: The reduced number of pipes generated less system interrupts. > > The only problem I noticed (so far) with this method is that if we have > more than 1 IP address to a single MAC address, each IP will be shaped > individually instead of share the same speed of the other(s) IP(s) with > the same MAC. > > Anyway, I am very curious about the result of test 2. Why do the pipes > have influence on system performance if there is nothing passing through > them?