From owner-freebsd-net@FreeBSD.ORG Thu Oct 9 15:34:17 2003 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 44F5A16A4B3; Thu, 9 Oct 2003 15:34:17 -0700 (PDT) Received: from mailout09.sul.t-online.com (mailout09.sul.t-online.com [194.25.134.84]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1795E43FE3; Thu, 9 Oct 2003 15:34:15 -0700 (PDT) (envelope-from Alexander@Leidinger.net) Received: from fwd01.aul.t-online.de by mailout09.sul.t-online.com with smtp id 1A7cXc-0004t8-05; Thu, 09 Oct 2003 17:17:24 +0200 Received: from Andro-Beta.Leidinger.net (E4RPvmZaZe38HVHF9ltwU2oIrHnCzL3wne3YWpQvXNxiSvimYlB+wU@[217.229.218.61]) by fmrl01.sul.t-online.com with esmtp id 1A7cX5-29J90a0; Thu, 9 Oct 2003 17:16:51 +0200 Received: from Magelan.Leidinger.net (Magellan [192.168.1.1]) h99FGkf5026509; Thu, 9 Oct 2003 17:16:46 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from Magelan.Leidinger.net (netchild@localhost [127.0.0.1]) h99FGjVx035946; Thu, 9 Oct 2003 17:16:45 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Date: Thu, 9 Oct 2003 17:16:45 +0200 From: Alexander Leidinger To: net@freebsd.org Message-Id: <20031009171645.33c63fa2.Alexander@Leidinger.net> X-Mailer: Sylpheed version 0.9.6claws (GTK+ 1.2.10; i386-portbld-freebsd5.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Seen: false X-ID: E4RPvmZaZe38HVHF9ltwU2oIrHnCzL3wne3YWpQvXNxiSvimYlB+wU@t-dialin.net cc: luigi@freebsd.org Subject: dummynet "OUCH! pipe should have been idle!"-message in 4.9-RC X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: netchild@FreeBSD.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Oct 2003 22:34:17 -0000 Hi, [please CC me] I noticed this log message on a 4.9-RC (src from ~Oct 1): ---snip--- Oct 9 15:57:42 Andro-Beta /kernel: dummynet: OUCH! pipe should have been idle! Oct 9 15:57:56 Andro-Beta /kernel: dummynet: OUCH! pipe should have been idle! ---snip--- I looked a little bit into the code, but I don't think I really understand what's happening here... I think I should worry about something if I see such a message. HZ is set to 1000, net.inet.ip.fw.one_pass=0 and dummynet/ipfw is set up like: ---snip--- [lo0 rules from rc.firewall] ############ # SetUp ${fwcmd} pipe 1 config bw 120Kbit/s queue 100 # 95% of real bandwith ${fwcmd} queue 1 config pipe 1 weight 50 mask all ${fwcmd} queue 2 config pipe 1 weight 1 queue 100 mask all ${fwcmd} queue 3 config pipe 1 weight 40 mask all # tcpflags ack iplen 66 ${fwcmd} add 10000 set 5 skipto 11000 tcp from any to any tcpflags ack out xmit ${outside_interface} ### iplen 1-66 # BW limit for edonkey ${fwcmd} add set 5 skipto 12000 tcp from any to any 2442,4661-4666 out xmit ${outside_interface} ${fwcmd} add set 5 skipto 13000 udp from any to any 4661-4666 out xmit ${outside_interface} ${fwcmd} add set 5 skipto 14000 tcp from any 2442,4661,4665 to any out xmit ${outside_interface} ${fwcmd} add set 5 skipto 15000 udp from any 4661,4665 to any out xmit ${outside_interface} # ssh: high priority ${fwcmd} add set 5 skipto 16000 tcp from any to any ssh out xmit ${outside_interface} # other outgoing traffic ${fwcmd} add set 5 skipto 17000 all from any to any out xmit ${outside_interface} # non outgoing traffic ${fwcmd} add set 5 skipto 20000 all from any to any ## shaping # VIP packets: tcpflags ack #iplen 66 ${fwcmd} add 11000 set 5 queue 1 tcp from any to any tcpflags ack out xmit ${outside_interface} ### iplen 1-66 ${fwcmd} add set 5 skipto 20000 tcp from any to any tcpflags ack out xmit ${outside_interface} ### iplen 1-66 # BW limit for mldonkey ${fwcmd} add 12000 set 5 queue 2 tcp from any to any 2442,4661-4666 out xmit ${outside_interface} ${fwcmd} add set 5 skipto 20000 tcp from any to any 2442,4661-4666 out xmit ${outside_interface} ${fwcmd} add 13000 set 5 queue 2 udp from any to any 4661-4666 out xmit ${outside_interface} ${fwcmd} add set 5 skipto 20000 udp from any to any 4661-4666 out xmit ${outside_interface} ${fwcmd} add 14000 set 5 queue 2 tcp from any 2442,4661,4665 to any out xmit ${outside_interface} ${fwcmd} add set 5 skipto 20000 tcp from any 2442,4661,4665 to any out xmit ${outside_interface} ${fwcmd} add 15000 set 5 queue 2 udp from any 4661,4665 to any out xmit ${outside_interface} ${fwcmd} add set 5 skipto 20000 udp from any 4661,4665 to any out xmit ${outside_interface} # VIP packets: ssh ${fwcmd} add 16000 set 5 queue 1 tcp from any to any ssh out xmit ${outside_interface} ${fwcmd} add set 5 skipto 20000 tcp from any to any ssh out xmit ${outside_interface} # normal traffic ${fwcmd} add 17000 set 5 queue 3 all from any to any out xmit ${outside_interface} ${fwcmd} add set 5 skipto 20000 all from any to any out xmit ${outside_interface} ## canary: We shouldn't get a log message from this rule! ${fwcmd} add set 5 skipto 20000 log all from any to any ############ # Allow packets with associated dynamic rules ${fwcmd} add 20000 check-state ############ # Don't allow spoofed "established" connections ${fwcmd} add 30000 set 6 deny tcp from any to any established ############ # Does the route agree with the incoming packet? ${fwcmd} add set 6 deny ip from any to any not verrevpath in ############ # Additional security rule ${fwcmd} add set 6 deny tcp from any to any tcpoptions !mss setup [some "allow ... setup keep-state" rules, ipfw defaults to deny] ---snip--- Some other things I noticed: The check-state rule doesn't show the number of matched packets/bytes (the dynamic rules show those stats, but I expected that the check-state rule displays the accumulated stats). Rule 30000 also shows a huge amount of matches, which surprised me. Are there this many spoofed packets out there (I removed it for now)? The "skipto 20000" rules after the queue rules show a large difference in matched packets. The queue stats don't show this many dropped packets. I also noticed a significant drop of throughtput if I have set 5 active, even if only one of the queues sees packets. Is the problem sitting in front of the keyboard or is there something else going on? Bye, Alexander. -- Speak softly and carry a cellular phone. http://www.Leidinger.net Alexander @ Leidinger.net GPG fingerprint = C518 BC70 E67F 143F BE91 3365 79E2 9C60 B006 3FE7