From owner-freebsd-net@FreeBSD.ORG Sun Feb 20 23:42:10 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A200D1065670; Sun, 20 Feb 2011 23:42:10 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 47E298FC12; Sun, 20 Feb 2011 23:42:10 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id C7FAA73098; Mon, 21 Feb 2011 00:55:40 +0100 (CET) Date: Mon, 21 Feb 2011 00:55:40 +0100 From: Luigi Rizzo To: Pawel Tyll Message-ID: <20110220235540.GA10655@onelab2.iet.unipi.it> References: <410175608.20110220013900@nitronet.pl> <1145317277.20110220045434@nitronet.pl> <20110220135855.GA4794@onelab2.iet.unipi.it> <288793167.20110220235028@nitronet.pl> <20110220231825.GA10566@onelab2.iet.unipi.it> <1167743969.20110221001312@nitronet.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1167743969.20110221001312@nitronet.pl> User-Agent: Mutt/1.4.2.3i Cc: Brandon Gooch , freebsd-ipfw@freebsd.org, Jack Vogel , freebsd-net@freebsd.org Subject: Re: problem analysys (Re: [Panic] Dummynet/IPFW related recurring crash.) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Feb 2011 23:42:10 -0000 On Mon, Feb 21, 2011 at 12:13:12AM +0100, Pawel Tyll wrote: > > understood. I am just saying that for instance the vlan presence and > > changes is quite significant in this context. > > You say vlans are "pretty much static" but can you tell us who adds/remove > > them, assign addresses ? > It's not that much work and changes are simple and far between. I do > that personally. IP addresses don't change, however I sometimes > (rarely) destroy and recreate vlans. Panics don't happen immediately > after this operation, or while it happens, and there were times from > panic to panic that I didn't touch a thing. > > > Also the ruleset must have something more than those two rules. > > From the stack trace, the panic seems to occur in a call to the > > "antispoof" option which presumably is somewhere in your ruleset. > > If not, then the stack is corrupt. > Full ruleset with IP addresses removed: > 00010 1691 128516 deny ip from any to any not antispoof in > 00020 87440010 6826835332 fwd [removed] ip from table(60) to table(61) > 00050 3246 156244 allow tcp from any to [removed] dst-port 53 // DNS Rules 50-59 > 00051 2463493 260607132 allow udp from any to [removed] // DNS Rules 50-59 > 00059 23891 1091822 deny ip from any to [removed] // DNS Rules 50-59 > 00100 32 2176 allow ip from any to any via lo0 > 00100 929493 48342523 deny ip from any to table(10) dst-port 131-139,445 > 00102 56574 2779124 fwd [removed] tcp from table(1) to not table(5) dst-port 80 > 00103 0 0 fwd [removed] tcp from table(2) to not table(5) dst-port 80 > 00104 427 17244 fwd [removed] tcp from table(3) to not table(5) > 00105 6 808 deny ip from table(3) to not table(5) > 00200 0 0 deny ip from any to 127.0.0.0/8 > 00300 0 0 deny ip from 127.0.0.0/8 to any > 00400 0 0 deny ip from any to ::1 > 00500 0 0 deny ip from ::1 to any > 00600 0 0 allow ipv6-icmp from :: to ff02::/16 > 00700 0 0 allow ipv6-icmp from fe80::/10 to fe80::/10 > 00800 0 0 allow ipv6-icmp from fe80::/10 to ff02::/16 > 00900 0 0 allow ipv6-icmp from any to any ip6 icmp6types 1 > 01000 0 0 allow ipv6-icmp from any to any ip6 icmp6types 2,135,136 > 30000 462392089 204487140826 pipe tablearg ip from table(100) to any in > 30001 535282183 461888428313 pipe tablearg ip from any to table(101) out > 34900 11650783 1216622001 skipto 35001 ip from table(10) to table(10) > 35000 597825867 244960831012 fwd [removed] ip from 192.168.0.0/16 to not 192.168.0.0/16 > 65534 1595697378 1254723485778 allow ip from any to any > 65535 0 0 allow ip from any to any > > 12:07AM up 1 day, 21 mins, 1 user, load averages: 0.08, 0.06, 0.01 > > Should IP addresses be required, I'll gladly send "uncensored" ruleset > to you privately. addresses not needed, thanks. From what i saw in the backtrace, the panic occurred on an incoming packet on the 'antispoof' option. The ruleset confirms the backtrace, but since 'antispoof' happens to be run on every packet given it is on the first rule, it apparently has nothing to do with dummynet because even if you reinjected the packets, they go to rule 34900. So, i'd still focus the attention on a corrupt interface list. Sure, that memory can be corrupt by anything including dummynet, but there is no reason to believe that dummynet is more likely than other subsystems to cause the breakage. Unfortunately i don't think I can be of more help. cheers luigi