From owner-freebsd-current@FreeBSD.ORG Tue Feb 24 23:10:25 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A8ACEEC3; Tue, 24 Feb 2015 23:10:25 +0000 (UTC) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 67C0E997; Tue, 24 Feb 2015 23:10:25 +0000 (UTC) Received: from vega.codepro.be (unknown [172.16.1.3]) by venus.codepro.be (Postfix) with ESMTP id 9F8A51B123; Wed, 25 Feb 2015 00:10:22 +0100 (CET) Received: by vega.codepro.be (Postfix, from userid 1001) id 99F14F257; Wed, 25 Feb 2015 00:10:22 +0100 (CET) Date: Wed, 25 Feb 2015 00:10:22 +0100 From: Kristof Provost To: Davide Italiano Subject: Re: pf crash on -current Message-ID: <20150224231022.GN2181@vega.codepro.be> References: <54EBD112.8050705@freebsd.org> <20150224070547.GH2181@vega.codepro.be> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20150224070547.GH2181@vega.codepro.be> X-PGP-Fingerprint: E114 D9EA 909E D469 8F57 17A5 7D15 91C6 9EFA F286 X-Checked-By-NSA: Probably User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-current , Allan Jude X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 23:10:25 -0000 On 2015-02-24 08:05:47 (+0100), Kristof Provost wrote: > On 2015-02-23 17:23:55 (-0800), Davide Italiano wrote: > > The bt you posted suggest this could be stack overflow, probably due > > to infinite recursion. > > Also, as a wild guess, just looking at the stacktrace, I think this > > might be related to the recent ipv6 fragment changes. Try to back them > > out, and see if things gets more stable ( r278831 and r278843). > > > That's almost certainly what it is. > After a bit of fiddling around I've managed to reproduce this locally. Essentially we get caught in a loop of defragmenting and refragmenting: Fragmented packets come in on one interface and get collected until we can defragment it. At that point the defragmented packet is handed back to the ip stack (at the pfil point in ip6_input(). Normal processing continues. Eventually we figure out that the packet has to be forwarded and we end up at the pfil hook in ip6_forward(). After doing the inspection on the defragmented packet we see that the packet has been defragmented and because we're forwarding we have to refragment it. That's indicated by the presence of the PF_REASSEMBLED tag. In pf_refragment6() we remove that tag, split the packet up again and then ip6_forward() the individual fragments. Those fragments hit the pfil hook on the way out, so they're collected until we can reconstruct the full packet, at which point we're right back where we left off and things continue until we run out of stack. There are two reasons Allan is seeing this and no one else has so far. The first is that he's scrubbing both on input and output. My own tests have always been done with 'scrub in all fragment reassemble', rather than 'scrub all fragment reassemble' so I didn't see this problem. The second is that he's got an internal interface with a higher MTU, so the refragmentation actually works for him. There's an open problem where ip6_forward() drops the defragmented packet before the pfil(PFIL_OUT) hook because it's too big for the output interface. If the last patch of my series (https://reviews.freebsd.org/D1815) had been merged as well more people would have been affected. One possible fix for Allan's problem would be to tag the fragments after refragmentation so that pf ignores them. After all, the defragmented packet has already been inspected so there's no point in checking the fragments again. I have the feeling there's a way to fix this problem and the issue D1815 tries to fix in one go though. I'll need to think about it a bit more. Regards, Kristof