From owner-freebsd-pf@freebsd.org Tue Oct 13 21:35:36 2020 Return-Path: Delivered-To: freebsd-pf@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1A05944C122 for ; Tue, 13 Oct 2020 21:35:36 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C9pkH6z5Cz4JXP; Tue, 13 Oct 2020 21:35:35 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.codepro.be", Issuer "Let's Encrypt Authority X3" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id B866B1A929; Tue, 13 Oct 2020 21:35:35 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: by venus.codepro.be (Postfix, authenticated sender kp) id CB6EF38795; Tue, 13 Oct 2020 23:35:33 +0200 (CEST) From: "Kristof Provost" To: "Andreas Longwitz" Cc: "J David" , freebsd-pf@freebsd.org Subject: Re: Packets passed by pf don't make it out? Date: Tue, 13 Oct 2020 23:35:29 +0200 X-Mailer: MailMate (1.13.2r5673) Message-ID: <0072D8A9-6ACE-47D0-AE94-124C4F955735@FreeBSD.org> In-Reply-To: <5F84CF18.1040905@incore.de> References: <5F8336C7.5020709@incore.de> <5F84CF18.1040905@incore.de> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Oct 2020 21:35:36 -0000 On 12 Oct 2020, at 23:48, Andreas Longwitz wrote: > Hello, > > now I can confirm (on FreeBSD 10 Stable) what you see on fb2 when your > program udp_client is running on fb1. pf creates a state for the first > packet only, for the other packets pf failes to create a state with > messages like > > pf: stack key attach failed on re0: UDP in wire: 192.168.14.10:23456 > 172.16.0.2:12345 stack: 192.168.14.10:23456 > 192.168.14.100:12345 1:0, existing: UDP in wire: 192.168.14.10:23456 > 172.16.0.1:12345 stack: 192.168.14.10:23456 192.168.14.100:12345 1:0 > > pf gives this messages in debug mode (pfctl -x loud). > > I do not know if we see a bug in pf or if your program udp_client does > something illegal, I think Kristof can tell us. > Your confidence is both flattering and misplaced :) I think I can reproduce the problem on CURRENT and with VNET jails, which is convenient. I see the same ‘stack key attach failed’ error message. My current thinking is that we’re hitting a state collision, because post-RDR our connection information is the same (192.168.14.10:23456 192.168.14.100:12345). That means we can’t create a new state, and the packet gets dropped. It’s a little unusual for a client to keep re-using src ports like that, but it’s not actually wrong. I’m not sure how we can fix this. Best, Kristof