From nobody Mon Dec 8 13:42:07 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4dQ37S75MRz6Jpv6 for ; Mon, 08 Dec 2025 13:42:12 +0000 (UTC) (envelope-from grembo@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R13" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4dQ37S5kCLz3jw6; Mon, 08 Dec 2025 13:42:12 +0000 (UTC) (envelope-from grembo@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1765201332; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EYaFgDGI5Sb3jc0nlQ6vIm/cQtYHDTYke5gMR61WUaI=; b=Hn0Vw/osNZCIUh5TFLmHD/Sez8K6GmJjjpa2bvDK2sSL2kaxzM5m3uJmK/R6Q4AiNcZ3fO oBah9RhWBu1EF8FZjbBKFRKVZ3rVLi9a81S5hLmSJ7QF3zSO3ITNt01iZKzYNHerQUFHmu E4aY3/5hav49OBQS5T9GCL1iDLk+S3SnATdzcljeyZawYf8dkd+n9ADv0T+SJ4RDUFCPEV /v4Z/3+Dodt231q1t+gqQAryc3QAf4EwVH7vaHEfdoiDfwVguI6ig7adexJ2UeTCayECpu NuTT/7ISXOuukVfXvnqr65zWPraeY/tdlVXTUCKA7rl60zvJ9QIOlESS9K7ZOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1765201332; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EYaFgDGI5Sb3jc0nlQ6vIm/cQtYHDTYke5gMR61WUaI=; b=yZ/KZE1yEz42qxZqXlqZsDjWVn4E4nP+D8GSfDQ5AKBGE8JV3F3fm0olbUQFTdWWa0nOgH ScsYYeIAUMEwThjH/5IhEmmim6r0d7z5axHKjpHGWu8xk0skP7gssLSvtVayAwYTrQD2Lb kkQ/AfafO5dxTBYpHI6AJ3MjEBpmPy6yW3TxW2bBcmmLP5ZV4a93xHF//7/DSmC6O91jpc 0prQISuO3EiKbcRzPvt0TGH7vMSk3QY9sbuqdO8B/3D697xIiJTejaPQ9QkvPb0uxMiWfH vKPWYvxCyOmWAGi2kiXdkdaxt3NV50YK/iioKpO+sRqS2uT0TbTu/feokkTJsg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1765201332; a=rsa-sha256; cv=none; b=jckI12+ONPcvzMT+FISeHYSLwm+KLGMKPoYTAPPLBlQ9FVrH97mSSNRdd10RwXhFtxLo/o oXlYC/3ZCHAF++EDfslj+7CSQdfScLE6z2Ql5EUrSZomtgVuEFIwk7rRa1sx913nupKiOp yBKPlDgvCLooz3qnLheLWxY2D/FCziugoYTcfnH+Fbd6llq54D+/YiZgqDQLpOXlKeWNPN T89ayHsW116QC+n4hA6ydTpvSuoUBdpzraEU20r1ktcoo7aabnXpbhA9al2X6G9pZEBTbe /iMeZ6ERXgIWpoRA8oRPhmP87ZPo2W6c+T9rJ18BOhQAZ2yiA9CnVnGL3xatYg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from mail.evolve.de (mail.evolve.de [213.239.217.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512) (Client did not present a certificate) (Authenticated sender: grembo/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4dQ37S1b0fz15b5; Mon, 08 Dec 2025 13:42:12 +0000 (UTC) (envelope-from grembo@freebsd.org) Received: by mail.evolve.de (OpenSMTPD) with ESMTPSA id 2f5d8a15 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 8 Dec 2025 13:42:10 +0000 (UTC) Date: Mon, 8 Dec 2025 14:42:07 +0100 From: Michael Gmelin To: Michael Butler Cc: Rozhuk Ivan , freebsd-current Subject: Re: fib selection and persistence using ipfw Message-ID: <20251208144207.6b1f4ea6.grembo@freebsd.org> In-Reply-To: <75037780-3748-4cf3-8a44-a0e9c0b76e06@protected-networks.net> References: <20350073-abc5-4116-9fd7-8e8f708a26d4@protected-networks.net> <20251208031147.393b2391@rimwks.local> <75037780-3748-4cf3-8a44-a0e9c0b76e06@protected-networks.net> X-Face: $wrgCtfdVw_H9WAY?S&9+/F"!41z'L$uo*WzT8miX?kZ~W~Lr5W7v?j0Sde\mwB&/ypo^}> +a'4xMc^^KroE~+v^&^#[B">soBo1y6(TW6#UZiC]o>C6`ej+i Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAJFBMVEWJBwe5BQDl LASZU0/LTEWEfHbyj0Txi32+sKrp1Mv944X8/fm1rS+cAAAACXBIWXMAAAsTAAAL EwEAmpwYAAAAB3RJTUUH3wESCxwC7OBhbgAAACFpVFh0Q29tbWVudAAAAAAAQ3Jl YXRlZCB3aXRoIFRoZSBHSU1QbbCXAAAAAghJREFUOMu11DFvEzEUAGCfEhBVFzuq AKkLd0O6VrIQsLXVSZXoWE5N1K3DobBBA9fQpRWc8OkWouaIjedWKiyREOKs+3PY fvalCNjgLVHeF7/3bMtBzV8C/VsQ8tecEgCcDgrzjekwKZ7TwsJZd/ywEKwwP+ZM 8P3drTsAwWn2mpWuDDuYiK1bFs6De0KUUFw0tWxm+D4AIhuuvZqtyWYeO7jQ4Aea 7jUqI+ixhQoHex4WshEvSXdood7stlv4oSuFOC4tqGcr0NjEqXgV4mMJO38nld4+ xKNxRDon7khyKVqY7YR4d+Cg0OMrkWXZOM7YDkEfKiilCn1qYv4mighZiynuHHOA Wq9QJq+BIES7lMFUtcikMnkDGHUoncA+uHgrP0ctIEqfwLHzeSo+eUA66AqzwN6n 2ZHJhw6Qh/PoyC/QENyEyC/AyNjq74Bs+3UH0xYwzDUC4B97HgLocg1QLYgDDO1v f3UX9Y307Ew4AHh67YAFFsxEpkXwpXY3eIgMhAAE3R19L919nNnuD2wlPcDE3UeT L2ytEICQib9BXgS2fU8PrD82ToYO1OEmMSnYTjSqSv9wdC0tPYC+rQRQD9ESnldF CyqfmiYW+tlALt8gH2xrMdC/youbjzPXEun+/ReXsMCDyve3dZc09fn2Oas8oXGc Jj6/fOeK5UmSMPmf/jL+GD8BEj0k/Fn6IO4AAAAASUVORK5CYII= List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit On Mon, 8 Dec 2025 08:38:22 -0500 Michael Butler wrote: > On 12/7/25 20:11, Rozhuk Ivan wrote: > > On Sun, 7 Dec 2025 17:28:49 -0500 > > Michael Butler wrote: > > > >> Having two upstream providers, I'm trying to enforce symmetric > >> routing which, in OpenBSD's pf config can be implemented using .. > >> > >> # Inbound control-plane to the firewall itself (per-WAN reply-to > >> for symmetry) > >> pass in on $wan_a proto { tcp, udp, icmp } to ($wan_a) \ > >> reply-to ($wan_a $gw_a) keep state > >> pass in on $wan_b proto { tcp, udp, icmp } to ($wan_b) \ > >> reply-to ($wan_b $gw_b) keep state > >> > >> I've tried all manner of ipfw packet tagging in the hope that it > >> would yield similar results, e.g. > >> > >> setfib 1 ip from any to any recv tap0 > >> setfib 1 ip from any to any tagged 1 > >> count tag 1 ip from any to any recv tap0 > >> > >> [ .. ] > >> > >> check-state > >> allow ip from .. keep-state > >> deny log ip from any to any > >> > >> Is anyone else doing something like this on -current? > > > > > > Actually no, but: > > ifconfig vlan1001 172.16.0.31/24 fib 1 > > ifconfig vlan1002 172.16.0.32/24 fib 2 > > > > Do not forget set fib to network interface like it done in examlpe. > > In my case if same IP+mask set on more than one net if - only last > > one will process packets to sockets. > > Interface FIBs only work when the connection stays on the same > machine. > > In my case, I want to sustain the routing state for packets > traversing it. > > ISP-A -> Border-GW -> Mail-Server > ^ > ISP-B-----| > > Border-GW has multiple FIBs defined and sets the relevant FIB as > packets arrive over their respective interfaces. > > Destination address is the same (Mail-Server). > > When a connection is established, there is an IPFW state table entry > in the kernel on Border-GW and which contains the FIB in > ipfw_dyn_rule->id->fib > > What isn't happening is that replies (e.g. SYN-ACK) don't go out the > interface on which the SYN arrived despite having that info :-( > > Is this possible with IPFW? If not, will it work with PF on FreeBSD? > I did see some historical notes about 'reply-to' and don't know if > they're relevant, > In general, reply-to works with FreeBSD's pf. On 14.3 it also works across multiple hosts when using pfsync (usually in combination with carp). Michael -- Michael Gmelin