From owner-freebsd-net@freebsd.org Fri Jan 29 15:15:52 2021 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 01E584EF0ED for ; Fri, 29 Jan 2021 15:15:52 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com [IPv6:2a00:1450:4864:20::530]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DS1BH1JKvz3Gkg for ; Fri, 29 Jan 2021 15:15:50 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: by mail-ed1-x530.google.com with SMTP id c2so10905765edr.11 for ; Fri, 29 Jan 2021 07:15:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tuxpowered-net.20150623.gappssmtp.com; s=20150623; h=to:from:subject:message-id:date:user-agent:mime-version; bh=ODHfhtzxB2kySmUY4ju+vIZofpzWTQHgx+MrzTbbMbw=; b=Dflx39hlifVKtXdHdGV93FPhpQg0YNTnsIly0YY3ZROc/1HgyO3K7bSPn0IdIi1kHv uz3SWV+RBuA2S3zXkgq6EUPIZCbhijuEKhwJduDI0mqp/hYNXO0KloNTNUdfVXNGDp/J F2HUetwIVOeQ7OR+38Jmg3IAtzLK4GbIAVY4JFApPTEOe196uyNZP+JZGXeEfp24cBHj Rtfojx6uUrQqxgHg/HlURVxGiGDMNVQTCw0FK3SyTstNGIslQZee0gRHYySio7+c4CGg kdhxQzrTyPb2HbBA6jc7yXCnn+LJi5E4s5hoUyruGxvMQtmcq+cxPns9nZYyRXdOa8k5 M+RQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version; bh=ODHfhtzxB2kySmUY4ju+vIZofpzWTQHgx+MrzTbbMbw=; b=nJ9cvZVSyOvQ+rW7N/qfyrY3Cpvv1IiuvOj1GJAOUO5uIyT6F2ehNZXlf5nS11/ET3 5UeR+p37/AZCk1we9V3QCkDSyisb3wblkCNsrPf4a+cOpoj7dZ6ewEkMbZaPrAyB8CrE wcanxm62c8l2NQ41ZKJwA0JEgmx3OEPLLYsYu5UcX//P5Yby5eq+2dr/JfE10UWHkazn SEkgEY3nlL51AxxN73OKyt2iRAaEw6pGjdis1NFzqdH1kDJjLKZx+NbrOofbrUkUmC6V 503V4Nw98xzsbqYSNKlnEm4OwqrgFTt21vRunX+OFMQTC/BqA7a7kgexNAvTojpdGups IIMg== X-Gm-Message-State: AOAM533T8Yf0YcPIVcYuhqHYIwNgocVblOtPcxv2wnbr3a5gGmJzlJ6z QjCNtoddn2M9Ax1AXWzeAH9mayscw+3ong== X-Google-Smtp-Source: ABdhPJwdJCKl2xr75ymIpHAreojh/eRx1TxwqJTRu3KubKVBCnC313D+W3SARsTm2ffnjpRRAoP0iQ== X-Received: by 2002:a05:6402:2694:: with SMTP id w20mr5661728edd.200.1611933348889; Fri, 29 Jan 2021 07:15:48 -0800 (PST) Received: from proton.tuxpowered.net ([95.81.15.36]) by smtp.gmail.com with ESMTPSA id r26sm5052020edc.95.2021.01.29.07.15.47 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 29 Jan 2021 07:15:48 -0800 (PST) To: freebsd-net@FreeBSD.org From: Kajetan Staszkiewicz Subject: How to not send traffic to TCP/IP stack Message-ID: Date: Fri, 29 Jan 2021 16:15:47 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="V1LVOj24wjUf6pEwspqkx46QZvp58I6wL" X-Rspamd-Queue-Id: 4DS1BH1JKvz3Gkg X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=tuxpowered-net.20150623.gappssmtp.com header.s=20150623 header.b=Dflx39hl; dmarc=none; spf=pass (mx1.freebsd.org: domain of vegeta@tuxpowered.net designates 2a00:1450:4864:20::530 as permitted sender) smtp.mailfrom=vegeta@tuxpowered.net X-Spamd-Result: default: False [-1.60 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[tuxpowered-net.20150623.gappssmtp.com:+]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::530:from]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[95.81.15.36:received]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[tuxpowered-net.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-net@freebsd.org]; DMARC_NA(0.00)[tuxpowered.net]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::530:from:127.0.2.255]; NEURAL_SPAM_SHORT(1.00)[1.000]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::530:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-net] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jan 2021 15:15:52 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --V1LVOj24wjUf6pEwspqkx46QZvp58I6wL Content-Type: multipart/mixed; boundary="fXZtMwzNkk58NaTHGcEsci053Y4Wi6UhF"; protected-headers="v1" From: Kajetan Staszkiewicz To: freebsd-net@FreeBSD.org Message-ID: Subject: How to not send traffic to TCP/IP stack --fXZtMwzNkk58NaTHGcEsci053Y4Wi6UhF Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: quoted-printable Hello group, On Linux traffic is directed to the IP stack by "local" entries in route table. By removing such entries, or rather not having them in the first place in additional routing tables it is possible to forward *all* traffic through a router, even if it is directed at router's own IP addresses. I have a situation where a FreeBSD pf-based Load Balancer must access a service which is hosted on itsef. For external clients this is trivial: SYN goes: 1. From client 2. Over BGP-managed network to the LB. The LB advertises public the public address over BGP to core routers. 3. It arrives at the public interface of LB where it is matched against a route-to pf rule. A public-side pf state is created, a tag is assigned.= 4. pf's rout-to routes it to a LB Node / target. 5. Leaves the LB over internal interface, matches the tag, another state is created. ACK: 1. From LB Node 2. Hits internal interface of LB, the state is already there. 3. Is routed to a default gateway learned from BGP. 4. Leaves via the public interface of LB, the state is already there. So far so good. But what if a LB wants to access the service? SYN: 1. LB sends out a packet through public interface becuase that's where the default gateway points. 2. Core router sends the packet to one of LBs, in this case the same one who originated the packet. 3. It arrives at the public interface of LB where it is matched against a route-to pf rule. A public-side pf state is created, a tag is assigned.= 4. pf's rout-to routes it to a LB Node / target. 5. Leaves the LB over internal interface, matches the tag, another state is created. ACK: 1. From LB Node 2. Hits internal interface of LB, the state is already there. 3. Normal routing decision of LB decides to send the packet to IP stack. 4. The packet never hits the pf state on the public side of LB. 5. The public side pf state never sees ACK from the LB Node, the state times out very fast. My goal is to have loadbalanced connections to *always* behave like they come from the Internet, that is to leave the LB and bounce off the core router. --=20 | pozdrawiam / greetings | Powered by macOS, Debian and FreeBSD | | Kajetan Staszkiewicz | www: http://vegeta.tuxpowered.net | `------------------------^--------------------------------------' --fXZtMwzNkk58NaTHGcEsci053Y4Wi6UhF-- --V1LVOj24wjUf6pEwspqkx46QZvp58I6wL Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wmMEABEIACMWIQSOEQZObv2B8mf0JbnjtFCvbXs6FAUCYBQmowUDAAAAAAAKCRDjtFCvbXs6FO75 AJ9Tcgxjgl1MVjBNNPV68kFr0hi2+gCdH8y9S4NxXcjrsZ4rzV7ldmMPxbQ= =FfAr -----END PGP SIGNATURE----- --V1LVOj24wjUf6pEwspqkx46QZvp58I6wL--