From nobody Mon Mar 13 13:58:58 2023 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PZytp6gTqz3xkxq for ; Mon, 13 Mar 2023 13:58:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PZytp3vVcz3vNh for ; Mon, 13 Mar 2023 13:58:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1678715938; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=t0LLBcZ0w7OQNePPMl02+Z3wuwdik6DGmGrX2W0LV20=; b=T7bz1/WRfNe7pd5ta93XoKA/lwYCwHUzG3btGICQyK/KvCT+ZSd0iJlfFBeDYiOOCZgea5 0dT7LOSveuyNAEdKFlJuf5JIgfaVuctKJc9ZJuM6g5JvdpscosMB/5wg93z150Q6FOfFQ8 iOyVGMXAI3TegofhgGdqyhj6QAznwgJ5EO742/zW1O9Of6Z5Ze9xEEHvfie29vKjvCKyLZ nVHXJQ5pFHc2W4PapjqOkbKa5n93Xi7ZNvzm8S5gRApEHTA/beAXYkVsjhx2K7UUt/docV YS5ws3ut3+L10uF2l9682JyueB3p2J4CGzN8fbUOHKlc7cv8l8cuWdH5EZ016w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1678715938; a=rsa-sha256; cv=none; b=ZJb4jUcnUoGnZ3bNnT+txmfiuADuud1a7KPCGIlMNxDYOagDVfeLwIHNzyEqDvlYi8PHRT YlRZlJh21u+Up4X+308+X5iQ1qH3TkVbF0t+HECYdWIzfn6kpgddMbuEEbe5kbdkRv2wwg 7RW1SC8hcFclmRDHefyjTcnHgw5dEoJKn6hlaYeqKNwjJ3iiyLItH0y/9K6e75hHU8MFzp t5CCU/P+tqYdTm7pxGwptcj+ImVd9zy4fz+VUrByDpdtYhMAx0mjCVh0rzl+MYy0o4I8gA 3ffa8M3vVTL+lzQ9KE+JGQM9LoFP+CyK27VyP5OaMQUvhIfbUxkwKzrWkIl/Kw== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4PZytp2rq4zw72 for ; Mon, 13 Mar 2023 13:58:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 32DDww0i038879 for ; Mon, 13 Mar 2023 13:58:58 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 32DDwwXE038878 for bugs@FreeBSD.org; Mon, 13 Mar 2023 13:58:58 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 270177] "ipfw fwd NEIGHBOUR" goes through default gateway. Date: Mon, 13 Mar 2023 13:58:58 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rx5670@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D270177 Bug ID: 270177 Summary: "ipfw fwd NEIGHBOUR" goes through default gateway. Product: Base System Version: 13.1-RELEASE Hardware: Any OS: Any Status: New Severity: Affects Only Me Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: rx5670@gmail.com Hello. I have a box (fbsd3) whith 2 internet connections: inet0 [1.1.1.253] and inet2 [2.2.2.253]. I installed 13.1-RELEASE and updated it to 13.1-p6. ##################################################################### When I ping this box's inet2 leg from outside it sends replies through its inet0 leg. >From my outside test box (deb2) I ping fbsd3's inet2 leg: root@deb2:~# ping -c 1 2.2.2.253 PING 2.2.2.253 (2.2.2.253) 56(84) bytes of data. 64 bytes from 2.2.2.253: icmp_seq=3D1 ttl=3D63 time=3D0.532 ms --- 2.2.2.253 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev =3D 0.532/0.532/0.532/0.000 ms root@deb2:~# tcpdump on fbsd3's inet2 shows this: root@fbsd3:~ # tcpdump -np -i inet2 16:07:20.224133 IP 3.3.3.254 > 2.2.2.253: ICMP echo request, id 16107, seq = 1, length 64 root@fbsd3:~ # at the same time tcpdump on fbsd3's inet0 shows this: root@fbsd3:~ # tcpdump -np -i inet0 16:07:20.224196 IP 2.2.2.253 > 3.3.3.254: ICMP echo reply, id 16107, seq 1, length 64 root@fbsd3:~ # ##################################################################### When initial tcp connection is established some packets not forwarded by "i= pfw fwd". On fbsd3 I use ncat to listen tcp 12345 port: root@fbsd3:~ # nc -kl 12345 and from my outside test box (deb2) I ncat fbsd3's inet2 leg: root@deb2:~# echo | nc 2.2.2.253 12345 root@deb2:~# tcpdump on fbsd3's inet2 shows this: root@fbsd3:~ # tcpdump -np -i inet2 16:24:45.215293 IP 3.3.3.254.59908 > 2.2.2.253.12345: Flags [S], seq 3626711860, win 64240, options [mss 1460,sackOK,TS val 3097103294 ecr 0,nop,wscale 7], length 0 16:24:45.215692 IP 3.3.3.254.59908 > 2.2.2.253.12345: Flags [.], ack 3088954691, win 502, options [nop,nop,TS val 3097103295 ecr 39579762], leng= th 0 16:24:45.215759 IP 3.3.3.254.59908 > 2.2.2.253.12345: Flags [P.], seq 0:1, = ack 1, win 502, options [nop,nop,TS val 3097103295 ecr 39579762], length 1 16:24:45.215806 IP 3.3.3.254.59908 > 2.2.2.253.12345: Flags [F.], seq 1, ac= k 1, win 502, options [nop,nop,TS val 3097103295 ecr 39579762], length 0 16:24:45.215826 IP 2.2.2.253.12345 > 3.3.3.254.59908: Flags [.], ack 2, win 1027, options [nop,nop,TS val 39579762 ecr 3097103295], length 0 16:24:45.215859 IP 2.2.2.253.12345 > 3.3.3.254.59908: Flags [F.], seq 1, ac= k 2, win 1027, options [nop,nop,TS val 39579762 ecr 3097103295], length 0 16:24:45.215966 IP 3.3.3.254.59908 > 2.2.2.253.12345: Flags [.], ack 2, win 502, options [nop,nop,TS val 3097103295 ecr 39579762], length 0 root@fbsd3:~ # at the same time tcpdump on fbsd3's inet0 shows this: root@fbsd3:~ # tcpdump -np -i inet0 16:24:45.215349 IP 2.2.2.253.12345 > 3.3.3.254.59908: Flags [S.], seq 3088954690, ack 3626711861, win 65535, options [mss 1460,nop,wscale 6,sackO= K,TS val 39579762 ecr 3097103294], length 0 root@fbsd3:~ # ##################################################################### "Ping" from 13.1-p6 not forwarded by "ipfw fwd". >From fbsd3 I ping my outside test box (deb2) and forwarding through inet2 d= oes not work: root@fbsd3:~ # root@fbsd3:~ # ping -c 1 3.3.3.254 PING 3.3.3.254 (3.3.3.254): 56 data bytes 64 bytes from 3.3.3.254: icmp_seq=3D0 ttl=3D63 time=3D0.393 ms --- 3.3.3.254 ping statistics --- 1 packets transmitted, 1 packets received, 0.0% packet loss round-trip min/avg/max/stddev =3D 0.393/0.393/0.393/0.000 ms root@fbsd3:~ # at the same time tcpdump on fbsd3's inet0 shows this: root@fbsd3:~ # tcpdump -np -i inet0 16:35:50.005649 IP 1.1.1.253 > 3.3.3.254: ICMP echo request, id 28962, seq = 0, length 64 16:35:50.005965 IP 3.3.3.254 > 1.1.1.253: ICMP echo reply, id 28962, seq 0, length 64 root@fbsd3:~ # ##################################################################### My configuration is: --------------------------------------------------------------------- My outside box (deb2) is Debian linux: root@deb2:~# ip a 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens161: mtu 1500 qdisc pfifo_fast stat= e UP group default qlen 1000 link/ether 00:50:56:92:eb:0d brd ff:ff:ff:ff:ff:ff altname enp4s0 inet 3.3.3.254/24 brd 3.3.3.255 scope global ens161 valid_lft forever preferred_lft forever inet6 fe80::250:56ff:fe92:eb0d/64 scope link valid_lft forever preferred_lft forever 3: ens192: mtu 1500 qdisc pfifo_fast stat= e UP group default qlen 1000 link/ether 00:50:56:92:d8:81 brd ff:ff:ff:ff:ff:ff altname enp11s0 inet 172.22.22.104/16 brd 172.22.255.255 scope global dynamic ens192 valid_lft 72827sec preferred_lft 72827sec inet6 fe80::250:56ff:fe92:d881/64 scope link valid_lft forever preferred_lft forever root@deb2:~# root@deb2:~# root@deb2:~# ip route show default via 172.22.1.254 dev ens192 1.1.1.0/24 via 3.3.3.3 dev ens161 2.2.2.0/24 via 3.3.3.3 dev ens161 3.3.3.0/24 dev ens161 proto kernel scope link src 3.3.3.254 172.22.0.0/16 dev ens192 proto kernel scope link src 172.22.22.104 --------------------------------------------------------------------- My intermediate box (deb1) is Debian linux: root@deb1:~# ip a 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens161: mtu 1500 qdisc pfifo_fast stat= e UP group default qlen 1000 link/ether 00:50:56:92:dd:1e brd ff:ff:ff:ff:ff:ff altname enp4s0 inet 3.3.3.3/24 brd 3.3.3.255 scope global ens161 valid_lft forever preferred_lft forever inet6 fe80::250:56ff:fe92:dd1e/64 scope link valid_lft forever preferred_lft forever 3: ens192: mtu 1500 qdisc pfifo_fast stat= e UP group default qlen 1000 link/ether 00:50:56:92:72:d5 brd ff:ff:ff:ff:ff:ff altname enp11s0 inet 172.22.22.102/16 brd 172.22.255.255 scope global dynamic ens192 valid_lft 53323sec preferred_lft 53323sec inet6 fe80::250:56ff:fe92:72d5/64 scope link valid_lft forever preferred_lft forever 4: ens224: mtu 1500 qdisc pfifo_fast stat= e UP group default qlen 1000 link/ether 00:50:56:92:ff:b3 brd ff:ff:ff:ff:ff:ff altname enp19s0 inet 2.2.2.2/24 brd 2.2.2.255 scope global ens224 valid_lft forever preferred_lft forever inet6 fe80::250:56ff:fe92:ffb3/64 scope link valid_lft forever preferred_lft forever 5: ens256: mtu 1500 qdisc pfifo_fast stat= e UP group default qlen 1000 link/ether 00:50:56:92:38:55 brd ff:ff:ff:ff:ff:ff altname enp27s0 inet 1.1.1.1/24 brd 1.1.1.255 scope global ens256 valid_lft forever preferred_lft forever inet6 fe80::250:56ff:fe92:3855/64 scope link valid_lft forever preferred_lft forever root@deb1:~# root@deb1:~# root@deb1:~# ip route sh default via 172.22.1.254 dev ens192 1.1.1.0/24 dev ens256 proto kernel scope link src 1.1.1.1 2.2.2.0/24 dev ens224 proto kernel scope link src 2.2.2.2 3.3.3.0/24 dev ens161 proto kernel scope link src 3.3.3.3 172.22.0.0/16 dev ens192 proto kernel scope link src 172.22.22.102 root@deb1:~# --------------------------------------------------------------------- My test freebsd box (fbsd3) root@fbsd3:~ # uname -a FreeBSD fbsd3 13.1-RELEASE-p6 FreeBSD 13.1-RELEASE-p6 GENERIC amd64 root@fbsd3:~ # root@fbsd3:~ # root@fbsd3:~ # cat /etc/rc.conf hostname=3D"fbsd3" # Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable dumpdev=3D"AUTO" # zfs_enable=3D"YES" clear_tmp_enable=3D"YES" sshd_enable=3D"YES" # firewall_enable=3D"YES" firewall_script=3D"/etc/rc.firewall.local" # gateway_enable=3D"YES" defaultrouter=3D"1.1.1.1" # ifconfig_em0_name=3Dlan0 ifconfig_vmx0_name=3Dinet0 ifconfig_vmx1_name=3Dinet2 # ifconfig_lan0=3D"inet 172.22.1.196 netmask 255.255.0.0" ifconfig_inet0=3D"inet 1.1.1.253 netmask 255.255.255.0" ifconfig_inet2=3D"inet 2.2.2.253 netmask 255.255.255.0" root@fbsd3:~ # root@fbsd3:~ # root@fbsd3:~ # ipfw -S list 00050 set 0 skipto 100 ip from any to any in // ## ALL_IN 00060 set 0 skipto 200 ip from any to any out // ## ALL_OUT 00110 set 0 skipto 300 ip from any to any in recv lan0 // ## IN_LAN0 00120 set 0 skipto 400 ip from any to any in recv inet0 // ## IN_INET0 00140 set 0 skipto 500 ip from any to any in recv inet2 // ## IN_INET2 00210 set 0 skipto 600 ip from any to any out xmit lan0 // ## OUT_LAN0 00220 set 0 skipto 700 ip from any to any out xmit inet0 // ## OUT_INET0 00230 set 0 skipto 800 ip from any to any out xmit inet2 // ## OUT_INET2 00310 set 0 allow ip from any to me keep-state :default 00390 set 0 deny ip from any to any 00410 set 0 allow ip from any to me keep-state :default 00490 set 0 deny ip from any to any 00510 set 0 allow ip from any to me keep-state :default 00590 set 0 deny ip from any to any 00610 set 0 allow ip from me to any keep-state :default 00690 set 0 deny ip from any to any 00700 set 0 fwd 2.2.2.2 ip from any to any 00710 set 0 allow ip from me to any keep-state :default 00790 set 0 deny ip from any to any 00810 set 0 allow ip from me to any keep-state :default 00900 set 0 deny ip from any to any 65535 set 31 deny ip from any to any root@fbsd3:~ # root@fbsd3:~ # root@fbsd3:~ # netstat -rn4 Routing tables Internet: Destination Gateway Flags Netif Expire default 1.1.1.1 UGS inet0 1.1.1.0/24 link#2 U inet0 1.1.1.253 link#2 UHS lo0 2.2.2.0/24 link#3 U inet2 2.2.2.253 link#3 UHS lo0 127.0.0.1 link#4 UH lo0 172.22.0.0/16 link#1 U em0 172.22.1.196 link#1 UHS lo0 root@fbsd3:~ # --=20 You are receiving this mail because: You are the assignee for the bug.=