From nobody Mon Nov 15 07:58:06 2021 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id E4C0F186C60B; Mon, 15 Nov 2021 07:58:21 +0000 (UTC) (envelope-from eugene@zhegan.in) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [128.127.146.8]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail.norma.perm.ru", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Ht1lc5X8gz4gBl; Mon, 15 Nov 2021 07:58:20 +0000 (UTC) (envelope-from eugene@zhegan.in) Received: from [192.168.243.7] ([192.168.243.7]) by elf.hq.norma.perm.ru (8.16.1/8.15.2) with ESMTP id 1AF7tsjK084117; Mon, 15 Nov 2021 12:55:54 +0500 (+05) (envelope-from eugene@zhegan.in) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=vivat-retail.ru; s=key; t=1636962955; bh=1EI8EMkLYl0nAoMPQdaWbjAh4JZvkoGqp3eF/hdiWxQ=; h=Date:Subject:To:References:From:Cc:In-Reply-To; b=PGraU8u0olOzUbuxPlMSzVuNabmOeediTwtjafFXmbf/YbD8D0XV1ocnFa+oWjkty He4ru3KC2uvROVf5N67JrOz/IMVLsNjQwKGaEDW8De68THheaWBkHRYicCALjo2d93 6rdfOovJcbrYBBz05a/LvkE6c9PUuiaBCOlluqQI= Message-ID: Date: Mon, 15 Nov 2021 12:58:06 +0500 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.3.0 Subject: Re: packet loss between interfaces on the router Content-Language: ru To: stable@freebsd.org References: <216340c2-795d-d7ad-87d8-e07d9336564d@zhegan.in> From: "Eugene M. Zheganin" Cc: freebsd-pf@freebsd.org In-Reply-To: <216340c2-795d-d7ad-87d8-e07d9336564d@zhegan.in> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4Ht1lc5X8gz4gBl X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=fail ("headers rsa verify failed") header.d=vivat-retail.ru header.s=key header.b=PGraU8u0; dmarc=none; spf=pass (mx1.freebsd.org: domain of eugene@zhegan.in designates 128.127.146.8 as permitted sender) smtp.mailfrom=eugene@zhegan.in X-Spamd-Result: default: False [-0.30 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; R_DKIM_REJECT(1.00)[vivat-retail.ru:s=key]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[zhegan.in]; R_SPF_ALLOW(-0.20)[+a]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_SPAM_SHORT(1.00)[1.000]; DKIM_TRACE(0.00)[vivat-retail.ru:-]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:212494, ipnet:128.127.146.0/24, country:RU]; RCVD_COUNT_TWO(0.00)[2]; MID_RHS_MATCH_FROM(0.00)[] X-Spam: Yes X-ThisMailContainsUnwantedMimeParts: N Hello, 15.11.2021 2:14, Eugene M. Zheganin пишет: > [...] > The host is running PF as a packet filter, several dozens of rules. I > disable the scrub on outer interface (since the lost packet wasn'ta  > fragment, I was sceptical about it, and it doesn't help indeed). > [...] > ...and seems like it's a PF problem (so I probably should've started this conversation in pf@) Here's another stalled session with PF debug turned "loud". Below are caprtures on outer and inner interfaces, along with PF debug messages. What is the "3" condition ? I only managed to find that this is some sort of ackskew clashing. Could something be done here via pf configuration ? Outer interface: ===Cut=== [...] 12:16:37.537660 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15260617, win 1652, options [nop,nop,TS val 2720237530 ecr 2777961105], length 0 12:16:37.537670 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15263513, win 1652, options [nop,nop,TS val 2720237530 ecr 2777961105], length 0 12:16:37.537737 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15266409, win 1652, options [nop,nop,TS val 2720237531 ecr 2777961105], length 0 12:16:37.537824 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15276545, win 1652, options [nop,nop,TS val 2720237532 ecr 2777961105], length 0 12:16:37.537940 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15282337, win 1652, options [nop,nop,TS val 2720237532 ecr 2777961105], length 0 12:16:37.538261 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15324329:15325777, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237529], length 1448: HTTP 12:16:37.538272 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15325777:15327225, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237529], length 1448: HTTP 12:16:37.538278 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15327225:15328673, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237530], length 1448: HTTP 12:16:37.538289 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15328673:15330121, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237530], length 1448: HTTP 12:16:37.538323 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15330121:15331569, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237530], length 1448: HTTP 12:16:37.538335 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15331569:15333017, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237530], length 1448: HTTP 12:16:37.544805 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15286681, win 1652, options [nop,nop,TS val 2720237533 ecr 2777961105], length 0 12:16:37.545209 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15291025, win 1652, options [nop,nop,TS val 2720237534 ecr 2777961105], length 0 12:16:37.545278 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15296817, win 1652, options [nop,nop,TS val 2720237534 ecr 2777961105], length 0 12:16:37.545436 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15304057, win 1652, options [nop,nop,TS val 2720237535 ecr 2777961105], length 0 12:16:37.545985 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15308401, win 1652, options [nop,nop,TS val 2720237535 ecr 2777961105], length 0 12:16:37.546086 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15314193, win 1652, options [nop,nop,TS val 2720237536 ecr 2777961105], length 0 12:16:37.563993 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15319985, win 1652, options [nop,nop,TS val 2720237554 ecr 2777961125], length 0 12:16:37.564063 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15324329, win 1652, options [nop,nop,TS val 2720237555 ecr 2777961125], length 0 12:16:37.564445 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15327225, win 1652, options [nop,nop,TS val 2720237556 ecr 2777961135], length 0 12:16:37.564696 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15331569, win 1652, options [nop,nop,TS val 2720237557 ecr 2777961135], length 0 12:16:37.607557 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15333017, win 1652, options [nop,nop,TS val 2720237598 ecr 2777961135], length 0 12:16:37.768865 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777961365 ecr 2720237530], length 1448: HTTP 12:16:37.797072 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15333017, win 1652, options [nop,nop,TS val 2720237787 ecr 2777961135,nop,nop,sack 1 {15260617:15262065}], length 0 12:16:38.026968 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777961625 ecr 2720237530], length 1448: HTTP 12:16:38.044806 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15333017, win 1652, options [nop,nop,TS val 2720238045 ecr 2777961135,nop,nop,sack 1 {15260617:15262065}], length 0 12:16:38.349576 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777961945 ecr 2720237530], length 1448: HTTP 12:16:38.385464 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15333017, win 1652, options [nop,nop,TS val 2720238368 ecr 2777961135,nop,nop,sack 1 {15260617:15262065}], length 0 12:16:38.798092 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777962395 ecr 2720237530], length 1448: HTTP 12:16:38.826845 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15333017, win 1652, options [nop,nop,TS val 2720238816 ecr 2777961135,nop,nop,sack 1 {15260617:15262065}], length 0 12:16:39.477080 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777963075 ecr 2720237530], length 1448: HTTP 12:16:39.510688 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15333017, win 1652, options [nop,nop,TS val 2720239497 ecr 2777961135,nop,nop,sack 1 {15260617:15262065}], length 0 12:16:40.643251 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777964235 ecr 2720237530], length 1448: HTTP 12:16:40.673700 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15333017, win 1652, options [nop,nop,TS val 2720240661 ecr 2777961135,nop,nop,sack 1 {15260617:15262065}], length 0 12:16:42.759423 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777966355 ecr 2720237530], length 1448: HTTP 12:16:42.790404 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15333017, win 1652, options [nop,nop,TS val 2720242777 ecr 2777961135,nop,nop,sack 1 {15260617:15262065}], length 0 12:16:46.809355 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777970405 ecr 2720237530], length 1448: HTTP 12:16:46.840079 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15333017, win 1652, options [nop,nop,TS val 2720246827 ecr 2777961135,nop,nop,sack 1 {15260617:15262065}], length 0 12:16:54.692330 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777978285 ecr 2720237530], length 1448: HTTP ^C ===Cut=== Inner interface: ===Cut=== [...] 12:16:37.537666 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15260617, win 1652, options [nop,nop,TS val 2720237530 ecr 2777961105], length 0 12:16:37.538251 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15324329:15325777, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237529], length 1448: HTTP 12:16:37.538266 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15325777:15327225, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237529], length 1448: HTTP 12:16:37.538274 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15327225:15328673, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237530], length 1448: HTTP 12:16:37.538282 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15328673:15330121, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237530], length 1448: HTTP 12:16:37.538312 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15330121:15331569, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237530], length 1448: HTTP 12:16:37.538328 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15331569:15333017, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237530], length 1448: HTTP 12:16:37.768852 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777961365 ecr 2720237530], length 1448: HTTP 12:16:38.026950 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777961625 ecr 2720237530], length 1448: HTTP 12:16:38.349553 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777961945 ecr 2720237530], length 1448: HTTP 12:16:38.798075 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777962395 ecr 2720237530], length 1448: HTTP 12:16:39.477063 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777963075 ecr 2720237530], length 1448: HTTP 12:16:40.643234 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777964235 ecr 2720237530], length 1448: HTTP 12:16:42.759404 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777966355 ecr 2720237530], length 1448: HTTP 12:16:46.809338 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777970405 ecr 2720237530], length 1448: HTTP 12:16:54.692313 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777978285 ecr 2720237530], length 1448: HTTP ^C ===Cut=== PF debug messages for this session: ===Cut=== # grep -A1 62.109.28.82:52982 messages Nov 15 12:16:25 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=782592198 high=782801846 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=782670390 len=0 ackskew=-78192 pkts=394:1710 dir=in,fwd Nov 15 12:16:25 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:25 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=782592198 high=782801846 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=782674734 len=0 ackskew=-82536 pkts=394:1710 dir=in,fwd Nov 15 12:16:25 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:25 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=782592198 high=782801846 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=782684870 len=0 ackskew=-92672 pkts=394:1710 dir=in,fwd Nov 15 12:16:25 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:25 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=782592198 high=782801846 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=782690662 len=0 ackskew=-98464 pkts=394:1710 dir=in,fwd Nov 15 12:16:25 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:25 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=782592198 high=782801846 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=782692110 len=0 ackskew=-99912 pkts=394:1710 dir=in,fwd Nov 15 12:16:25 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:32 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=786691630 high=786937614 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=786766926 len=0 ackskew=-75296 pkts=1069:4605 dir=in,fwd Nov 15 12:16:32 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:32 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=786691630 high=786937614 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=786788646 len=0 ackskew=-97016 pkts=1072:4636 dir=in,fwd Nov 15 12:16:32 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:32 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=786691630 high=786937614 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=786792990 len=0 ackskew=-101360 pkts=1072:4636 dir=in,fwd Nov 15 12:16:32 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:32 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=786691630 high=786937614 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=786804574 len=0 ackskew=-112944 pkts=1072:4636 dir=in,fwd Nov 15 12:16:32 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:32 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=786691630 high=786937614 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=786810366 len=0 ackskew=-118736 pkts=1072:4636 dir=in,fwd Nov 15 12:16:32 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:32 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=786691630 high=786937614 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=786829190 len=0 ackskew=-137560 pkts=1072:4636 dir=in,fwd Nov 15 12:16:32 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:34 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1494 modulator=0 wscale=7] [lo=789374774 high=789626822 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=789444278 len=0 ackskew=-69504 pkts=1494:6459 dir=in,fwd Nov 15 12:16:34 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795483886 len=0 ackskew=-68056 pkts=2452:10662 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795486782 len=0 ackskew=-70952 pkts=2452:10662 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4pf: state reuse TCP outA wire:  seq=1741467272 (1741467272) ack=795496918 len=0 ackskew=-81088 pkts=2452:10662 dir=in,fwd Nov 15 12:16:37 gw0 kernel: 192.168.0.247pf: State failure on: 3   | -- Nov 15 12:16:37 gw0 kernel:  wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795502710 len=0 ackskew=-86880 pkts=2452:10662 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795507054 len=0 ackskew=-91224 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795511398 len=0 ackskew=-95568 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795517190 len=0 ackskew=-101360 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795524430 len=0 ackskew=-108600 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795528774 len=0 ackskew=-112944 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795534566 len=0 ackskew=-118736 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795540358 len=0 ackskew=-124528 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795544702 len=0 ackskew=-128872 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795547598 len=0 ackskew=-131768 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795551942 len=0 ackskew=-136112 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-137560 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10669 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:38 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10670 dir=in,fwd Nov 15 12:16:38 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:38 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10671 dir=in,fwd Nov 15 12:16:38 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:39 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10672 dir=in,fwd Nov 15 12:16:39 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:39 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10673 dir=in,fwd Nov 15 12:16:39 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:40 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10674 dir=in,fwd Nov 15 12:16:40 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:42 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10675 dir=in,fwd Nov 15 12:16:42 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:46 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10676 dir=in,fwd Nov 15 12:16:46 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:54 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10677 dir=in,fwd Nov 15 12:16:54 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:17:09 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 FA seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10677 dir=in,fwd Nov 15 12:17:09 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:17:09 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 FA seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10677 dir=in,fwd Nov 15 12:17:09 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:17:09 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 FA seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10677 dir=in,fwd Nov 15 12:17:09 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:17:10 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 FA seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10677 dir=in,fwd Nov 15 12:17:10 gw0 kernel: pf: State failure on:     3   | Nov 15 12:17:10 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467273) ack=795553390 len=0 ackskew=-70952 pkts=2452:10678 dir=in,fwd Nov 15 12:17:10 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:17:11 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 FA seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10678 dir=in,fwd Nov 15 12:17:11 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:17:13 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 FA seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10678 dir=in,fwd Nov 15 12:17:13 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:17:16 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 FA seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10678 dir=in,fwd Nov 15 12:17:16 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:17:24 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 FA seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10678 dir=in,fwd Nov 15 12:17:24 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:17:25 gw0 kernel: pf: BAD state: pf: OK ICMP 5:1 TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6192.168.0.248 -> 192.168.0.230 state: TCP out wire: 192.168.0.230:3340 192.168.11.4] 4:4 A seq=1741467272 (1741467273) ack=795553390 len=0 ackskew=-70952 pkts=2452:10679 dir=in,fwd Nov 15 12:17:25 gw0 kernel: :59016pf: State failure on:     3   | -- Nov 15 12:17:39 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 FA seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10679 dir=in,fwd Nov 15 12:17:39 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:17:41 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467273) ack=795553390 len=0 ackskew=-70952 pkts=2452:10680 dir=in,fwd Nov 15 12:17:41 gw0 kernel: pf: State failure on:     3   | ===Cut=== Eugene. From nobody Mon Nov 15 08:06:11 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id C04BF184B1DA for ; Mon, 15 Nov 2021 08:08:17 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from smtprelay08.ispgateway.de (smtprelay08.ispgateway.de [134.119.228.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Ht1z4627Gz4lh9 for ; Mon, 15 Nov 2021 08:08:16 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from [91.20.68.89] (helo=fabiankeil.de) by smtprelay08.ispgateway.de with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1mmX28-0002rp-PX for freebsd-stable@freebsd.org; Mon, 15 Nov 2021 09:08:08 +0100 Date: Mon, 15 Nov 2021 09:06:11 +0100 From: Fabian Keil To: freebsd-stable@freebsd.org Subject: Re: stable/12: jail(2) failures after ca9ab8ea1774 Message-ID: <20211115085227.56bd2255@fabiankeil.de> In-Reply-To: <8ef505b5-2d3b-8f94-7de1-fa9eaa1f2e2f@bruelltuete.com> References: <20211110074613.6b81f85a@fabiankeil.de> <8ef505b5-2d3b-8f94-7de1-fa9eaa1f2e2f@bruelltuete.com> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; boundary="Sig_/gI62DcKeT=JdXkCo+8aZm60"; protocol="application/pgp-signature"; micalg=pgp-sha1 X-Df-Sender: Nzc1MDY3 X-Rspamd-Queue-Id: 4Ht1z4627Gz4lh9 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of freebsd-listen@fabiankeil.de has no SPF policy when checking 134.119.228.98) smtp.mailfrom=freebsd-listen@fabiankeil.de X-Spamd-Result: default: False [-0.73 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(1.00)[1.000]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[fabiankeil.de]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; RECEIVED_SPAMHAUS_PBL(0.00)[91.20.68.89:received]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_SPAM_LONG(0.47)[0.467]; RCVD_IN_DNSWL_NONE(0.00)[134.119.228.98:from]; SIGNED_PGP(-2.00)[]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:34011, ipnet:134.119.228.0/24, country:DE]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[134.119.228.98:from] X-ThisMailContainsUnwantedMimeParts: N --Sig_/gI62DcKeT=JdXkCo+8aZm60 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Johannes Totz via freebsd-stable wrote on 2021= -11-11: > On 10/11/2021 06:46, Fabian Keil wrote: > > ElectroBSD's ggate[cd]-related patches are available at: > > > > They should apply cleanly on stable/12 e644c87aa. >=20 > Lots of great stuff in that patch file! Thanks for sharing! You're welcome. As it turned out, jail(2) works as advertised when the forked process closes the pidfile which it doesn't need. I've updated the patch set: > Which ones of those should we back-port to fbsd? Not my call. The security fixes should be uncontroversial so I'd recommend looking at them first. > I've got patches open on ggate too, see > https://reviews.freebsd.org/D31722 > https://reviews.freebsd.org/D31727 > https://reviews.freebsd.org/D31709 Interesting. Thanks for the links. Fabian --Sig_/gI62DcKeT=JdXkCo+8aZm60 Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iF0EARECAB0WIQTKUNd6H/m3+ByGULIFiohV/3dUnQUCYZIU8wAKCRAFiohV/3dU nYJNAJ91Zt/wF/MfLn/XKUDj6qj0LXne9ACfRUQQa/yKpv/P8rxDgDr6WAtnmWk= =X7cP -----END PGP SIGNATURE----- --Sig_/gI62DcKeT=JdXkCo+8aZm60--