From owner-freebsd-net@freebsd.org Sun Jul 14 12:57:46 2019 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A517815E1DA5 for ; Sun, 14 Jul 2019 12:57:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 236C86FF3F for ; Sun, 14 Jul 2019 12:57:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id D14DB15E1DA2; Sun, 14 Jul 2019 12:57:45 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 900AE15E1DA1 for ; Sun, 14 Jul 2019 12:57:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 315CE6FF38 for ; Sun, 14 Jul 2019 12:57:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id ED8735E0C for ; Sun, 14 Jul 2019 12:57:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6ECvi1P048951 for ; Sun, 14 Jul 2019 12:57:44 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6ECvivc048950 for net@FreeBSD.org; Sun, 14 Jul 2019 12:57:44 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 219428] em network driver broken in current Date: Sun, 14 Jul 2019 12:57:36 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: arkadiusz.majewski@iptrace.pl X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 315CE6FF38 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.980,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Jul 2019 12:57:47 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D219428 --- Comment #24 from IPTRACE --- My sysctl.conf file. net.inet.tcp.blackhole=3D2 net.inet.udp.blackhole=3D1 net.inet.icmp.log_redirect=3D1 net.inet.icmp.drop_redirect=3D1 net.inet.ip.random_id=3D1 net.link.tap.up_on_open=3D1 net.inet.tcp.mssdflt=3D1440 net.inet.tcp.nolocaltimewait=3D1 net.inet.ip.check_interface=3D1 net.inet.ip.redirect=3D0 net.inet.tcp.drop_synfin=3D1 net.inet.tcp.msl=3D15000 net.inet.tcp.icmp_may_rst=3D0 net.inet.tcp.path_mtu_discovery=3D0 net.inet6.icmp6.rediraccept=3D0 net.inet6.ip6.redirect=3D0 kern.ipc.maxsockbuf=3D16777216 net.inet.tcp.sendspace=3D1048576 net.inet.tcp.recvspace=3D1048576 net.inet.tcp.sendbuf_max=3D16777216 net.inet.tcp.recvbuf_max=3D16777216 net.inet.tcp.sendbuf_inc=3D524288 net.inet.tcp.recvbuf_inc=3D524288 net.inet.tcp.cc.algorithm=3Dcubic net.inet.tcp.tso=3D0 net.inet.tcp.rexmit_slop=3D50 net.inet.tcp.msl=3D5000 net.inet.tcp.keepinit=3D5000 net.inet.tcp.finwait2_timeout=3D5000 net.inet.tcp.fast_finwait2_recycle=3D1 net.inet.tcp.always_keepalive=3D0 net.route.netisr_maxqlen=3D2048 net.inet.ip.process_options=3D0 net.inet.sctp.blackhole=3D2 net.inet.tcp.abc_l_var=3D44 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Sun Jul 14 18:09:14 2019 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C7041A1B74 for ; Sun, 14 Jul 2019 18:09:14 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from shell1.rawbw.com (shell1.rawbw.com [198.144.192.42]) by mx1.freebsd.org (Postfix) with ESMTP id 341ED776CD for ; Sun, 14 Jul 2019 16:58:00 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from yv.noip.me (c-67-180-169-236.hsd1.ca.comcast.net [67.180.169.236]) (authenticated bits=0) by shell1.rawbw.com (8.15.1/8.15.1) with ESMTPSA id x6EGvwgv005773 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Sun, 14 Jul 2019 09:57:58 -0700 (PDT) (envelope-from yuri@rawbw.com) X-Authentication-Warning: shell1.rawbw.com: Host c-67-180-169-236.hsd1.ca.comcast.net [67.180.169.236] claimed to be yv.noip.me Subject: Re: How to set up ipfw(8) NAT between an alias and the main IP address, when the alias is in another network? To: Michael Sierchio , "freebsd-net@freebsd.org" References: <8e388abc-f2ac-b070-cf86-a4d3971ac095@rawbw.com> From: Yuri Message-ID: Date: Sun, 14 Jul 2019 09:57:57 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 341ED776CD X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of yuri@rawbw.com designates 198.144.192.42 as permitted sender) smtp.mailfrom=yuri@rawbw.com X-Spamd-Result: default: False [-4.66 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:198.144.192.32/27]; HAS_XAW(0.00)[]; MX_GOOD(-0.01)[mx.rawbw.net]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.87)[-0.868,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:7961, ipnet:198.144.192.0/20, country:US]; SUBJECT_ENDS_QUESTION(1.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[236.169.180.67.zen.spamhaus.org : 127.0.0.10]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[rawbw.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[42.192.144.198.list.dnswl.org : 127.0.10.0]; IP_SCORE(-2.58)[ip: (-5.89), ipnet: 198.144.192.0/20(-3.26), asn: 7961(-3.71), country: US(-0.06)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Jul 2019 18:09:17 -0000 On 2019-07-08 10:17, Michael Sierchio wrote: > NAT is already maintaining state – it is possible to combine stateful rules > and NAT, but don't. ;-) > > Are you really proposing to NAT twice, or is 192.168.1.2 a phony address > for the purposes of discussion here? > > In any case, consider something like the following: > > #!/bin/sh > > fw="/sbin/ipfw -q" > sysctl net.inet.ip.fw.one_pass=0 > > IP_JAIL="192.168.100.2" > IP_EXIF="192.168.1.2" > > OIF="sk0" > > ################################################################################ > # If 192.168.1.2 is really your interface address, you'll be nat'ing twice > # on the way to the internet, which is ugly. You don't need the *unreg_only* > # directive if you only have RFC1918 addresses anyway. You should clarify > # if this is the case. *reset* kills all active nat sessions if you run this > # script again. > > $fw flush > > $fw nat 1 config \ > redirect_addr ${IP_EXIF} ${IP_JAIL} \ > redirect_addr ${IP_JAIL} ${IP_EXIF} \ > if ${OIF} unreg_only reset > > ################################################################################ > # separate in and out as a matter of habit - don't mention protocol in nat > # statement, do this in subsequent rules > > $fw add 01000 nat 1 ip from any to any in recv ${OIF} > > # check-state isn't needed, really, since it gets performed at the next > # rule that mentions state > > #$#$#$#$# $fw add 01500 check-state > > # these will match traffic to/from external IP and not jail > > $fw add 02000 allow tcp from ${IP_EXIF} to any out setup keep-state > $fw add 02010 allow udp from ${IP_EXIF} to any out keep-state > $fw add 02020 allow icmp from ${IP_EXIF} to any out keep-state > > ################################################################################ > # Why is this safe? because it will only match NAT return packets. > # It only permits traffic to your jail in this case. Also, for TCP to > # function properly, to need to accept ICMP error messages, esp. need-frag > > $fw add 02000 allow ip from any to ${IP_JAIL} > > ################################################################################ > # outbound packets pass through nat > > $fw add 03000 nat 1 ip from any to any out xmit ${OIF} > > ################################################################################ > # if your default rule (65535) is DENY, you need something like this. This > will > # match only NAT'd traffic > > $fw add 50000 allow ip from any to any out xmit ${OIF} Thanks for this script, Michael. I tried it on both my host system and in the virtual machine, with a different set of addresses, but it blocks outgoing connections from the original system, while allowing connections from jail. I think that the reason is that NAT translates the return packets that are destined for the original system, as if they were going into the jail. I'm also not looking for a script altering the whole firewall table, I only need to add particular rules for the jail without changing existing rules. The precise definition of what I need is this: jail IP is in a different net. All outgoing traffic from jail has to be natted when its destination is outside. Return traffic should be natted only when it corresponds to the sent traffic from jail. Host connections should not be affected by NAT. I think that this is possible, because firewalls are generally able to match incoming packets to the outgoing connections. Yuri From owner-freebsd-net@freebsd.org Sun Jul 14 19:25:53 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CD87BA3FBF for ; Sun, 14 Jul 2019 19:25:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id B01FF88BED for ; Sun, 14 Jul 2019 19:25:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id AFAE5A3FBE; Sun, 14 Jul 2019 19:25:53 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AF71EA3FBD for ; Sun, 14 Jul 2019 19:25:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 929DF88BEC for ; Sun, 14 Jul 2019 19:25:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 750ACA215 for ; Sun, 14 Jul 2019 19:25:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6EJPrTq068064 for ; Sun, 14 Jul 2019 19:25:53 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6EJPrj7068063 for net@FreeBSD.org; Sun, 14 Jul 2019 19:25:53 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 234026] [panic] [dummynet] Repeatable panic in dummynet due to locking issues and use-after-free Date: Sun, 14 Jul 2019 19:25:53 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-STABLE X-Bugzilla-Keywords: crash X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: noresponse@yandex.ru X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: B01FF88BED X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.984,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Jul 2019 19:25:53 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D234026 --- Comment #3 from Stanislav Trofimov --- (In reply to Stanislav Trofimov from comment #1) Latest p6 update fixed my problem --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Sun Jul 14 21:00:34 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 18E59A65B3 for ; Sun, 14 Jul 2019 21:00:34 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id D4CF68C6F4 for ; Sun, 14 Jul 2019 21:00:33 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: by mailman.nyi.freebsd.org (Postfix) id C1927A659E; Sun, 14 Jul 2019 21:00:33 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B0779A6598 for ; Sun, 14 Jul 2019 21:00:33 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5841A8C6DC for ; Sun, 14 Jul 2019 21:00:33 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1EA05B241 for ; Sun, 14 Jul 2019 21:00:33 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6EL0WH5072785 for ; Sun, 14 Jul 2019 21:00:32 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6EL0WHv072778 for net@FreeBSD.org; Sun, 14 Jul 2019 21:00:32 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201907142100.x6EL0WHv072778@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: net@FreeBSD.org Subject: Problem reports for net@FreeBSD.org that need special attention Date: Sun, 14 Jul 2019 21:00:32 +0000 MIME-Version: 1.0 X-Rspamd-Queue-Id: 5841A8C6DC X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.99 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.99)[-0.988,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Jul 2019 21:00:34 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- In Progress | 221146 | [ixgbe] Problem with second laggport In Progress | 235700 | oce(4) driver causes fatal trap 12 on boot with e New | 204438 | setsockopt() handling of kern.ipc.maxsockbuf limi New | 205592 | TCP processing in IPSec causes kernel panic New | 213410 | [carp] service netif restart causes hang only whe Open | 193452 | Dell PowerEdge 210 II -- Kernel panic bce (broadc Open | 194485 | Userland cannot add IPv6 prefix routes Open | 200319 | Bridge+CARP crashes/freezes Open | 202510 | [CARP] advertisements sourced from CARP IP cause Open | 222273 | igb(4): Kernel panic (fatal trap 12) due to netwo Open | 225438 | panic in6_unlink_ifa() due to race Open | 227720 | Kernel panic in ppp server Open | 233952 | jme NICs non functional after 11.2 to 12.0 upgrad Open | 236888 | ppp daemon: Allow MTU to be overridden for PPPoE Open | 236983 | bnxt(4) VLAN not operational unless explicit "ifc Open | 237072 | netgraph(4): performance issue [on HardenedBSD]? Open | 237391 | route get returns no result for network addresses Open | 237840 | Removed dummynet dependency on ipfw 18 problems total for which you should take action. From owner-freebsd-net@freebsd.org Mon Jul 15 08:42:31 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B6901B4193 for ; Mon, 15 Jul 2019 08:42:31 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8E76E858B2; Mon, 15 Jul 2019 08:42:31 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from reviews.nyi.freebsd.org (reviews.nyi.freebsd.org [IPv6:2610:1c1:1:607c::16:b]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id D398E1B16A; Mon, 15 Jul 2019 08:42:30 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by reviews.nyi.freebsd.org (Postfix, from userid 1346) id D2BEED4D62; Mon, 15 Jul 2019 08:42:30 +0000 (UTC) Date: Mon, 15 Jul 2019 08:42:30 +0000 To: Phabricator From: "aleksandr.fedorov_itglobal.com (Aleksandr Fedorov)" Cc: freebsd-net@freebsd.org Reply-to: "aleksandr.fedorov_itglobal.com (Aleksandr Fedorov)" Subject: [Differential] D19422: if_vxlan(4) Allow set MTU more than 1500 bytes. Message-ID: <3abfc4f42b2894e21966a73e96dc51fe@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , X-Herald-Rules: <81>, <110> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk Thread-Topic: PHID-DREV-bvtxxu4jwhzdkwqxxgd7 X-Phabricator-Mail-ID: 1538931 X-Phabricator-Send-Attempt: oks5uaaq2rtseie5 In-Reply-To: References: Thread-Index: MzYyMWIxNGMwOTE4YmQzMWVhNTU2NTFhMGQ2IF0sPHY= X-Phabricator-Stamps: actor(@aleksandr.fedorov_itglobal.com) application(Differential) author(@aleksandr.fedorov_itglobal.com) herald(H81) herald(H110) monogram(D19422) object-type(DREV) phid(PHID-DREV-bvtxxu4jwhzdkwqxxgd7) reviewer(#network) reviewer(@bryanv) reviewer(@hrs) reviewer(@jhb) reviewer(@krion) reviewer(@rgrimes) revision-status(needs-review) subscriber(@ae) subscriber(@evgueni.gavrilov_itglobal.com) subscriber(@freebsd-net-list) subscriber(@olevole_olevole.ru) via(web) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_3abfc4f42b2894e21966a73e96dc51fe" X-Rspamd-Queue-Id: 8E76E858B2 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.94 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.94)[-0.942,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jul 2019 08:42:31 -0000 --b1_3abfc4f42b2894e21966a73e96dc51fe Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: base64 YWxla3NhbmRyLmZlZG9yb3ZfaXRnbG9iYWwuY29tIHVwZGF0ZWQgdGhpcyByZXZpc2lvbiB0byBE aWZmIDU5NzUzLgphbGVrc2FuZHIuZmVkb3Jvdl9pdGdsb2JhbC5jb20gYWRkZWQgYSBjb21tZW50 LgoKCiAgVlhMQU4gZW5jYXBzdWxhdGUgZXRoZXJuZXQgZnJhbWVzIHdpdGhpbiBVRFAvSVAgcGFj a2V0cy4gU28sIHdlIGNhbiBjYWxjdWxhdGUgbWF4aW11bSBvdmVyaGVhZCBmb3IgSVB2NDoKICAK ICAtIElQX01BWFBBQ0tFVCA9IDY1SyAtIGNvbnN0YW50IGZyb20gbmV0aW5ldC9pcC5oLgogIC0g TWF4aW11bSBJUCBoZWFkZXIgbGVuZ3RoIElQX01BWF9IRFJfTEVOIChUaGVyZSBpcyBubyBzdWl0 YWJsZSBjb25zdGFudCBmb3IgaXQuKSA9IDYwIGFzIHRoZSBJbnRlcm5ldCBIZWFkZXIgTGVuZ3Ro IGZpZWxkIGlzIHRoZSB1bnNpZ25lZCA0LWJpdCBudW1iZXIgb2YgMzItYml0IHdvcmRzIC0gMTUg KiAzMiA9IDQ4MCBiaXQgPSA2MCBieXRlcy4KICAtIHNpemVvZihzdHJ1Y3QgdWRwaGRyKSA9IDgg Ynl0ZXMuCiAgLSBzaXplb2Yoc3RydWN0IHZ4bGFuX2hlYWRlcikgPSA4IGJ5dGVzLgogIC0gSW5u ZXIgZnJhbWUgRVRIRVJfSERSX0xFTiA9IDE0IGJ5dGVzLgogIC0gSW5uZXIgZnJhbWUgRVRIRVJf Q1JDX0xFTiA9IDQgYnl0ZXMuCiAgLSBJbm5lciBmcmFtZSBFVEhFUl9WTEFOX0VOQ0FQX0xFTiA9 IDQgYnl0ZXMuCiAgCiAgVGhlIHJlc3VsdCBWWExBTl9NQVhfTVRVID0gSVBfTUFYUEFDS0VUIC0g SVBfTUFYX0hEUl9MRU4gLSBzaXplb2Yoc3RydWN0IHVkcGhkcikgLSBzaXplb2Yoc3RydWN0IHZ4 bGFuX2hlYWRlcikgLSBFVEhFUl9IRFJfTEVOIC0gRVRIRVJfQ1JDX0xFTiAtIEVUSEVSX1ZMQU5f RU5DQVBfTEVOID0gNjU1MzUgLSA2MCAtIDggLSA4IC0gMTQgLSA0IC0gNCA9IDY1NDM3CiAgU28g b3ZlcmhlYWQgaXMgOTggYnl0ZXMuCiAgCiAgVW5mb3J0dW5hdGVseSBmb3IgSVB2NiwgdGhlIG1h eGltdW0gaGVhZGVyIGxlbmd0aCBpcyBub3QgZml4ZWQgYW5kIGNhbiBiZSBleHRlbmRlZCBpbiB0 aGUgZnV0dXJlLiBJdCBpcyB1c3VhbGx5IDQwIGJ5dGVzLgogIAogIFJldmlzaW9uIGNoYW5nZXM6 CiAgCiAgLSBDYWxjdWxhdGUgVlhMQU5fTUFYX01UVSB1c2luZyBleGlzdGluZyBjb25zdGFudHMu CgpDSEFOR0VTIFNJTkNFIExBU1QgVVBEQVRFCiAgaHR0cHM6Ly9yZXZpZXdzLmZyZWVic2Qub3Jn L0QxOTQyMj92cz01OTY4NCZpZD01OTc1MwoKQ0hBTkdFUyBTSU5DRSBMQVNUIEFDVElPTgogIGh0 dHBzOi8vcmV2aWV3cy5mcmVlYnNkLm9yZy9EMTk0MjIvbmV3LwoKUkVWSVNJT04gREVUQUlMCiAg aHR0cHM6Ly9yZXZpZXdzLmZyZWVic2Qub3JnL0QxOTQyMgoKQUZGRUNURUQgRklMRVMKICBzeXMv bmV0L2lmX3Z4bGFuLmMKCkVNQUlMIFBSRUZFUkVOQ0VTCiAgaHR0cHM6Ly9yZXZpZXdzLmZyZWVi c2Qub3JnL3NldHRpbmdzL3BhbmVsL2VtYWlscHJlZmVyZW5jZXMvCgpUbzogYWxla3NhbmRyLmZl ZG9yb3ZfaXRnbG9iYWwuY29tLCBicnlhbnYsIGhycywgI25ldHdvcmssIHJncmltZXMsIGtyaW9u LCBqaGIKQ2M6IGV2Z3VlbmkuZ2F2cmlsb3ZfaXRnbG9iYWwuY29tLCBvbGV2b2xlX29sZXZvbGUu cnUsIGFlLCBmcmVlYnNkLW5ldC1saXN0LCBrcnp5c3p0b2YuZ2FsYXprYV9pbnRlbC5jb20K --b1_3abfc4f42b2894e21966a73e96dc51fe Content-Type: text/x-patch; charset=utf-8; name="D19422.59753.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D19422.59753.patch" ZGlmZiAtLWdpdCBhL3N5cy9uZXQvaWZfdnhsYW4uYyBiL3N5cy9uZXQvaWZfdnhsYW4uYwotLS0g YS9zeXMvbmV0L2lmX3Z4bGFuLmMKKysrIGIvc3lzL25ldC9pZl92eGxhbi5jCkBAIC04NCw2ICs4 NCwxNSBAQAogCWludAkJCQkgdnhsc29tY191c2VyczsKIH07CiAKKy8qCisgKiBUaGUgbWF4aW11 bSBNVFUgb2YgZW5jYXBzdWxhdGVkIGV0aGVybmV0IGZyYW1lIHdpdGhpbiBJUHY0L1VEUCBwYWNr ZXQuCisgKi8KKyNkZWZpbmUgVlhMQU5fTUFYX01UVQkoSVBfTUFYUEFDS0VUIC0gXAorCQk2MCAv KiBNYXhpbXVtIElQdjQgaGVhZGVyIGxlbiAqLyAtIFwKKwkJc2l6ZW9mKHN0cnVjdCB1ZHBoZHIp IC0gXAorCQlzaXplb2Yoc3RydWN0IHZ4bGFuX2hlYWRlcikgLSBcCisJCUVUSEVSX0hEUl9MRU4g LSBFVEhFUl9DUkNfTEVOIC0gRVRIRVJfVkxBTl9FTkNBUF9MRU4pCisKICNkZWZpbmUgVlhMQU5f U09fTUNfTUFYX0dST1VQUwkJMzIKIAogI2RlZmluZSBWWExBTl9TT19WTklfSEFTSF9TSElGVAkJ NgpAQCAtMjI0NywxMCArMjI1NiwxMSBAQAogCWlmciA9IChzdHJ1Y3QgaWZyZXEgKikgZGF0YTsK IAlpZmQgPSAoc3RydWN0IGlmZHJ2ICopIGRhdGE7CiAKKwllcnJvciA9IDA7CisKIAlzd2l0Y2gg KGNtZCkgewogCWNhc2UgU0lPQ0FERE1VTFRJOgogCWNhc2UgU0lPQ0RFTE1VTFRJOgotCQllcnJv ciA9IDA7CiAJCWJyZWFrOwogCiAJY2FzZSBTSU9DR0RSVlNQRUM6CkBAIC0yMjY3LDYgKzIyNzcs MTMgQEAKIAkJZXJyb3IgPSBpZm1lZGlhX2lvY3RsKGlmcCwgaWZyLCAmc2MtPnZ4bF9tZWRpYSwg Y21kKTsKIAkJYnJlYWs7CiAKKwljYXNlIFNJT0NTSUZNVFU6CisJCWlmIChpZnItPmlmcl9tdHUg PCBFVEhFUk1JTiB8fCBpZnItPmlmcl9tdHUgPiBWWExBTl9NQVhfTVRVKQorCQkJZXJyb3IgPSBF SU5WQUw7CisJCWVsc2UKKwkJCWlmcC0+aWZfbXR1ID0gaWZyLT5pZnJfbXR1OworCQlicmVhazsK KwogCWRlZmF1bHQ6CiAJCWVycm9yID0gZXRoZXJfaW9jdGwoaWZwLCBjbWQsIGRhdGEpOwogCQli cmVhazsKQEAgLTI3NDcsOCArMjc2NCw4IEBACiAJaWZwLT5pZl9pb2N0bCA9IHZ4bGFuX2lvY3Rs OwogCWlmcC0+aWZfdHJhbnNtaXQgPSB2eGxhbl90cmFuc21pdDsKIAlpZnAtPmlmX3FmbHVzaCA9 IHZ4bGFuX3FmbHVzaDsKLQlpZnAtPmlmX2NhcGFiaWxpdGllcyB8PSBJRkNBUF9MSU5LU1RBVEU7 Ci0JaWZwLT5pZl9jYXBlbmFibGUgfD0gSUZDQVBfTElOS1NUQVRFOworCWlmcC0+aWZfY2FwYWJp bGl0aWVzIHw9IElGQ0FQX0xJTktTVEFURSB8IElGQ0FQX0pVTUJPX01UVTsKKwlpZnAtPmlmX2Nh cGVuYWJsZSB8PSBJRkNBUF9MSU5LU1RBVEUgfCBJRkNBUF9KVU1CT19NVFU7CiAKIAlpZm1lZGlh X2luaXQoJnNjLT52eGxfbWVkaWEsIDAsIHZ4bGFuX21lZGlhX2NoYW5nZSwgdnhsYW5fbWVkaWFf c3RhdHVzKTsKIAlpZm1lZGlhX2FkZCgmc2MtPnZ4bF9tZWRpYSwgSUZNX0VUSEVSIHwgSUZNX0FV VE8sIDAsIE5VTEwpOwoK --b1_3abfc4f42b2894e21966a73e96dc51fe-- From owner-freebsd-net@freebsd.org Mon Jul 15 08:43:17 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 05427B429D for ; Mon, 15 Jul 2019 08:43:17 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D827285A6A; Mon, 15 Jul 2019 08:43:16 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from reviews.nyi.freebsd.org (reviews.nyi.freebsd.org [IPv6:2610:1c1:1:607c::16:b]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 0FDC61B198; Mon, 15 Jul 2019 08:43:16 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by reviews.nyi.freebsd.org (Postfix, from userid 1346) id 0D4D9D4F57; Mon, 15 Jul 2019 08:43:16 +0000 (UTC) Date: Mon, 15 Jul 2019 08:43:16 +0000 To: Phabricator From: "aleksandr.fedorov_itglobal.com (Aleksandr Fedorov)" Cc: freebsd-net@freebsd.org Reply-to: "aleksandr.fedorov_itglobal.com (Aleksandr Fedorov)" Subject: [Differential] D19422: if_vxlan(4) Allow set MTU more than 1500 bytes. Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: X-Herald-Rules: <81>, <110> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk Thread-Topic: PHID-DREV-bvtxxu4jwhzdkwqxxgd7 X-Phabricator-Mail-ID: 1538982 X-Phabricator-Send-Attempt: ujplxadw3oe2x3gn In-Reply-To: References: Thread-Index: MzYyMWIxNGMwOTE4YmQzMWVhNTU2NTFhMGQ2IF0sPKQ= X-Phabricator-Stamps: actor(@aleksandr.fedorov_itglobal.com) application(Differential) author(@aleksandr.fedorov_itglobal.com) herald(H81) herald(H110) monogram(D19422) object-type(DREV) phid(PHID-DREV-bvtxxu4jwhzdkwqxxgd7) reviewer(#network) reviewer(@bryanv) reviewer(@hrs) reviewer(@jhb) reviewer(@krion) reviewer(@rgrimes) revision-status(needs-review) subscriber(@ae) subscriber(@evgueni.gavrilov_itglobal.com) subscriber(@freebsd-net-list) subscriber(@olevole_olevole.ru) via(web) MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8" X-Rspamd-Queue-Id: D827285A6A X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.94 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.94)[-0.943,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jul 2019 08:43:17 -0000 YWxla3NhbmRyLmZlZG9yb3ZfaXRnbG9iYWwuY29tIG1hcmtlZCBhbiBpbmxpbmUgY29tbWVudCBh cyBkb25lLgoKQ0hBTkdFUyBTSU5DRSBMQVNUIEFDVElPTgogIGh0dHBzOi8vcmV2aWV3cy5mcmVl YnNkLm9yZy9EMTk0MjIvbmV3LwoKUkVWSVNJT04gREVUQUlMCiAgaHR0cHM6Ly9yZXZpZXdzLmZy ZWVic2Qub3JnL0QxOTQyMgoKRU1BSUwgUFJFRkVSRU5DRVMKICBodHRwczovL3Jldmlld3MuZnJl ZWJzZC5vcmcvc2V0dGluZ3MvcGFuZWwvZW1haWxwcmVmZXJlbmNlcy8KClRvOiBhbGVrc2FuZHIu ZmVkb3Jvdl9pdGdsb2JhbC5jb20sIGJyeWFudiwgaHJzLCAjbmV0d29yaywgcmdyaW1lcywga3Jp b24sIGpoYgpDYzogZXZndWVuaS5nYXZyaWxvdl9pdGdsb2JhbC5jb20sIG9sZXZvbGVfb2xldm9s ZS5ydSwgYWUsIGZyZWVic2QtbmV0LWxpc3QsIGtyenlzenRvZi5nYWxhemthX2ludGVsLmNvbQo= From owner-freebsd-net@freebsd.org Mon Jul 15 13:33:29 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 888AFBA263 for ; Mon, 15 Jul 2019 13:33:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 669B890A58 for ; Mon, 15 Jul 2019 13:33:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 66335BA262; Mon, 15 Jul 2019 13:33:29 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 65E2EBA261 for ; Mon, 15 Jul 2019 13:33:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48C7C90A56 for ; Mon, 15 Jul 2019 13:33:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1C78F1E526 for ; Mon, 15 Jul 2019 13:33:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6FDXSpS024035 for ; Mon, 15 Jul 2019 13:33:28 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6FDXS38024034 for net@FreeBSD.org; Mon, 15 Jul 2019 13:33:28 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 239216] Problem with ix(4) driver on systems with two Intel X520 cards Date: Mon, 15 Jul 2019 13:33:27 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: IntelNetworking X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 48C7C90A56 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.985,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jul 2019 13:33:29 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D239216 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|ports-bugs@FreeBSD.org |net@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Mon Jul 15 13:35:46 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4CC96BA3B6 for ; Mon, 15 Jul 2019 13:35:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 26D5D90CF0 for ; Mon, 15 Jul 2019 13:35:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 266D2BA3B3; Mon, 15 Jul 2019 13:35:46 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 26171BA3B2 for ; Mon, 15 Jul 2019 13:35:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0941D90CEF for ; Mon, 15 Jul 2019 13:35:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D4C661E531 for ; Mon, 15 Jul 2019 13:35:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6FDZjNn027334 for ; Mon, 15 Jul 2019 13:35:45 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6FDZjUB027333 for net@FreeBSD.org; Mon, 15 Jul 2019 13:35:45 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 239224] Virtual Interface: Not Working in Netmap Mode on 11.3 STABLE Date: Mon, 15 Jul 2019 13:35:45 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.3-STABLE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: keywords assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 26D5D90CF0 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.99 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.99)[-0.989,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jul 2019 13:35:46 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D239224 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |regression Assignee|bugs@FreeBSD.org |net@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Mon Jul 15 16:07:18 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2FC36BE051 for ; Mon, 15 Jul 2019 16:07:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 12ADB6A1E4 for ; Mon, 15 Jul 2019 16:07:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 107ACBE050; Mon, 15 Jul 2019 16:07:18 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 103EBBE04F for ; Mon, 15 Jul 2019 16:07:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E74336A1E3 for ; Mon, 15 Jul 2019 16:07:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D3DBD20097 for ; Mon, 15 Jul 2019 16:07:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6FG7HQx013399 for ; Mon, 15 Jul 2019 16:07:17 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6FG7HD1013398 for net@FreeBSD.org; Mon, 15 Jul 2019 16:07:17 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 239224] Virtual Interface: Not Working in Netmap Mode on 11.3 STABLE Date: Mon, 15 Jul 2019 16:07:17 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.3-STABLE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: vmaffione@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: E74336A1E3 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.983,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jul 2019 16:07:18 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D239224 --- Comment #1 from Vincenzo Maffione --- Hi, Maybe you hit the bug introduced by r349922 (fixed in HEAD, will MFC toda= y)? https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238642=20 Can you check if the stack trace is the same? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Mon Jul 15 18:55:24 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A68C8C1272 for ; Mon, 15 Jul 2019 18:55:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 8A902716B8 for ; Mon, 15 Jul 2019 18:55:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 8826FC1271; Mon, 15 Jul 2019 18:55:24 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 87E8FC1270 for ; Mon, 15 Jul 2019 18:55:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 69457716B7 for ; Mon, 15 Jul 2019 18:55:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3DE0B21F43 for ; Mon, 15 Jul 2019 18:55:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6FItOBD031541 for ; Mon, 15 Jul 2019 18:55:24 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6FItOt7031535 for net@FreeBSD.org; Mon, 15 Jul 2019 18:55:24 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 239224] Virtual Interface: Not Working in Netmap Mode on 11.3 STABLE Date: Mon, 15 Jul 2019 18:55:24 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.3-STABLE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: halfling@halfling.com.br X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 8A902716B8 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.99 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.99)[-0.988,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jul 2019 18:55:24 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D239224 --- Comment #2 from Charles Goncalves --- (In reply to Vincenzo Maffione from comment #1) Yeah it=E2=80=99s the same bug when a run command like Aleksandr Fedorov: root@current:~ # ifconfig vlan0 create up=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20 root@current:~ # vale-ctl -a vale0:vlan0 I=E2=80=99m running on FreeBSD 11.3-STABLE #0 r349942 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Mon Jul 15 19:52:19 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3C54EC24C0 for ; Mon, 15 Jul 2019 19:52:19 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1E2E873A98; Mon, 15 Jul 2019 19:52:19 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from reviews.nyi.freebsd.org (reviews.nyi.freebsd.org [IPv6:2610:1c1:1:607c::16:b]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 90A0122A12; Mon, 15 Jul 2019 19:52:18 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by reviews.nyi.freebsd.org (Postfix, from userid 1346) id 8F891BD446; Mon, 15 Jul 2019 19:52:18 +0000 (UTC) Date: Mon, 15 Jul 2019 19:52:18 +0000 To: Phabricator From: "jhb (John Baldwin)" Cc: freebsd-net@freebsd.org Reply-to: "jhb (John Baldwin)" Subject: [Differential] D19422: if_vxlan(4) Allow set MTU more than 1500 bytes. Message-ID: <91694a3a45b2a834a321c76de3800ae8@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , X-Herald-Rules: <81>, <110> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk Thread-Topic: PHID-DREV-bvtxxu4jwhzdkwqxxgd7 X-Phabricator-Mail-ID: 1539941 X-Phabricator-Send-Attempt: byvfxss4577mudy4 In-Reply-To: References: Thread-Index: MzYyMWIxNGMwOTE4YmQzMWVhNTU2NTFhMGQ2IF0s2XI= X-Phabricator-Stamps: actor(@jhb) application(Differential) author(@aleksandr.fedorov_itglobal.com) herald(H81) herald(H110) monogram(D19422) object-type(DREV) phid(PHID-DREV-bvtxxu4jwhzdkwqxxgd7) reviewer(#network) reviewer(@bryanv) reviewer(@hrs) reviewer(@jhb) reviewer(@krion) reviewer(@rgrimes) revision-status(accepted) subscriber(@ae) subscriber(@evgueni.gavrilov_itglobal.com) subscriber(@freebsd-net-list) subscriber(@olevole_olevole.ru) via(web) MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8" X-Rspamd-Queue-Id: 1E2E873A98 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.94 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-0.99)[-0.993,0]; NEURAL_HAM_SHORT(-0.95)[-0.948,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jul 2019 19:52:19 -0000 amhiIGFjY2VwdGVkIHRoaXMgcmV2aXNpb24uCmpoYiBhZGRlZCBhIGNvbW1lbnQuClRoaXMgcmV2 aXNpb24gaXMgbm93IGFjY2VwdGVkIGFuZCByZWFkeSB0byBsYW5kLgoKCiAgVGhhbmtzIQoKQ0hB TkdFUyBTSU5DRSBMQVNUIEFDVElPTgogIGh0dHBzOi8vcmV2aWV3cy5mcmVlYnNkLm9yZy9EMTk0 MjIvbmV3LwoKUkVWSVNJT04gREVUQUlMCiAgaHR0cHM6Ly9yZXZpZXdzLmZyZWVic2Qub3JnL0Qx OTQyMgoKRU1BSUwgUFJFRkVSRU5DRVMKICBodHRwczovL3Jldmlld3MuZnJlZWJzZC5vcmcvc2V0 dGluZ3MvcGFuZWwvZW1haWxwcmVmZXJlbmNlcy8KClRvOiBhbGVrc2FuZHIuZmVkb3Jvdl9pdGds b2JhbC5jb20sIGJyeWFudiwgaHJzLCAjbmV0d29yaywgcmdyaW1lcywga3Jpb24sIGpoYgpDYzog ZXZndWVuaS5nYXZyaWxvdl9pdGdsb2JhbC5jb20sIG9sZXZvbGVfb2xldm9sZS5ydSwgYWUsIGZy ZWVic2QtbmV0LWxpc3QsIGtyenlzenRvZi5nYWxhemthX2ludGVsLmNvbQo= From owner-freebsd-net@freebsd.org Mon Jul 15 20:12:42 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6715EC2A0F for ; Mon, 15 Jul 2019 20:12:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 498B874D59 for ; Mon, 15 Jul 2019 20:12:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 491C3C2A0E; Mon, 15 Jul 2019 20:12:42 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 48DABC2A0D for ; Mon, 15 Jul 2019 20:12:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2B84274D58 for ; Mon, 15 Jul 2019 20:12:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 04E4C22DC5 for ; Mon, 15 Jul 2019 20:12:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6FKCfAm008577 for ; Mon, 15 Jul 2019 20:12:41 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6FKCfDF008576 for net@FreeBSD.org; Mon, 15 Jul 2019 20:12:41 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 238796] ipfilter: fix unremovable rules and rules checksum for comparison Date: Mon, 15 Jul 2019 20:12:41 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: cy@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: cy@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.isobsolete attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 498B874D59 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.98)[-0.976,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jul 2019 20:12:42 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238796 Cy Schubert changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #205744|0 |1 is obsolete| | --- Comment #20 from Cy Schubert --- Created attachment 205808 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D205808&action= =3Dedit Removal of all listed patches. I've cleaned up some of the pointer arithmetic in fil.c and ip_state.c (my = last three commits), which have resolved another issue but also resolve this one. Please svnup to the latest -current and try reproducing the bug again (with= out any other patches applied). --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Mon Jul 15 20:16:07 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 186B2C2C05 for ; Mon, 15 Jul 2019 20:16:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id F08D9751F8 for ; Mon, 15 Jul 2019 20:16:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id F0271C2C02; Mon, 15 Jul 2019 20:16:06 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EFD50C2C01 for ; Mon, 15 Jul 2019 20:16:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D0D0F751F7 for ; Mon, 15 Jul 2019 20:16:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id AB6DD22DCF for ; Mon, 15 Jul 2019 20:16:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6FKG6QW013198 for ; Mon, 15 Jul 2019 20:16:06 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6FKG6po013197 for net@FreeBSD.org; Mon, 15 Jul 2019 20:16:06 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 239224] Virtual Interface: Not Working in Netmap Mode on 11.3 STABLE Date: Mon, 15 Jul 2019 20:16:05 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.3-STABLE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: vmaffione@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: D0D0F751F7 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_SHORT(-0.97)[-0.970,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jul 2019 20:16:07 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D239224 Vincenzo Maffione changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |In Progress --- Comment #3 from Vincenzo Maffione --- I just committed the fix as r350007. Can you please try again? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Jul 16 01:27:59 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 84592A229A for ; Tue, 16 Jul 2019 01:27:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 6577F89EBE for ; Tue, 16 Jul 2019 01:27:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 65129A2299; Tue, 16 Jul 2019 01:27:59 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 64DA7A2298 for ; Tue, 16 Jul 2019 01:27:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48B2489EBD for ; Tue, 16 Jul 2019 01:27:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1F665264B1 for ; Tue, 16 Jul 2019 01:27:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6G1RxTS018943 for ; Tue, 16 Jul 2019 01:27:59 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6G1RxAC018942 for net@FreeBSD.org; Tue, 16 Jul 2019 01:27:59 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 239224] Virtual Interface: Not Working in Netmap Mode on 11.3 STABLE Date: Tue, 16 Jul 2019 01:27:58 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.3-STABLE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: DUPLICATE X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: vmaffione@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution assigned_to bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 48B2489EBD X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_SHORT(-0.98)[-0.979,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jul 2019 01:27:59 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D239224 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |DUPLICATE Assignee|net@FreeBSD.org |vmaffione@FreeBSD.org Status|In Progress |Closed --- Comment #4 from Kubilay Kocak --- Panic report & regression reported in bug 238642 (which was/is still open at the time of this report), so close this as a duplicate *** This bug has been marked as a duplicate of bug 238642 *** --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Jul 16 01:28:03 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6A9F8A22AC for ; Tue, 16 Jul 2019 01:28:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4265189EF0 for ; Tue, 16 Jul 2019 01:28:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 41E2CA22AB; Tue, 16 Jul 2019 01:28:03 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 41523A22AA for ; Tue, 16 Jul 2019 01:28:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1D04C89EEB for ; Tue, 16 Jul 2019 01:28:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id ADAD0264B9 for ; Tue, 16 Jul 2019 01:28:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6G1S2bQ019025 for ; Tue, 16 Jul 2019 01:28:02 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6G1S2UK019024 for net@FreeBSD.org; Tue, 16 Jul 2019 01:28:02 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 238642] netmap: fix kernel pointer printing in netmap_generic.c Date: Tue, 16 Jul 2019 01:27:58 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch, security X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: vmaffione@FreeBSD.org X-Bugzilla-Flags: mfc-stable11? mfc-stable12? X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 1D04C89EEB X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_SHORT(-0.98)[-0.979,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jul 2019 01:28:03 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238642 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |halfling@halfling.com.br --- Comment #7 from Kubilay Kocak --- *** Bug 239224 has been marked as a duplicate of this bug. *** --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Tue Jul 16 01:31:01 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D1CADA2541 for ; Tue, 16 Jul 2019 01:31:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id B4EEC8A1AF for ; Tue, 16 Jul 2019 01:31:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id B4610A2540; Tue, 16 Jul 2019 01:31:01 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B41D8A253F for ; Tue, 16 Jul 2019 01:31:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 96CA58A1AB for ; Tue, 16 Jul 2019 01:31:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6FD37264F8 for ; Tue, 16 Jul 2019 01:31:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6G1V18c025846 for ; Tue, 16 Jul 2019 01:31:01 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6G1V1ex025845 for net@FreeBSD.org; Tue, 16 Jul 2019 01:31:01 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 238642] netmap: fix kernel pointer printing in netmap_generic.c Date: Tue, 16 Jul 2019 01:31:01 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch, security X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: vmaffione@FreeBSD.org X-Bugzilla-Flags: mfc-stable11? mfc-stable12? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: B4EEC8A1AF X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_SHORT(-0.98)[-0.981,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jul 2019 01:31:01 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238642 --- Comment #8 from Kubilay Kocak --- Regression fix committed in base r349966 MFC'd to stable/11 in base r350007= and stable/12 in base r350006 @Vincenzo If you could include PR: references in all commits (and mfc's) wh= ere possible that would be great! If this is now resolved, (no further commits needed), please set mfc-stable* flags to + where appropriate and close issue at your leisure --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Tue Jul 16 05:20:42 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 75F1FA5D2D for ; Tue, 16 Jul 2019 05:20:42 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 571536ABE6; Tue, 16 Jul 2019 05:20:42 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from reviews.nyi.freebsd.org (reviews.nyi.freebsd.org [IPv6:2610:1c1:1:607c::16:b]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id DED5FEDC; Tue, 16 Jul 2019 05:20:41 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by reviews.nyi.freebsd.org (Postfix, from userid 1346) id DE2A8249727; Tue, 16 Jul 2019 05:20:41 +0000 (UTC) Date: Tue, 16 Jul 2019 05:20:41 +0000 To: Phabricator From: "aleksandr.fedorov_itglobal.com (Aleksandr Fedorov)" Cc: freebsd-net@freebsd.org Reply-to: "aleksandr.fedorov_itglobal.com (Aleksandr Fedorov)" Subject: [Differential] D19422: if_vxlan(4) Allow set MTU more than 1500 bytes. Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: X-Herald-Rules: <81>, <110> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk Thread-Topic: PHID-DREV-bvtxxu4jwhzdkwqxxgd7 X-Phabricator-Mail-ID: 1540319 X-Phabricator-Send-Attempt: zmqcycwpse6ddq6r In-Reply-To: References: Thread-Index: MzYyMWIxNGMwOTE4YmQzMWVhNTU2NTFhMGQ2IF0tXqk= X-Phabricator-Stamps: actor(@aleksandr.fedorov_itglobal.com) application(Differential) author(@aleksandr.fedorov_itglobal.com) herald(H81) herald(H110) monogram(D19422) object-type(DREV) phid(PHID-DREV-bvtxxu4jwhzdkwqxxgd7) reviewer(#network) reviewer(@bryanv) reviewer(@hrs) reviewer(@jhb) reviewer(@krion) reviewer(@rgrimes) revision-status(accepted) subscriber(@ae) subscriber(@evgueni.gavrilov_itglobal.com) subscriber(@freebsd-net-list) subscriber(@olevole_olevole.ru) via(web) MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8" X-Rspamd-Queue-Id: 571536ABE6 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.90 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-0.99)[-0.993,0]; NEURAL_HAM_SHORT(-0.91)[-0.911,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jul 2019 05:20:42 -0000 YWxla3NhbmRyLmZlZG9yb3ZfaXRnbG9iYWwuY29tIGFkZGVkIGEgY29tbWVudC4KCgogIENhbiBh bnlvbmUgY29tbWl0IHRoaXMgcGF0Y2g/CgpDSEFOR0VTIFNJTkNFIExBU1QgQUNUSU9OCiAgaHR0 cHM6Ly9yZXZpZXdzLmZyZWVic2Qub3JnL0QxOTQyMi9uZXcvCgpSRVZJU0lPTiBERVRBSUwKICBo dHRwczovL3Jldmlld3MuZnJlZWJzZC5vcmcvRDE5NDIyCgpFTUFJTCBQUkVGRVJFTkNFUwogIGh0 dHBzOi8vcmV2aWV3cy5mcmVlYnNkLm9yZy9zZXR0aW5ncy9wYW5lbC9lbWFpbHByZWZlcmVuY2Vz LwoKVG86IGFsZWtzYW5kci5mZWRvcm92X2l0Z2xvYmFsLmNvbSwgYnJ5YW52LCBocnMsICNuZXR3 b3JrLCByZ3JpbWVzLCBrcmlvbiwgamhiCkNjOiBldmd1ZW5pLmdhdnJpbG92X2l0Z2xvYmFsLmNv bSwgb2xldm9sZV9vbGV2b2xlLnJ1LCBhZSwgZnJlZWJzZC1uZXQtbGlzdCwga3J6eXN6dG9mLmdh bGF6a2FfaW50ZWwuY29tCg== From owner-freebsd-net@freebsd.org Tue Jul 16 05:37:44 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 234B7A62F2 for ; Tue, 16 Jul 2019 05:37:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 0662E6B563 for ; Tue, 16 Jul 2019 05:37:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 04EFCA62F1; Tue, 16 Jul 2019 05:37:44 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 04A5CA62F0 for ; Tue, 16 Jul 2019 05:37:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DBA6B6B561 for ; Tue, 16 Jul 2019 05:37:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8E20F124D for ; Tue, 16 Jul 2019 05:37:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6G5bhHn065995 for ; Tue, 16 Jul 2019 05:37:43 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6G5bhjb065994 for net@FreeBSD.org; Tue, 16 Jul 2019 05:37:43 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 235787] ixgbe no carrier problem - TX(7) desc avail = 2048, pidx = 0 Date: Tue, 16 Jul 2019 05:37:42 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-STABLE X-Bugzilla-Keywords: IntelNetworking X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: ultima@freebsd.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Unable to Reproduce X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: see_also Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: DBA6B6B561 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_SHORT(-0.98)[-0.982,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jul 2019 05:37:44 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235787 Richard Gallamore changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.freebsd.org/bu | |gzilla/show_bug.cgi?id=3D2= 392 | |40 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Jul 16 05:39:39 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C6D15A644C for ; Tue, 16 Jul 2019 05:39:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id A96B66B739 for ; Tue, 16 Jul 2019 05:39:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id A90AAA6446; Tue, 16 Jul 2019 05:39:39 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A8CFEA6445 for ; Tue, 16 Jul 2019 05:39:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 884F76B738 for ; Tue, 16 Jul 2019 05:39:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0903F125B for ; Tue, 16 Jul 2019 05:39:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6G5dcTS068234 for ; Tue, 16 Jul 2019 05:39:38 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6G5dcNM068233 for net@FreeBSD.org; Tue, 16 Jul 2019 05:39:38 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 239240] igb: TX(2) desc avail = 1024, pidx = 0 Date: Tue, 16 Jul 2019 05:39:38 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ultima@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: A96B66B739 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_SHORT(-0.99)[-0.985,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jul 2019 05:39:39 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D239240 Richard Gallamore changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|bugs@FreeBSD.org |net@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Jul 16 09:39:03 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C9F7DAAADC for ; Tue, 16 Jul 2019 09:39:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id AAAE67431C for ; Tue, 16 Jul 2019 09:39:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id AA43CAAADB; Tue, 16 Jul 2019 09:39:03 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A9FC1AAADA for ; Tue, 16 Jul 2019 09:39:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8C9207431B for ; Tue, 16 Jul 2019 09:39:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 805B23F41 for ; Tue, 16 Jul 2019 09:39:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6G9d3IZ098109 for ; Tue, 16 Jul 2019 09:39:03 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6G9d3xk098108 for net@FreeBSD.org; Tue, 16 Jul 2019 09:39:03 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 238796] ipfilter: fix unremovable rules and rules checksum for comparison Date: Tue, 16 Jul 2019 09:39:03 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: msl0000023508@gmail.com X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: cy@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 8C9207431B X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.976,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jul 2019 09:39:03 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238796 --- Comment #21 from WHR --- (In reply to Cy Schubert from comment #20) Unfortunately this bug is still happening in r350024. I has verified that the source code I compiling contain your recent 3 commi= ts (349978, 349979, 349980); and I don't think those changes could possibly fix this bug. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Tue Jul 16 11:37:13 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5BFF0AD5A7 for ; Tue, 16 Jul 2019 11:37:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 3BD3380F85 for ; Tue, 16 Jul 2019 11:37:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 3B623AD5A6; Tue, 16 Jul 2019 11:37:13 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3B265AD5A5 for ; Tue, 16 Jul 2019 11:37:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1DC8780F82 for ; Tue, 16 Jul 2019 11:37:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DFDC854DA for ; Tue, 16 Jul 2019 11:37:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6GBbCgu056043 for ; Tue, 16 Jul 2019 11:37:12 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6GBbCkS056042 for net@FreeBSD.org; Tue, 16 Jul 2019 11:37:12 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 238796] ipfilter: fix unremovable rules and rules checksum for comparison Date: Tue, 16 Jul 2019 11:37:13 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: cy@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: cy@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 1DC8780F82 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.97)[-0.969,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jul 2019 11:37:13 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238796 --- Comment #22 from Cy Schubert --- Hmmm. I am not able to reproduce the bug on real hardware or in a VM any mo= re. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Tue Jul 16 16:38:23 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4FC43B4714 for ; Tue, 16 Jul 2019 16:38:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 33E508F171 for ; Tue, 16 Jul 2019 16:38:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 337EDB4713; Tue, 16 Jul 2019 16:38:23 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 32F0EB4712 for ; Tue, 16 Jul 2019 16:38:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 155608F170 for ; Tue, 16 Jul 2019 16:38:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DA1B48B2D for ; Tue, 16 Jul 2019 16:38:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6GGcM6S081959 for ; Tue, 16 Jul 2019 16:38:22 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6GGcMB7081958 for net@FreeBSD.org; Tue, 16 Jul 2019 16:38:22 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 238642] netmap: fix kernel pointer printing in netmap_generic.c Date: Tue, 16 Jul 2019 16:38:22 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch, security X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: vmaffione@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: vmaffione@FreeBSD.org X-Bugzilla-Flags: mfc-stable11+ mfc-stable12+ X-Bugzilla-Changed-Fields: flagtypes.name resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 155608F170 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.99)[-0.986,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jul 2019 16:38:23 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238642 Vincenzo Maffione changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|mfc-stable11?, |mfc-stable11+, |mfc-stable12? |mfc-stable12+ Resolution|--- |FIXED Status|In Progress |Closed --- Comment #9 from Vincenzo Maffione --- You're right, I will include PR: references in the futures. I also set the flags. Closing, since ci.freebsd.org tells me that the regression is fixed. Thanks --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Tue Jul 16 17:00:36 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0F7EDB4E9B for ; Tue, 16 Jul 2019 17:00:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id E5E8D6831C for ; Tue, 16 Jul 2019 17:00:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id E5807B4E9A; Tue, 16 Jul 2019 17:00:35 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E5497B4E99 for ; Tue, 16 Jul 2019 17:00:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C72C868318 for ; Tue, 16 Jul 2019 17:00:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B9EAA8EDE for ; Tue, 16 Jul 2019 17:00:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6GH0Zo9028261 for ; Tue, 16 Jul 2019 17:00:35 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6GH0Z0g028260 for net@FreeBSD.org; Tue, 16 Jul 2019 17:00:35 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 239240] igb: TX(2) desc avail = 1024, pidx = 0 Date: Tue, 16 Jul 2019 17:00:35 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: arkadiusz.majewski@iptrace.pl X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: E5E8D6831C X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.98)[-0.980,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jul 2019 17:00:36 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D239240 IPTRACE changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |arkadiusz.majewski@iptrace. | |pl --- Comment #1 from IPTRACE --- Hello! I'm affected as well for FreeBSD 12.0-RELEASE-p7. kernel: igb0: TX(0) desc avail =3D 1024, pidx =3D 0 ... kernel: igb2: TX(5) desc avail =3D 1024, pidx =3D 0 igb0@pci0:129:0:0: class=3D0x020000 card=3D0x00001458 chip=3D0x1521808= 6 rev=3D0x01 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'I350 Gigabit Network Connection' class =3D network subclass =3D ethernet igb2@pci0:132:0:0: class=3D0x020000 card=3D0xa02c8086 chip=3D0x10e8808= 6 rev=3D0x01 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82576 Gigabit Network Connection' class =3D network subclass =3D ethernet Use case (forced by me but generally it happens randomly): 1. All was working. 2. I detached Ethernet cable from modem side. 3. Connected again and saw "no carrier" on igb0 interface. 4. System logged as follow: Jul 5 22:31:35 hpv kernel: igb0: TX(0) desc avail =3D 1024, pidx =3D= 0 Jul 5 22:31:38 hpv kernel: igb0: TX(0) desc avail =3D 1024, pidx =3D= 0 Jul 5 22:31:40 hpv kernel: igb0: TX(0) desc avail =3D 1024, pidx =3D= 0 5. I was trying "server:# service netif restart igb0 && service routing restart" without success. 6. Resarted OS to back to normal. Is there any workaround instead of restart a system? I was trying below without progress as well. # ifconfig igb0 down # ifconfig igb0 up I think the problem is when the network card loses ethernet link and then errors occurs with non-working interface?! --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Jul 16 19:25:29 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6D3D0B83B1 for ; Tue, 16 Jul 2019 19:25:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4AA3770956 for ; Tue, 16 Jul 2019 19:25:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 4A177B83B0; Tue, 16 Jul 2019 19:25:29 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 49D8DB83AF for ; Tue, 16 Jul 2019 19:25:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2B4E370955 for ; Tue, 16 Jul 2019 19:25:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 06922AA35 for ; Tue, 16 Jul 2019 19:25:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6GJPSXA007413 for ; Tue, 16 Jul 2019 19:25:28 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6GJPSK3007412 for net@FreeBSD.org; Tue, 16 Jul 2019 19:25:28 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 239240] igb: TX(2) desc avail = 1024, pidx = 0 messages appear when the network card loses ethernet link Date: Tue, 16 Jul 2019 19:25:29 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-RELEASE X-Bugzilla-Keywords: IntelNetworking X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: keywords short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 4AA3770956 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.99 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.99)[-0.991,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jul 2019 19:25:29 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D239240 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |IntelNetworking Summary|igb: TX(2) desc avail =3D |igb: TX(2) desc avail =3D |1024, pidx =3D 0 |1024, pidx =3D 0 messages | |appear when the network | |card loses ethernet link --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Wed Jul 17 07:42:55 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D320EA5C1B for ; Wed, 17 Jul 2019 07:42:55 +0000 (UTC) (envelope-from satan@ukr.net) Received: from hell.ukr.net (hell.ukr.net [212.42.67.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.ukr.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 45EEF6C5B1; Wed, 17 Jul 2019 07:42:54 +0000 (UTC) (envelope-from satan@ukr.net) Received: from satan by hell.ukr.net with local ID 1hneaJ-000H6B-LZ ; Wed, 17 Jul 2019 10:42:43 +0300 Date: Wed, 17 Jul 2019 10:42:43 +0300 From: Vitalij Satanivskij To: Michael Tuexen Cc: Paul , freebsd-net@freebsd.org Subject: Re: Issues with TCP Timestamps allocation Message-ID: <20190717074243.GA65665@hell.ukr.net> References: <1562579483.67527000.24rw4xi5@frv39.fwdcdn.com> <32FD061B-245C-41D2-81DE-1B4756A7173D@freebsd.org> <1562591379.369129000.gpmxvurq@frv39.fwdcdn.com> <1562599181.734953000.1l9a1d23@frv39.fwdcdn.com> <0C475A01-9BCD-4E4A-9731-09AB919CA9BE@freebsd.org> <1562676414.933145000.z3zteyqp@frv39.fwdcdn.com> <1E9F3F99-C3E9-44DD-AA70-9B11E19D4769@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1E9F3F99-C3E9-44DD-AA70-9B11E19D4769@freebsd.org> User-Agent: Mutt/1.12.1 (2019-06-15) X-Rspamd-Queue-Id: 45EEF6C5B1 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dmarc=pass (policy=none) header.from=ukr.net; spf=pass (mx1.freebsd.org: domain of satan@ukr.net designates 212.42.67.68 as permitted sender) smtp.mailfrom=satan@ukr.net X-Spamd-Result: default: False [-2.90 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.96)[-0.965,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:212.42.67.64/27]; FREEMAIL_FROM(0.00)[ukr.net]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.61)[0.608,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[mxs.ukr.net]; DMARC_POLICY_ALLOW(-0.50)[ukr.net,none]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[ukr.net]; ASN(0.00)[asn:8856, ipnet:212.42.66.0/23, country:UA]; FREEMAIL_CC(0.00)[ukr.net]; IP_SCORE(-0.73)[asn: 8856(-3.76), country: UA(0.08)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jul 2019 07:42:55 -0000 Hello. Is there any changes about this problem I'm using FreeBSD 12 on my desktop and can confirm problem occur with some hosts. Michael Tuexen wrote: MT> MT> MT> > On 9. Jul 2019, at 14:58, Paul wrote: MT> > MT> > Hi Michael, MT> > MT> > 9 July 2019, 15:34:29, by "Michael Tuexen" : MT> > MT> >> MT> >> MT> >>> On 8. Jul 2019, at 17:22, Paul wrote: MT> >>> MT> >>> MT> >>> MT> >>> 8 July 2019, 17:12:21, by "Michael Tuexen" : MT> >>> MT> >>>>> On 8. Jul 2019, at 15:24, Paul wrote: MT> >>>>> MT> >>>>> Hi Michael, MT> >>>>> MT> >>>>> 8 July 2019, 15:53:15, by "Michael Tuexen" : MT> >>>>> MT> >>>>>>> On 8. Jul 2019, at 12:37, Paul wrote: MT> >>>>>>> MT> >>>>>>> Hi team, MT> >>>>>>> MT> >>>>>>> Recently we had an upgrade to 12 Stable. Immediately after, we have started MT> >>>>>>> seeing some strange connection establishment timeouts to some fixed number MT> >>>>>>> of external (world) hosts. The issue was persistent and easy to reproduce. MT> >>>>>>> Thanks to a patience and dedication of our system engineer we have tracked MT> >>>>>>> this issue down to a specific commit: MT> >>>>>>> MT> >>>>>>> https://svnweb.freebsd.org/base?view=revision&revision=338053 MT> >>>>>>> MT> >>>>>>> This patch was also back-ported into 11 Stable: MT> >>>>>>> MT> >>>>>>> https://svnweb.freebsd.org/base?view=revision&revision=348435 MT> >>>>>>> MT> >>>>>>> Among other things this patch changes the timestamp allocation strategy, MT> >>>>>>> by introducing a deterministic randomness via a hash function that takes MT> >>>>>>> into account a random key as well as source address, source port, dest MT> >>>>>>> address and dest port. As the result, timestamp offsets of different MT> >>>>>>> tuples (SA,SP,DA,DP) will be wildly different and will jump from small MT> >>>>>>> to large numbers and back, as long as something in the tuple changes. MT> >>>>>> Hi Paul, MT> >>>>>> MT> >>>>>> this is correct. MT> >>>>>> MT> >>>>>> Please note that the same happens with the old method, if two hosts with MT> >>>>>> different uptimes are bind a consumer grade NAT. MT> >>>>> MT> >>>>> If NAT does not replace timestamps then yes, it should be the case. MT> >>>>> MT> >>>>>>> MT> >>>>>>> After performing various tests of hosts that produce the above mentioned MT> >>>>>>> issue we came to conclusion that there are some interesting implementations MT> >>>>>>> that drop SYN packets with timestamps smaller than the largest timestamp MT> >>>>>>> value from streams of all recent or current connections from a specific MT> >>>>>>> address. This looks as some kind of SYN flood protection. MT> >>>>>> This also breaks multiple hosts with different uptimes behind a consumer MT> >>>>>> level NAT talking to such a server. MT> >>>>>>> MT> >>>>>>> To ensure that each external host is not going to see a wild jumps of MT> >>>>>>> timestamp values I propose a patch that removes ports from the equation MT> >>>>>>> all together, when calculating the timestamp offset: MT> >>>>>>> MT> >>>>>>> Index: sys/netinet/tcp_subr.c MT> >>>>>>> =================================================================== MT> >>>>>>> --- sys/netinet/tcp_subr.c (revision 348435) MT> >>>>>>> +++ sys/netinet/tcp_subr.c (working copy) MT> >>>>>>> @@ -2224,7 +2224,22 @@ MT> >>>>>>> uint32_t MT> >>>>>>> tcp_new_ts_offset(struct in_conninfo *inc) MT> >>>>>>> { MT> >>>>>>> - return (tcp_keyed_hash(inc, V_ts_offset_secret)); MT> >>>>>>> + /* MT> >>>>>>> + * Some implementations show a strange behaviour when a wildly random MT> >>>>>>> + * timestamps allocated for different streams. It seems that only the MT> >>>>>>> + * SYN packets are affected. Observed implementations drop SYN packets MT> >>>>>>> + * with timestamps smaller than the largest timestamp value of all MT> >>>>>>> + * recent or current connections from specific a address. To mitigate MT> >>>>>>> + * this we are going to ensure that each host will always observe MT> >>>>>>> + * timestamps as increasing no matter the stream: by dropping ports MT> >>>>>>> + * from the equation. MT> >>>>>>> + */ MT> >>>>>>> + struct in_conninfo inc_copy = *inc; MT> >>>>>>> + MT> >>>>>>> + inc_copy.inc_fport = 0; MT> >>>>>>> + inc_copy.inc_lport = 0; MT> >>>>>>> + MT> >>>>>>> + return (tcp_keyed_hash(&inc_copy, V_ts_offset_secret)); MT> >>>>>>> } MT> >>>>>>> MT> >>>>>>> /* MT> >>>>>>> MT> >>>>>>> In any case, the solution of the uptime leak, implemented in rev338053 is MT> >>>>>>> not going to suffer, because a supposed attacker is currently able to use MT> >>>>>>> any fixed values of SP and DP, albeit not 0, anyway, to remove them out MT> >>>>>>> of the equation. MT> >>>>>> Can you describe how a peer can compute the uptime from two observed timestamps? MT> >>>>>> I don't see how you can do that... MT> >>>>> MT> >>>>> Supposed attacker could run a script that continuously monitors timestamps, MT> >>>>> for example via a periodic TCP connection from a fixed local port (eg 12345) MT> >>>>> and a fixed local address to the fixed victim's address and port (eg 80). MT> >>>>> Whenever large discrepancy is observed, attacker can assume that reboot has MT> >>>>> happened (due to V_ts_offset_secret re-generation), hence the received MT> >>>>> timestamp is considered an approximate point of reboot from which the uptime MT> >>>>> can be calculated, until the next reboot and so on. MT> >>>> Ahh, I see. The patch we are talking about is not intended to protect against MT> >>>> continuous monitoring, which is something you can always do. You could even MT> >>>> watch for service availability and detect reboots. A change of the local key MT> >>>> would also look similar to a reboot without a temporary loss of connectivity. MT> >>>> MT> >>>> Thanks for the clarification. MT> >>>>> MT> >>>>>>> MT> >>>>>>> There is the list of example hosts that we were able to reproduce the MT> >>>>>>> issue with: MT> >>>>>>> MT> >>>>>>> curl -v http://88.99.60.171:80 MT> >>>>>>> curl -v http://163.172.71.252:80 MT> >>>>>>> curl -v http://5.9.242.150:80 MT> >>>>>>> curl -v https://185.134.205.105:443 MT> >>>>>>> curl -v https://136.243.1.231:443 MT> >>>>>>> curl -v https://144.76.196.4:443 MT> >>>>>>> curl -v http://94.127.191.194:80 MT> >>>>>>> MT> >>>>>>> To reproduce, call curl repeatedly with a same URL some number of times. MT> >>>>>>> You are going to see some of the requests stuck in MT> >>>>>>> `* Trying XXX.XXX.XXX.XXX...` MT> >>>>>>> MT> >>>>>>> For some reason, the easiest way to reproduce the issue is with nc: MT> >>>>>>> MT> >>>>>>> $ echo "foooooo" | nc -v 88.99.60.171 80 MT> >>>>>>> MT> >>>>>>> Only a few such calls are required until one of them is stuck on connect(): MT> >>>>>>> issuing SYN packets with an exponential backoff. MT> >>>>>> Thanks for providing an end-point to test with. I'll take a look. MT> >>>>>> Just to be clear: You are running a FreeBSD client against one of the above MT> >>>>>> servers and experience the problem with the new timestamp computations. MT> >>>>>> MT> >>>>>> You are not running arbitrary clients against a FreeBSD server... MT> >>>>> MT> >>>>> We are talking about FreeBSD being the client. Peers that yield this unwanted MT> >>>>> behaviour are unknown. Little bit of tinkering showed that some of them run MT> >>>>> Debian: MT> >>>>> MT> >>>>> telnet 88.99.60.171 22 MT> >>>>> Trying 88.99.60.171... MT> >>>>> Connected to 88.99.60.171. MT> >>>>> Escape character is '^]'. MT> >>>>> SSH-2.0-OpenSSH_6.7p1 Debian-5+deb8u3 MT> >>>> Also some are hosted by Hetzner, but not all. I'll will look into MT> >>>> this tomorrow, since I'm on a deadline today (well it is 2am tomorrow MT> >>>> morning, to be precise)... MT> >>> MT> >>> Thanks a lot, I would appreciate that. MT> >> Hi Paul, MT> >> MT> >> I have looked into this. MT> >> MT> >> * The FreeBSD behaviour is the one which is specified in the last bullet item MT> >> in https://tools.ietf.org/html/rfc7323#section-5.4 MT> >> It is also the one, which is RECOMMENDED in MT> >> https://tools.ietf.org/html/rfc7323#section-7.1 MT> >> MT> >> * My NAT box (a popular one in Germany) does NOT rewrite TCP timestamps. MT> >> MT> >> This means that the host you are referring to have some sort of protection, MT> >> which makes incorrect assumptions. It will also break multiple hosts behind MT> >> a NAT. MT> >> MT> >> I can run MT> >> curl -v http://88.99.60.171:80 MT> >> in a loop without any problems from a FreeBSD head system. I tested 1000 MT> >> iterations or so. The TS.val is jumping up and down as expected. MT> >> I'm wondering why you are observing errors in this case, too. MT> >> MT> >> However, doing something like MT> >> echo "foooooo" | nc -v 88.99.60.171 80 MT> >> triggers the problem. MT> >> MT> >> So I think there is some functionality (in a middlebox or running on the host), MT> >> which incorrectly assume monotonic timestamps between multiple TCP connections MT> >> coming from the same IP address, but only in case of errors at the application layer. MT> > MT> > Yeah, exactly, some hosts seem to enable this only in case of an error in HTTP MT> > communication (some smart proxy?). However, there are some that behave this way MT> > regardless of errors, for example these: MT> > MT> > curl -v https://185.134.205.105:443 MT> > curl -v https://136.243.1.231:443 MT> Wireshark sees an Encrypted Alert in both cases. So I guess this is another indication MT> of "error at the application layer". MT> > MT> >> MT> >> Do you have any insights whether the hosts you are listed share something in MT> >> common. Some of them are hosted by Hetzner, but not all. MT> > MT> > Nope. A whole set of endpoints that we have detected so far is pretty diverse, MT> > containing a lot of different locations geographically, as well as different MT> > hosters. MT> OK. Thanks for the clarification. MT> > MT> >> MT> >> I think in general, it is the correct thing to include the port numbers in MT> >> the offset computation. We might add a sysctl variable to control the inclusion. MT> >> This would allow interworking with broken middleboxes. MT> > MT> > Yeah, I completely agree that these rare cases should not dictate the implementation. MT> > But an ability to enable a work-around via sysctl would be greatly appreciated. MT> > Currently we are unable to roll-out the upgrade across all servers because of this MT> > issue: even though it happens not so often, a lot of requests from our users MT> > get stuck or fail all together. For example, a host 185.134.205.105 is a kind of MT> > social network that our proxy servers connect to so securely access to content, MT> > such as images, on behalf of our users. MT> > MT> >> MT> >> Please note, this does not fix the case of multiple clients behind a NAT. MT> > MT> > Yeah, that's true. Fortunately we don't use NAT. MT> > MT> >> MT> >> I'm also trying to figure out how and why Linux and Windows are handling this. MT> > MT> > Thanks for bothering! MT> Will let you know what I figure out. MT> MT> Best regards MT> Michael MT> > MT> >> MT> >> Best regards MT> >> Michael MT> >> MT> >>> MT> >>>> MT> >>>> Best regards MT> >>>> Michael MT> >>>>> MT> >>>>> MT> >>>>>> MT> >>>>>> Best regards MT> >>>>>> Michael MT> >>>>>> MT> >>>>>> MT> >>>> MT> >>>> MT> >> MT> >> MT> MT> _______________________________________________ MT> freebsd-net@freebsd.org mailing list MT> https://lists.freebsd.org/mailman/listinfo/freebsd-net MT> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Wed Jul 17 07:47:50 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 68E65A5DAD for ; Wed, 17 Jul 2019 07:47:50 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2F94D6C770 for ; Wed, 17 Jul 2019 07:47:50 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from [IPv6:2a02:8109:1140:c3d:7843:89ae:ec92:371] (unknown [IPv6:2a02:8109:1140:c3d:7843:89ae:ec92:371]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTPSA id A180F71B631AA; Wed, 17 Jul 2019 09:47:45 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: Issues with TCP Timestamps allocation From: Michael Tuexen In-Reply-To: <20190717074243.GA65665@hell.ukr.net> Date: Wed, 17 Jul 2019 09:47:45 +0200 Cc: Paul , freebsd-net@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <1562579483.67527000.24rw4xi5@frv39.fwdcdn.com> <32FD061B-245C-41D2-81DE-1B4756A7173D@freebsd.org> <1562591379.369129000.gpmxvurq@frv39.fwdcdn.com> <1562599181.734953000.1l9a1d23@frv39.fwdcdn.com> <0C475A01-9BCD-4E4A-9731-09AB919CA9BE@freebsd.org> <1562676414.933145000.z3zteyqp@frv39.fwdcdn.com> <1E9F3F99-C3E9-44DD-AA70-9B11E19D4769@freebsd.org> <20190717074243.GA65665@hell.ukr.net> To: Vitalij Satanivskij X-Mailer: Apple Mail (2.3445.104.11) X-Spam-Status: No, score=-1.7 required=5.0 tests=ALL_TRUSTED,BAYES_00, NORMAL_HTTP_TO_IP,NUMERIC_HTTP_ADDR autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jul 2019 07:47:50 -0000 > On 17. Jul 2019, at 09:42, Vitalij Satanivskij wrote: >=20 >=20 >=20 > Hello.=20 >=20 > Is there any changes about this problem >=20 >=20 > I'm using FreeBSD 12 on my desktop and can confirm problem occur with = some hosts. Can you provide a list of some of these hosts? I'll put up a change for review later today. In the meantime you can deal with the buggy hosts by disabling the = timestamps or dropping extensions on SYN retransmits. Best regards Michael >=20 >=20 >=20 > Michael Tuexen wrote: > MT>=20 > MT>=20 > MT> > On 9. Jul 2019, at 14:58, Paul wrote: > MT> >=20 > MT> > Hi Michael, > MT> >=20 > MT> > 9 July 2019, 15:34:29, by "Michael Tuexen" : > MT> >=20 > MT> >>=20 > MT> >>=20 > MT> >>> On 8. Jul 2019, at 17:22, Paul wrote: > MT> >>>=20 > MT> >>>=20 > MT> >>>=20 > MT> >>> 8 July 2019, 17:12:21, by "Michael Tuexen" = : > MT> >>>=20 > MT> >>>>> On 8. Jul 2019, at 15:24, Paul wrote: > MT> >>>>>=20 > MT> >>>>> Hi Michael, > MT> >>>>>=20 > MT> >>>>> 8 July 2019, 15:53:15, by "Michael Tuexen" = : > MT> >>>>>=20 > MT> >>>>>>> On 8. Jul 2019, at 12:37, Paul wrote: > MT> >>>>>>>=20 > MT> >>>>>>> Hi team, > MT> >>>>>>>=20 > MT> >>>>>>> Recently we had an upgrade to 12 Stable. Immediately = after, we have started=20 > MT> >>>>>>> seeing some strange connection establishment timeouts to = some fixed number > MT> >>>>>>> of external (world) hosts. The issue was persistent and = easy to reproduce. > MT> >>>>>>> Thanks to a patience and dedication of our system engineer = we have tracked =20 > MT> >>>>>>> this issue down to a specific commit: > MT> >>>>>>>=20 > MT> >>>>>>> = https://svnweb.freebsd.org/base?view=3Drevision&revision=3D338053 > MT> >>>>>>>=20 > MT> >>>>>>> This patch was also back-ported into 11 Stable: > MT> >>>>>>>=20 > MT> >>>>>>> = https://svnweb.freebsd.org/base?view=3Drevision&revision=3D348435 > MT> >>>>>>>=20 > MT> >>>>>>> Among other things this patch changes the timestamp = allocation strategy, > MT> >>>>>>> by introducing a deterministic randomness via a hash = function that takes > MT> >>>>>>> into account a random key as well as source address, = source port, dest > MT> >>>>>>> address and dest port. As the result, timestamp offsets of = different > MT> >>>>>>> tuples (SA,SP,DA,DP) will be wildly different and will = jump from small=20 > MT> >>>>>>> to large numbers and back, as long as something in the = tuple changes. > MT> >>>>>> Hi Paul, > MT> >>>>>>=20 > MT> >>>>>> this is correct. > MT> >>>>>>=20 > MT> >>>>>> Please note that the same happens with the old method, if = two hosts with > MT> >>>>>> different uptimes are bind a consumer grade NAT. > MT> >>>>>=20 > MT> >>>>> If NAT does not replace timestamps then yes, it should be = the case. > MT> >>>>>=20 > MT> >>>>>>>=20 > MT> >>>>>>> After performing various tests of hosts that produce the = above mentioned=20 > MT> >>>>>>> issue we came to conclusion that there are some = interesting implementations=20 > MT> >>>>>>> that drop SYN packets with timestamps smaller than the = largest timestamp=20 > MT> >>>>>>> value from streams of all recent or current connections = from a specific=20 > MT> >>>>>>> address. This looks as some kind of SYN flood protection. > MT> >>>>>> This also breaks multiple hosts with different uptimes = behind a consumer > MT> >>>>>> level NAT talking to such a server. > MT> >>>>>>>=20 > MT> >>>>>>> To ensure that each external host is not going to see a = wild jumps of=20 > MT> >>>>>>> timestamp values I propose a patch that removes ports from = the equation > MT> >>>>>>> all together, when calculating the timestamp offset: > MT> >>>>>>>=20 > MT> >>>>>>> Index: sys/netinet/tcp_subr.c > MT> >>>>>>> = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > MT> >>>>>>> --- sys/netinet/tcp_subr.c (revision 348435) > MT> >>>>>>> +++ sys/netinet/tcp_subr.c (working copy) > MT> >>>>>>> @@ -2224,7 +2224,22 @@ > MT> >>>>>>> uint32_t > MT> >>>>>>> tcp_new_ts_offset(struct in_conninfo *inc) > MT> >>>>>>> { > MT> >>>>>>> - return (tcp_keyed_hash(inc, V_ts_offset_secret)); > MT> >>>>>>> + /*=20 > MT> >>>>>>> + * Some implementations show a strange behaviour = when a wildly random=20 > MT> >>>>>>> + * timestamps allocated for different streams. It = seems that only the > MT> >>>>>>> + * SYN packets are affected. Observed = implementations drop SYN packets > MT> >>>>>>> + * with timestamps smaller than the largest = timestamp value of all=20 > MT> >>>>>>> + * recent or current connections from specific a = address. To mitigate=20 > MT> >>>>>>> + * this we are going to ensure that each host = will always observe=20 > MT> >>>>>>> + * timestamps as increasing no matter the stream: = by dropping ports > MT> >>>>>>> + * from the equation. > MT> >>>>>>> + */=20 > MT> >>>>>>> + struct in_conninfo inc_copy =3D *inc; > MT> >>>>>>> + > MT> >>>>>>> + inc_copy.inc_fport =3D 0; > MT> >>>>>>> + inc_copy.inc_lport =3D 0; > MT> >>>>>>> + > MT> >>>>>>> + return (tcp_keyed_hash(&inc_copy, V_ts_offset_secret)); > MT> >>>>>>> } > MT> >>>>>>>=20 > MT> >>>>>>> /* > MT> >>>>>>>=20 > MT> >>>>>>> In any case, the solution of the uptime leak, implemented = in rev338053 is=20 > MT> >>>>>>> not going to suffer, because a supposed attacker is = currently able to use=20 > MT> >>>>>>> any fixed values of SP and DP, albeit not 0, anyway, to = remove them out=20 > MT> >>>>>>> of the equation. > MT> >>>>>> Can you describe how a peer can compute the uptime from two = observed timestamps? > MT> >>>>>> I don't see how you can do that... > MT> >>>>>=20 > MT> >>>>> Supposed attacker could run a script that continuously = monitors timestamps, > MT> >>>>> for example via a periodic TCP connection from a fixed local = port (eg 12345)=20 > MT> >>>>> and a fixed local address to the fixed victim's address and = port (eg 80). > MT> >>>>> Whenever large discrepancy is observed, attacker can assume = that reboot has=20 > MT> >>>>> happened (due to V_ts_offset_secret re-generation), hence = the received=20 > MT> >>>>> timestamp is considered an approximate point of reboot from = which the uptime > MT> >>>>> can be calculated, until the next reboot and so on. > MT> >>>> Ahh, I see. The patch we are talking about is not intended to = protect against > MT> >>>> continuous monitoring, which is something you can always do. = You could even > MT> >>>> watch for service availability and detect reboots. A change = of the local key > MT> >>>> would also look similar to a reboot without a temporary loss = of connectivity. > MT> >>>>=20 > MT> >>>> Thanks for the clarification. > MT> >>>>>=20 > MT> >>>>>>>=20 > MT> >>>>>>> There is the list of example hosts that we were able to = reproduce the=20 > MT> >>>>>>> issue with: > MT> >>>>>>>=20 > MT> >>>>>>> curl -v http://88.99.60.171:80 > MT> >>>>>>> curl -v http://163.172.71.252:80 > MT> >>>>>>> curl -v http://5.9.242.150:80 > MT> >>>>>>> curl -v https://185.134.205.105:443 > MT> >>>>>>> curl -v https://136.243.1.231:443 > MT> >>>>>>> curl -v https://144.76.196.4:443 > MT> >>>>>>> curl -v http://94.127.191.194:80 > MT> >>>>>>>=20 > MT> >>>>>>> To reproduce, call curl repeatedly with a same URL some = number of times.=20 > MT> >>>>>>> You are going to see some of the requests stuck in=20 > MT> >>>>>>> `* Trying XXX.XXX.XXX.XXX...` > MT> >>>>>>>=20 > MT> >>>>>>> For some reason, the easiest way to reproduce the issue is = with nc: > MT> >>>>>>>=20 > MT> >>>>>>> $ echo "foooooo" | nc -v 88.99.60.171 80 > MT> >>>>>>>=20 > MT> >>>>>>> Only a few such calls are required until one of them is = stuck on connect(): > MT> >>>>>>> issuing SYN packets with an exponential backoff. > MT> >>>>>> Thanks for providing an end-point to test with. I'll take a = look. > MT> >>>>>> Just to be clear: You are running a FreeBSD client against = one of the above > MT> >>>>>> servers and experience the problem with the new timestamp = computations. > MT> >>>>>>=20 > MT> >>>>>> You are not running arbitrary clients against a FreeBSD = server... > MT> >>>>>=20 > MT> >>>>> We are talking about FreeBSD being the client. Peers that = yield this unwanted > MT> >>>>> behaviour are unknown. Little bit of tinkering showed that = some of them run=20 > MT> >>>>> Debian: > MT> >>>>>=20 > MT> >>>>> telnet 88.99.60.171 22 > MT> >>>>> Trying 88.99.60.171... > MT> >>>>> Connected to 88.99.60.171. > MT> >>>>> Escape character is '^]'. > MT> >>>>> SSH-2.0-OpenSSH_6.7p1 Debian-5+deb8u3 > MT> >>>> Also some are hosted by Hetzner, but not all. I'll will look = into > MT> >>>> this tomorrow, since I'm on a deadline today (well it is 2am = tomorrow > MT> >>>> morning, to be precise)... > MT> >>>=20 > MT> >>> Thanks a lot, I would appreciate that. > MT> >> Hi Paul, > MT> >>=20 > MT> >> I have looked into this. > MT> >>=20 > MT> >> * The FreeBSD behaviour is the one which is specified in the = last bullet item > MT> >> in https://tools.ietf.org/html/rfc7323#section-5.4 > MT> >> It is also the one, which is RECOMMENDED in > MT> >> https://tools.ietf.org/html/rfc7323#section-7.1=20 > MT> >>=20 > MT> >> * My NAT box (a popular one in Germany) does NOT rewrite TCP = timestamps. > MT> >>=20 > MT> >> This means that the host you are referring to have some sort of = protection, > MT> >> which makes incorrect assumptions. It will also break multiple = hosts behind > MT> >> a NAT. > MT> >>=20 > MT> >> I can run > MT> >> curl -v http://88.99.60.171:80 > MT> >> in a loop without any problems from a FreeBSD head system. I = tested 1000 > MT> >> iterations or so. The TS.val is jumping up and down as = expected. > MT> >> I'm wondering why you are observing errors in this case, too. > MT> >>=20 > MT> >> However, doing something like > MT> >> echo "foooooo" | nc -v 88.99.60.171 80 > MT> >> triggers the problem. > MT> >>=20 > MT> >> So I think there is some functionality (in a middlebox or = running on the host), > MT> >> which incorrectly assume monotonic timestamps between multiple = TCP connections > MT> >> coming from the same IP address, but only in case of errors at = the application layer. > MT> >=20 > MT> > Yeah, exactly, some hosts seem to enable this only in case of an = error in HTTP > MT> > communication (some smart proxy?). However, there are some that = behave this way > MT> > regardless of errors, for example these: > MT> >=20 > MT> > curl -v https://185.134.205.105:443 > MT> > curl -v https://136.243.1.231:443 > MT> Wireshark sees an Encrypted Alert in both cases. So I guess this = is another indication > MT> of "error at the application layer". > MT> >=20 > MT> >>=20 > MT> >> Do you have any insights whether the hosts you are listed share = something in > MT> >> common. Some of them are hosted by Hetzner, but not all. > MT> >=20 > MT> > Nope. A whole set of endpoints that we have detected so far is = pretty diverse, > MT> > containing a lot of different locations geographically, as well = as different > MT> > hosters. > MT> OK. Thanks for the clarification. > MT> >=20 > MT> >>=20 > MT> >> I think in general, it is the correct thing to include the port = numbers in > MT> >> the offset computation. We might add a sysctl variable to = control the inclusion. > MT> >> This would allow interworking with broken middleboxes. > MT> >=20 > MT> > Yeah, I completely agree that these rare cases should not = dictate the implementation. > MT> > But an ability to enable a work-around via sysctl would be = greatly appreciated. > MT> > Currently we are unable to roll-out the upgrade across all = servers because of this > MT> > issue: even though it happens not so often, a lot of requests = from our users=20 > MT> > get stuck or fail all together. For example, a host = 185.134.205.105 is a kind of > MT> > social network that our proxy servers connect to so securely = access to content, > MT> > such as images, on behalf of our users. > MT> >=20 > MT> >>=20 > MT> >> Please note, this does not fix the case of multiple clients = behind a NAT. > MT> >=20 > MT> > Yeah, that's true. Fortunately we don't use NAT. > MT> >=20 > MT> >>=20 > MT> >> I'm also trying to figure out how and why Linux and Windows are = handling this. > MT> >=20 > MT> > Thanks for bothering! > MT> Will let you know what I figure out. > MT>=20 > MT> Best regards > MT> Michael > MT> >=20 > MT> >>=20 > MT> >> Best regards > MT> >> Michael > MT> >>=20 > MT> >>>=20 > MT> >>>>=20 > MT> >>>> Best regards > MT> >>>> Michael=20 > MT> >>>>>=20 > MT> >>>>>=20 > MT> >>>>>>=20 > MT> >>>>>> Best regards > MT> >>>>>> Michael > MT> >>>>>>=20 > MT> >>>>>>=20 > MT> >>>>=20 > MT> >>>>=20 > MT> >>=20 > MT> >>=20 > MT>=20 > MT> _______________________________________________ > MT> freebsd-net@freebsd.org mailing list > MT> https://lists.freebsd.org/mailman/listinfo/freebsd-net > MT> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Wed Jul 17 08:42:31 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 49E92A6B6C for ; Wed, 17 Jul 2019 08:42:31 +0000 (UTC) (envelope-from egypcio@gmail.com) Received: from mail-yw1-xc35.google.com (mail-yw1-xc35.google.com [IPv6:2607:f8b0:4864:20::c35]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 747C36E702 for ; Wed, 17 Jul 2019 08:42:30 +0000 (UTC) (envelope-from egypcio@gmail.com) Received: by mail-yw1-xc35.google.com with SMTP id z63so10263328ywz.9 for ; Wed, 17 Jul 2019 01:42:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=JSrR1MW9tLv2wPhZleUGGwsp8VRyjBH7D7yq4qECEWs=; b=EoyI69PBGZTGqdLz86+DTRLzu/NA21frUJn3JvMSu4X7OKpjt6mdDBUJVSheeO1u5H N9G20ueFhxCXaNDVj7S8PG8YaDTL+b8zSX3w/ANkgnmtzoTq0z34r0mBNCswC6B/Hptf l8q2uAnZMVup779ysJRenGwAfsljgsRMabp/mc4TwRAGj3wYvfZQifr69S+QtjFoOxKg NkScDDyJO3CudpNizPX+FxNttZ0WbptwUq99GVFKYwTYN3een1SVAHOUnOPZTctVNQam nu8GeFMd4hG48+vEMVAsSbnMU/v0A3IlZHIL4ZjeAuEZ1ThAxykA6bzs1Z961ne76uWB UYcg== X-Gm-Message-State: APjAAAX3VeJAVNhHEAvUWTJvqc1N+lo/ccoj+ru5XCQrsRCpq7IxLzHX wLcM8OrcRrBg0gkrTZbFqxId5D71/XCJIugzA0ljY+kj X-Google-Smtp-Source: APXvYqy8wlSzJU+3OierCFkcUKb3AX7mbLyNZlulf97DsUFy0/+mZxhPDoTzqMXIVBLLV6/I5misn0po9rjx4XBBWaw= X-Received: by 2002:a81:4fd4:: with SMTP id d203mr21596260ywb.166.1563352949725; Wed, 17 Jul 2019 01:42:29 -0700 (PDT) MIME-Version: 1.0 References: <8e388abc-f2ac-b070-cf86-a4d3971ac095@rawbw.com> In-Reply-To: <8e388abc-f2ac-b070-cf86-a4d3971ac095@rawbw.com> From: =?UTF-8?Q?Vin=C3=ADcius_Zavam?= Date: Wed, 17 Jul 2019 08:42:18 +0000 Message-ID: Subject: Re: How to set up ipfw(8) NAT between an alias and the main IP address, when the alias is in another network? To: Yuri Cc: "freebsd-net@freebsd.org" X-Rspamd-Queue-Id: 747C36E702 X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.90 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[googlemail.com]; URI_COUNT_ODD(1.00)[3]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DKIM_TRACE(0.00)[googlemail.com:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.36)[-0.362,0]; DMARC_POLICY_ALLOW(-0.50)[googlemail.com,quarantine]; FORGED_SENDER(0.30)[egypcio@googlemail.com,egypcio@gmail.com]; MIME_TRACE(0.00)[0:+,1:+]; SUBJECT_ENDS_QUESTION(1.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[egypcio@googlemail.com,egypcio@gmail.com]; DWL_DNSWL_NONE(0.00)[googlemail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.993,0]; R_DKIM_ALLOW(-0.20)[googlemail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-net@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[5.3.c.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; HTTP_TO_IP(1.00)[]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; IP_SCORE(-2.83)[ip: (-8.51), ipnet: 2607:f8b0::/32(-3.15), asn: 15169(-2.44), country: US(-0.06)] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jul 2019 08:42:31 -0000 Am Sa., 6. Juli 2019 um 08:02 Uhr schrieb Yuri : > My network interface looks like this: > > sk0: flags=3D8843 metric 0 mtu 15= 00 > options=3D80009 > ether 01:3c:47:8a:17:12 > inet 192.168.1.2 netmask 0xffffff00 broadcast 192.168.1.255 > inet 192.168.100.2 netmask 0xffffffff broadcast 192.168.100.2 > media: Ethernet autoselect (100baseTX ) > status: active > nd6 options=3D29 > > The second IP address is an alias that is used for jail. > > I would like to set up NAT so that this jail would access the internet > through the same interface. > > > I tried this script: > > > fw=3D"/sbin/ipfw -q" > > $fw nat 1 config redirect_addr 192.168.100.2 192.168.1.2 redirect_addr > 192.168.1.2 192.168.100.2 if sk0 unreg_only reset > > $fw add 1001 nat 1 tcp from 192.168.100.2/32 to any via sk0 keep-state > > $fw add 1002 check-state > > > The rule 1001 has keep-state, therefore it should process both outgoing > tcp and incoming response packets. But the outbound packets are NATted, > but the inbound ones are not. > > What is wrong, and how to fix this script? > > > Thank you, > > Yuri > jail ... ip4=3Dinherit ? --=20 Vin=C3=ADcius Zavam keybase.io/egypcio From owner-freebsd-net@freebsd.org Wed Jul 17 10:09:30 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8F613A87A2 for ; Wed, 17 Jul 2019 10:09:30 +0000 (UTC) (envelope-from satan@ukr.net) Received: from hell.ukr.net (hell.ukr.net [212.42.67.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.ukr.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B7F671797; Wed, 17 Jul 2019 10:09:30 +0000 (UTC) (envelope-from satan@ukr.net) Received: from satan by hell.ukr.net with local ID 1hngsI-0003SV-Tf ; Wed, 17 Jul 2019 13:09:26 +0300 Date: Wed, 17 Jul 2019 13:09:26 +0300 From: Vitalij Satanivskij To: Michael Tuexen Cc: Vitalij Satanivskij , freebsd-net@freebsd.org, Paul Subject: Re: Issues with TCP Timestamps allocation Message-ID: <20190717100926.GA24984@hell.ukr.net> References: <1562579483.67527000.24rw4xi5@frv39.fwdcdn.com> <32FD061B-245C-41D2-81DE-1B4756A7173D@freebsd.org> <1562591379.369129000.gpmxvurq@frv39.fwdcdn.com> <1562599181.734953000.1l9a1d23@frv39.fwdcdn.com> <0C475A01-9BCD-4E4A-9731-09AB919CA9BE@freebsd.org> <1562676414.933145000.z3zteyqp@frv39.fwdcdn.com> <1E9F3F99-C3E9-44DD-AA70-9B11E19D4769@freebsd.org> <20190717074243.GA65665@hell.ukr.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) X-Rspamd-Queue-Id: 4B7F671797 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.92 / 15.00]; NEURAL_HAM_MEDIUM(-0.99)[-0.993,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.92)[-0.922,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jul 2019 10:09:30 -0000 Hello again Michael Tuexen wrote: MT> > On 17. Jul 2019, at 09:42, Vitalij Satanivskij wrote: MT> > MT> > MT> > MT> > Hello. MT> > MT> > Is there any changes about this problem MT> > MT> > MT> > I'm using FreeBSD 12 on my desktop and can confirm problem occur with some hosts. MT> Can you provide a list of some of these hosts? MT> I'll put up a change for review later today. Here some hosts. 5.9.242.150 https://vitagramma.com 77.120.8.194 https://volia.com 31.41.220.92 https://moemisto.ua 185.5.72.33 https://fotostrana.ru Problem can be seen by sending curl request to hosts in serial (manual, so delay it's from few msec to few sec) Or by using proxy on machine with parallel/serial request's (eq squid or reverse proxy in nginx) On system before https://svnweb.freebsd.org/base?view=revision&revision=338053 such behavior not seen. MT> MT> In the meantime you can deal with the buggy hosts by disabling the timestamps MT> or dropping extensions on SYN retransmits. You meen by some code changes? MT> MT> Best regards MT> Michael MT> > MT> > MT> > MT> > Michael Tuexen wrote: MT> > MT> MT> > MT> MT> > MT> > On 9. Jul 2019, at 14:58, Paul wrote: MT> > MT> > MT> > MT> > Hi Michael, MT> > MT> > MT> > MT> > 9 July 2019, 15:34:29, by "Michael Tuexen" : MT> > MT> > MT> > MT> >> MT> > MT> >> MT> > MT> >>> On 8. Jul 2019, at 17:22, Paul wrote: MT> > MT> >>> MT> > MT> >>> MT> > MT> >>> MT> > MT> >>> 8 July 2019, 17:12:21, by "Michael Tuexen" : MT> > MT> >>> MT> > MT> >>>>> On 8. Jul 2019, at 15:24, Paul wrote: MT> > MT> >>>>> MT> > MT> >>>>> Hi Michael, MT> > MT> >>>>> MT> > MT> >>>>> 8 July 2019, 15:53:15, by "Michael Tuexen" : MT> > MT> >>>>> MT> > MT> >>>>>>> On 8. Jul 2019, at 12:37, Paul wrote: MT> > MT> >>>>>>> MT> > MT> >>>>>>> Hi team, MT> > MT> >>>>>>> MT> > MT> >>>>>>> Recently we had an upgrade to 12 Stable. Immediately after, we have started MT> > MT> >>>>>>> seeing some strange connection establishment timeouts to some fixed number MT> > MT> >>>>>>> of external (world) hosts. The issue was persistent and easy to reproduce. MT> > MT> >>>>>>> Thanks to a patience and dedication of our system engineer we have tracked MT> > MT> >>>>>>> this issue down to a specific commit: MT> > MT> >>>>>>> MT> > MT> >>>>>>> https://svnweb.freebsd.org/base?view=revision&revision=338053 MT> > MT> >>>>>>> MT> > MT> >>>>>>> This patch was also back-ported into 11 Stable: MT> > MT> >>>>>>> MT> > MT> >>>>>>> https://svnweb.freebsd.org/base?view=revision&revision=348435 MT> > MT> >>>>>>> MT> > MT> >>>>>>> Among other things this patch changes the timestamp allocation strategy, MT> > MT> >>>>>>> by introducing a deterministic randomness via a hash function that takes MT> > MT> >>>>>>> into account a random key as well as source address, source port, dest MT> > MT> >>>>>>> address and dest port. As the result, timestamp offsets of different MT> > MT> >>>>>>> tuples (SA,SP,DA,DP) will be wildly different and will jump from small MT> > MT> >>>>>>> to large numbers and back, as long as something in the tuple changes. MT> > MT> >>>>>> Hi Paul, MT> > MT> >>>>>> MT> > MT> >>>>>> this is correct. MT> > MT> >>>>>> MT> > MT> >>>>>> Please note that the same happens with the old method, if two hosts with MT> > MT> >>>>>> different uptimes are bind a consumer grade NAT. MT> > MT> >>>>> MT> > MT> >>>>> If NAT does not replace timestamps then yes, it should be the case. MT> > MT> >>>>> MT> > MT> >>>>>>> MT> > MT> >>>>>>> After performing various tests of hosts that produce the above mentioned MT> > MT> >>>>>>> issue we came to conclusion that there are some interesting implementations MT> > MT> >>>>>>> that drop SYN packets with timestamps smaller than the largest timestamp MT> > MT> >>>>>>> value from streams of all recent or current connections from a specific MT> > MT> >>>>>>> address. This looks as some kind of SYN flood protection. MT> > MT> >>>>>> This also breaks multiple hosts with different uptimes behind a consumer MT> > MT> >>>>>> level NAT talking to such a server. MT> > MT> >>>>>>> MT> > MT> >>>>>>> To ensure that each external host is not going to see a wild jumps of MT> > MT> >>>>>>> timestamp values I propose a patch that removes ports from the equation MT> > MT> >>>>>>> all together, when calculating the timestamp offset: MT> > MT> >>>>>>> MT> > MT> >>>>>>> Index: sys/netinet/tcp_subr.c MT> > MT> >>>>>>> =================================================================== MT> > MT> >>>>>>> --- sys/netinet/tcp_subr.c (revision 348435) MT> > MT> >>>>>>> +++ sys/netinet/tcp_subr.c (working copy) MT> > MT> >>>>>>> @@ -2224,7 +2224,22 @@ MT> > MT> >>>>>>> uint32_t MT> > MT> >>>>>>> tcp_new_ts_offset(struct in_conninfo *inc) MT> > MT> >>>>>>> { MT> > MT> >>>>>>> - return (tcp_keyed_hash(inc, V_ts_offset_secret)); MT> > MT> >>>>>>> + /* MT> > MT> >>>>>>> + * Some implementations show a strange behaviour when a wildly random MT> > MT> >>>>>>> + * timestamps allocated for different streams. It seems that only the MT> > MT> >>>>>>> + * SYN packets are affected. Observed implementations drop SYN packets MT> > MT> >>>>>>> + * with timestamps smaller than the largest timestamp value of all MT> > MT> >>>>>>> + * recent or current connections from specific a address. To mitigate MT> > MT> >>>>>>> + * this we are going to ensure that each host will always observe MT> > MT> >>>>>>> + * timestamps as increasing no matter the stream: by dropping ports MT> > MT> >>>>>>> + * from the equation. MT> > MT> >>>>>>> + */ MT> > MT> >>>>>>> + struct in_conninfo inc_copy = *inc; MT> > MT> >>>>>>> + MT> > MT> >>>>>>> + inc_copy.inc_fport = 0; MT> > MT> >>>>>>> + inc_copy.inc_lport = 0; MT> > MT> >>>>>>> + MT> > MT> >>>>>>> + return (tcp_keyed_hash(&inc_copy, V_ts_offset_secret)); MT> > MT> >>>>>>> } MT> > MT> >>>>>>> MT> > MT> >>>>>>> /* MT> > MT> >>>>>>> MT> > MT> >>>>>>> In any case, the solution of the uptime leak, implemented in rev338053 is MT> > MT> >>>>>>> not going to suffer, because a supposed attacker is currently able to use MT> > MT> >>>>>>> any fixed values of SP and DP, albeit not 0, anyway, to remove them out MT> > MT> >>>>>>> of the equation. MT> > MT> >>>>>> Can you describe how a peer can compute the uptime from two observed timestamps? MT> > MT> >>>>>> I don't see how you can do that... MT> > MT> >>>>> MT> > MT> >>>>> Supposed attacker could run a script that continuously monitors timestamps, MT> > MT> >>>>> for example via a periodic TCP connection from a fixed local port (eg 12345) MT> > MT> >>>>> and a fixed local address to the fixed victim's address and port (eg 80). MT> > MT> >>>>> Whenever large discrepancy is observed, attacker can assume that reboot has MT> > MT> >>>>> happened (due to V_ts_offset_secret re-generation), hence the received MT> > MT> >>>>> timestamp is considered an approximate point of reboot from which the uptime MT> > MT> >>>>> can be calculated, until the next reboot and so on. MT> > MT> >>>> Ahh, I see. The patch we are talking about is not intended to protect against MT> > MT> >>>> continuous monitoring, which is something you can always do. You could even MT> > MT> >>>> watch for service availability and detect reboots. A change of the local key MT> > MT> >>>> would also look similar to a reboot without a temporary loss of connectivity. MT> > MT> >>>> MT> > MT> >>>> Thanks for the clarification. MT> > MT> >>>>> MT> > MT> >>>>>>> MT> > MT> >>>>>>> There is the list of example hosts that we were able to reproduce the MT> > MT> >>>>>>> issue with: MT> > MT> >>>>>>> MT> > MT> >>>>>>> curl -v http://88.99.60.171:80 MT> > MT> >>>>>>> curl -v http://163.172.71.252:80 MT> > MT> >>>>>>> curl -v http://5.9.242.150:80 MT> > MT> >>>>>>> curl -v https://185.134.205.105:443 MT> > MT> >>>>>>> curl -v https://136.243.1.231:443 MT> > MT> >>>>>>> curl -v https://144.76.196.4:443 MT> > MT> >>>>>>> curl -v http://94.127.191.194:80 MT> > MT> >>>>>>> MT> > MT> >>>>>>> To reproduce, call curl repeatedly with a same URL some number of times. MT> > MT> >>>>>>> You are going to see some of the requests stuck in MT> > MT> >>>>>>> `* Trying XXX.XXX.XXX.XXX...` MT> > MT> >>>>>>> MT> > MT> >>>>>>> For some reason, the easiest way to reproduce the issue is with nc: MT> > MT> >>>>>>> MT> > MT> >>>>>>> $ echo "foooooo" | nc -v 88.99.60.171 80 MT> > MT> >>>>>>> MT> > MT> >>>>>>> Only a few such calls are required until one of them is stuck on connect(): MT> > MT> >>>>>>> issuing SYN packets with an exponential backoff. MT> > MT> >>>>>> Thanks for providing an end-point to test with. I'll take a look. MT> > MT> >>>>>> Just to be clear: You are running a FreeBSD client against one of the above MT> > MT> >>>>>> servers and experience the problem with the new timestamp computations. MT> > MT> >>>>>> MT> > MT> >>>>>> You are not running arbitrary clients against a FreeBSD server... MT> > MT> >>>>> MT> > MT> >>>>> We are talking about FreeBSD being the client. Peers that yield this unwanted MT> > MT> >>>>> behaviour are unknown. Little bit of tinkering showed that some of them run MT> > MT> >>>>> Debian: MT> > MT> >>>>> MT> > MT> >>>>> telnet 88.99.60.171 22 MT> > MT> >>>>> Trying 88.99.60.171... MT> > MT> >>>>> Connected to 88.99.60.171. MT> > MT> >>>>> Escape character is '^]'. MT> > MT> >>>>> SSH-2.0-OpenSSH_6.7p1 Debian-5+deb8u3 MT> > MT> >>>> Also some are hosted by Hetzner, but not all. I'll will look into MT> > MT> >>>> this tomorrow, since I'm on a deadline today (well it is 2am tomorrow MT> > MT> >>>> morning, to be precise)... MT> > MT> >>> MT> > MT> >>> Thanks a lot, I would appreciate that. MT> > MT> >> Hi Paul, MT> > MT> >> MT> > MT> >> I have looked into this. MT> > MT> >> MT> > MT> >> * The FreeBSD behaviour is the one which is specified in the last bullet item MT> > MT> >> in https://tools.ietf.org/html/rfc7323#section-5.4 MT> > MT> >> It is also the one, which is RECOMMENDED in MT> > MT> >> https://tools.ietf.org/html/rfc7323#section-7.1 MT> > MT> >> MT> > MT> >> * My NAT box (a popular one in Germany) does NOT rewrite TCP timestamps. MT> > MT> >> MT> > MT> >> This means that the host you are referring to have some sort of protection, MT> > MT> >> which makes incorrect assumptions. It will also break multiple hosts behind MT> > MT> >> a NAT. MT> > MT> >> MT> > MT> >> I can run MT> > MT> >> curl -v http://88.99.60.171:80 MT> > MT> >> in a loop without any problems from a FreeBSD head system. I tested 1000 MT> > MT> >> iterations or so. The TS.val is jumping up and down as expected. MT> > MT> >> I'm wondering why you are observing errors in this case, too. MT> > MT> >> MT> > MT> >> However, doing something like MT> > MT> >> echo "foooooo" | nc -v 88.99.60.171 80 MT> > MT> >> triggers the problem. MT> > MT> >> MT> > MT> >> So I think there is some functionality (in a middlebox or running on the host), MT> > MT> >> which incorrectly assume monotonic timestamps between multiple TCP connections MT> > MT> >> coming from the same IP address, but only in case of errors at the application layer. MT> > MT> > MT> > MT> > Yeah, exactly, some hosts seem to enable this only in case of an error in HTTP MT> > MT> > communication (some smart proxy?). However, there are some that behave this way MT> > MT> > regardless of errors, for example these: MT> > MT> > MT> > MT> > curl -v https://185.134.205.105:443 MT> > MT> > curl -v https://136.243.1.231:443 MT> > MT> Wireshark sees an Encrypted Alert in both cases. So I guess this is another indication MT> > MT> of "error at the application layer". MT> > MT> > MT> > MT> >> MT> > MT> >> Do you have any insights whether the hosts you are listed share something in MT> > MT> >> common. Some of them are hosted by Hetzner, but not all. MT> > MT> > MT> > MT> > Nope. A whole set of endpoints that we have detected so far is pretty diverse, MT> > MT> > containing a lot of different locations geographically, as well as different MT> > MT> > hosters. MT> > MT> OK. Thanks for the clarification. MT> > MT> > MT> > MT> >> MT> > MT> >> I think in general, it is the correct thing to include the port numbers in MT> > MT> >> the offset computation. We might add a sysctl variable to control the inclusion. MT> > MT> >> This would allow interworking with broken middleboxes. MT> > MT> > MT> > MT> > Yeah, I completely agree that these rare cases should not dictate the implementation. MT> > MT> > But an ability to enable a work-around via sysctl would be greatly appreciated. MT> > MT> > Currently we are unable to roll-out the upgrade across all servers because of this MT> > MT> > issue: even though it happens not so often, a lot of requests from our users MT> > MT> > get stuck or fail all together. For example, a host 185.134.205.105 is a kind of MT> > MT> > social network that our proxy servers connect to so securely access to content, MT> > MT> > such as images, on behalf of our users. MT> > MT> > MT> > MT> >> MT> > MT> >> Please note, this does not fix the case of multiple clients behind a NAT. MT> > MT> > MT> > MT> > Yeah, that's true. Fortunately we don't use NAT. MT> > MT> > MT> > MT> >> MT> > MT> >> I'm also trying to figure out how and why Linux and Windows are handling this. MT> > MT> > MT> > MT> > Thanks for bothering! MT> > MT> Will let you know what I figure out. MT> > MT> MT> > MT> Best regards MT> > MT> Michael MT> > MT> > MT> > MT> >> MT> > MT> >> Best regards MT> > MT> >> Michael MT> > MT> >> MT> > MT> >>> MT> > MT> >>>> MT> > MT> >>>> Best regards MT> > MT> >>>> Michael MT> > MT> >>>>> MT> > MT> >>>>> MT> > MT> >>>>>> MT> > MT> >>>>>> Best regards MT> > MT> >>>>>> Michael MT> > MT> >>>>>> MT> > MT> >>>>>> MT> > MT> >>>> MT> > MT> >>>> MT> > MT> >> MT> > MT> >> MT> > MT> MT> > MT> _______________________________________________ MT> > MT> freebsd-net@freebsd.org mailing list MT> > MT> https://lists.freebsd.org/mailman/listinfo/freebsd-net MT> > MT> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" MT> MT> _______________________________________________ MT> freebsd-net@freebsd.org mailing list MT> https://lists.freebsd.org/mailman/listinfo/freebsd-net MT> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Wed Jul 17 11:03:02 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 15210A991D for ; Wed, 17 Jul 2019 11:03:02 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D4C8A73B26 for ; Wed, 17 Jul 2019 11:03:01 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from [IPv6:2a02:8109:1140:c3d:7843:89ae:ec92:371] (unknown [IPv6:2a02:8109:1140:c3d:7843:89ae:ec92:371]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTPSA id 277EA72106C25; Wed, 17 Jul 2019 13:02:57 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: Issues with TCP Timestamps allocation From: Michael Tuexen In-Reply-To: <20190717100926.GA24984@hell.ukr.net> Date: Wed, 17 Jul 2019 13:02:56 +0200 Cc: freebsd-net@freebsd.org, Paul Content-Transfer-Encoding: quoted-printable Message-Id: <48817BF6-AEDD-4D28-95F8-A4D53E4999B1@freebsd.org> References: <1562579483.67527000.24rw4xi5@frv39.fwdcdn.com> <32FD061B-245C-41D2-81DE-1B4756A7173D@freebsd.org> <1562591379.369129000.gpmxvurq@frv39.fwdcdn.com> <1562599181.734953000.1l9a1d23@frv39.fwdcdn.com> <0C475A01-9BCD-4E4A-9731-09AB919CA9BE@freebsd.org> <1562676414.933145000.z3zteyqp@frv39.fwdcdn.com> <1E9F3F99-C3E9-44DD-AA70-9B11E19D4769@freebsd.org> <20190717074243.GA65665@hell.ukr.net> <20190717100926.GA24984@hell.ukr.net> To: Vitalij Satanivskij X-Mailer: Apple Mail (2.3445.104.11) X-Spam-Status: No, score=-1.7 required=5.0 tests=ALL_TRUSTED,BAYES_00, NORMAL_HTTP_TO_IP,NUMERIC_HTTP_ADDR autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jul 2019 11:03:02 -0000 > On 17. Jul 2019, at 12:09, Vitalij Satanivskij wrote: >=20 > Hello again >=20 > Michael Tuexen wrote: > MT> > On 17. Jul 2019, at 09:42, Vitalij Satanivskij = wrote: > MT> >=20 > MT> >=20 > MT> >=20 > MT> > Hello.=20 > MT> >=20 > MT> > Is there any changes about this problem > MT> >=20 > MT> >=20 > MT> > I'm using FreeBSD 12 on my desktop and can confirm problem occur = with some hosts. > MT> Can you provide a list of some of these hosts? > MT> I'll put up a change for review later today. >=20 >=20 > Here some hosts. >=20 > 5.9.242.150 https://vitagramma.com > 77.120.8.194 https://volia.com > 31.41.220.92 https://moemisto.ua > 185.5.72.33 https://fotostrana.ru OK, thanks. That might help to figure out what is broken exactly. I'm = not yet sure if it is a broken end point implementation or a middlebox making false = assumptions. >=20 > Problem can be seen by sending curl request to hosts in serial = (manual, so delay it's from few msec to few sec) >=20 > Or by using proxy on machine with parallel/serial request's (eq squid = or reverse proxy in nginx) >=20 > On system before = https://svnweb.freebsd.org/base?view=3Drevision&revision=3D338053 such = behavior not seen. >=20 > MT>=20 > MT> In the meantime you can deal with the buggy hosts by disabling the = timestamps > MT> or dropping extensions on SYN retransmits. >=20 > You meen by some code changes? No. Two options: Option 1: Drop the TCP timestamp option on the third retransmission To enable this, you configure on the client sudo sysctl -w net.inet.tcp.rexmit_drop_options=3D1 or put net.inet.tcp.rexmit_drop_options=3D1 in /etc/sysctl.conf and reboot In case of the broken host, the first SYN retransmission will happen 1 = second after the initial SYN segment, the second retransmission will happen 1.2 seconds = after the first. On the third retransmission, which happens again 1.2 seconds later, the TCP = timestamp option is dropped and the connection setup will succeed. This gives you a total = delay of 3.4 seconds on connection setup instead of the longer timeout. Option 2: Disable the TCP timestamps (and window scaling) To enable this, you configure on the client sudo sysctl -w net.inet.tcp.rfc1323=3D0 or put net.inet.tcp.rfc1323=3D0 in /etc/sysctl.conf and reboot. This disables the timestamp option and window scaling completely. This = allows you to setup the connections without any delay. However, you don't have the = benefits of the extension. Both options don't require any code changes. Best regards Michael >=20 >=20 > MT>=20 > MT> Best regards > MT> Michael > MT> >=20 > MT> >=20 > MT> >=20 > MT> > Michael Tuexen wrote: > MT> > MT>=20 > MT> > MT>=20 > MT> > MT> > On 9. Jul 2019, at 14:58, Paul wrote: > MT> > MT> >=20 > MT> > MT> > Hi Michael, > MT> > MT> >=20 > MT> > MT> > 9 July 2019, 15:34:29, by "Michael Tuexen" = : > MT> > MT> >=20 > MT> > MT> >>=20 > MT> > MT> >>=20 > MT> > MT> >>> On 8. Jul 2019, at 17:22, Paul wrote: > MT> > MT> >>>=20 > MT> > MT> >>>=20 > MT> > MT> >>>=20 > MT> > MT> >>> 8 July 2019, 17:12:21, by "Michael Tuexen" = : > MT> > MT> >>>=20 > MT> > MT> >>>>> On 8. Jul 2019, at 15:24, Paul wrote: > MT> > MT> >>>>>=20 > MT> > MT> >>>>> Hi Michael, > MT> > MT> >>>>>=20 > MT> > MT> >>>>> 8 July 2019, 15:53:15, by "Michael Tuexen" = : > MT> > MT> >>>>>=20 > MT> > MT> >>>>>>> On 8. Jul 2019, at 12:37, Paul = wrote: > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> Hi team, > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> Recently we had an upgrade to 12 Stable. Immediately = after, we have started=20 > MT> > MT> >>>>>>> seeing some strange connection establishment = timeouts to some fixed number > MT> > MT> >>>>>>> of external (world) hosts. The issue was persistent = and easy to reproduce. > MT> > MT> >>>>>>> Thanks to a patience and dedication of our system = engineer we have tracked =20 > MT> > MT> >>>>>>> this issue down to a specific commit: > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> = https://svnweb.freebsd.org/base?view=3Drevision&revision=3D338053 > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> This patch was also back-ported into 11 Stable: > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> = https://svnweb.freebsd.org/base?view=3Drevision&revision=3D348435 > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> Among other things this patch changes the timestamp = allocation strategy, > MT> > MT> >>>>>>> by introducing a deterministic randomness via a hash = function that takes > MT> > MT> >>>>>>> into account a random key as well as source address, = source port, dest > MT> > MT> >>>>>>> address and dest port. As the result, timestamp = offsets of different > MT> > MT> >>>>>>> tuples (SA,SP,DA,DP) will be wildly different and = will jump from small=20 > MT> > MT> >>>>>>> to large numbers and back, as long as something in = the tuple changes. > MT> > MT> >>>>>> Hi Paul, > MT> > MT> >>>>>>=20 > MT> > MT> >>>>>> this is correct. > MT> > MT> >>>>>>=20 > MT> > MT> >>>>>> Please note that the same happens with the old = method, if two hosts with > MT> > MT> >>>>>> different uptimes are bind a consumer grade NAT. > MT> > MT> >>>>>=20 > MT> > MT> >>>>> If NAT does not replace timestamps then yes, it should = be the case. > MT> > MT> >>>>>=20 > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> After performing various tests of hosts that produce = the above mentioned=20 > MT> > MT> >>>>>>> issue we came to conclusion that there are some = interesting implementations=20 > MT> > MT> >>>>>>> that drop SYN packets with timestamps smaller than = the largest timestamp=20 > MT> > MT> >>>>>>> value from streams of all recent or current = connections from a specific=20 > MT> > MT> >>>>>>> address. This looks as some kind of SYN flood = protection. > MT> > MT> >>>>>> This also breaks multiple hosts with different = uptimes behind a consumer > MT> > MT> >>>>>> level NAT talking to such a server. > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> To ensure that each external host is not going to = see a wild jumps of=20 > MT> > MT> >>>>>>> timestamp values I propose a patch that removes = ports from the equation > MT> > MT> >>>>>>> all together, when calculating the timestamp offset: > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> Index: sys/netinet/tcp_subr.c > MT> > MT> >>>>>>> = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > MT> > MT> >>>>>>> --- sys/netinet/tcp_subr.c (revision 348435) > MT> > MT> >>>>>>> +++ sys/netinet/tcp_subr.c (working copy) > MT> > MT> >>>>>>> @@ -2224,7 +2224,22 @@ > MT> > MT> >>>>>>> uint32_t > MT> > MT> >>>>>>> tcp_new_ts_offset(struct in_conninfo *inc) > MT> > MT> >>>>>>> { > MT> > MT> >>>>>>> - return (tcp_keyed_hash(inc, = V_ts_offset_secret)); > MT> > MT> >>>>>>> + /*=20 > MT> > MT> >>>>>>> + * Some implementations show a strange = behaviour when a wildly random=20 > MT> > MT> >>>>>>> + * timestamps allocated for different = streams. It seems that only the > MT> > MT> >>>>>>> + * SYN packets are affected. Observed = implementations drop SYN packets > MT> > MT> >>>>>>> + * with timestamps smaller than the largest = timestamp value of all=20 > MT> > MT> >>>>>>> + * recent or current connections from = specific a address. To mitigate=20 > MT> > MT> >>>>>>> + * this we are going to ensure that each = host will always observe=20 > MT> > MT> >>>>>>> + * timestamps as increasing no matter the = stream: by dropping ports > MT> > MT> >>>>>>> + * from the equation. > MT> > MT> >>>>>>> + */=20 > MT> > MT> >>>>>>> + struct in_conninfo inc_copy =3D *inc; > MT> > MT> >>>>>>> + > MT> > MT> >>>>>>> + inc_copy.inc_fport =3D 0; > MT> > MT> >>>>>>> + inc_copy.inc_lport =3D 0; > MT> > MT> >>>>>>> + > MT> > MT> >>>>>>> + return (tcp_keyed_hash(&inc_copy, = V_ts_offset_secret)); > MT> > MT> >>>>>>> } > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> /* > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> In any case, the solution of the uptime leak, = implemented in rev338053 is=20 > MT> > MT> >>>>>>> not going to suffer, because a supposed attacker is = currently able to use=20 > MT> > MT> >>>>>>> any fixed values of SP and DP, albeit not 0, anyway, = to remove them out=20 > MT> > MT> >>>>>>> of the equation. > MT> > MT> >>>>>> Can you describe how a peer can compute the uptime = from two observed timestamps? > MT> > MT> >>>>>> I don't see how you can do that... > MT> > MT> >>>>>=20 > MT> > MT> >>>>> Supposed attacker could run a script that continuously = monitors timestamps, > MT> > MT> >>>>> for example via a periodic TCP connection from a fixed = local port (eg 12345)=20 > MT> > MT> >>>>> and a fixed local address to the fixed victim's = address and port (eg 80). > MT> > MT> >>>>> Whenever large discrepancy is observed, attacker can = assume that reboot has=20 > MT> > MT> >>>>> happened (due to V_ts_offset_secret re-generation), = hence the received=20 > MT> > MT> >>>>> timestamp is considered an approximate point of reboot = from which the uptime > MT> > MT> >>>>> can be calculated, until the next reboot and so on. > MT> > MT> >>>> Ahh, I see. The patch we are talking about is not = intended to protect against > MT> > MT> >>>> continuous monitoring, which is something you can = always do. You could even > MT> > MT> >>>> watch for service availability and detect reboots. A = change of the local key > MT> > MT> >>>> would also look similar to a reboot without a temporary = loss of connectivity. > MT> > MT> >>>>=20 > MT> > MT> >>>> Thanks for the clarification. > MT> > MT> >>>>>=20 > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> There is the list of example hosts that we were able = to reproduce the=20 > MT> > MT> >>>>>>> issue with: > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> curl -v http://88.99.60.171:80 > MT> > MT> >>>>>>> curl -v http://163.172.71.252:80 > MT> > MT> >>>>>>> curl -v http://5.9.242.150:80 > MT> > MT> >>>>>>> curl -v https://185.134.205.105:443 > MT> > MT> >>>>>>> curl -v https://136.243.1.231:443 > MT> > MT> >>>>>>> curl -v https://144.76.196.4:443 > MT> > MT> >>>>>>> curl -v http://94.127.191.194:80 > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> To reproduce, call curl repeatedly with a same URL = some number of times.=20 > MT> > MT> >>>>>>> You are going to see some of the requests stuck in=20= > MT> > MT> >>>>>>> `* Trying XXX.XXX.XXX.XXX...` > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> For some reason, the easiest way to reproduce the = issue is with nc: > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> $ echo "foooooo" | nc -v 88.99.60.171 80 > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> Only a few such calls are required until one of them = is stuck on connect(): > MT> > MT> >>>>>>> issuing SYN packets with an exponential backoff. > MT> > MT> >>>>>> Thanks for providing an end-point to test with. I'll = take a look. > MT> > MT> >>>>>> Just to be clear: You are running a FreeBSD client = against one of the above > MT> > MT> >>>>>> servers and experience the problem with the new = timestamp computations. > MT> > MT> >>>>>>=20 > MT> > MT> >>>>>> You are not running arbitrary clients against a = FreeBSD server... > MT> > MT> >>>>>=20 > MT> > MT> >>>>> We are talking about FreeBSD being the client. Peers = that yield this unwanted > MT> > MT> >>>>> behaviour are unknown. Little bit of tinkering showed = that some of them run=20 > MT> > MT> >>>>> Debian: > MT> > MT> >>>>>=20 > MT> > MT> >>>>> telnet 88.99.60.171 22 > MT> > MT> >>>>> Trying 88.99.60.171... > MT> > MT> >>>>> Connected to 88.99.60.171. > MT> > MT> >>>>> Escape character is '^]'. > MT> > MT> >>>>> SSH-2.0-OpenSSH_6.7p1 Debian-5+deb8u3 > MT> > MT> >>>> Also some are hosted by Hetzner, but not all. I'll will = look into > MT> > MT> >>>> this tomorrow, since I'm on a deadline today (well it = is 2am tomorrow > MT> > MT> >>>> morning, to be precise)... > MT> > MT> >>>=20 > MT> > MT> >>> Thanks a lot, I would appreciate that. > MT> > MT> >> Hi Paul, > MT> > MT> >>=20 > MT> > MT> >> I have looked into this. > MT> > MT> >>=20 > MT> > MT> >> * The FreeBSD behaviour is the one which is specified in = the last bullet item > MT> > MT> >> in https://tools.ietf.org/html/rfc7323#section-5.4 > MT> > MT> >> It is also the one, which is RECOMMENDED in > MT> > MT> >> https://tools.ietf.org/html/rfc7323#section-7.1=20 > MT> > MT> >>=20 > MT> > MT> >> * My NAT box (a popular one in Germany) does NOT rewrite = TCP timestamps. > MT> > MT> >>=20 > MT> > MT> >> This means that the host you are referring to have some = sort of protection, > MT> > MT> >> which makes incorrect assumptions. It will also break = multiple hosts behind > MT> > MT> >> a NAT. > MT> > MT> >>=20 > MT> > MT> >> I can run > MT> > MT> >> curl -v http://88.99.60.171:80 > MT> > MT> >> in a loop without any problems from a FreeBSD head = system. I tested 1000 > MT> > MT> >> iterations or so. The TS.val is jumping up and down as = expected. > MT> > MT> >> I'm wondering why you are observing errors in this case, = too. > MT> > MT> >>=20 > MT> > MT> >> However, doing something like > MT> > MT> >> echo "foooooo" | nc -v 88.99.60.171 80 > MT> > MT> >> triggers the problem. > MT> > MT> >>=20 > MT> > MT> >> So I think there is some functionality (in a middlebox or = running on the host), > MT> > MT> >> which incorrectly assume monotonic timestamps between = multiple TCP connections > MT> > MT> >> coming from the same IP address, but only in case of = errors at the application layer. > MT> > MT> >=20 > MT> > MT> > Yeah, exactly, some hosts seem to enable this only in case = of an error in HTTP > MT> > MT> > communication (some smart proxy?). However, there are some = that behave this way > MT> > MT> > regardless of errors, for example these: > MT> > MT> >=20 > MT> > MT> > curl -v https://185.134.205.105:443 > MT> > MT> > curl -v https://136.243.1.231:443 > MT> > MT> Wireshark sees an Encrypted Alert in both cases. So I guess = this is another indication > MT> > MT> of "error at the application layer". > MT> > MT> >=20 > MT> > MT> >>=20 > MT> > MT> >> Do you have any insights whether the hosts you are listed = share something in > MT> > MT> >> common. Some of them are hosted by Hetzner, but not all. > MT> > MT> >=20 > MT> > MT> > Nope. A whole set of endpoints that we have detected so = far is pretty diverse, > MT> > MT> > containing a lot of different locations geographically, as = well as different > MT> > MT> > hosters. > MT> > MT> OK. Thanks for the clarification. > MT> > MT> >=20 > MT> > MT> >>=20 > MT> > MT> >> I think in general, it is the correct thing to include = the port numbers in > MT> > MT> >> the offset computation. We might add a sysctl variable to = control the inclusion. > MT> > MT> >> This would allow interworking with broken middleboxes. > MT> > MT> >=20 > MT> > MT> > Yeah, I completely agree that these rare cases should not = dictate the implementation. > MT> > MT> > But an ability to enable a work-around via sysctl would be = greatly appreciated. > MT> > MT> > Currently we are unable to roll-out the upgrade across all = servers because of this > MT> > MT> > issue: even though it happens not so often, a lot of = requests from our users=20 > MT> > MT> > get stuck or fail all together. For example, a host = 185.134.205.105 is a kind of > MT> > MT> > social network that our proxy servers connect to so = securely access to content, > MT> > MT> > such as images, on behalf of our users. > MT> > MT> >=20 > MT> > MT> >>=20 > MT> > MT> >> Please note, this does not fix the case of multiple = clients behind a NAT. > MT> > MT> >=20 > MT> > MT> > Yeah, that's true. Fortunately we don't use NAT. > MT> > MT> >=20 > MT> > MT> >>=20 > MT> > MT> >> I'm also trying to figure out how and why Linux and = Windows are handling this. > MT> > MT> >=20 > MT> > MT> > Thanks for bothering! > MT> > MT> Will let you know what I figure out. > MT> > MT>=20 > MT> > MT> Best regards > MT> > MT> Michael > MT> > MT> >=20 > MT> > MT> >>=20 > MT> > MT> >> Best regards > MT> > MT> >> Michael > MT> > MT> >>=20 > MT> > MT> >>>=20 > MT> > MT> >>>>=20 > MT> > MT> >>>> Best regards > MT> > MT> >>>> Michael=20 > MT> > MT> >>>>>=20 > MT> > MT> >>>>>=20 > MT> > MT> >>>>>>=20 > MT> > MT> >>>>>> Best regards > MT> > MT> >>>>>> Michael > MT> > MT> >>>>>>=20 > MT> > MT> >>>>>>=20 > MT> > MT> >>>>=20 > MT> > MT> >>>>=20 > MT> > MT> >>=20 > MT> > MT> >>=20 > MT> > MT>=20 > MT> > MT> _______________________________________________ > MT> > MT> freebsd-net@freebsd.org mailing list > MT> > MT> https://lists.freebsd.org/mailman/listinfo/freebsd-net > MT> > MT> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" > MT>=20 > MT> _______________________________________________ > MT> freebsd-net@freebsd.org mailing list > MT> https://lists.freebsd.org/mailman/listinfo/freebsd-net > MT> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Wed Jul 17 11:55:06 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 02902AA71C for ; Wed, 17 Jul 2019 11:55:06 +0000 (UTC) (envelope-from satan@ukr.net) Received: from hell.ukr.net (hell.ukr.net [212.42.67.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.ukr.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ABC4775D5D; Wed, 17 Jul 2019 11:55:05 +0000 (UTC) (envelope-from satan@ukr.net) Received: from satan by hell.ukr.net with local ID 1hniWU-000DqS-Ap ; Wed, 17 Jul 2019 14:55:02 +0300 Date: Wed, 17 Jul 2019 14:55:02 +0300 From: Vitalij Satanivskij To: Michael Tuexen Cc: Vitalij Satanivskij , freebsd-net@freebsd.org, Paul Subject: Re: Issues with TCP Timestamps allocation Message-ID: <20190717115502.GA53155@hell.ukr.net> Reply-To: satan@ukr.net References: <1562591379.369129000.gpmxvurq@frv39.fwdcdn.com> <1562599181.734953000.1l9a1d23@frv39.fwdcdn.com> <0C475A01-9BCD-4E4A-9731-09AB919CA9BE@freebsd.org> <1562676414.933145000.z3zteyqp@frv39.fwdcdn.com> <1E9F3F99-C3E9-44DD-AA70-9B11E19D4769@freebsd.org> <20190717074243.GA65665@hell.ukr.net> <20190717100926.GA24984@hell.ukr.net> <48817BF6-AEDD-4D28-95F8-A4D53E4999B1@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <48817BF6-AEDD-4D28-95F8-A4D53E4999B1@freebsd.org> User-Agent: Mutt/1.12.1 (2019-06-15) X-Rspamd-Queue-Id: ABC4775D5D X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.92 / 15.00]; NEURAL_HAM_MEDIUM(-0.99)[-0.993,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.92)[-0.924,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jul 2019 11:55:06 -0000 MT> > MT> In the meantime you can deal with the buggy hosts by disabling the timestamps MT> > MT> or dropping extensions on SYN retransmits. MT> > MT> > You meen by some code changes? MT> No. MT> MT> Two options: MT> MT> Option 1: Drop the TCP timestamp option on the third retransmission MT> To enable this, you configure on the client MT> sudo sysctl -w net.inet.tcp.rexmit_drop_options=1 MT> or put MT> net.inet.tcp.rexmit_drop_options=1 MT> in /etc/sysctl.conf MT> and reboot MT> In case of the broken host, the first SYN retransmission will happen 1 second after the MT> initial SYN segment, the second retransmission will happen 1.2 seconds after the first. On the MT> third retransmission, which happens again 1.2 seconds later, the TCP timestamp option is MT> dropped and the connection setup will succeed. This gives you a total delay of 3.4 seconds MT> on connection setup instead of the longer timeout. First Option is not working. Steel see same behave. MT> MT> Option 2: Disable the TCP timestamps (and window scaling) MT> To enable this, you configure on the client MT> sudo sysctl -w net.inet.tcp.rfc1323=0 MT> or put MT> net.inet.tcp.rfc1323=0 MT> in /etc/sysctl.conf MT> and reboot. MT> This disables the timestamp option and window scaling completely. This allows you to MT> setup the connections without any delay. However, you don't have the benefits of the MT> extension. MT> MT> Both options don't require any code changes. This option was tested some time before. Yep it's help. But overal performance of tcp networking ... Let's say to bad :( MT> Best regards MT> Michael MT> MT> MT> > MT> > MT> > MT> MT> > MT> Best regards MT> > MT> Michael MT> > MT> > MT> > MT> > MT> > MT> > MT> > MT> > Michael Tuexen wrote: MT> > MT> > MT> MT> > MT> > MT> MT> > MT> > MT> > On 9. Jul 2019, at 14:58, Paul wrote: MT> > MT> > MT> > MT> > MT> > MT> > Hi Michael, MT> > MT> > MT> > MT> > MT> > MT> > 9 July 2019, 15:34:29, by "Michael Tuexen" : MT> > MT> > MT> > MT> > MT> > MT> >> MT> > MT> > MT> >> MT> > MT> > MT> >>> On 8. Jul 2019, at 17:22, Paul wrote: MT> > MT> > MT> >>> MT> > MT> > MT> >>> MT> > MT> > MT> >>> MT> > MT> > MT> >>> 8 July 2019, 17:12:21, by "Michael Tuexen" : MT> > MT> > MT> >>> MT> > MT> > MT> >>>>> On 8. Jul 2019, at 15:24, Paul wrote: MT> > MT> > MT> >>>>> MT> > MT> > MT> >>>>> Hi Michael, MT> > MT> > MT> >>>>> MT> > MT> > MT> >>>>> 8 July 2019, 15:53:15, by "Michael Tuexen" : MT> > MT> > MT> >>>>> MT> > MT> > MT> >>>>>>> On 8. Jul 2019, at 12:37, Paul wrote: MT> > MT> > MT> >>>>>>> MT> > MT> > MT> >>>>>>> Hi team, MT> > MT> > MT> >>>>>>> MT> > MT> > MT> >>>>>>> Recently we had an upgrade to 12 Stable. Immediately after, we have started MT> > MT> > MT> >>>>>>> seeing some strange connection establishment timeouts to some fixed number MT> > MT> > MT> >>>>>>> of external (world) hosts. The issue was persistent and easy to reproduce. MT> > MT> > MT> >>>>>>> Thanks to a patience and dedication of our system engineer we have tracked MT> > MT> > MT> >>>>>>> this issue down to a specific commit: MT> > MT> > MT> >>>>>>> MT> > MT> > MT> >>>>>>> https://svnweb.freebsd.org/base?view=revision&revision=338053 MT> > MT> > MT> >>>>>>> MT> > MT> > MT> >>>>>>> This patch was also back-ported into 11 Stable: MT> > MT> > MT> >>>>>>> MT> > MT> > MT> >>>>>>> https://svnweb.freebsd.org/base?view=revision&revision=348435 MT> > MT> > MT> >>>>>>> MT> > MT> > MT> >>>>>>> Among other things this patch changes the timestamp allocation strategy, MT> > MT> > MT> >>>>>>> by introducing a deterministic randomness via a hash function that takes MT> > MT> > MT> >>>>>>> into account a random key as well as source address, source port, dest MT> > MT> > MT> >>>>>>> address and dest port. As the result, timestamp offsets of different MT> > MT> > MT> >>>>>>> tuples (SA,SP,DA,DP) will be wildly different and will jump from small MT> > MT> > MT> >>>>>>> to large numbers and back, as long as something in the tuple changes. MT> > MT> > MT> >>>>>> Hi Paul, MT> > MT> > MT> >>>>>> MT> > MT> > MT> >>>>>> this is correct. MT> > MT> > MT> >>>>>> MT> > MT> > MT> >>>>>> Please note that the same happens with the old method, if two hosts with MT> > MT> > MT> >>>>>> different uptimes are bind a consumer grade NAT. MT> > MT> > MT> >>>>> MT> > MT> > MT> >>>>> If NAT does not replace timestamps then yes, it should be the case. MT> > MT> > MT> >>>>> MT> > MT> > MT> >>>>>>> MT> > MT> > MT> >>>>>>> After performing various tests of hosts that produce the above mentioned MT> > MT> > MT> >>>>>>> issue we came to conclusion that there are some interesting implementations MT> > MT> > MT> >>>>>>> that drop SYN packets with timestamps smaller than the largest timestamp MT> > MT> > MT> >>>>>>> value from streams of all recent or current connections from a specific MT> > MT> > MT> >>>>>>> address. This looks as some kind of SYN flood protection. MT> > MT> > MT> >>>>>> This also breaks multiple hosts with different uptimes behind a consumer MT> > MT> > MT> >>>>>> level NAT talking to such a server. MT> > MT> > MT> >>>>>>> MT> > MT> > MT> >>>>>>> To ensure that each external host is not going to see a wild jumps of MT> > MT> > MT> >>>>>>> timestamp values I propose a patch that removes ports from the equation MT> > MT> > MT> >>>>>>> all together, when calculating the timestamp offset: MT> > MT> > MT> >>>>>>> MT> > MT> > MT> >>>>>>> Index: sys/netinet/tcp_subr.c MT> > MT> > MT> >>>>>>> =================================================================== MT> > MT> > MT> >>>>>>> --- sys/netinet/tcp_subr.c (revision 348435) MT> > MT> > MT> >>>>>>> +++ sys/netinet/tcp_subr.c (working copy) MT> > MT> > MT> >>>>>>> @@ -2224,7 +2224,22 @@ MT> > MT> > MT> >>>>>>> uint32_t MT> > MT> > MT> >>>>>>> tcp_new_ts_offset(struct in_conninfo *inc) MT> > MT> > MT> >>>>>>> { MT> > MT> > MT> >>>>>>> - return (tcp_keyed_hash(inc, V_ts_offset_secret)); MT> > MT> > MT> >>>>>>> + /* MT> > MT> > MT> >>>>>>> + * Some implementations show a strange behaviour when a wildly random MT> > MT> > MT> >>>>>>> + * timestamps allocated for different streams. It seems that only the MT> > MT> > MT> >>>>>>> + * SYN packets are affected. Observed implementations drop SYN packets MT> > MT> > MT> >>>>>>> + * with timestamps smaller than the largest timestamp value of all MT> > MT> > MT> >>>>>>> + * recent or current connections from specific a address. To mitigate MT> > MT> > MT> >>>>>>> + * this we are going to ensure that each host will always observe MT> > MT> > MT> >>>>>>> + * timestamps as increasing no matter the stream: by dropping ports MT> > MT> > MT> >>>>>>> + * from the equation. MT> > MT> > MT> >>>>>>> + */ MT> > MT> > MT> >>>>>>> + struct in_conninfo inc_copy = *inc; MT> > MT> > MT> >>>>>>> + MT> > MT> > MT> >>>>>>> + inc_copy.inc_fport = 0; MT> > MT> > MT> >>>>>>> + inc_copy.inc_lport = 0; MT> > MT> > MT> >>>>>>> + MT> > MT> > MT> >>>>>>> + return (tcp_keyed_hash(&inc_copy, V_ts_offset_secret)); MT> > MT> > MT> >>>>>>> } MT> > MT> > MT> >>>>>>> MT> > MT> > MT> >>>>>>> /* MT> > MT> > MT> >>>>>>> MT> > MT> > MT> >>>>>>> In any case, the solution of the uptime leak, implemented in rev338053 is MT> > MT> > MT> >>>>>>> not going to suffer, because a supposed attacker is currently able to use MT> > MT> > MT> >>>>>>> any fixed values of SP and DP, albeit not 0, anyway, to remove them out MT> > MT> > MT> >>>>>>> of the equation. MT> > MT> > MT> >>>>>> Can you describe how a peer can compute the uptime from two observed timestamps? MT> > MT> > MT> >>>>>> I don't see how you can do that... MT> > MT> > MT> >>>>> MT> > MT> > MT> >>>>> Supposed attacker could run a script that continuously monitors timestamps, MT> > MT> > MT> >>>>> for example via a periodic TCP connection from a fixed local port (eg 12345) MT> > MT> > MT> >>>>> and a fixed local address to the fixed victim's address and port (eg 80). MT> > MT> > MT> >>>>> Whenever large discrepancy is observed, attacker can assume that reboot has MT> > MT> > MT> >>>>> happened (due to V_ts_offset_secret re-generation), hence the received MT> > MT> > MT> >>>>> timestamp is considered an approximate point of reboot from which the uptime MT> > MT> > MT> >>>>> can be calculated, until the next reboot and so on. MT> > MT> > MT> >>>> Ahh, I see. The patch we are talking about is not intended to protect against MT> > MT> > MT> >>>> continuous monitoring, which is something you can always do. You could even MT> > MT> > MT> >>>> watch for service availability and detect reboots. A change of the local key MT> > MT> > MT> >>>> would also look similar to a reboot without a temporary loss of connectivity. MT> > MT> > MT> >>>> MT> > MT> > MT> >>>> Thanks for the clarification. MT> > MT> > MT> >>>>> MT> > MT> > MT> >>>>>>> MT> > MT> > MT> >>>>>>> There is the list of example hosts that we were able to reproduce the MT> > MT> > MT> >>>>>>> issue with: MT> > MT> > MT> >>>>>>> MT> > MT> > MT> >>>>>>> curl -v http://88.99.60.171:80 MT> > MT> > MT> >>>>>>> curl -v http://163.172.71.252:80 MT> > MT> > MT> >>>>>>> curl -v http://5.9.242.150:80 MT> > MT> > MT> >>>>>>> curl -v https://185.134.205.105:443 MT> > MT> > MT> >>>>>>> curl -v https://136.243.1.231:443 MT> > MT> > MT> >>>>>>> curl -v https://144.76.196.4:443 MT> > MT> > MT> >>>>>>> curl -v http://94.127.191.194:80 MT> > MT> > MT> >>>>>>> MT> > MT> > MT> >>>>>>> To reproduce, call curl repeatedly with a same URL some number of times. MT> > MT> > MT> >>>>>>> You are going to see some of the requests stuck in MT> > MT> > MT> >>>>>>> `* Trying XXX.XXX.XXX.XXX...` MT> > MT> > MT> >>>>>>> MT> > MT> > MT> >>>>>>> For some reason, the easiest way to reproduce the issue is with nc: MT> > MT> > MT> >>>>>>> MT> > MT> > MT> >>>>>>> $ echo "foooooo" | nc -v 88.99.60.171 80 MT> > MT> > MT> >>>>>>> MT> > MT> > MT> >>>>>>> Only a few such calls are required until one of them is stuck on connect(): MT> > MT> > MT> >>>>>>> issuing SYN packets with an exponential backoff. MT> > MT> > MT> >>>>>> Thanks for providing an end-point to test with. I'll take a look. MT> > MT> > MT> >>>>>> Just to be clear: You are running a FreeBSD client against one of the above MT> > MT> > MT> >>>>>> servers and experience the problem with the new timestamp computations. MT> > MT> > MT> >>>>>> MT> > MT> > MT> >>>>>> You are not running arbitrary clients against a FreeBSD server... MT> > MT> > MT> >>>>> MT> > MT> > MT> >>>>> We are talking about FreeBSD being the client. Peers that yield this unwanted MT> > MT> > MT> >>>>> behaviour are unknown. Little bit of tinkering showed that some of them run MT> > MT> > MT> >>>>> Debian: MT> > MT> > MT> >>>>> MT> > MT> > MT> >>>>> telnet 88.99.60.171 22 MT> > MT> > MT> >>>>> Trying 88.99.60.171... MT> > MT> > MT> >>>>> Connected to 88.99.60.171. MT> > MT> > MT> >>>>> Escape character is '^]'. MT> > MT> > MT> >>>>> SSH-2.0-OpenSSH_6.7p1 Debian-5+deb8u3 MT> > MT> > MT> >>>> Also some are hosted by Hetzner, but not all. I'll will look into MT> > MT> > MT> >>>> this tomorrow, since I'm on a deadline today (well it is 2am tomorrow MT> > MT> > MT> >>>> morning, to be precise)... MT> > MT> > MT> >>> MT> > MT> > MT> >>> Thanks a lot, I would appreciate that. MT> > MT> > MT> >> Hi Paul, MT> > MT> > MT> >> MT> > MT> > MT> >> I have looked into this. MT> > MT> > MT> >> MT> > MT> > MT> >> * The FreeBSD behaviour is the one which is specified in the last bullet item MT> > MT> > MT> >> in https://tools.ietf.org/html/rfc7323#section-5.4 MT> > MT> > MT> >> It is also the one, which is RECOMMENDED in MT> > MT> > MT> >> https://tools.ietf.org/html/rfc7323#section-7.1 MT> > MT> > MT> >> MT> > MT> > MT> >> * My NAT box (a popular one in Germany) does NOT rewrite TCP timestamps. MT> > MT> > MT> >> MT> > MT> > MT> >> This means that the host you are referring to have some sort of protection, MT> > MT> > MT> >> which makes incorrect assumptions. It will also break multiple hosts behind MT> > MT> > MT> >> a NAT. MT> > MT> > MT> >> MT> > MT> > MT> >> I can run MT> > MT> > MT> >> curl -v http://88.99.60.171:80 MT> > MT> > MT> >> in a loop without any problems from a FreeBSD head system. I tested 1000 MT> > MT> > MT> >> iterations or so. The TS.val is jumping up and down as expected. MT> > MT> > MT> >> I'm wondering why you are observing errors in this case, too. MT> > MT> > MT> >> MT> > MT> > MT> >> However, doing something like MT> > MT> > MT> >> echo "foooooo" | nc -v 88.99.60.171 80 MT> > MT> > MT> >> triggers the problem. MT> > MT> > MT> >> MT> > MT> > MT> >> So I think there is some functionality (in a middlebox or running on the host), MT> > MT> > MT> >> which incorrectly assume monotonic timestamps between multiple TCP connections MT> > MT> > MT> >> coming from the same IP address, but only in case of errors at the application layer. MT> > MT> > MT> > MT> > MT> > MT> > Yeah, exactly, some hosts seem to enable this only in case of an error in HTTP MT> > MT> > MT> > communication (some smart proxy?). However, there are some that behave this way MT> > MT> > MT> > regardless of errors, for example these: MT> > MT> > MT> > MT> > MT> > MT> > curl -v https://185.134.205.105:443 MT> > MT> > MT> > curl -v https://136.243.1.231:443 MT> > MT> > MT> Wireshark sees an Encrypted Alert in both cases. So I guess this is another indication MT> > MT> > MT> of "error at the application layer". MT> > MT> > MT> > MT> > MT> > MT> >> MT> > MT> > MT> >> Do you have any insights whether the hosts you are listed share something in MT> > MT> > MT> >> common. Some of them are hosted by Hetzner, but not all. MT> > MT> > MT> > MT> > MT> > MT> > Nope. A whole set of endpoints that we have detected so far is pretty diverse, MT> > MT> > MT> > containing a lot of different locations geographically, as well as different MT> > MT> > MT> > hosters. MT> > MT> > MT> OK. Thanks for the clarification. MT> > MT> > MT> > MT> > MT> > MT> >> MT> > MT> > MT> >> I think in general, it is the correct thing to include the port numbers in MT> > MT> > MT> >> the offset computation. We might add a sysctl variable to control the inclusion. MT> > MT> > MT> >> This would allow interworking with broken middleboxes. MT> > MT> > MT> > MT> > MT> > MT> > Yeah, I completely agree that these rare cases should not dictate the implementation. MT> > MT> > MT> > But an ability to enable a work-around via sysctl would be greatly appreciated. MT> > MT> > MT> > Currently we are unable to roll-out the upgrade across all servers because of this MT> > MT> > MT> > issue: even though it happens not so often, a lot of requests from our users MT> > MT> > MT> > get stuck or fail all together. For example, a host 185.134.205.105 is a kind of MT> > MT> > MT> > social network that our proxy servers connect to so securely access to content, MT> > MT> > MT> > such as images, on behalf of our users. MT> > MT> > MT> > MT> > MT> > MT> >> MT> > MT> > MT> >> Please note, this does not fix the case of multiple clients behind a NAT. MT> > MT> > MT> > MT> > MT> > MT> > Yeah, that's true. Fortunately we don't use NAT. MT> > MT> > MT> > MT> > MT> > MT> >> MT> > MT> > MT> >> I'm also trying to figure out how and why Linux and Windows are handling this. MT> > MT> > MT> > MT> > MT> > MT> > Thanks for bothering! MT> > MT> > MT> Will let you know what I figure out. MT> > MT> > MT> MT> > MT> > MT> Best regards MT> > MT> > MT> Michael MT> > MT> > MT> > MT> > MT> > MT> >> MT> > MT> > MT> >> Best regards MT> > MT> > MT> >> Michael MT> > MT> > MT> >> MT> > MT> > MT> >>> MT> > MT> > MT> >>>> MT> > MT> > MT> >>>> Best regards MT> > MT> > MT> >>>> Michael MT> > MT> > MT> >>>>> MT> > MT> > MT> >>>>> MT> > MT> > MT> >>>>>> MT> > MT> > MT> >>>>>> Best regards MT> > MT> > MT> >>>>>> Michael MT> > MT> > MT> >>>>>> MT> > MT> > MT> >>>>>> MT> > MT> > MT> >>>> MT> > MT> > MT> >>>> MT> > MT> > MT> >> MT> > MT> > MT> >> MT> > MT> > MT> MT> > MT> > MT> _______________________________________________ MT> > MT> > MT> freebsd-net@freebsd.org mailing list MT> > MT> > MT> https://lists.freebsd.org/mailman/listinfo/freebsd-net MT> > MT> > MT> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" MT> > MT> MT> > MT> _______________________________________________ MT> > MT> freebsd-net@freebsd.org mailing list MT> > MT> https://lists.freebsd.org/mailman/listinfo/freebsd-net MT> > MT> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" MT> From owner-freebsd-net@freebsd.org Wed Jul 17 12:16:42 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 798EBAB482 for ; Wed, 17 Jul 2019 12:16:42 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (mail-n.franken.de [193.175.24.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0F1B876BD2 for ; Wed, 17 Jul 2019 12:16:41 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from [IPv6:2a02:8109:1140:c3d:7843:89ae:ec92:371] (unknown [IPv6:2a02:8109:1140:c3d:7843:89ae:ec92:371]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTPSA id 37F3C72106C28; Wed, 17 Jul 2019 14:16:37 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: Issues with TCP Timestamps allocation From: Michael Tuexen In-Reply-To: <20190717115502.GA53155@hell.ukr.net> Date: Wed, 17 Jul 2019 14:16:36 +0200 Cc: Paul , freebsd-net@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <8763FDC7-8B71-41C3-8D1C-10416DA9A871@freebsd.org> References: <1562591379.369129000.gpmxvurq@frv39.fwdcdn.com> <1562599181.734953000.1l9a1d23@frv39.fwdcdn.com> <0C475A01-9BCD-4E4A-9731-09AB919CA9BE@freebsd.org> <1562676414.933145000.z3zteyqp@frv39.fwdcdn.com> <1E9F3F99-C3E9-44DD-AA70-9B11E19D4769@freebsd.org> <20190717074243.GA65665@hell.ukr.net> <20190717100926.GA24984@hell.ukr.net> <48817BF6-AEDD-4D28-95F8-A4D53E4999B1@freebsd.org> <20190717115502.GA53155@hell.ukr.net> To: Vitalij Satanivskij X-Mailer: Apple Mail (2.3445.104.11) X-Spam-Status: No, score=-1.7 required=5.0 tests=ALL_TRUSTED,BAYES_00, NORMAL_HTTP_TO_IP,NUMERIC_HTTP_ADDR autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-Rspamd-Queue-Id: 0F1B876BD2 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.97)[-0.971,0]; ASN(0.00)[asn:680, ipnet:193.174.0.0/15, country:DE] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jul 2019 12:16:42 -0000 > On 17. Jul 2019, at 13:55, Vitalij Satanivskij wrote: >=20 > MT> > MT> In the meantime you can deal with the buggy hosts by = disabling the timestamps > MT> > MT> or dropping extensions on SYN retransmits. > MT> >=20 > MT> > You meen by some code changes? > MT> No. > MT>=20 > MT> Two options: > MT>=20 > MT> Option 1: Drop the TCP timestamp option on the third = retransmission > MT> To enable this, you configure on the client > MT> sudo sysctl -w net.inet.tcp.rexmit_drop_options=3D1 > MT> or put > MT> net.inet.tcp.rexmit_drop_options=3D1 > MT> in /etc/sysctl.conf > MT> and reboot > MT> In case of the broken host, the first SYN retransmission will = happen 1 second after the > MT> initial SYN segment, the second retransmission will happen 1.2 = seconds after the first. On the > MT> third retransmission, which happens again 1.2 seconds later, the = TCP timestamp option is > MT> dropped and the connection setup will succeed. This gives you a = total delay of 3.4 seconds > MT> on connection setup instead of the longer timeout. >=20 > First Option is not working. Steel see same behave. Interesting. It works for me: tuexen@head:~ % curl https://vitagramma.com > /dev/null % Total % Received % Xferd Average Speed Time Time Time = Current Dload Upload Total Spent Left = Speed 100 18265 0 18265 0 0 33637 0 --:--:-- --:--:-- --:--:-- = 33575 tuexen@head:~ % curl https://vitagramma.com > /dev/null % Total % Received % Xferd Average Speed Time Time Time = Current Dload Upload Total Spent Left = Speed 100 18265 0 18265 0 0 4834 0 --:--:-- 0:00:03 --:--:-- = 4833 tuexen@head:~ % curl https://vitagramma.com > /dev/null % Total % Received % Xferd Average Speed Time Time Time = Current Dload Upload Total Spent Left = Speed 100 18265 0 18265 0 0 35813 0 --:--:-- --:--:-- --:--:-- = 35813 tuexen@head:~ % time curl https://vitagramma.com > /dev/null % Total % Received % Xferd Average Speed Time Time Time = Current Dload Upload Total Spent Left = Speed 100 18265 0 18265 0 0 48320 0 --:--:-- --:--:-- --:--:-- = 48320 0.012u 0.031s 0:00.39 10.2% 140+245k 0+0io 0pf+0w tuexen@head:~ % time curl https://vitagramma.com > /dev/null % Total % Received % Xferd Average Speed Time Time Time = Current Dload Upload Total Spent Left = Speed 100 18265 0 18265 0 0 4592 0 --:--:-- 0:00:03 --:--:-- = 4591 0.031u 0.010s 0:03.99 1.0% 80+140k 0+0io 0pf+0w tuexen@head:~ % curl https://vitagramma.com > /dev/null % Total % Received % Xferd Average Speed Time Time Time = Current Dload Upload Total Spent Left = Speed 100 18265 0 18265 0 0 37815 0 --:--:-- --:--:-- --:--:-- = 37737 tuexen@head:~ % curl https://vitagramma.com > /dev/null % Total % Received % Xferd Average Speed Time Time Time = Current Dload Upload Total Spent Left = Speed 100 18265 0 18265 0 0 27261 0 --:--:-- --:--:-- --:--:-- = 27220 tuexen@head:~ % curl https://vitagramma.com > /dev/null % Total % Received % Xferd Average Speed Time Time Time = Current Dload Upload Total Spent Left = Speed 100 18265 0 18265 0 0 4533 0 --:--:-- 0:00:04 --:--:-- = 4533 tuexen@head:~ % curl https://vitagramma.com > /dev/null % Total % Received % Xferd Average Speed Time Time Time = Current Dload Upload Total Spent Left = Speed 100 18265 0 18265 0 0 48320 0 --:--:-- --:--:-- --:--:-- = 48192 tuexen@head:~ % curl https://vitagramma.com > /dev/null % Total % Received % Xferd Average Speed Time Time Time = Current Dload Upload Total Spent Left = Speed 100 18265 0 18265 0 0 4746 0 --:--:-- 0:00:03 --:--:-- = 4745 tuexen@head:~ % curl https://vitagramma.com > /dev/null % Total % Received % Xferd Average Speed Time Time Time = Current Dload Upload Total Spent Left = Speed 100 18265 0 18265 0 0 4500 0 --:--:-- 0:00:04 --:--:-- = 4767 tuexen@head:~ % curl https://vitagramma.com > /dev/null % Total % Received % Xferd Average Speed Time Time Time = Current Dload Upload Total Spent Left = Speed 100 18265 0 18265 0 0 4726 0 --:--:-- 0:00:03 --:--:-- = 4726 tuexen@head:~ % curl https://vitagramma.com > /dev/null % Total % Received % Xferd Average Speed Time Time Time = Current Dload Upload Total Spent Left = Speed 100 18265 0 18265 0 0 34268 0 --:--:-- --:--:-- --:--:-- = 34332 tuexen@head:~ %=20 So it either works immediately or with a delay of 3 to 4 seconds... Best regards Michael >=20 >=20 > MT>=20 > MT> Option 2: Disable the TCP timestamps (and window scaling) > MT> To enable this, you configure on the client > MT> sudo sysctl -w net.inet.tcp.rfc1323=3D0 > MT> or put > MT> net.inet.tcp.rfc1323=3D0 > MT> in /etc/sysctl.conf > MT> and reboot. > MT> This disables the timestamp option and window scaling completely. = This allows you to > MT> setup the connections without any delay. However, you don't have = the benefits of the > MT> extension. > MT>=20 > MT> Both options don't require any code changes. >=20 > This option was tested some time before. Yep it's help. But overal = performance of tcp networking ... Let's say to bad :( >=20 >=20 >=20 >=20 > MT> Best regards > MT> Michael > MT>=20 > MT>=20 > MT> >=20 > MT> >=20 > MT> > MT>=20 > MT> > MT> Best regards > MT> > MT> Michael > MT> > MT> >=20 > MT> > MT> >=20 > MT> > MT> >=20 > MT> > MT> > Michael Tuexen wrote: > MT> > MT> > MT>=20 > MT> > MT> > MT>=20 > MT> > MT> > MT> > On 9. Jul 2019, at 14:58, Paul = wrote: > MT> > MT> > MT> >=20 > MT> > MT> > MT> > Hi Michael, > MT> > MT> > MT> >=20 > MT> > MT> > MT> > 9 July 2019, 15:34:29, by "Michael Tuexen" = : > MT> > MT> > MT> >=20 > MT> > MT> > MT> >>=20 > MT> > MT> > MT> >>=20 > MT> > MT> > MT> >>> On 8. Jul 2019, at 17:22, Paul = wrote: > MT> > MT> > MT> >>>=20 > MT> > MT> > MT> >>>=20 > MT> > MT> > MT> >>>=20 > MT> > MT> > MT> >>> 8 July 2019, 17:12:21, by "Michael Tuexen" = : > MT> > MT> > MT> >>>=20 > MT> > MT> > MT> >>>>> On 8. Jul 2019, at 15:24, Paul = wrote: > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> >>>>> Hi Michael, > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> >>>>> 8 July 2019, 15:53:15, by "Michael Tuexen" = : > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> >>>>>>> On 8. Jul 2019, at 12:37, Paul = wrote: > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> >>>>>>> Hi team, > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> >>>>>>> Recently we had an upgrade to 12 Stable. = Immediately after, we have started=20 > MT> > MT> > MT> >>>>>>> seeing some strange connection establishment = timeouts to some fixed number > MT> > MT> > MT> >>>>>>> of external (world) hosts. The issue was = persistent and easy to reproduce. > MT> > MT> > MT> >>>>>>> Thanks to a patience and dedication of our = system engineer we have tracked =20 > MT> > MT> > MT> >>>>>>> this issue down to a specific commit: > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> >>>>>>> = https://svnweb.freebsd.org/base?view=3Drevision&revision=3D338053 > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> >>>>>>> This patch was also back-ported into 11 = Stable: > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> >>>>>>> = https://svnweb.freebsd.org/base?view=3Drevision&revision=3D348435 > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> >>>>>>> Among other things this patch changes the = timestamp allocation strategy, > MT> > MT> > MT> >>>>>>> by introducing a deterministic randomness via = a hash function that takes > MT> > MT> > MT> >>>>>>> into account a random key as well as source = address, source port, dest > MT> > MT> > MT> >>>>>>> address and dest port. As the result, = timestamp offsets of different > MT> > MT> > MT> >>>>>>> tuples (SA,SP,DA,DP) will be wildly different = and will jump from small=20 > MT> > MT> > MT> >>>>>>> to large numbers and back, as long as = something in the tuple changes. > MT> > MT> > MT> >>>>>> Hi Paul, > MT> > MT> > MT> >>>>>>=20 > MT> > MT> > MT> >>>>>> this is correct. > MT> > MT> > MT> >>>>>>=20 > MT> > MT> > MT> >>>>>> Please note that the same happens with the old = method, if two hosts with > MT> > MT> > MT> >>>>>> different uptimes are bind a consumer grade = NAT. > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> >>>>> If NAT does not replace timestamps then yes, it = should be the case. > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> >>>>>>> After performing various tests of hosts that = produce the above mentioned=20 > MT> > MT> > MT> >>>>>>> issue we came to conclusion that there are = some interesting implementations=20 > MT> > MT> > MT> >>>>>>> that drop SYN packets with timestamps smaller = than the largest timestamp=20 > MT> > MT> > MT> >>>>>>> value from streams of all recent or current = connections from a specific=20 > MT> > MT> > MT> >>>>>>> address. This looks as some kind of SYN flood = protection. > MT> > MT> > MT> >>>>>> This also breaks multiple hosts with different = uptimes behind a consumer > MT> > MT> > MT> >>>>>> level NAT talking to such a server. > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> >>>>>>> To ensure that each external host is not going = to see a wild jumps of=20 > MT> > MT> > MT> >>>>>>> timestamp values I propose a patch that = removes ports from the equation > MT> > MT> > MT> >>>>>>> all together, when calculating the timestamp = offset: > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> >>>>>>> Index: sys/netinet/tcp_subr.c > MT> > MT> > MT> >>>>>>> = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > MT> > MT> > MT> >>>>>>> --- sys/netinet/tcp_subr.c (revision = 348435) > MT> > MT> > MT> >>>>>>> +++ sys/netinet/tcp_subr.c (working copy) > MT> > MT> > MT> >>>>>>> @@ -2224,7 +2224,22 @@ > MT> > MT> > MT> >>>>>>> uint32_t > MT> > MT> > MT> >>>>>>> tcp_new_ts_offset(struct in_conninfo *inc) > MT> > MT> > MT> >>>>>>> { > MT> > MT> > MT> >>>>>>> - return (tcp_keyed_hash(inc, = V_ts_offset_secret)); > MT> > MT> > MT> >>>>>>> + /*=20 > MT> > MT> > MT> >>>>>>> + * Some implementations show a = strange behaviour when a wildly random=20 > MT> > MT> > MT> >>>>>>> + * timestamps allocated for different = streams. It seems that only the > MT> > MT> > MT> >>>>>>> + * SYN packets are affected. Observed = implementations drop SYN packets > MT> > MT> > MT> >>>>>>> + * with timestamps smaller than the = largest timestamp value of all=20 > MT> > MT> > MT> >>>>>>> + * recent or current connections from = specific a address. To mitigate=20 > MT> > MT> > MT> >>>>>>> + * this we are going to ensure that = each host will always observe=20 > MT> > MT> > MT> >>>>>>> + * timestamps as increasing no matter = the stream: by dropping ports > MT> > MT> > MT> >>>>>>> + * from the equation. > MT> > MT> > MT> >>>>>>> + */=20 > MT> > MT> > MT> >>>>>>> + struct in_conninfo inc_copy =3D *inc; > MT> > MT> > MT> >>>>>>> + > MT> > MT> > MT> >>>>>>> + inc_copy.inc_fport =3D 0; > MT> > MT> > MT> >>>>>>> + inc_copy.inc_lport =3D 0; > MT> > MT> > MT> >>>>>>> + > MT> > MT> > MT> >>>>>>> + return (tcp_keyed_hash(&inc_copy, = V_ts_offset_secret)); > MT> > MT> > MT> >>>>>>> } > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> >>>>>>> /* > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> >>>>>>> In any case, the solution of the uptime leak, = implemented in rev338053 is=20 > MT> > MT> > MT> >>>>>>> not going to suffer, because a supposed = attacker is currently able to use=20 > MT> > MT> > MT> >>>>>>> any fixed values of SP and DP, albeit not 0, = anyway, to remove them out=20 > MT> > MT> > MT> >>>>>>> of the equation. > MT> > MT> > MT> >>>>>> Can you describe how a peer can compute the = uptime from two observed timestamps? > MT> > MT> > MT> >>>>>> I don't see how you can do that... > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> >>>>> Supposed attacker could run a script that = continuously monitors timestamps, > MT> > MT> > MT> >>>>> for example via a periodic TCP connection from a = fixed local port (eg 12345)=20 > MT> > MT> > MT> >>>>> and a fixed local address to the fixed victim's = address and port (eg 80). > MT> > MT> > MT> >>>>> Whenever large discrepancy is observed, attacker = can assume that reboot has=20 > MT> > MT> > MT> >>>>> happened (due to V_ts_offset_secret = re-generation), hence the received=20 > MT> > MT> > MT> >>>>> timestamp is considered an approximate point of = reboot from which the uptime > MT> > MT> > MT> >>>>> can be calculated, until the next reboot and so = on. > MT> > MT> > MT> >>>> Ahh, I see. The patch we are talking about is not = intended to protect against > MT> > MT> > MT> >>>> continuous monitoring, which is something you can = always do. You could even > MT> > MT> > MT> >>>> watch for service availability and detect = reboots. A change of the local key > MT> > MT> > MT> >>>> would also look similar to a reboot without a = temporary loss of connectivity. > MT> > MT> > MT> >>>>=20 > MT> > MT> > MT> >>>> Thanks for the clarification. > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> >>>>>>> There is the list of example hosts that we = were able to reproduce the=20 > MT> > MT> > MT> >>>>>>> issue with: > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> >>>>>>> curl -v http://88.99.60.171:80 > MT> > MT> > MT> >>>>>>> curl -v http://163.172.71.252:80 > MT> > MT> > MT> >>>>>>> curl -v http://5.9.242.150:80 > MT> > MT> > MT> >>>>>>> curl -v https://185.134.205.105:443 > MT> > MT> > MT> >>>>>>> curl -v https://136.243.1.231:443 > MT> > MT> > MT> >>>>>>> curl -v https://144.76.196.4:443 > MT> > MT> > MT> >>>>>>> curl -v http://94.127.191.194:80 > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> >>>>>>> To reproduce, call curl repeatedly with a same = URL some number of times.=20 > MT> > MT> > MT> >>>>>>> You are going to see some of the requests = stuck in=20 > MT> > MT> > MT> >>>>>>> `* Trying XXX.XXX.XXX.XXX...` > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> >>>>>>> For some reason, the easiest way to reproduce = the issue is with nc: > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> >>>>>>> $ echo "foooooo" | nc -v 88.99.60.171 80 > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> >>>>>>> Only a few such calls are required until one = of them is stuck on connect(): > MT> > MT> > MT> >>>>>>> issuing SYN packets with an exponential = backoff. > MT> > MT> > MT> >>>>>> Thanks for providing an end-point to test with. = I'll take a look. > MT> > MT> > MT> >>>>>> Just to be clear: You are running a FreeBSD = client against one of the above > MT> > MT> > MT> >>>>>> servers and experience the problem with the new = timestamp computations. > MT> > MT> > MT> >>>>>>=20 > MT> > MT> > MT> >>>>>> You are not running arbitrary clients against a = FreeBSD server... > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> >>>>> We are talking about FreeBSD being the client. = Peers that yield this unwanted > MT> > MT> > MT> >>>>> behaviour are unknown. Little bit of tinkering = showed that some of them run=20 > MT> > MT> > MT> >>>>> Debian: > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> >>>>> telnet 88.99.60.171 22 > MT> > MT> > MT> >>>>> Trying 88.99.60.171... > MT> > MT> > MT> >>>>> Connected to 88.99.60.171. > MT> > MT> > MT> >>>>> Escape character is '^]'. > MT> > MT> > MT> >>>>> SSH-2.0-OpenSSH_6.7p1 Debian-5+deb8u3 > MT> > MT> > MT> >>>> Also some are hosted by Hetzner, but not all. = I'll will look into > MT> > MT> > MT> >>>> this tomorrow, since I'm on a deadline today = (well it is 2am tomorrow > MT> > MT> > MT> >>>> morning, to be precise)... > MT> > MT> > MT> >>>=20 > MT> > MT> > MT> >>> Thanks a lot, I would appreciate that. > MT> > MT> > MT> >> Hi Paul, > MT> > MT> > MT> >>=20 > MT> > MT> > MT> >> I have looked into this. > MT> > MT> > MT> >>=20 > MT> > MT> > MT> >> * The FreeBSD behaviour is the one which is = specified in the last bullet item > MT> > MT> > MT> >> in https://tools.ietf.org/html/rfc7323#section-5.4 > MT> > MT> > MT> >> It is also the one, which is RECOMMENDED in > MT> > MT> > MT> >> https://tools.ietf.org/html/rfc7323#section-7.1=20 > MT> > MT> > MT> >>=20 > MT> > MT> > MT> >> * My NAT box (a popular one in Germany) does NOT = rewrite TCP timestamps. > MT> > MT> > MT> >>=20 > MT> > MT> > MT> >> This means that the host you are referring to have = some sort of protection, > MT> > MT> > MT> >> which makes incorrect assumptions. It will also = break multiple hosts behind > MT> > MT> > MT> >> a NAT. > MT> > MT> > MT> >>=20 > MT> > MT> > MT> >> I can run > MT> > MT> > MT> >> curl -v http://88.99.60.171:80 > MT> > MT> > MT> >> in a loop without any problems from a FreeBSD head = system. I tested 1000 > MT> > MT> > MT> >> iterations or so. The TS.val is jumping up and down = as expected. > MT> > MT> > MT> >> I'm wondering why you are observing errors in this = case, too. > MT> > MT> > MT> >>=20 > MT> > MT> > MT> >> However, doing something like > MT> > MT> > MT> >> echo "foooooo" | nc -v 88.99.60.171 80 > MT> > MT> > MT> >> triggers the problem. > MT> > MT> > MT> >>=20 > MT> > MT> > MT> >> So I think there is some functionality (in a = middlebox or running on the host), > MT> > MT> > MT> >> which incorrectly assume monotonic timestamps = between multiple TCP connections > MT> > MT> > MT> >> coming from the same IP address, but only in case = of errors at the application layer. > MT> > MT> > MT> >=20 > MT> > MT> > MT> > Yeah, exactly, some hosts seem to enable this only = in case of an error in HTTP > MT> > MT> > MT> > communication (some smart proxy?). However, there = are some that behave this way > MT> > MT> > MT> > regardless of errors, for example these: > MT> > MT> > MT> >=20 > MT> > MT> > MT> > curl -v https://185.134.205.105:443 > MT> > MT> > MT> > curl -v https://136.243.1.231:443 > MT> > MT> > MT> Wireshark sees an Encrypted Alert in both cases. So I = guess this is another indication > MT> > MT> > MT> of "error at the application layer". > MT> > MT> > MT> >=20 > MT> > MT> > MT> >>=20 > MT> > MT> > MT> >> Do you have any insights whether the hosts you are = listed share something in > MT> > MT> > MT> >> common. Some of them are hosted by Hetzner, but not = all. > MT> > MT> > MT> >=20 > MT> > MT> > MT> > Nope. A whole set of endpoints that we have detected = so far is pretty diverse, > MT> > MT> > MT> > containing a lot of different locations = geographically, as well as different > MT> > MT> > MT> > hosters. > MT> > MT> > MT> OK. Thanks for the clarification. > MT> > MT> > MT> >=20 > MT> > MT> > MT> >>=20 > MT> > MT> > MT> >> I think in general, it is the correct thing to = include the port numbers in > MT> > MT> > MT> >> the offset computation. We might add a sysctl = variable to control the inclusion. > MT> > MT> > MT> >> This would allow interworking with broken = middleboxes. > MT> > MT> > MT> >=20 > MT> > MT> > MT> > Yeah, I completely agree that these rare cases = should not dictate the implementation. > MT> > MT> > MT> > But an ability to enable a work-around via sysctl = would be greatly appreciated. > MT> > MT> > MT> > Currently we are unable to roll-out the upgrade = across all servers because of this > MT> > MT> > MT> > issue: even though it happens not so often, a lot of = requests from our users=20 > MT> > MT> > MT> > get stuck or fail all together. For example, a host = 185.134.205.105 is a kind of > MT> > MT> > MT> > social network that our proxy servers connect to so = securely access to content, > MT> > MT> > MT> > such as images, on behalf of our users. > MT> > MT> > MT> >=20 > MT> > MT> > MT> >>=20 > MT> > MT> > MT> >> Please note, this does not fix the case of multiple = clients behind a NAT. > MT> > MT> > MT> >=20 > MT> > MT> > MT> > Yeah, that's true. Fortunately we don't use NAT. > MT> > MT> > MT> >=20 > MT> > MT> > MT> >>=20 > MT> > MT> > MT> >> I'm also trying to figure out how and why Linux and = Windows are handling this. > MT> > MT> > MT> >=20 > MT> > MT> > MT> > Thanks for bothering! > MT> > MT> > MT> Will let you know what I figure out. > MT> > MT> > MT>=20 > MT> > MT> > MT> Best regards > MT> > MT> > MT> Michael > MT> > MT> > MT> >=20 > MT> > MT> > MT> >>=20 > MT> > MT> > MT> >> Best regards > MT> > MT> > MT> >> Michael > MT> > MT> > MT> >>=20 > MT> > MT> > MT> >>>=20 > MT> > MT> > MT> >>>>=20 > MT> > MT> > MT> >>>> Best regards > MT> > MT> > MT> >>>> Michael=20 > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> >>>>>>=20 > MT> > MT> > MT> >>>>>> Best regards > MT> > MT> > MT> >>>>>> Michael > MT> > MT> > MT> >>>>>>=20 > MT> > MT> > MT> >>>>>>=20 > MT> > MT> > MT> >>>>=20 > MT> > MT> > MT> >>>>=20 > MT> > MT> > MT> >>=20 > MT> > MT> > MT> >>=20 > MT> > MT> > MT>=20 > MT> > MT> > MT> _______________________________________________ > MT> > MT> > MT> freebsd-net@freebsd.org mailing list > MT> > MT> > MT> https://lists.freebsd.org/mailman/listinfo/freebsd-net > MT> > MT> > MT> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" > MT> > MT>=20 > MT> > MT> _______________________________________________ > MT> > MT> freebsd-net@freebsd.org mailing list > MT> > MT> https://lists.freebsd.org/mailman/listinfo/freebsd-net > MT> > MT> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" > MT>=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Wed Jul 17 12:32:55 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A9E13ABC14 for ; Wed, 17 Jul 2019 12:32:55 +0000 (UTC) (envelope-from satan@ukr.net) Received: from hell.ukr.net (hell.ukr.net [212.42.67.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.ukr.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BEE57764C; Wed, 17 Jul 2019 12:32:54 +0000 (UTC) (envelope-from satan@ukr.net) Received: from satan by hell.ukr.net with local ID 1hnj75-000Dy2-Pq ; Wed, 17 Jul 2019 15:32:51 +0300 Date: Wed, 17 Jul 2019 15:32:51 +0300 From: Vitalij Satanivskij To: Michael Tuexen Cc: Vitalij Satanivskij , Paul , freebsd-net@freebsd.org Subject: Re: Issues with TCP Timestamps allocation Message-ID: <20190717123251.GA53638@hell.ukr.net> Reply-To: satan@ukr.net References: <1562599181.734953000.1l9a1d23@frv39.fwdcdn.com> <0C475A01-9BCD-4E4A-9731-09AB919CA9BE@freebsd.org> <1562676414.933145000.z3zteyqp@frv39.fwdcdn.com> <1E9F3F99-C3E9-44DD-AA70-9B11E19D4769@freebsd.org> <20190717074243.GA65665@hell.ukr.net> <20190717100926.GA24984@hell.ukr.net> <48817BF6-AEDD-4D28-95F8-A4D53E4999B1@freebsd.org> <20190717115502.GA53155@hell.ukr.net> <8763FDC7-8B71-41C3-8D1C-10416DA9A871@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <8763FDC7-8B71-41C3-8D1C-10416DA9A871@freebsd.org> User-Agent: Mutt/1.12.1 (2019-06-15) X-Rspamd-Queue-Id: 4BEE57764C X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dmarc=pass (policy=none) header.from=ukr.net; spf=pass (mx1.freebsd.org: domain of satan@ukr.net designates 212.42.67.68 as permitted sender) smtp.mailfrom=satan@ukr.net X-Spamd-Result: default: False [-3.53 / 15.00]; ARC_NA(0.00)[]; HAS_REPLYTO(0.00)[satan@ukr.net]; NEURAL_HAM_MEDIUM(-0.96)[-0.959,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip4:212.42.67.64/27]; FREEMAIL_FROM(0.00)[ukr.net]; MIME_GOOD(-0.10)[text/plain]; FREEMAIL_REPLYTO(0.00)[ukr.net]; REPLYTO_ADDR_EQ_FROM(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: mxs.ukr.net]; DMARC_POLICY_ALLOW(-0.50)[ukr.net,none]; NEURAL_HAM_SHORT(-0.02)[-0.019,0]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[ukr.net]; ASN(0.00)[asn:8856, ipnet:212.42.66.0/23, country:UA]; FREEMAIL_CC(0.00)[ukr.net]; IP_SCORE(-0.74)[asn: 8856(-3.78), country: UA(0.08)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jul 2019 12:32:55 -0000 Hmm, looks like with some host's work but not with another Wed/17.07:/home/satan hell:-1522/15:28>curl https://volia.com > /dev/null % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 41519 0 41519 0 0 137k 0 --:--:-- --:--:-- --:--:-- 137k Wed/17.07:/home/satan hell:-1523/15:28>curl https://volia.com > /dev/null % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- 0:00:53 --:--:-- 0^C Wed/17.07:/home/satan hell:-1524/15:29>sysctl net.inet.tcp.rexmit_drop_options net.inet.tcp.rexmit_drop_options: 1 But MT> Interesting. It works for me: MT> MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null MT> % Total % Received % Xferd Average Speed Time Time Time Current MT> Dload Upload Total Spent Left Speed MT> 100 18265 0 18265 0 0 33637 0 --:--:-- --:--:-- --:--:-- 33575 MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null MT> % Total % Received % Xferd Average Speed Time Time Time Current MT> Dload Upload Total Spent Left Speed MT> 100 18265 0 18265 0 0 4834 0 --:--:-- 0:00:03 --:--:-- 4833 MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null MT> % Total % Received % Xferd Average Speed Time Time Time Current MT> Dload Upload Total Spent Left Speed MT> 100 18265 0 18265 0 0 35813 0 --:--:-- --:--:-- --:--:-- 35813 MT> tuexen@head:~ % time curl https://vitagramma.com > /dev/null MT> % Total % Received % Xferd Average Speed Time Time Time Current MT> Dload Upload Total Spent Left Speed MT> 100 18265 0 18265 0 0 48320 0 --:--:-- --:--:-- --:--:-- 48320 MT> 0.012u 0.031s 0:00.39 10.2% 140+245k 0+0io 0pf+0w MT> tuexen@head:~ % time curl https://vitagramma.com > /dev/null MT> % Total % Received % Xferd Average Speed Time Time Time Current MT> Dload Upload Total Spent Left Speed MT> 100 18265 0 18265 0 0 4592 0 --:--:-- 0:00:03 --:--:-- 4591 MT> 0.031u 0.010s 0:03.99 1.0% 80+140k 0+0io 0pf+0w MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null MT> % Total % Received % Xferd Average Speed Time Time Time Current MT> Dload Upload Total Spent Left Speed MT> 100 18265 0 18265 0 0 37815 0 --:--:-- --:--:-- --:--:-- 37737 MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null MT> % Total % Received % Xferd Average Speed Time Time Time Current MT> Dload Upload Total Spent Left Speed MT> 100 18265 0 18265 0 0 27261 0 --:--:-- --:--:-- --:--:-- 27220 MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null MT> % Total % Received % Xferd Average Speed Time Time Time Current MT> Dload Upload Total Spent Left Speed MT> 100 18265 0 18265 0 0 4533 0 --:--:-- 0:00:04 --:--:-- 4533 MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null MT> % Total % Received % Xferd Average Speed Time Time Time Current MT> Dload Upload Total Spent Left Speed MT> 100 18265 0 18265 0 0 48320 0 --:--:-- --:--:-- --:--:-- 48192 MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null MT> % Total % Received % Xferd Average Speed Time Time Time Current MT> Dload Upload Total Spent Left Speed MT> 100 18265 0 18265 0 0 4746 0 --:--:-- 0:00:03 --:--:-- 4745 MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null MT> % Total % Received % Xferd Average Speed Time Time Time Current MT> Dload Upload Total Spent Left Speed MT> 100 18265 0 18265 0 0 4500 0 --:--:-- 0:00:04 --:--:-- 4767 MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null MT> % Total % Received % Xferd Average Speed Time Time Time Current MT> Dload Upload Total Spent Left Speed MT> 100 18265 0 18265 0 0 4726 0 --:--:-- 0:00:03 --:--:-- 4726 MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null MT> % Total % Received % Xferd Average Speed Time Time Time Current MT> Dload Upload Total Spent Left Speed MT> 100 18265 0 18265 0 0 34268 0 --:--:-- --:--:-- --:--:-- 34332 MT> tuexen@head:~ % MT> MT> So it either works immediately or with a delay of 3 to 4 seconds... MT> MT> Best regards MT> Michael MT> > MT> > MT> > MT> MT> > MT> Option 2: Disable the TCP timestamps (and window scaling) MT> > MT> To enable this, you configure on the client MT> > MT> sudo sysctl -w net.inet.tcp.rfc1323=0 MT> > MT> or put MT> > MT> net.inet.tcp.rfc1323=0 MT> > MT> in /etc/sysctl.conf MT> > MT> and reboot. MT> > MT> This disables the timestamp option and window scaling completely. This allows you to MT> > MT> setup the connections without any delay. However, you don't have the benefits of the MT> > MT> extension. MT> > MT> MT> > MT> Both options don't require any code changes. MT> > MT> > This option was tested some time before. Yep it's help. But overal performance of tcp networking ... Let's say to bad :( MT> > MT> > MT> > MT> > MT> > MT> Best regards MT> > MT> Michael MT> > MT> MT> > MT> MT> > MT> > MT> > MT> > MT> > MT> > MT> MT> > MT> > MT> Best regards MT> > MT> > MT> Michael MT> > MT> > MT> > MT> > MT> > MT> > MT> > MT> > MT> > MT> > MT> > MT> > Michael Tuexen wrote: MT> > MT> > MT> > MT> MT> > MT> > MT> > MT> MT> > MT> > MT> > MT> > On 9. Jul 2019, at 14:58, Paul wrote: MT> > MT> > MT> > MT> > MT> > MT> > MT> > MT> > Hi Michael, MT> > MT> > MT> > MT> > MT> > MT> > MT> > MT> > 9 July 2019, 15:34:29, by "Michael Tuexen" : MT> > MT> > MT> > MT> > MT> > MT> > MT> > MT> >> MT> > MT> > MT> > MT> >> MT> > MT> > MT> > MT> >>> On 8. Jul 2019, at 17:22, Paul wrote: MT> > MT> > MT> > MT> >>> MT> > MT> > MT> > MT> >>> MT> > MT> > MT> > MT> >>> MT> > MT> > MT> > MT> >>> 8 July 2019, 17:12:21, by "Michael Tuexen" : MT> > MT> > MT> > MT> >>> MT> > MT> > MT> > MT> >>>>> On 8. Jul 2019, at 15:24, Paul wrote: MT> > MT> > MT> > MT> >>>>> MT> > MT> > MT> > MT> >>>>> Hi Michael, MT> > MT> > MT> > MT> >>>>> MT> > MT> > MT> > MT> >>>>> 8 July 2019, 15:53:15, by "Michael Tuexen" : MT> > MT> > MT> > MT> >>>>> MT> > MT> > MT> > MT> >>>>>>> On 8. Jul 2019, at 12:37, Paul wrote: MT> > MT> > MT> > MT> >>>>>>> MT> > MT> > MT> > MT> >>>>>>> Hi team, MT> > MT> > MT> > MT> >>>>>>> MT> > MT> > MT> > MT> >>>>>>> Recently we had an upgrade to 12 Stable. Immediately after, we have started MT> > MT> > MT> > MT> >>>>>>> seeing some strange connection establishment timeouts to some fixed number MT> > MT> > MT> > MT> >>>>>>> of external (world) hosts. The issue was persistent and easy to reproduce. MT> > MT> > MT> > MT> >>>>>>> Thanks to a patience and dedication of our system engineer we have tracked MT> > MT> > MT> > MT> >>>>>>> this issue down to a specific commit: MT> > MT> > MT> > MT> >>>>>>> MT> > MT> > MT> > MT> >>>>>>> https://svnweb.freebsd.org/base?view=revision&revision=338053 MT> > MT> > MT> > MT> >>>>>>> MT> > MT> > MT> > MT> >>>>>>> This patch was also back-ported into 11 Stable: MT> > MT> > MT> > MT> >>>>>>> MT> > MT> > MT> > MT> >>>>>>> https://svnweb.freebsd.org/base?view=revision&revision=348435 MT> > MT> > MT> > MT> >>>>>>> MT> > MT> > MT> > MT> >>>>>>> Among other things this patch changes the timestamp allocation strategy, MT> > MT> > MT> > MT> >>>>>>> by introducing a deterministic randomness via a hash function that takes MT> > MT> > MT> > MT> >>>>>>> into account a random key as well as source address, source port, dest MT> > MT> > MT> > MT> >>>>>>> address and dest port. As the result, timestamp offsets of different MT> > MT> > MT> > MT> >>>>>>> tuples (SA,SP,DA,DP) will be wildly different and will jump from small MT> > MT> > MT> > MT> >>>>>>> to large numbers and back, as long as something in the tuple changes. MT> > MT> > MT> > MT> >>>>>> Hi Paul, MT> > MT> > MT> > MT> >>>>>> MT> > MT> > MT> > MT> >>>>>> this is correct. MT> > MT> > MT> > MT> >>>>>> MT> > MT> > MT> > MT> >>>>>> Please note that the same happens with the old method, if two hosts with MT> > MT> > MT> > MT> >>>>>> different uptimes are bind a consumer grade NAT. MT> > MT> > MT> > MT> >>>>> MT> > MT> > MT> > MT> >>>>> If NAT does not replace timestamps then yes, it should be the case. MT> > MT> > MT> > MT> >>>>> MT> > MT> > MT> > MT> >>>>>>> MT> > MT> > MT> > MT> >>>>>>> After performing various tests of hosts that produce the above mentioned MT> > MT> > MT> > MT> >>>>>>> issue we came to conclusion that there are some interesting implementations MT> > MT> > MT> > MT> >>>>>>> that drop SYN packets with timestamps smaller than the largest timestamp MT> > MT> > MT> > MT> >>>>>>> value from streams of all recent or current connections from a specific MT> > MT> > MT> > MT> >>>>>>> address. This looks as some kind of SYN flood protection. MT> > MT> > MT> > MT> >>>>>> This also breaks multiple hosts with different uptimes behind a consumer MT> > MT> > MT> > MT> >>>>>> level NAT talking to such a server. MT> > MT> > MT> > MT> >>>>>>> MT> > MT> > MT> > MT> >>>>>>> To ensure that each external host is not going to see a wild jumps of MT> > MT> > MT> > MT> >>>>>>> timestamp values I propose a patch that removes ports from the equation MT> > MT> > MT> > MT> >>>>>>> all together, when calculating the timestamp offset: MT> > MT> > MT> > MT> >>>>>>> MT> > MT> > MT> > MT> >>>>>>> Index: sys/netinet/tcp_subr.c MT> > MT> > MT> > MT> >>>>>>> =================================================================== MT> > MT> > MT> > MT> >>>>>>> --- sys/netinet/tcp_subr.c (revision 348435) MT> > MT> > MT> > MT> >>>>>>> +++ sys/netinet/tcp_subr.c (working copy) MT> > MT> > MT> > MT> >>>>>>> @@ -2224,7 +2224,22 @@ MT> > MT> > MT> > MT> >>>>>>> uint32_t MT> > MT> > MT> > MT> >>>>>>> tcp_new_ts_offset(struct in_conninfo *inc) MT> > MT> > MT> > MT> >>>>>>> { MT> > MT> > MT> > MT> >>>>>>> - return (tcp_keyed_hash(inc, V_ts_offset_secret)); MT> > MT> > MT> > MT> >>>>>>> + /* MT> > MT> > MT> > MT> >>>>>>> + * Some implementations show a strange behaviour when a wildly random MT> > MT> > MT> > MT> >>>>>>> + * timestamps allocated for different streams. It seems that only the MT> > MT> > MT> > MT> >>>>>>> + * SYN packets are affected. Observed implementations drop SYN packets MT> > MT> > MT> > MT> >>>>>>> + * with timestamps smaller than the largest timestamp value of all MT> > MT> > MT> > MT> >>>>>>> + * recent or current connections from specific a address. To mitigate MT> > MT> > MT> > MT> >>>>>>> + * this we are going to ensure that each host will always observe MT> > MT> > MT> > MT> >>>>>>> + * timestamps as increasing no matter the stream: by dropping ports MT> > MT> > MT> > MT> >>>>>>> + * from the equation. MT> > MT> > MT> > MT> >>>>>>> + */ MT> > MT> > MT> > MT> >>>>>>> + struct in_conninfo inc_copy = *inc; MT> > MT> > MT> > MT> >>>>>>> + MT> > MT> > MT> > MT> >>>>>>> + inc_copy.inc_fport = 0; MT> > MT> > MT> > MT> >>>>>>> + inc_copy.inc_lport = 0; MT> > MT> > MT> > MT> >>>>>>> + MT> > MT> > MT> > MT> >>>>>>> + return (tcp_keyed_hash(&inc_copy, V_ts_offset_secret)); MT> > MT> > MT> > MT> >>>>>>> } MT> > MT> > MT> > MT> >>>>>>> MT> > MT> > MT> > MT> >>>>>>> /* MT> > MT> > MT> > MT> >>>>>>> MT> > MT> > MT> > MT> >>>>>>> In any case, the solution of the uptime leak, implemented in rev338053 is MT> > MT> > MT> > MT> >>>>>>> not going to suffer, because a supposed attacker is currently able to use MT> > MT> > MT> > MT> >>>>>>> any fixed values of SP and DP, albeit not 0, anyway, to remove them out MT> > MT> > MT> > MT> >>>>>>> of the equation. MT> > MT> > MT> > MT> >>>>>> Can you describe how a peer can compute the uptime from two observed timestamps? MT> > MT> > MT> > MT> >>>>>> I don't see how you can do that... MT> > MT> > MT> > MT> >>>>> MT> > MT> > MT> > MT> >>>>> Supposed attacker could run a script that continuously monitors timestamps, MT> > MT> > MT> > MT> >>>>> for example via a periodic TCP connection from a fixed local port (eg 12345) MT> > MT> > MT> > MT> >>>>> and a fixed local address to the fixed victim's address and port (eg 80). MT> > MT> > MT> > MT> >>>>> Whenever large discrepancy is observed, attacker can assume that reboot has MT> > MT> > MT> > MT> >>>>> happened (due to V_ts_offset_secret re-generation), hence the received MT> > MT> > MT> > MT> >>>>> timestamp is considered an approximate point of reboot from which the uptime MT> > MT> > MT> > MT> >>>>> can be calculated, until the next reboot and so on. MT> > MT> > MT> > MT> >>>> Ahh, I see. The patch we are talking about is not intended to protect against MT> > MT> > MT> > MT> >>>> continuous monitoring, which is something you can always do. You could even MT> > MT> > MT> > MT> >>>> watch for service availability and detect reboots. A change of the local key MT> > MT> > MT> > MT> >>>> would also look similar to a reboot without a temporary loss of connectivity. MT> > MT> > MT> > MT> >>>> MT> > MT> > MT> > MT> >>>> Thanks for the clarification. MT> > MT> > MT> > MT> >>>>> MT> > MT> > MT> > MT> >>>>>>> MT> > MT> > MT> > MT> >>>>>>> There is the list of example hosts that we were able to reproduce the MT> > MT> > MT> > MT> >>>>>>> issue with: MT> > MT> > MT> > MT> >>>>>>> MT> > MT> > MT> > MT> >>>>>>> curl -v http://88.99.60.171:80 MT> > MT> > MT> > MT> >>>>>>> curl -v http://163.172.71.252:80 MT> > MT> > MT> > MT> >>>>>>> curl -v http://5.9.242.150:80 MT> > MT> > MT> > MT> >>>>>>> curl -v https://185.134.205.105:443 MT> > MT> > MT> > MT> >>>>>>> curl -v https://136.243.1.231:443 MT> > MT> > MT> > MT> >>>>>>> curl -v https://144.76.196.4:443 MT> > MT> > MT> > MT> >>>>>>> curl -v http://94.127.191.194:80 MT> > MT> > MT> > MT> >>>>>>> MT> > MT> > MT> > MT> >>>>>>> To reproduce, call curl repeatedly with a same URL some number of times. MT> > MT> > MT> > MT> >>>>>>> You are going to see some of the requests stuck in MT> > MT> > MT> > MT> >>>>>>> `* Trying XXX.XXX.XXX.XXX...` MT> > MT> > MT> > MT> >>>>>>> MT> > MT> > MT> > MT> >>>>>>> For some reason, the easiest way to reproduce the issue is with nc: MT> > MT> > MT> > MT> >>>>>>> MT> > MT> > MT> > MT> >>>>>>> $ echo "foooooo" | nc -v 88.99.60.171 80 MT> > MT> > MT> > MT> >>>>>>> MT> > MT> > MT> > MT> >>>>>>> Only a few such calls are required until one of them is stuck on connect(): MT> > MT> > MT> > MT> >>>>>>> issuing SYN packets with an exponential backoff. MT> > MT> > MT> > MT> >>>>>> Thanks for providing an end-point to test with. I'll take a look. MT> > MT> > MT> > MT> >>>>>> Just to be clear: You are running a FreeBSD client against one of the above MT> > MT> > MT> > MT> >>>>>> servers and experience the problem with the new timestamp computations. MT> > MT> > MT> > MT> >>>>>> MT> > MT> > MT> > MT> >>>>>> You are not running arbitrary clients against a FreeBSD server... MT> > MT> > MT> > MT> >>>>> MT> > MT> > MT> > MT> >>>>> We are talking about FreeBSD being the client. Peers that yield this unwanted MT> > MT> > MT> > MT> >>>>> behaviour are unknown. Little bit of tinkering showed that some of them run MT> > MT> > MT> > MT> >>>>> Debian: MT> > MT> > MT> > MT> >>>>> MT> > MT> > MT> > MT> >>>>> telnet 88.99.60.171 22 MT> > MT> > MT> > MT> >>>>> Trying 88.99.60.171... MT> > MT> > MT> > MT> >>>>> Connected to 88.99.60.171. MT> > MT> > MT> > MT> >>>>> Escape character is '^]'. MT> > MT> > MT> > MT> >>>>> SSH-2.0-OpenSSH_6.7p1 Debian-5+deb8u3 MT> > MT> > MT> > MT> >>>> Also some are hosted by Hetzner, but not all. I'll will look into MT> > MT> > MT> > MT> >>>> this tomorrow, since I'm on a deadline today (well it is 2am tomorrow MT> > MT> > MT> > MT> >>>> morning, to be precise)... MT> > MT> > MT> > MT> >>> MT> > MT> > MT> > MT> >>> Thanks a lot, I would appreciate that. MT> > MT> > MT> > MT> >> Hi Paul, MT> > MT> > MT> > MT> >> MT> > MT> > MT> > MT> >> I have looked into this. MT> > MT> > MT> > MT> >> MT> > MT> > MT> > MT> >> * The FreeBSD behaviour is the one which is specified in the last bullet item MT> > MT> > MT> > MT> >> in https://tools.ietf.org/html/rfc7323#section-5.4 MT> > MT> > MT> > MT> >> It is also the one, which is RECOMMENDED in MT> > MT> > MT> > MT> >> https://tools.ietf.org/html/rfc7323#section-7.1 MT> > MT> > MT> > MT> >> MT> > MT> > MT> > MT> >> * My NAT box (a popular one in Germany) does NOT rewrite TCP timestamps. MT> > MT> > MT> > MT> >> MT> > MT> > MT> > MT> >> This means that the host you are referring to have some sort of protection, MT> > MT> > MT> > MT> >> which makes incorrect assumptions. It will also break multiple hosts behind MT> > MT> > MT> > MT> >> a NAT. MT> > MT> > MT> > MT> >> MT> > MT> > MT> > MT> >> I can run MT> > MT> > MT> > MT> >> curl -v http://88.99.60.171:80 MT> > MT> > MT> > MT> >> in a loop without any problems from a FreeBSD head system. I tested 1000 MT> > MT> > MT> > MT> >> iterations or so. The TS.val is jumping up and down as expected. MT> > MT> > MT> > MT> >> I'm wondering why you are observing errors in this case, too. MT> > MT> > MT> > MT> >> MT> > MT> > MT> > MT> >> However, doing something like MT> > MT> > MT> > MT> >> echo "foooooo" | nc -v 88.99.60.171 80 MT> > MT> > MT> > MT> >> triggers the problem. MT> > MT> > MT> > MT> >> MT> > MT> > MT> > MT> >> So I think there is some functionality (in a middlebox or running on the host), MT> > MT> > MT> > MT> >> which incorrectly assume monotonic timestamps between multiple TCP connections MT> > MT> > MT> > MT> >> coming from the same IP address, but only in case of errors at the application layer. MT> > MT> > MT> > MT> > MT> > MT> > MT> > MT> > Yeah, exactly, some hosts seem to enable this only in case of an error in HTTP MT> > MT> > MT> > MT> > communication (some smart proxy?). However, there are some that behave this way MT> > MT> > MT> > MT> > regardless of errors, for example these: MT> > MT> > MT> > MT> > MT> > MT> > MT> > MT> > curl -v https://185.134.205.105:443 MT> > MT> > MT> > MT> > curl -v https://136.243.1.231:443 MT> > MT> > MT> > MT> Wireshark sees an Encrypted Alert in both cases. So I guess this is another indication MT> > MT> > MT> > MT> of "error at the application layer". MT> > MT> > MT> > MT> > MT> > MT> > MT> > MT> >> MT> > MT> > MT> > MT> >> Do you have any insights whether the hosts you are listed share something in MT> > MT> > MT> > MT> >> common. Some of them are hosted by Hetzner, but not all. MT> > MT> > MT> > MT> > MT> > MT> > MT> > MT> > Nope. A whole set of endpoints that we have detected so far is pretty diverse, MT> > MT> > MT> > MT> > containing a lot of different locations geographically, as well as different MT> > MT> > MT> > MT> > hosters. MT> > MT> > MT> > MT> OK. Thanks for the clarification. MT> > MT> > MT> > MT> > MT> > MT> > MT> > MT> >> MT> > MT> > MT> > MT> >> I think in general, it is the correct thing to include the port numbers in MT> > MT> > MT> > MT> >> the offset computation. We might add a sysctl variable to control the inclusion. MT> > MT> > MT> > MT> >> This would allow interworking with broken middleboxes. MT> > MT> > MT> > MT> > MT> > MT> > MT> > MT> > Yeah, I completely agree that these rare cases should not dictate the implementation. MT> > MT> > MT> > MT> > But an ability to enable a work-around via sysctl would be greatly appreciated. MT> > MT> > MT> > MT> > Currently we are unable to roll-out the upgrade across all servers because of this MT> > MT> > MT> > MT> > issue: even though it happens not so often, a lot of requests from our users MT> > MT> > MT> > MT> > get stuck or fail all together. For example, a host 185.134.205.105 is a kind of MT> > MT> > MT> > MT> > social network that our proxy servers connect to so securely access to content, MT> > MT> > MT> > MT> > such as images, on behalf of our users. MT> > MT> > MT> > MT> > MT> > MT> > MT> > MT> >> MT> > MT> > MT> > MT> >> Please note, this does not fix the case of multiple clients behind a NAT. MT> > MT> > MT> > MT> > MT> > MT> > MT> > MT> > Yeah, that's true. Fortunately we don't use NAT. MT> > MT> > MT> > MT> > MT> > MT> > MT> > MT> >> MT> > MT> > MT> > MT> >> I'm also trying to figure out how and why Linux and Windows are handling this. MT> > MT> > MT> > MT> > MT> > MT> > MT> > MT> > Thanks for bothering! MT> > MT> > MT> > MT> Will let you know what I figure out. MT> > MT> > MT> > MT> MT> > MT> > MT> > MT> Best regards MT> > MT> > MT> > MT> Michael MT> > MT> > MT> > MT> > MT> > MT> > MT> > MT> >> MT> > MT> > MT> > MT> >> Best regards MT> > MT> > MT> > MT> >> Michael MT> > MT> > MT> > MT> >> MT> > MT> > MT> > MT> >>> MT> > MT> > MT> > MT> >>>> MT> > MT> > MT> > MT> >>>> Best regards MT> > MT> > MT> > MT> >>>> Michael MT> > MT> > MT> > MT> >>>>> MT> > MT> > MT> > MT> >>>>> MT> > MT> > MT> > MT> >>>>>> MT> > MT> > MT> > MT> >>>>>> Best regards MT> > MT> > MT> > MT> >>>>>> Michael MT> > MT> > MT> > MT> >>>>>> MT> > MT> > MT> > MT> >>>>>> MT> > MT> > MT> > MT> >>>> MT> > MT> > MT> > MT> >>>> MT> > MT> > MT> > MT> >> MT> > MT> > MT> > MT> >> MT> > MT> > MT> > MT> MT> > MT> > MT> > MT> _______________________________________________ MT> > MT> > MT> > MT> freebsd-net@freebsd.org mailing list MT> > MT> > MT> > MT> https://lists.freebsd.org/mailman/listinfo/freebsd-net MT> > MT> > MT> > MT> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" MT> > MT> > MT> MT> > MT> > MT> _______________________________________________ MT> > MT> > MT> freebsd-net@freebsd.org mailing list MT> > MT> > MT> https://lists.freebsd.org/mailman/listinfo/freebsd-net MT> > MT> > MT> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" MT> > MT> MT> > _______________________________________________ MT> > freebsd-net@freebsd.org mailing list MT> > https://lists.freebsd.org/mailman/listinfo/freebsd-net MT> > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" MT> From owner-freebsd-net@freebsd.org Wed Jul 17 12:42:05 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4DE65AC003 for ; Wed, 17 Jul 2019 12:42:05 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 18B1077C31 for ; Wed, 17 Jul 2019 12:42:05 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from [IPv6:2a02:8109:1140:c3d:d878:d920:c09d:25c5] (unknown [IPv6:2a02:8109:1140:c3d:d878:d920:c09d:25c5]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTPSA id 26F6272106C2B; Wed, 17 Jul 2019 14:42:01 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: Issues with TCP Timestamps allocation From: Michael Tuexen In-Reply-To: <20190717123251.GA53638@hell.ukr.net> Date: Wed, 17 Jul 2019 14:42:00 +0200 Cc: Paul , freebsd-net@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <1562599181.734953000.1l9a1d23@frv39.fwdcdn.com> <0C475A01-9BCD-4E4A-9731-09AB919CA9BE@freebsd.org> <1562676414.933145000.z3zteyqp@frv39.fwdcdn.com> <1E9F3F99-C3E9-44DD-AA70-9B11E19D4769@freebsd.org> <20190717074243.GA65665@hell.ukr.net> <20190717100926.GA24984@hell.ukr.net> <48817BF6-AEDD-4D28-95F8-A4D53E4999B1@freebsd.org> <20190717115502.GA53155@hell.ukr.net> <8763FDC7-8B71-41C3-8D1C-10416DA9A871@freebsd.org> <20190717123251.GA53638@hell.ukr.net> To: Vitalij Satanivskij X-Mailer: Apple Mail (2.3445.104.11) X-Spam-Status: No, score=-1.7 required=5.0 tests=ALL_TRUSTED,BAYES_00, NORMAL_HTTP_TO_IP,NUMERIC_HTTP_ADDR autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jul 2019 12:42:05 -0000 > On 17. Jul 2019, at 14:32, Vitalij Satanivskij wrote: >=20 > Hmm, looks like with some host's work but not with another >=20 > Wed/17.07:/home/satan > hell:-1522/15:28>curl https://volia.com > /dev/null > % Total % Received % Xferd Average Speed Time Time Time = Current > Dload Upload Total Spent Left = Speed > 100 41519 0 41519 0 0 137k 0 --:--:-- --:--:-- = --:--:-- 137k > Wed/17.07:/home/satan > hell:-1523/15:28>curl https://volia.com > /dev/null > % Total % Received % Xferd Average Speed Time Time Time = Current > Dload Upload Total Spent Left = Speed > 0 0 0 0 0 0 0 0 --:--:-- 0:00:53 = --:--:-- 0^C > Wed/17.07:/home/satan > hell:-1524/15:29>sysctl net.inet.tcp.rexmit_drop_options > net.inet.tcp.rexmit_drop_options: 1 OK, I can confirm that for https://volia.com only a timeout helps. What I observed for now is that for the "blocking" to occur is it = crucial that the server sends the FIN and therefore goes into the TIMEWAIT state. The = timeout seems to be 60 seconds. The blocking is also not limited to a single server port. I'm not sure yet whether it is a broken end point or a broken middle = box. Best regards Michael >=20 > But=20 >=20 > MT> Interesting. It works for me: > MT>=20 > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > MT> Dload Upload Total Spent = Left Speed > MT> 100 18265 0 18265 0 0 33637 0 --:--:-- --:--:-- = --:--:-- 33575 > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > MT> Dload Upload Total Spent = Left Speed > MT> 100 18265 0 18265 0 0 4834 0 --:--:-- 0:00:03 = --:--:-- 4833 > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > MT> Dload Upload Total Spent = Left Speed > MT> 100 18265 0 18265 0 0 35813 0 --:--:-- --:--:-- = --:--:-- 35813 > MT> tuexen@head:~ % time curl https://vitagramma.com > /dev/null > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > MT> Dload Upload Total Spent = Left Speed > MT> 100 18265 0 18265 0 0 48320 0 --:--:-- --:--:-- = --:--:-- 48320 > MT> 0.012u 0.031s 0:00.39 10.2% 140+245k 0+0io 0pf+0w > MT> tuexen@head:~ % time curl https://vitagramma.com > /dev/null > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > MT> Dload Upload Total Spent = Left Speed > MT> 100 18265 0 18265 0 0 4592 0 --:--:-- 0:00:03 = --:--:-- 4591 > MT> 0.031u 0.010s 0:03.99 1.0% 80+140k 0+0io 0pf+0w > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > MT> Dload Upload Total Spent = Left Speed > MT> 100 18265 0 18265 0 0 37815 0 --:--:-- --:--:-- = --:--:-- 37737 > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > MT> Dload Upload Total Spent = Left Speed > MT> 100 18265 0 18265 0 0 27261 0 --:--:-- --:--:-- = --:--:-- 27220 > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > MT> Dload Upload Total Spent = Left Speed > MT> 100 18265 0 18265 0 0 4533 0 --:--:-- 0:00:04 = --:--:-- 4533 > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > MT> Dload Upload Total Spent = Left Speed > MT> 100 18265 0 18265 0 0 48320 0 --:--:-- --:--:-- = --:--:-- 48192 > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > MT> Dload Upload Total Spent = Left Speed > MT> 100 18265 0 18265 0 0 4746 0 --:--:-- 0:00:03 = --:--:-- 4745 > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > MT> Dload Upload Total Spent = Left Speed > MT> 100 18265 0 18265 0 0 4500 0 --:--:-- 0:00:04 = --:--:-- 4767 > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > MT> Dload Upload Total Spent = Left Speed > MT> 100 18265 0 18265 0 0 4726 0 --:--:-- 0:00:03 = --:--:-- 4726 > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > MT> Dload Upload Total Spent = Left Speed > MT> 100 18265 0 18265 0 0 34268 0 --:--:-- --:--:-- = --:--:-- 34332 > MT> tuexen@head:~ %=20 > MT>=20 > MT> So it either works immediately or with a delay of 3 to 4 = seconds... > MT>=20 > MT> Best regards > MT> Michael > MT> >=20 > MT> >=20 > MT> > MT>=20 > MT> > MT> Option 2: Disable the TCP timestamps (and window scaling) > MT> > MT> To enable this, you configure on the client > MT> > MT> sudo sysctl -w net.inet.tcp.rfc1323=3D0 > MT> > MT> or put > MT> > MT> net.inet.tcp.rfc1323=3D0 > MT> > MT> in /etc/sysctl.conf > MT> > MT> and reboot. > MT> > MT> This disables the timestamp option and window scaling = completely. This allows you to > MT> > MT> setup the connections without any delay. However, you don't = have the benefits of the > MT> > MT> extension. > MT> > MT>=20 > MT> > MT> Both options don't require any code changes. > MT> >=20 > MT> > This option was tested some time before. Yep it's help. But = overal performance of tcp networking ... Let's say to bad :( > MT> >=20 > MT> >=20 > MT> >=20 > MT> >=20 > MT> > MT> Best regards > MT> > MT> Michael > MT> > MT>=20 > MT> > MT>=20 > MT> > MT> >=20 > MT> > MT> >=20 > MT> > MT> > MT>=20 > MT> > MT> > MT> Best regards > MT> > MT> > MT> Michael > MT> > MT> > MT> >=20 > MT> > MT> > MT> >=20 > MT> > MT> > MT> >=20 > MT> > MT> > MT> > Michael Tuexen wrote: > MT> > MT> > MT> > MT>=20 > MT> > MT> > MT> > MT>=20 > MT> > MT> > MT> > MT> > On 9. Jul 2019, at 14:58, Paul = wrote: > MT> > MT> > MT> > MT> >=20 > MT> > MT> > MT> > MT> > Hi Michael, > MT> > MT> > MT> > MT> >=20 > MT> > MT> > MT> > MT> > 9 July 2019, 15:34:29, by "Michael Tuexen" = : > MT> > MT> > MT> > MT> >=20 > MT> > MT> > MT> > MT> >>=20 > MT> > MT> > MT> > MT> >>=20 > MT> > MT> > MT> > MT> >>> On 8. Jul 2019, at 17:22, Paul = wrote: > MT> > MT> > MT> > MT> >>>=20 > MT> > MT> > MT> > MT> >>>=20 > MT> > MT> > MT> > MT> >>>=20 > MT> > MT> > MT> > MT> >>> 8 July 2019, 17:12:21, by "Michael Tuexen" = : > MT> > MT> > MT> > MT> >>>=20 > MT> > MT> > MT> > MT> >>>>> On 8. Jul 2019, at 15:24, Paul = wrote: > MT> > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> > MT> >>>>> Hi Michael, > MT> > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> > MT> >>>>> 8 July 2019, 15:53:15, by "Michael Tuexen" = : > MT> > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> > MT> >>>>>>> On 8. Jul 2019, at 12:37, Paul = wrote: > MT> > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> > MT> >>>>>>> Hi team, > MT> > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> > MT> >>>>>>> Recently we had an upgrade to 12 Stable. = Immediately after, we have started=20 > MT> > MT> > MT> > MT> >>>>>>> seeing some strange connection = establishment timeouts to some fixed number > MT> > MT> > MT> > MT> >>>>>>> of external (world) hosts. The issue was = persistent and easy to reproduce. > MT> > MT> > MT> > MT> >>>>>>> Thanks to a patience and dedication of = our system engineer we have tracked =20 > MT> > MT> > MT> > MT> >>>>>>> this issue down to a specific commit: > MT> > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> > MT> >>>>>>> = https://svnweb.freebsd.org/base?view=3Drevision&revision=3D338053 > MT> > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> > MT> >>>>>>> This patch was also back-ported into 11 = Stable: > MT> > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> > MT> >>>>>>> = https://svnweb.freebsd.org/base?view=3Drevision&revision=3D348435 > MT> > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> > MT> >>>>>>> Among other things this patch changes = the timestamp allocation strategy, > MT> > MT> > MT> > MT> >>>>>>> by introducing a deterministic = randomness via a hash function that takes > MT> > MT> > MT> > MT> >>>>>>> into account a random key as well as = source address, source port, dest > MT> > MT> > MT> > MT> >>>>>>> address and dest port. As the result, = timestamp offsets of different > MT> > MT> > MT> > MT> >>>>>>> tuples (SA,SP,DA,DP) will be wildly = different and will jump from small=20 > MT> > MT> > MT> > MT> >>>>>>> to large numbers and back, as long as = something in the tuple changes. > MT> > MT> > MT> > MT> >>>>>> Hi Paul, > MT> > MT> > MT> > MT> >>>>>>=20 > MT> > MT> > MT> > MT> >>>>>> this is correct. > MT> > MT> > MT> > MT> >>>>>>=20 > MT> > MT> > MT> > MT> >>>>>> Please note that the same happens with = the old method, if two hosts with > MT> > MT> > MT> > MT> >>>>>> different uptimes are bind a consumer = grade NAT. > MT> > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> > MT> >>>>> If NAT does not replace timestamps then = yes, it should be the case. > MT> > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> > MT> >>>>>>> After performing various tests of hosts = that produce the above mentioned=20 > MT> > MT> > MT> > MT> >>>>>>> issue we came to conclusion that there = are some interesting implementations=20 > MT> > MT> > MT> > MT> >>>>>>> that drop SYN packets with timestamps = smaller than the largest timestamp=20 > MT> > MT> > MT> > MT> >>>>>>> value from streams of all recent or = current connections from a specific=20 > MT> > MT> > MT> > MT> >>>>>>> address. This looks as some kind of SYN = flood protection. > MT> > MT> > MT> > MT> >>>>>> This also breaks multiple hosts with = different uptimes behind a consumer > MT> > MT> > MT> > MT> >>>>>> level NAT talking to such a server. > MT> > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> > MT> >>>>>>> To ensure that each external host is not = going to see a wild jumps of=20 > MT> > MT> > MT> > MT> >>>>>>> timestamp values I propose a patch that = removes ports from the equation > MT> > MT> > MT> > MT> >>>>>>> all together, when calculating the = timestamp offset: > MT> > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> > MT> >>>>>>> Index: sys/netinet/tcp_subr.c > MT> > MT> > MT> > MT> >>>>>>> = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > MT> > MT> > MT> > MT> >>>>>>> --- sys/netinet/tcp_subr.c = (revision 348435) > MT> > MT> > MT> > MT> >>>>>>> +++ sys/netinet/tcp_subr.c (working = copy) > MT> > MT> > MT> > MT> >>>>>>> @@ -2224,7 +2224,22 @@ > MT> > MT> > MT> > MT> >>>>>>> uint32_t > MT> > MT> > MT> > MT> >>>>>>> tcp_new_ts_offset(struct in_conninfo = *inc) > MT> > MT> > MT> > MT> >>>>>>> { > MT> > MT> > MT> > MT> >>>>>>> - return (tcp_keyed_hash(inc, = V_ts_offset_secret)); > MT> > MT> > MT> > MT> >>>>>>> + /*=20 > MT> > MT> > MT> > MT> >>>>>>> + * Some implementations show a = strange behaviour when a wildly random=20 > MT> > MT> > MT> > MT> >>>>>>> + * timestamps allocated for = different streams. It seems that only the > MT> > MT> > MT> > MT> >>>>>>> + * SYN packets are affected. = Observed implementations drop SYN packets > MT> > MT> > MT> > MT> >>>>>>> + * with timestamps smaller than = the largest timestamp value of all=20 > MT> > MT> > MT> > MT> >>>>>>> + * recent or current = connections from specific a address. To mitigate=20 > MT> > MT> > MT> > MT> >>>>>>> + * this we are going to ensure = that each host will always observe=20 > MT> > MT> > MT> > MT> >>>>>>> + * timestamps as increasing no = matter the stream: by dropping ports > MT> > MT> > MT> > MT> >>>>>>> + * from the equation. > MT> > MT> > MT> > MT> >>>>>>> + */=20 > MT> > MT> > MT> > MT> >>>>>>> + struct in_conninfo inc_copy =3D = *inc; > MT> > MT> > MT> > MT> >>>>>>> + > MT> > MT> > MT> > MT> >>>>>>> + inc_copy.inc_fport =3D 0; > MT> > MT> > MT> > MT> >>>>>>> + inc_copy.inc_lport =3D 0; > MT> > MT> > MT> > MT> >>>>>>> + > MT> > MT> > MT> > MT> >>>>>>> + return = (tcp_keyed_hash(&inc_copy, V_ts_offset_secret)); > MT> > MT> > MT> > MT> >>>>>>> } > MT> > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> > MT> >>>>>>> /* > MT> > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> > MT> >>>>>>> In any case, the solution of the uptime = leak, implemented in rev338053 is=20 > MT> > MT> > MT> > MT> >>>>>>> not going to suffer, because a supposed = attacker is currently able to use=20 > MT> > MT> > MT> > MT> >>>>>>> any fixed values of SP and DP, albeit = not 0, anyway, to remove them out=20 > MT> > MT> > MT> > MT> >>>>>>> of the equation. > MT> > MT> > MT> > MT> >>>>>> Can you describe how a peer can compute = the uptime from two observed timestamps? > MT> > MT> > MT> > MT> >>>>>> I don't see how you can do that... > MT> > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> > MT> >>>>> Supposed attacker could run a script that = continuously monitors timestamps, > MT> > MT> > MT> > MT> >>>>> for example via a periodic TCP connection = from a fixed local port (eg 12345)=20 > MT> > MT> > MT> > MT> >>>>> and a fixed local address to the fixed = victim's address and port (eg 80). > MT> > MT> > MT> > MT> >>>>> Whenever large discrepancy is observed, = attacker can assume that reboot has=20 > MT> > MT> > MT> > MT> >>>>> happened (due to V_ts_offset_secret = re-generation), hence the received=20 > MT> > MT> > MT> > MT> >>>>> timestamp is considered an approximate = point of reboot from which the uptime > MT> > MT> > MT> > MT> >>>>> can be calculated, until the next reboot = and so on. > MT> > MT> > MT> > MT> >>>> Ahh, I see. The patch we are talking about = is not intended to protect against > MT> > MT> > MT> > MT> >>>> continuous monitoring, which is something = you can always do. You could even > MT> > MT> > MT> > MT> >>>> watch for service availability and detect = reboots. A change of the local key > MT> > MT> > MT> > MT> >>>> would also look similar to a reboot without = a temporary loss of connectivity. > MT> > MT> > MT> > MT> >>>>=20 > MT> > MT> > MT> > MT> >>>> Thanks for the clarification. > MT> > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> > MT> >>>>>>> There is the list of example hosts that = we were able to reproduce the=20 > MT> > MT> > MT> > MT> >>>>>>> issue with: > MT> > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> > MT> >>>>>>> curl -v http://88.99.60.171:80 > MT> > MT> > MT> > MT> >>>>>>> curl -v http://163.172.71.252:80 > MT> > MT> > MT> > MT> >>>>>>> curl -v http://5.9.242.150:80 > MT> > MT> > MT> > MT> >>>>>>> curl -v https://185.134.205.105:443 > MT> > MT> > MT> > MT> >>>>>>> curl -v https://136.243.1.231:443 > MT> > MT> > MT> > MT> >>>>>>> curl -v https://144.76.196.4:443 > MT> > MT> > MT> > MT> >>>>>>> curl -v http://94.127.191.194:80 > MT> > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> > MT> >>>>>>> To reproduce, call curl repeatedly with = a same URL some number of times.=20 > MT> > MT> > MT> > MT> >>>>>>> You are going to see some of the = requests stuck in=20 > MT> > MT> > MT> > MT> >>>>>>> `* Trying XXX.XXX.XXX.XXX...` > MT> > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> > MT> >>>>>>> For some reason, the easiest way to = reproduce the issue is with nc: > MT> > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> > MT> >>>>>>> $ echo "foooooo" | nc -v 88.99.60.171 80 > MT> > MT> > MT> > MT> >>>>>>>=20 > MT> > MT> > MT> > MT> >>>>>>> Only a few such calls are required until = one of them is stuck on connect(): > MT> > MT> > MT> > MT> >>>>>>> issuing SYN packets with an exponential = backoff. > MT> > MT> > MT> > MT> >>>>>> Thanks for providing an end-point to test = with. I'll take a look. > MT> > MT> > MT> > MT> >>>>>> Just to be clear: You are running a = FreeBSD client against one of the above > MT> > MT> > MT> > MT> >>>>>> servers and experience the problem with = the new timestamp computations. > MT> > MT> > MT> > MT> >>>>>>=20 > MT> > MT> > MT> > MT> >>>>>> You are not running arbitrary clients = against a FreeBSD server... > MT> > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> > MT> >>>>> We are talking about FreeBSD being the = client. Peers that yield this unwanted > MT> > MT> > MT> > MT> >>>>> behaviour are unknown. Little bit of = tinkering showed that some of them run=20 > MT> > MT> > MT> > MT> >>>>> Debian: > MT> > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> > MT> >>>>> telnet 88.99.60.171 22 > MT> > MT> > MT> > MT> >>>>> Trying 88.99.60.171... > MT> > MT> > MT> > MT> >>>>> Connected to 88.99.60.171. > MT> > MT> > MT> > MT> >>>>> Escape character is '^]'. > MT> > MT> > MT> > MT> >>>>> SSH-2.0-OpenSSH_6.7p1 Debian-5+deb8u3 > MT> > MT> > MT> > MT> >>>> Also some are hosted by Hetzner, but not = all. I'll will look into > MT> > MT> > MT> > MT> >>>> this tomorrow, since I'm on a deadline = today (well it is 2am tomorrow > MT> > MT> > MT> > MT> >>>> morning, to be precise)... > MT> > MT> > MT> > MT> >>>=20 > MT> > MT> > MT> > MT> >>> Thanks a lot, I would appreciate that. > MT> > MT> > MT> > MT> >> Hi Paul, > MT> > MT> > MT> > MT> >>=20 > MT> > MT> > MT> > MT> >> I have looked into this. > MT> > MT> > MT> > MT> >>=20 > MT> > MT> > MT> > MT> >> * The FreeBSD behaviour is the one which is = specified in the last bullet item > MT> > MT> > MT> > MT> >> in = https://tools.ietf.org/html/rfc7323#section-5.4 > MT> > MT> > MT> > MT> >> It is also the one, which is RECOMMENDED in > MT> > MT> > MT> > MT> >> = https://tools.ietf.org/html/rfc7323#section-7.1=20 > MT> > MT> > MT> > MT> >>=20 > MT> > MT> > MT> > MT> >> * My NAT box (a popular one in Germany) does = NOT rewrite TCP timestamps. > MT> > MT> > MT> > MT> >>=20 > MT> > MT> > MT> > MT> >> This means that the host you are referring to = have some sort of protection, > MT> > MT> > MT> > MT> >> which makes incorrect assumptions. It will = also break multiple hosts behind > MT> > MT> > MT> > MT> >> a NAT. > MT> > MT> > MT> > MT> >>=20 > MT> > MT> > MT> > MT> >> I can run > MT> > MT> > MT> > MT> >> curl -v http://88.99.60.171:80 > MT> > MT> > MT> > MT> >> in a loop without any problems from a FreeBSD = head system. I tested 1000 > MT> > MT> > MT> > MT> >> iterations or so. The TS.val is jumping up = and down as expected. > MT> > MT> > MT> > MT> >> I'm wondering why you are observing errors in = this case, too. > MT> > MT> > MT> > MT> >>=20 > MT> > MT> > MT> > MT> >> However, doing something like > MT> > MT> > MT> > MT> >> echo "foooooo" | nc -v 88.99.60.171 80 > MT> > MT> > MT> > MT> >> triggers the problem. > MT> > MT> > MT> > MT> >>=20 > MT> > MT> > MT> > MT> >> So I think there is some functionality (in a = middlebox or running on the host), > MT> > MT> > MT> > MT> >> which incorrectly assume monotonic timestamps = between multiple TCP connections > MT> > MT> > MT> > MT> >> coming from the same IP address, but only in = case of errors at the application layer. > MT> > MT> > MT> > MT> >=20 > MT> > MT> > MT> > MT> > Yeah, exactly, some hosts seem to enable this = only in case of an error in HTTP > MT> > MT> > MT> > MT> > communication (some smart proxy?). However, = there are some that behave this way > MT> > MT> > MT> > MT> > regardless of errors, for example these: > MT> > MT> > MT> > MT> >=20 > MT> > MT> > MT> > MT> > curl -v https://185.134.205.105:443 > MT> > MT> > MT> > MT> > curl -v https://136.243.1.231:443 > MT> > MT> > MT> > MT> Wireshark sees an Encrypted Alert in both cases. = So I guess this is another indication > MT> > MT> > MT> > MT> of "error at the application layer". > MT> > MT> > MT> > MT> >=20 > MT> > MT> > MT> > MT> >>=20 > MT> > MT> > MT> > MT> >> Do you have any insights whether the hosts = you are listed share something in > MT> > MT> > MT> > MT> >> common. Some of them are hosted by Hetzner, = but not all. > MT> > MT> > MT> > MT> >=20 > MT> > MT> > MT> > MT> > Nope. A whole set of endpoints that we have = detected so far is pretty diverse, > MT> > MT> > MT> > MT> > containing a lot of different locations = geographically, as well as different > MT> > MT> > MT> > MT> > hosters. > MT> > MT> > MT> > MT> OK. Thanks for the clarification. > MT> > MT> > MT> > MT> >=20 > MT> > MT> > MT> > MT> >>=20 > MT> > MT> > MT> > MT> >> I think in general, it is the correct thing = to include the port numbers in > MT> > MT> > MT> > MT> >> the offset computation. We might add a sysctl = variable to control the inclusion. > MT> > MT> > MT> > MT> >> This would allow interworking with broken = middleboxes. > MT> > MT> > MT> > MT> >=20 > MT> > MT> > MT> > MT> > Yeah, I completely agree that these rare cases = should not dictate the implementation. > MT> > MT> > MT> > MT> > But an ability to enable a work-around via = sysctl would be greatly appreciated. > MT> > MT> > MT> > MT> > Currently we are unable to roll-out the = upgrade across all servers because of this > MT> > MT> > MT> > MT> > issue: even though it happens not so often, a = lot of requests from our users=20 > MT> > MT> > MT> > MT> > get stuck or fail all together. For example, a = host 185.134.205.105 is a kind of > MT> > MT> > MT> > MT> > social network that our proxy servers connect = to so securely access to content, > MT> > MT> > MT> > MT> > such as images, on behalf of our users. > MT> > MT> > MT> > MT> >=20 > MT> > MT> > MT> > MT> >>=20 > MT> > MT> > MT> > MT> >> Please note, this does not fix the case of = multiple clients behind a NAT. > MT> > MT> > MT> > MT> >=20 > MT> > MT> > MT> > MT> > Yeah, that's true. Fortunately we don't use = NAT. > MT> > MT> > MT> > MT> >=20 > MT> > MT> > MT> > MT> >>=20 > MT> > MT> > MT> > MT> >> I'm also trying to figure out how and why = Linux and Windows are handling this. > MT> > MT> > MT> > MT> >=20 > MT> > MT> > MT> > MT> > Thanks for bothering! > MT> > MT> > MT> > MT> Will let you know what I figure out. > MT> > MT> > MT> > MT>=20 > MT> > MT> > MT> > MT> Best regards > MT> > MT> > MT> > MT> Michael > MT> > MT> > MT> > MT> >=20 > MT> > MT> > MT> > MT> >>=20 > MT> > MT> > MT> > MT> >> Best regards > MT> > MT> > MT> > MT> >> Michael > MT> > MT> > MT> > MT> >>=20 > MT> > MT> > MT> > MT> >>>=20 > MT> > MT> > MT> > MT> >>>>=20 > MT> > MT> > MT> > MT> >>>> Best regards > MT> > MT> > MT> > MT> >>>> Michael=20 > MT> > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> > MT> >>>>>=20 > MT> > MT> > MT> > MT> >>>>>>=20 > MT> > MT> > MT> > MT> >>>>>> Best regards > MT> > MT> > MT> > MT> >>>>>> Michael > MT> > MT> > MT> > MT> >>>>>>=20 > MT> > MT> > MT> > MT> >>>>>>=20 > MT> > MT> > MT> > MT> >>>>=20 > MT> > MT> > MT> > MT> >>>>=20 > MT> > MT> > MT> > MT> >>=20 > MT> > MT> > MT> > MT> >>=20 > MT> > MT> > MT> > MT>=20 > MT> > MT> > MT> > MT> _______________________________________________ > MT> > MT> > MT> > MT> freebsd-net@freebsd.org mailing list > MT> > MT> > MT> > MT> = https://lists.freebsd.org/mailman/listinfo/freebsd-net > MT> > MT> > MT> > MT> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" > MT> > MT> > MT>=20 > MT> > MT> > MT> _______________________________________________ > MT> > MT> > MT> freebsd-net@freebsd.org mailing list > MT> > MT> > MT> https://lists.freebsd.org/mailman/listinfo/freebsd-net > MT> > MT> > MT> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" > MT> > MT>=20 > MT> > _______________________________________________ > MT> > freebsd-net@freebsd.org mailing list > MT> > https://lists.freebsd.org/mailman/listinfo/freebsd-net > MT> > To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" > MT>=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Wed Jul 17 16:10:11 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E1359B0139 for ; Wed, 17 Jul 2019 16:10:11 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: from mail-wm1-x342.google.com (mail-wm1-x342.google.com [IPv6:2a00:1450:4864:20::342]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7C988883EF for ; Wed, 17 Jul 2019 16:10:11 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: by mail-wm1-x342.google.com with SMTP id s3so22744667wms.2 for ; Wed, 17 Jul 2019 09:10:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kev009.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=GNNin4lEhkwWSa0gCBeV0+O7Zkg17m0bcAzc6G4XirU=; b=lqzezjMoAXzxodimIIugz4Gjs+h0XGV0zs7ll9/atflA5iGIyRG1vQDo2W0IeJHsQY 7qYR75g5hQa+YcZBAUYjMwO4/NfKIvAheN3YUPJQGBV7tGgV00CAi7i2yse+MyKJo3xv jSVPh2ktXsWhh97p60ubH49bs2YsXGCc6CDIE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=GNNin4lEhkwWSa0gCBeV0+O7Zkg17m0bcAzc6G4XirU=; b=FF00NBenGDvJ6NtMn+xshH5CbbBGIltYbtNtP/YRGo3/o1NfHJadqY7TBEqLPOTPsr W/du/QnoxB3MTcp3o/PjHakxzPkjJ9OBNGPHHFjdYMklF9uItbuzv5OkhAs6c+vW0LIR C7HLJjepIBXxsdVG8gcb/B45gMbswb8tf+o5wkh4TvWriHcRdJPcCgtVTu9RmnOfxixF +0natYe7z7SOiWQ/kjbmJioJKT9S9bf/jtVfeUdX35lGDVj2qJvwm0IyIsLO2lD+rriU sf3g01i4m9RCJT5k93FrD9oAR4by+Lacl5biATf320DWB+koPOh0LklH7wIFcreQAuvu Gcrg== X-Gm-Message-State: APjAAAVSsG0uyrEs7qWoqInDBvFh/7qE2s2PSNONSJnpzP48gdztwTiw X54cAkSh329Xl1JBNcQstxS7nFbW1zzQuXFkvgU= X-Google-Smtp-Source: APXvYqxIZLBQVlFFCfVxFyMfguDvcj/CBtZZauiJFveCDsecdfcJNfLywFz7inLbm2M9YDnQT/CLor1NWM+g5KG9upE= X-Received: by 2002:a1c:6a0e:: with SMTP id f14mr39555487wmc.154.1563379809865; Wed, 17 Jul 2019 09:10:09 -0700 (PDT) MIME-Version: 1.0 References: <1562599181.734953000.1l9a1d23@frv39.fwdcdn.com> <0C475A01-9BCD-4E4A-9731-09AB919CA9BE@freebsd.org> <1562676414.933145000.z3zteyqp@frv39.fwdcdn.com> <1E9F3F99-C3E9-44DD-AA70-9B11E19D4769@freebsd.org> <20190717074243.GA65665@hell.ukr.net> <20190717100926.GA24984@hell.ukr.net> <48817BF6-AEDD-4D28-95F8-A4D53E4999B1@freebsd.org> <20190717115502.GA53155@hell.ukr.net> <8763FDC7-8B71-41C3-8D1C-10416DA9A871@freebsd.org> <20190717123251.GA53638@hell.ukr.net> In-Reply-To: From: Kevin Bowling Date: Wed, 17 Jul 2019 09:09:58 -0700 Message-ID: Subject: Re: Issues with TCP Timestamps allocation To: Michael Tuexen Cc: Paul , Vitalij Satanivskij , freebsd-net@freebsd.org X-Rspamd-Queue-Id: 7C988883EF X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.96 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_SHORT(-0.96)[-0.965,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jul 2019 16:10:11 -0000 Any knowledge of the endpoints, Linux boxes misconfigured with tcp_tw_recycle? On Wed, Jul 17, 2019 at 5:42 AM Michael Tuexen wrote: > > On 17. Jul 2019, at 14:32, Vitalij Satanivskij wrote: > > > > Hmm, looks like with some host's work but not with another > > > > Wed/17.07:/home/satan > > hell:-1522/15:28>curl https://volia.com > /dev/null > > % Total % Received % Xferd Average Speed Time Time Time > Current > > Dload Upload Total Spent Left > Speed > > 100 41519 0 41519 0 0 137k 0 --:--:-- --:--:-- > --:--:-- 137k > > Wed/17.07:/home/satan > > hell:-1523/15:28>curl https://volia.com > /dev/null > > % Total % Received % Xferd Average Speed Time Time Time > Current > > Dload Upload Total Spent Left > Speed > > 0 0 0 0 0 0 0 0 --:--:-- 0:00:53 --:--:-- > 0^C > > Wed/17.07:/home/satan > > hell:-1524/15:29>sysctl net.inet.tcp.rexmit_drop_options > > net.inet.tcp.rexmit_drop_options: 1 > OK, I can confirm that for https://volia.com only a timeout helps. > > What I observed for now is that for the "blocking" to occur is it crucial > that > the server sends the FIN and therefore goes into the TIMEWAIT state. The > timeout > seems to be 60 seconds. > The blocking is also not limited to a single server port. > > I'm not sure yet whether it is a broken end point or a broken middle box. > > Best regards > Michael > > > > But > > > > MT> Interesting. It works for me: > > MT> > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time > Time Current > > MT> Dload Upload Total Spent > Left Speed > > MT> 100 18265 0 18265 0 0 33637 0 --:--:-- --:--:-- > --:--:-- 33575 > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time > Time Current > > MT> Dload Upload Total Spent > Left Speed > > MT> 100 18265 0 18265 0 0 4834 0 --:--:-- 0:00:03 > --:--:-- 4833 > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time > Time Current > > MT> Dload Upload Total Spent > Left Speed > > MT> 100 18265 0 18265 0 0 35813 0 --:--:-- --:--:-- > --:--:-- 35813 > > MT> tuexen@head:~ % time curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time > Time Current > > MT> Dload Upload Total Spent > Left Speed > > MT> 100 18265 0 18265 0 0 48320 0 --:--:-- --:--:-- > --:--:-- 48320 > > MT> 0.012u 0.031s 0:00.39 10.2% 140+245k 0+0io 0pf+0w > > MT> tuexen@head:~ % time curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time > Time Current > > MT> Dload Upload Total Spent > Left Speed > > MT> 100 18265 0 18265 0 0 4592 0 --:--:-- 0:00:03 > --:--:-- 4591 > > MT> 0.031u 0.010s 0:03.99 1.0% 80+140k 0+0io 0pf+0w > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time > Time Current > > MT> Dload Upload Total Spent > Left Speed > > MT> 100 18265 0 18265 0 0 37815 0 --:--:-- --:--:-- > --:--:-- 37737 > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time > Time Current > > MT> Dload Upload Total Spent > Left Speed > > MT> 100 18265 0 18265 0 0 27261 0 --:--:-- --:--:-- > --:--:-- 27220 > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time > Time Current > > MT> Dload Upload Total Spent > Left Speed > > MT> 100 18265 0 18265 0 0 4533 0 --:--:-- 0:00:04 > --:--:-- 4533 > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time > Time Current > > MT> Dload Upload Total Spent > Left Speed > > MT> 100 18265 0 18265 0 0 48320 0 --:--:-- --:--:-- > --:--:-- 48192 > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time > Time Current > > MT> Dload Upload Total Spent > Left Speed > > MT> 100 18265 0 18265 0 0 4746 0 --:--:-- 0:00:03 > --:--:-- 4745 > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time > Time Current > > MT> Dload Upload Total Spent > Left Speed > > MT> 100 18265 0 18265 0 0 4500 0 --:--:-- 0:00:04 > --:--:-- 4767 > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time > Time Current > > MT> Dload Upload Total Spent > Left Speed > > MT> 100 18265 0 18265 0 0 4726 0 --:--:-- 0:00:03 > --:--:-- 4726 > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time > Time Current > > MT> Dload Upload Total Spent > Left Speed > > MT> 100 18265 0 18265 0 0 34268 0 --:--:-- --:--:-- > --:--:-- 34332 > > MT> tuexen@head:~ % > > MT> > > MT> So it either works immediately or with a delay of 3 to 4 seconds... > > MT> > > MT> Best regards > > MT> Michael > > MT> > > > MT> > > > MT> > MT> > > MT> > MT> Option 2: Disable the TCP timestamps (and window scaling) > > MT> > MT> To enable this, you configure on the client > > MT> > MT> sudo sysctl -w net.inet.tcp.rfc1323=0 > > MT> > MT> or put > > MT> > MT> net.inet.tcp.rfc1323=0 > > MT> > MT> in /etc/sysctl.conf > > MT> > MT> and reboot. > > MT> > MT> This disables the timestamp option and window scaling > completely. This allows you to > > MT> > MT> setup the connections without any delay. However, you don't > have the benefits of the > > MT> > MT> extension. > > MT> > MT> > > MT> > MT> Both options don't require any code changes. > > MT> > > > MT> > This option was tested some time before. Yep it's help. But overal > performance of tcp networking ... Let's say to bad :( > > MT> > > > MT> > > > MT> > > > MT> > > > MT> > MT> Best regards > > MT> > MT> Michael > > MT> > MT> > > MT> > MT> > > MT> > MT> > > > MT> > MT> > > > MT> > MT> > MT> > > MT> > MT> > MT> Best regards > > MT> > MT> > MT> Michael > > MT> > MT> > MT> > > > MT> > MT> > MT> > > > MT> > MT> > MT> > > > MT> > MT> > MT> > Michael Tuexen wrote: > > MT> > MT> > MT> > MT> > > MT> > MT> > MT> > MT> > > MT> > MT> > MT> > MT> > On 9. Jul 2019, at 14:58, Paul > wrote: > > MT> > MT> > MT> > MT> > > > MT> > MT> > MT> > MT> > Hi Michael, > > MT> > MT> > MT> > MT> > > > MT> > MT> > MT> > MT> > 9 July 2019, 15:34:29, by "Michael Tuexen" < > tuexen@freebsd.org>: > > MT> > MT> > MT> > MT> > > > MT> > MT> > MT> > MT> >> > > MT> > MT> > MT> > MT> >> > > MT> > MT> > MT> > MT> >>> On 8. Jul 2019, at 17:22, Paul > wrote: > > MT> > MT> > MT> > MT> >>> > > MT> > MT> > MT> > MT> >>> > > MT> > MT> > MT> > MT> >>> > > MT> > MT> > MT> > MT> >>> 8 July 2019, 17:12:21, by "Michael Tuexen" < > tuexen@freebsd.org>: > > MT> > MT> > MT> > MT> >>> > > MT> > MT> > MT> > MT> >>>>> On 8. Jul 2019, at 15:24, Paul < > devgs@ukr.net> wrote: > > MT> > MT> > MT> > MT> >>>>> > > MT> > MT> > MT> > MT> >>>>> Hi Michael, > > MT> > MT> > MT> > MT> >>>>> > > MT> > MT> > MT> > MT> >>>>> 8 July 2019, 15:53:15, by "Michael Tuexen" < > tuexen@freebsd.org>: > > MT> > MT> > MT> > MT> >>>>> > > MT> > MT> > MT> > MT> >>>>>>> On 8. Jul 2019, at 12:37, Paul < > devgs@ukr.net> wrote: > > MT> > MT> > MT> > MT> >>>>>>> > > MT> > MT> > MT> > MT> >>>>>>> Hi team, > > MT> > MT> > MT> > MT> >>>>>>> > > MT> > MT> > MT> > MT> >>>>>>> Recently we had an upgrade to 12 Stable. > Immediately after, we have started > > MT> > MT> > MT> > MT> >>>>>>> seeing some strange connection > establishment timeouts to some fixed number > > MT> > MT> > MT> > MT> >>>>>>> of external (world) hosts. The issue was > persistent and easy to reproduce. > > MT> > MT> > MT> > MT> >>>>>>> Thanks to a patience and dedication of our > system engineer we have tracked > > MT> > MT> > MT> > MT> >>>>>>> this issue down to a specific commit: > > MT> > MT> > MT> > MT> >>>>>>> > > MT> > MT> > MT> > MT> >>>>>>> > https://svnweb.freebsd.org/base?view=revision&revision=338053 > > MT> > MT> > MT> > MT> >>>>>>> > > MT> > MT> > MT> > MT> >>>>>>> This patch was also back-ported into 11 > Stable: > > MT> > MT> > MT> > MT> >>>>>>> > > MT> > MT> > MT> > MT> >>>>>>> > https://svnweb.freebsd.org/base?view=revision&revision=348435 > > MT> > MT> > MT> > MT> >>>>>>> > > MT> > MT> > MT> > MT> >>>>>>> Among other things this patch changes the > timestamp allocation strategy, > > MT> > MT> > MT> > MT> >>>>>>> by introducing a deterministic randomness > via a hash function that takes > > MT> > MT> > MT> > MT> >>>>>>> into account a random key as well as > source address, source port, dest > > MT> > MT> > MT> > MT> >>>>>>> address and dest port. As the result, > timestamp offsets of different > > MT> > MT> > MT> > MT> >>>>>>> tuples (SA,SP,DA,DP) will be wildly > different and will jump from small > > MT> > MT> > MT> > MT> >>>>>>> to large numbers and back, as long as > something in the tuple changes. > > MT> > MT> > MT> > MT> >>>>>> Hi Paul, > > MT> > MT> > MT> > MT> >>>>>> > > MT> > MT> > MT> > MT> >>>>>> this is correct. > > MT> > MT> > MT> > MT> >>>>>> > > MT> > MT> > MT> > MT> >>>>>> Please note that the same happens with the > old method, if two hosts with > > MT> > MT> > MT> > MT> >>>>>> different uptimes are bind a consumer grade > NAT. > > MT> > MT> > MT> > MT> >>>>> > > MT> > MT> > MT> > MT> >>>>> If NAT does not replace timestamps then yes, > it should be the case. > > MT> > MT> > MT> > MT> >>>>> > > MT> > MT> > MT> > MT> >>>>>>> > > MT> > MT> > MT> > MT> >>>>>>> After performing various tests of hosts > that produce the above mentioned > > MT> > MT> > MT> > MT> >>>>>>> issue we came to conclusion that there are > some interesting implementations > > MT> > MT> > MT> > MT> >>>>>>> that drop SYN packets with timestamps > smaller than the largest timestamp > > MT> > MT> > MT> > MT> >>>>>>> value from streams of all recent or > current connections from a specific > > MT> > MT> > MT> > MT> >>>>>>> address. This looks as some kind of SYN > flood protection. > > MT> > MT> > MT> > MT> >>>>>> This also breaks multiple hosts with > different uptimes behind a consumer > > MT> > MT> > MT> > MT> >>>>>> level NAT talking to such a server. > > MT> > MT> > MT> > MT> >>>>>>> > > MT> > MT> > MT> > MT> >>>>>>> To ensure that each external host is not > going to see a wild jumps of > > MT> > MT> > MT> > MT> >>>>>>> timestamp values I propose a patch that > removes ports from the equation > > MT> > MT> > MT> > MT> >>>>>>> all together, when calculating the > timestamp offset: > > MT> > MT> > MT> > MT> >>>>>>> > > MT> > MT> > MT> > MT> >>>>>>> Index: sys/netinet/tcp_subr.c > > MT> > MT> > MT> > MT> >>>>>>> > =================================================================== > > MT> > MT> > MT> > MT> >>>>>>> --- sys/netinet/tcp_subr.c (revision > 348435) > > MT> > MT> > MT> > MT> >>>>>>> +++ sys/netinet/tcp_subr.c (working > copy) > > MT> > MT> > MT> > MT> >>>>>>> @@ -2224,7 +2224,22 @@ > > MT> > MT> > MT> > MT> >>>>>>> uint32_t > > MT> > MT> > MT> > MT> >>>>>>> tcp_new_ts_offset(struct in_conninfo *inc) > > MT> > MT> > MT> > MT> >>>>>>> { > > MT> > MT> > MT> > MT> >>>>>>> - return (tcp_keyed_hash(inc, > V_ts_offset_secret)); > > MT> > MT> > MT> > MT> >>>>>>> + /* > > MT> > MT> > MT> > MT> >>>>>>> + * Some implementations show a > strange behaviour when a wildly random > > MT> > MT> > MT> > MT> >>>>>>> + * timestamps allocated for > different streams. It seems that only the > > MT> > MT> > MT> > MT> >>>>>>> + * SYN packets are affected. > Observed implementations drop SYN packets > > MT> > MT> > MT> > MT> >>>>>>> + * with timestamps smaller than > the largest timestamp value of all > > MT> > MT> > MT> > MT> >>>>>>> + * recent or current connections > from specific a address. To mitigate > > MT> > MT> > MT> > MT> >>>>>>> + * this we are going to ensure > that each host will always observe > > MT> > MT> > MT> > MT> >>>>>>> + * timestamps as increasing no > matter the stream: by dropping ports > > MT> > MT> > MT> > MT> >>>>>>> + * from the equation. > > MT> > MT> > MT> > MT> >>>>>>> + */ > > MT> > MT> > MT> > MT> >>>>>>> + struct in_conninfo inc_copy = > *inc; > > MT> > MT> > MT> > MT> >>>>>>> + > > MT> > MT> > MT> > MT> >>>>>>> + inc_copy.inc_fport = 0; > > MT> > MT> > MT> > MT> >>>>>>> + inc_copy.inc_lport = 0; > > MT> > MT> > MT> > MT> >>>>>>> + > > MT> > MT> > MT> > MT> >>>>>>> + return (tcp_keyed_hash(&inc_copy, > V_ts_offset_secret)); > > MT> > MT> > MT> > MT> >>>>>>> } > > MT> > MT> > MT> > MT> >>>>>>> > > MT> > MT> > MT> > MT> >>>>>>> /* > > MT> > MT> > MT> > MT> >>>>>>> > > MT> > MT> > MT> > MT> >>>>>>> In any case, the solution of the uptime > leak, implemented in rev338053 is > > MT> > MT> > MT> > MT> >>>>>>> not going to suffer, because a supposed > attacker is currently able to use > > MT> > MT> > MT> > MT> >>>>>>> any fixed values of SP and DP, albeit not > 0, anyway, to remove them out > > MT> > MT> > MT> > MT> >>>>>>> of the equation. > > MT> > MT> > MT> > MT> >>>>>> Can you describe how a peer can compute the > uptime from two observed timestamps? > > MT> > MT> > MT> > MT> >>>>>> I don't see how you can do that... > > MT> > MT> > MT> > MT> >>>>> > > MT> > MT> > MT> > MT> >>>>> Supposed attacker could run a script that > continuously monitors timestamps, > > MT> > MT> > MT> > MT> >>>>> for example via a periodic TCP connection > from a fixed local port (eg 12345) > > MT> > MT> > MT> > MT> >>>>> and a fixed local address to the fixed > victim's address and port (eg 80). > > MT> > MT> > MT> > MT> >>>>> Whenever large discrepancy is observed, > attacker can assume that reboot has > > MT> > MT> > MT> > MT> >>>>> happened (due to V_ts_offset_secret > re-generation), hence the received > > MT> > MT> > MT> > MT> >>>>> timestamp is considered an approximate point > of reboot from which the uptime > > MT> > MT> > MT> > MT> >>>>> can be calculated, until the next reboot and > so on. > > MT> > MT> > MT> > MT> >>>> Ahh, I see. The patch we are talking about is > not intended to protect against > > MT> > MT> > MT> > MT> >>>> continuous monitoring, which is something you > can always do. You could even > > MT> > MT> > MT> > MT> >>>> watch for service availability and detect > reboots. A change of the local key > > MT> > MT> > MT> > MT> >>>> would also look similar to a reboot without a > temporary loss of connectivity. > > MT> > MT> > MT> > MT> >>>> > > MT> > MT> > MT> > MT> >>>> Thanks for the clarification. > > MT> > MT> > MT> > MT> >>>>> > > MT> > MT> > MT> > MT> >>>>>>> > > MT> > MT> > MT> > MT> >>>>>>> There is the list of example hosts that we > were able to reproduce the > > MT> > MT> > MT> > MT> >>>>>>> issue with: > > MT> > MT> > MT> > MT> >>>>>>> > > MT> > MT> > MT> > MT> >>>>>>> curl -v http://88.99.60.171:80 > > MT> > MT> > MT> > MT> >>>>>>> curl -v http://163.172.71.252:80 > > MT> > MT> > MT> > MT> >>>>>>> curl -v http://5.9.242.150:80 > > MT> > MT> > MT> > MT> >>>>>>> curl -v https://185.134.205.105:443 > > MT> > MT> > MT> > MT> >>>>>>> curl -v https://136.243.1.231:443 > > MT> > MT> > MT> > MT> >>>>>>> curl -v https://144.76.196.4:443 > > MT> > MT> > MT> > MT> >>>>>>> curl -v http://94.127.191.194:80 > > MT> > MT> > MT> > MT> >>>>>>> > > MT> > MT> > MT> > MT> >>>>>>> To reproduce, call curl repeatedly with a > same URL some number of times. > > MT> > MT> > MT> > MT> >>>>>>> You are going to see some of the requests > stuck in > > MT> > MT> > MT> > MT> >>>>>>> `* Trying XXX.XXX.XXX.XXX...` > > MT> > MT> > MT> > MT> >>>>>>> > > MT> > MT> > MT> > MT> >>>>>>> For some reason, the easiest way to > reproduce the issue is with nc: > > MT> > MT> > MT> > MT> >>>>>>> > > MT> > MT> > MT> > MT> >>>>>>> $ echo "foooooo" | nc -v 88.99.60.171 80 > > MT> > MT> > MT> > MT> >>>>>>> > > MT> > MT> > MT> > MT> >>>>>>> Only a few such calls are required until > one of them is stuck on connect(): > > MT> > MT> > MT> > MT> >>>>>>> issuing SYN packets with an exponential > backoff. > > MT> > MT> > MT> > MT> >>>>>> Thanks for providing an end-point to test > with. I'll take a look. > > MT> > MT> > MT> > MT> >>>>>> Just to be clear: You are running a FreeBSD > client against one of the above > > MT> > MT> > MT> > MT> >>>>>> servers and experience the problem with the > new timestamp computations. > > MT> > MT> > MT> > MT> >>>>>> > > MT> > MT> > MT> > MT> >>>>>> You are not running arbitrary clients > against a FreeBSD server... > > MT> > MT> > MT> > MT> >>>>> > > MT> > MT> > MT> > MT> >>>>> We are talking about FreeBSD being the > client. Peers that yield this unwanted > > MT> > MT> > MT> > MT> >>>>> behaviour are unknown. Little bit of > tinkering showed that some of them run > > MT> > MT> > MT> > MT> >>>>> Debian: > > MT> > MT> > MT> > MT> >>>>> > > MT> > MT> > MT> > MT> >>>>> telnet 88.99.60.171 22 > > MT> > MT> > MT> > MT> >>>>> Trying 88.99.60.171... > > MT> > MT> > MT> > MT> >>>>> Connected to 88.99.60.171. > > MT> > MT> > MT> > MT> >>>>> Escape character is '^]'. > > MT> > MT> > MT> > MT> >>>>> SSH-2.0-OpenSSH_6.7p1 Debian-5+deb8u3 > > MT> > MT> > MT> > MT> >>>> Also some are hosted by Hetzner, but not all. > I'll will look into > > MT> > MT> > MT> > MT> >>>> this tomorrow, since I'm on a deadline today > (well it is 2am tomorrow > > MT> > MT> > MT> > MT> >>>> morning, to be precise)... > > MT> > MT> > MT> > MT> >>> > > MT> > MT> > MT> > MT> >>> Thanks a lot, I would appreciate that. > > MT> > MT> > MT> > MT> >> Hi Paul, > > MT> > MT> > MT> > MT> >> > > MT> > MT> > MT> > MT> >> I have looked into this. > > MT> > MT> > MT> > MT> >> > > MT> > MT> > MT> > MT> >> * The FreeBSD behaviour is the one which is > specified in the last bullet item > > MT> > MT> > MT> > MT> >> in > https://tools.ietf.org/html/rfc7323#section-5.4 > > MT> > MT> > MT> > MT> >> It is also the one, which is RECOMMENDED in > > MT> > MT> > MT> > MT> >> > https://tools.ietf.org/html/rfc7323#section-7.1 > > MT> > MT> > MT> > MT> >> > > MT> > MT> > MT> > MT> >> * My NAT box (a popular one in Germany) does > NOT rewrite TCP timestamps. > > MT> > MT> > MT> > MT> >> > > MT> > MT> > MT> > MT> >> This means that the host you are referring to > have some sort of protection, > > MT> > MT> > MT> > MT> >> which makes incorrect assumptions. It will also > break multiple hosts behind > > MT> > MT> > MT> > MT> >> a NAT. > > MT> > MT> > MT> > MT> >> > > MT> > MT> > MT> > MT> >> I can run > > MT> > MT> > MT> > MT> >> curl -v http://88.99.60.171:80 > > MT> > MT> > MT> > MT> >> in a loop without any problems from a FreeBSD > head system. I tested 1000 > > MT> > MT> > MT> > MT> >> iterations or so. The TS.val is jumping up and > down as expected. > > MT> > MT> > MT> > MT> >> I'm wondering why you are observing errors in > this case, too. > > MT> > MT> > MT> > MT> >> > > MT> > MT> > MT> > MT> >> However, doing something like > > MT> > MT> > MT> > MT> >> echo "foooooo" | nc -v 88.99.60.171 80 > > MT> > MT> > MT> > MT> >> triggers the problem. > > MT> > MT> > MT> > MT> >> > > MT> > MT> > MT> > MT> >> So I think there is some functionality (in a > middlebox or running on the host), > > MT> > MT> > MT> > MT> >> which incorrectly assume monotonic timestamps > between multiple TCP connections > > MT> > MT> > MT> > MT> >> coming from the same IP address, but only in > case of errors at the application layer. > > MT> > MT> > MT> > MT> > > > MT> > MT> > MT> > MT> > Yeah, exactly, some hosts seem to enable this > only in case of an error in HTTP > > MT> > MT> > MT> > MT> > communication (some smart proxy?). However, > there are some that behave this way > > MT> > MT> > MT> > MT> > regardless of errors, for example these: > > MT> > MT> > MT> > MT> > > > MT> > MT> > MT> > MT> > curl -v https://185.134.205.105:443 > > MT> > MT> > MT> > MT> > curl -v https://136.243.1.231:443 > > MT> > MT> > MT> > MT> Wireshark sees an Encrypted Alert in both cases. > So I guess this is another indication > > MT> > MT> > MT> > MT> of "error at the application layer". > > MT> > MT> > MT> > MT> > > > MT> > MT> > MT> > MT> >> > > MT> > MT> > MT> > MT> >> Do you have any insights whether the hosts you > are listed share something in > > MT> > MT> > MT> > MT> >> common. Some of them are hosted by Hetzner, but > not all. > > MT> > MT> > MT> > MT> > > > MT> > MT> > MT> > MT> > Nope. A whole set of endpoints that we have > detected so far is pretty diverse, > > MT> > MT> > MT> > MT> > containing a lot of different locations > geographically, as well as different > > MT> > MT> > MT> > MT> > hosters. > > MT> > MT> > MT> > MT> OK. Thanks for the clarification. > > MT> > MT> > MT> > MT> > > > MT> > MT> > MT> > MT> >> > > MT> > MT> > MT> > MT> >> I think in general, it is the correct thing to > include the port numbers in > > MT> > MT> > MT> > MT> >> the offset computation. We might add a sysctl > variable to control the inclusion. > > MT> > MT> > MT> > MT> >> This would allow interworking with broken > middleboxes. > > MT> > MT> > MT> > MT> > > > MT> > MT> > MT> > MT> > Yeah, I completely agree that these rare cases > should not dictate the implementation. > > MT> > MT> > MT> > MT> > But an ability to enable a work-around via > sysctl would be greatly appreciated. > > MT> > MT> > MT> > MT> > Currently we are unable to roll-out the upgrade > across all servers because of this > > MT> > MT> > MT> > MT> > issue: even though it happens not so often, a > lot of requests from our users > > MT> > MT> > MT> > MT> > get stuck or fail all together. For example, a > host 185.134.205.105 is a kind of > > MT> > MT> > MT> > MT> > social network that our proxy servers connect to > so securely access to content, > > MT> > MT> > MT> > MT> > such as images, on behalf of our users. > > MT> > MT> > MT> > MT> > > > MT> > MT> > MT> > MT> >> > > MT> > MT> > MT> > MT> >> Please note, this does not fix the case of > multiple clients behind a NAT. > > MT> > MT> > MT> > MT> > > > MT> > MT> > MT> > MT> > Yeah, that's true. Fortunately we don't use NAT. > > MT> > MT> > MT> > MT> > > > MT> > MT> > MT> > MT> >> > > MT> > MT> > MT> > MT> >> I'm also trying to figure out how and why Linux > and Windows are handling this. > > MT> > MT> > MT> > MT> > > > MT> > MT> > MT> > MT> > Thanks for bothering! > > MT> > MT> > MT> > MT> Will let you know what I figure out. > > MT> > MT> > MT> > MT> > > MT> > MT> > MT> > MT> Best regards > > MT> > MT> > MT> > MT> Michael > > MT> > MT> > MT> > MT> > > > MT> > MT> > MT> > MT> >> > > MT> > MT> > MT> > MT> >> Best regards > > MT> > MT> > MT> > MT> >> Michael > > MT> > MT> > MT> > MT> >> > > MT> > MT> > MT> > MT> >>> > > MT> > MT> > MT> > MT> >>>> > > MT> > MT> > MT> > MT> >>>> Best regards > > MT> > MT> > MT> > MT> >>>> Michael > > MT> > MT> > MT> > MT> >>>>> > > MT> > MT> > MT> > MT> >>>>> > > MT> > MT> > MT> > MT> >>>>>> > > MT> > MT> > MT> > MT> >>>>>> Best regards > > MT> > MT> > MT> > MT> >>>>>> Michael > > MT> > MT> > MT> > MT> >>>>>> > > MT> > MT> > MT> > MT> >>>>>> > > MT> > MT> > MT> > MT> >>>> > > MT> > MT> > MT> > MT> >>>> > > MT> > MT> > MT> > MT> >> > > MT> > MT> > MT> > MT> >> > > MT> > MT> > MT> > MT> > > MT> > MT> > MT> > MT> _______________________________________________ > > MT> > MT> > MT> > MT> freebsd-net@freebsd.org mailing list > > MT> > MT> > MT> > MT> > https://lists.freebsd.org/mailman/listinfo/freebsd-net > > MT> > MT> > MT> > MT> To unsubscribe, send any mail to " > freebsd-net-unsubscribe@freebsd.org" > > MT> > MT> > MT> > > MT> > MT> > MT> _______________________________________________ > > MT> > MT> > MT> freebsd-net@freebsd.org mailing list > > MT> > MT> > MT> https://lists.freebsd.org/mailman/listinfo/freebsd-net > > MT> > MT> > MT> To unsubscribe, send any mail to " > freebsd-net-unsubscribe@freebsd.org" > > MT> > MT> > > MT> > _______________________________________________ > > MT> > freebsd-net@freebsd.org mailing list > > MT> > https://lists.freebsd.org/mailman/listinfo/freebsd-net > > MT> > To unsubscribe, send any mail to " > freebsd-net-unsubscribe@freebsd.org" > > MT> > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@freebsd.org Wed Jul 17 17:34:23 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4D908B20BA for ; Wed, 17 Jul 2019 17:34:23 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 21F4A8BFD3; Wed, 17 Jul 2019 17:34:23 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from reviews.nyi.freebsd.org (reviews.nyi.freebsd.org [IPv6:2610:1c1:1:607c::16:b]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 8276C21E2C; Wed, 17 Jul 2019 17:34:22 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by reviews.nyi.freebsd.org (Postfix, from userid 1346) id 81E4F18AEB2; Wed, 17 Jul 2019 17:34:22 +0000 (UTC) Date: Wed, 17 Jul 2019 17:34:22 +0000 To: Phabricator From: "aleksandr.fedorov_itglobal.com (Aleksandr Fedorov)" Cc: freebsd-net@freebsd.org Reply-to: "aleksandr.fedorov_itglobal.com (Aleksandr Fedorov)" Subject: [Differential] D19422: if_vxlan(4) Allow set MTU more than 1500 bytes. Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: X-Herald-Rules: <81>, <110> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk Thread-Topic: PHID-DREV-bvtxxu4jwhzdkwqxxgd7 X-Phabricator-Mail-ID: 1541685 X-Phabricator-Send-Attempt: l6emmdfpus6iyj7h In-Reply-To: References: Thread-Index: MzYyMWIxNGMwOTE4YmQzMWVhNTU2NTFhMGQ2IF0vXB4= X-Phabricator-Stamps: actor(@aleksandr.fedorov_itglobal.com) application(Differential) author(@aleksandr.fedorov_itglobal.com) herald(H81) herald(H110) monogram(D19422) object-type(DREV) phid(PHID-DREV-bvtxxu4jwhzdkwqxxgd7) reviewer(#network) reviewer(@bryanv) reviewer(@hrs) reviewer(@jhb) reviewer(@krion) reviewer(@rgrimes) revision-status(accepted) subscriber(@ae) subscriber(@evgueni.gavrilov_itglobal.com) subscriber(@freebsd-net-list) subscriber(@olevole_olevole.ru) via(web) MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8" X-Rspamd-Queue-Id: 21F4A8BFD3 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.95 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_SHORT(-0.96)[-0.956,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jul 2019 17:34:23 -0000 YWxla3NhbmRyLmZlZG9yb3ZfaXRnbG9iYWwuY29tIGFkZGVkIGEgY29tbWVudC4KCgogIHBpbmc/ CgpDSEFOR0VTIFNJTkNFIExBU1QgQUNUSU9OCiAgaHR0cHM6Ly9yZXZpZXdzLmZyZWVic2Qub3Jn L0QxOTQyMi9uZXcvCgpSRVZJU0lPTiBERVRBSUwKICBodHRwczovL3Jldmlld3MuZnJlZWJzZC5v cmcvRDE5NDIyCgpFTUFJTCBQUkVGRVJFTkNFUwogIGh0dHBzOi8vcmV2aWV3cy5mcmVlYnNkLm9y Zy9zZXR0aW5ncy9wYW5lbC9lbWFpbHByZWZlcmVuY2VzLwoKVG86IGFsZWtzYW5kci5mZWRvcm92 X2l0Z2xvYmFsLmNvbSwgYnJ5YW52LCBocnMsICNuZXR3b3JrLCByZ3JpbWVzLCBrcmlvbiwgamhi CkNjOiBldmd1ZW5pLmdhdnJpbG92X2l0Z2xvYmFsLmNvbSwgb2xldm9sZV9vbGV2b2xlLnJ1LCBh ZSwgZnJlZWJzZC1uZXQtbGlzdCwga3J6eXN6dG9mLmdhbGF6a2FfaW50ZWwuY29tCg== From owner-freebsd-net@freebsd.org Wed Jul 17 18:01:04 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 98817B27C6 for ; Wed, 17 Jul 2019 18:01:04 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (mail-n.franken.de [193.175.24.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2BDC58D23A for ; Wed, 17 Jul 2019 18:01:03 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from [IPv6:2a02:8109:1140:c3d:d878:d920:c09d:25c5] (unknown [IPv6:2a02:8109:1140:c3d:d878:d920:c09d:25c5]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTPSA id CC78471B18AEE; Wed, 17 Jul 2019 20:00:58 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: Issues with TCP Timestamps allocation From: Michael Tuexen In-Reply-To: Date: Wed, 17 Jul 2019 20:00:58 +0200 Cc: Paul , Vitalij Satanivskij , freebsd-net@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <1562599181.734953000.1l9a1d23@frv39.fwdcdn.com> <0C475A01-9BCD-4E4A-9731-09AB919CA9BE@freebsd.org> <1562676414.933145000.z3zteyqp@frv39.fwdcdn.com> <1E9F3F99-C3E9-44DD-AA70-9B11E19D4769@freebsd.org> <20190717074243.GA65665@hell.ukr.net> <20190717100926.GA24984@hell.ukr.net> <48817BF6-AEDD-4D28-95F8-A4D53E4999B1@freebsd.org> <20190717115502.GA53155@hell.ukr.net> <8763FDC7-8B71-41C3-8D1C-10416DA9A871@freebsd.org> <20190717123251.GA53638@hell.ukr.net> To: Kevin Bowling X-Mailer: Apple Mail (2.3445.104.11) X-Spam-Status: No, score=-1.7 required=5.0 tests=ALL_TRUSTED,BAYES_00, NORMAL_HTTP_TO_IP,NUMERIC_HTTP_ADDR autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-Rspamd-Queue-Id: 2BDC58D23A X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.92 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.92)[-0.923,0]; ASN(0.00)[asn:680, ipnet:193.174.0.0/15, country:DE]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jul 2019 18:01:04 -0000 > On 17. Jul 2019, at 18:09, Kevin Bowling = wrote: >=20 > Any knowledge of the endpoints, Linux boxes misconfigured with = tcp_tw_recycle? I contacted some Linux guys and they told me that tcp_tw_recycle is = specific to the 5 tuple. Is that not correct? The servers can be running Linux, I haven't checked all of them... For the problem we are seeing here, it port numbers are irrelevant. If = the server sends a FIN (and therefore goes to TIMEWAIT), one can experience = the problem, even if the client changes the port number and even if you talk to other server ports. I tested with the ssh port in addition to the web traffic. Best regards Michael >=20 > On Wed, Jul 17, 2019 at 5:42 AM Michael Tuexen = wrote: > > On 17. Jul 2019, at 14:32, Vitalij Satanivskij = wrote: > >=20 > > Hmm, looks like with some host's work but not with another > >=20 > > Wed/17.07:/home/satan > > hell:-1522/15:28>curl https://volia.com > /dev/null > > % Total % Received % Xferd Average Speed Time Time = Time Current > > Dload Upload Total Spent = Left Speed > > 100 41519 0 41519 0 0 137k 0 --:--:-- --:--:-- = --:--:-- 137k > > Wed/17.07:/home/satan > > hell:-1523/15:28>curl https://volia.com > /dev/null > > % Total % Received % Xferd Average Speed Time Time = Time Current > > Dload Upload Total Spent = Left Speed > > 0 0 0 0 0 0 0 0 --:--:-- 0:00:53 = --:--:-- 0^C > > Wed/17.07:/home/satan > > hell:-1524/15:29>sysctl net.inet.tcp.rexmit_drop_options > > net.inet.tcp.rexmit_drop_options: 1 > OK, I can confirm that for https://volia.com only a timeout helps. >=20 > What I observed for now is that for the "blocking" to occur is it = crucial that > the server sends the FIN and therefore goes into the TIMEWAIT state. = The timeout > seems to be 60 seconds. > The blocking is also not limited to a single server port. >=20 > I'm not sure yet whether it is a broken end point or a broken middle = box. >=20 > Best regards > Michael > >=20 > > But=20 > >=20 > > MT> Interesting. It works for me: > > MT>=20 > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > > MT> Dload Upload Total Spent = Left Speed > > MT> 100 18265 0 18265 0 0 33637 0 --:--:-- --:--:-- = --:--:-- 33575 > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > > MT> Dload Upload Total Spent = Left Speed > > MT> 100 18265 0 18265 0 0 4834 0 --:--:-- 0:00:03 = --:--:-- 4833 > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > > MT> Dload Upload Total Spent = Left Speed > > MT> 100 18265 0 18265 0 0 35813 0 --:--:-- --:--:-- = --:--:-- 35813 > > MT> tuexen@head:~ % time curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > > MT> Dload Upload Total Spent = Left Speed > > MT> 100 18265 0 18265 0 0 48320 0 --:--:-- --:--:-- = --:--:-- 48320 > > MT> 0.012u 0.031s 0:00.39 10.2% 140+245k 0+0io 0pf+0w > > MT> tuexen@head:~ % time curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > > MT> Dload Upload Total Spent = Left Speed > > MT> 100 18265 0 18265 0 0 4592 0 --:--:-- 0:00:03 = --:--:-- 4591 > > MT> 0.031u 0.010s 0:03.99 1.0% 80+140k 0+0io 0pf+0w > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > > MT> Dload Upload Total Spent = Left Speed > > MT> 100 18265 0 18265 0 0 37815 0 --:--:-- --:--:-- = --:--:-- 37737 > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > > MT> Dload Upload Total Spent = Left Speed > > MT> 100 18265 0 18265 0 0 27261 0 --:--:-- --:--:-- = --:--:-- 27220 > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > > MT> Dload Upload Total Spent = Left Speed > > MT> 100 18265 0 18265 0 0 4533 0 --:--:-- 0:00:04 = --:--:-- 4533 > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > > MT> Dload Upload Total Spent = Left Speed > > MT> 100 18265 0 18265 0 0 48320 0 --:--:-- --:--:-- = --:--:-- 48192 > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > > MT> Dload Upload Total Spent = Left Speed > > MT> 100 18265 0 18265 0 0 4746 0 --:--:-- 0:00:03 = --:--:-- 4745 > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > > MT> Dload Upload Total Spent = Left Speed > > MT> 100 18265 0 18265 0 0 4500 0 --:--:-- 0:00:04 = --:--:-- 4767 > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > > MT> Dload Upload Total Spent = Left Speed > > MT> 100 18265 0 18265 0 0 4726 0 --:--:-- 0:00:03 = --:--:-- 4726 > > MT> tuexen@head:~ % curl https://vitagramma.com > /dev/null > > MT> % Total % Received % Xferd Average Speed Time Time = Time Current > > MT> Dload Upload Total Spent = Left Speed > > MT> 100 18265 0 18265 0 0 34268 0 --:--:-- --:--:-- = --:--:-- 34332 > > MT> tuexen@head:~ %=20 > > MT>=20 > > MT> So it either works immediately or with a delay of 3 to 4 = seconds... > > MT>=20 > > MT> Best regards > > MT> Michael > > MT> >=20 > > MT> >=20 > > MT> > MT>=20 > > MT> > MT> Option 2: Disable the TCP timestamps (and window scaling) > > MT> > MT> To enable this, you configure on the client > > MT> > MT> sudo sysctl -w net.inet.tcp.rfc1323=3D0 > > MT> > MT> or put > > MT> > MT> net.inet.tcp.rfc1323=3D0 > > MT> > MT> in /etc/sysctl.conf > > MT> > MT> and reboot. > > MT> > MT> This disables the timestamp option and window scaling = completely. This allows you to > > MT> > MT> setup the connections without any delay. However, you = don't have the benefits of the > > MT> > MT> extension. > > MT> > MT>=20 > > MT> > MT> Both options don't require any code changes. > > MT> >=20 > > MT> > This option was tested some time before. Yep it's help. But = overal performance of tcp networking ... Let's say to bad :( > > MT> >=20 > > MT> >=20 > > MT> >=20 > > MT> >=20 > > MT> > MT> Best regards > > MT> > MT> Michael > > MT> > MT>=20 > > MT> > MT>=20 > > MT> > MT> >=20 > > MT> > MT> >=20 > > MT> > MT> > MT>=20 > > MT> > MT> > MT> Best regards > > MT> > MT> > MT> Michael > > MT> > MT> > MT> >=20 > > MT> > MT> > MT> >=20 > > MT> > MT> > MT> >=20 > > MT> > MT> > MT> > Michael Tuexen wrote: > > MT> > MT> > MT> > MT>=20 > > MT> > MT> > MT> > MT>=20 > > MT> > MT> > MT> > MT> > On 9. Jul 2019, at 14:58, Paul = wrote: > > MT> > MT> > MT> > MT> >=20 > > MT> > MT> > MT> > MT> > Hi Michael, > > MT> > MT> > MT> > MT> >=20 > > MT> > MT> > MT> > MT> > 9 July 2019, 15:34:29, by "Michael Tuexen" = : > > MT> > MT> > MT> > MT> >=20 > > MT> > MT> > MT> > MT> >>=20 > > MT> > MT> > MT> > MT> >>=20 > > MT> > MT> > MT> > MT> >>> On 8. Jul 2019, at 17:22, Paul = wrote: > > MT> > MT> > MT> > MT> >>>=20 > > MT> > MT> > MT> > MT> >>>=20 > > MT> > MT> > MT> > MT> >>>=20 > > MT> > MT> > MT> > MT> >>> 8 July 2019, 17:12:21, by "Michael Tuexen" = : > > MT> > MT> > MT> > MT> >>>=20 > > MT> > MT> > MT> > MT> >>>>> On 8. Jul 2019, at 15:24, Paul = wrote: > > MT> > MT> > MT> > MT> >>>>>=20 > > MT> > MT> > MT> > MT> >>>>> Hi Michael, > > MT> > MT> > MT> > MT> >>>>>=20 > > MT> > MT> > MT> > MT> >>>>> 8 July 2019, 15:53:15, by "Michael = Tuexen" : > > MT> > MT> > MT> > MT> >>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>> On 8. Jul 2019, at 12:37, Paul = wrote: > > MT> > MT> > MT> > MT> >>>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>> Hi team, > > MT> > MT> > MT> > MT> >>>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>> Recently we had an upgrade to 12 = Stable. Immediately after, we have started=20 > > MT> > MT> > MT> > MT> >>>>>>> seeing some strange connection = establishment timeouts to some fixed number > > MT> > MT> > MT> > MT> >>>>>>> of external (world) hosts. The issue = was persistent and easy to reproduce. > > MT> > MT> > MT> > MT> >>>>>>> Thanks to a patience and dedication of = our system engineer we have tracked =20 > > MT> > MT> > MT> > MT> >>>>>>> this issue down to a specific commit: > > MT> > MT> > MT> > MT> >>>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>> = https://svnweb.freebsd.org/base?view=3Drevision&revision=3D338053 > > MT> > MT> > MT> > MT> >>>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>> This patch was also back-ported into = 11 Stable: > > MT> > MT> > MT> > MT> >>>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>> = https://svnweb.freebsd.org/base?view=3Drevision&revision=3D348435 > > MT> > MT> > MT> > MT> >>>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>> Among other things this patch changes = the timestamp allocation strategy, > > MT> > MT> > MT> > MT> >>>>>>> by introducing a deterministic = randomness via a hash function that takes > > MT> > MT> > MT> > MT> >>>>>>> into account a random key as well as = source address, source port, dest > > MT> > MT> > MT> > MT> >>>>>>> address and dest port. As the result, = timestamp offsets of different > > MT> > MT> > MT> > MT> >>>>>>> tuples (SA,SP,DA,DP) will be wildly = different and will jump from small=20 > > MT> > MT> > MT> > MT> >>>>>>> to large numbers and back, as long as = something in the tuple changes. > > MT> > MT> > MT> > MT> >>>>>> Hi Paul, > > MT> > MT> > MT> > MT> >>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>> this is correct. > > MT> > MT> > MT> > MT> >>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>> Please note that the same happens with = the old method, if two hosts with > > MT> > MT> > MT> > MT> >>>>>> different uptimes are bind a consumer = grade NAT. > > MT> > MT> > MT> > MT> >>>>>=20 > > MT> > MT> > MT> > MT> >>>>> If NAT does not replace timestamps then = yes, it should be the case. > > MT> > MT> > MT> > MT> >>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>> After performing various tests of = hosts that produce the above mentioned=20 > > MT> > MT> > MT> > MT> >>>>>>> issue we came to conclusion that there = are some interesting implementations=20 > > MT> > MT> > MT> > MT> >>>>>>> that drop SYN packets with timestamps = smaller than the largest timestamp=20 > > MT> > MT> > MT> > MT> >>>>>>> value from streams of all recent or = current connections from a specific=20 > > MT> > MT> > MT> > MT> >>>>>>> address. This looks as some kind of = SYN flood protection. > > MT> > MT> > MT> > MT> >>>>>> This also breaks multiple hosts with = different uptimes behind a consumer > > MT> > MT> > MT> > MT> >>>>>> level NAT talking to such a server. > > MT> > MT> > MT> > MT> >>>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>> To ensure that each external host is = not going to see a wild jumps of=20 > > MT> > MT> > MT> > MT> >>>>>>> timestamp values I propose a patch = that removes ports from the equation > > MT> > MT> > MT> > MT> >>>>>>> all together, when calculating the = timestamp offset: > > MT> > MT> > MT> > MT> >>>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>> Index: sys/netinet/tcp_subr.c > > MT> > MT> > MT> > MT> >>>>>>> = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > MT> > MT> > MT> > MT> >>>>>>> --- sys/netinet/tcp_subr.c = (revision 348435) > > MT> > MT> > MT> > MT> >>>>>>> +++ sys/netinet/tcp_subr.c = (working copy) > > MT> > MT> > MT> > MT> >>>>>>> @@ -2224,7 +2224,22 @@ > > MT> > MT> > MT> > MT> >>>>>>> uint32_t > > MT> > MT> > MT> > MT> >>>>>>> tcp_new_ts_offset(struct in_conninfo = *inc) > > MT> > MT> > MT> > MT> >>>>>>> { > > MT> > MT> > MT> > MT> >>>>>>> - return (tcp_keyed_hash(inc, = V_ts_offset_secret)); > > MT> > MT> > MT> > MT> >>>>>>> + /*=20 > > MT> > MT> > MT> > MT> >>>>>>> + * Some implementations show = a strange behaviour when a wildly random=20 > > MT> > MT> > MT> > MT> >>>>>>> + * timestamps allocated for = different streams. It seems that only the > > MT> > MT> > MT> > MT> >>>>>>> + * SYN packets are affected. = Observed implementations drop SYN packets > > MT> > MT> > MT> > MT> >>>>>>> + * with timestamps smaller = than the largest timestamp value of all=20 > > MT> > MT> > MT> > MT> >>>>>>> + * recent or current = connections from specific a address. To mitigate=20 > > MT> > MT> > MT> > MT> >>>>>>> + * this we are going to = ensure that each host will always observe=20 > > MT> > MT> > MT> > MT> >>>>>>> + * timestamps as increasing = no matter the stream: by dropping ports > > MT> > MT> > MT> > MT> >>>>>>> + * from the equation. > > MT> > MT> > MT> > MT> >>>>>>> + */=20 > > MT> > MT> > MT> > MT> >>>>>>> + struct in_conninfo inc_copy =3D= *inc; > > MT> > MT> > MT> > MT> >>>>>>> + > > MT> > MT> > MT> > MT> >>>>>>> + inc_copy.inc_fport =3D 0; > > MT> > MT> > MT> > MT> >>>>>>> + inc_copy.inc_lport =3D 0; > > MT> > MT> > MT> > MT> >>>>>>> + > > MT> > MT> > MT> > MT> >>>>>>> + return = (tcp_keyed_hash(&inc_copy, V_ts_offset_secret)); > > MT> > MT> > MT> > MT> >>>>>>> } > > MT> > MT> > MT> > MT> >>>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>> /* > > MT> > MT> > MT> > MT> >>>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>> In any case, the solution of the = uptime leak, implemented in rev338053 is=20 > > MT> > MT> > MT> > MT> >>>>>>> not going to suffer, because a = supposed attacker is currently able to use=20 > > MT> > MT> > MT> > MT> >>>>>>> any fixed values of SP and DP, albeit = not 0, anyway, to remove them out=20 > > MT> > MT> > MT> > MT> >>>>>>> of the equation. > > MT> > MT> > MT> > MT> >>>>>> Can you describe how a peer can compute = the uptime from two observed timestamps? > > MT> > MT> > MT> > MT> >>>>>> I don't see how you can do that... > > MT> > MT> > MT> > MT> >>>>>=20 > > MT> > MT> > MT> > MT> >>>>> Supposed attacker could run a script = that continuously monitors timestamps, > > MT> > MT> > MT> > MT> >>>>> for example via a periodic TCP = connection from a fixed local port (eg 12345)=20 > > MT> > MT> > MT> > MT> >>>>> and a fixed local address to the fixed = victim's address and port (eg 80). > > MT> > MT> > MT> > MT> >>>>> Whenever large discrepancy is observed, = attacker can assume that reboot has=20 > > MT> > MT> > MT> > MT> >>>>> happened (due to V_ts_offset_secret = re-generation), hence the received=20 > > MT> > MT> > MT> > MT> >>>>> timestamp is considered an approximate = point of reboot from which the uptime > > MT> > MT> > MT> > MT> >>>>> can be calculated, until the next reboot = and so on. > > MT> > MT> > MT> > MT> >>>> Ahh, I see. The patch we are talking = about is not intended to protect against > > MT> > MT> > MT> > MT> >>>> continuous monitoring, which is something = you can always do. You could even > > MT> > MT> > MT> > MT> >>>> watch for service availability and detect = reboots. A change of the local key > > MT> > MT> > MT> > MT> >>>> would also look similar to a reboot = without a temporary loss of connectivity. > > MT> > MT> > MT> > MT> >>>>=20 > > MT> > MT> > MT> > MT> >>>> Thanks for the clarification. > > MT> > MT> > MT> > MT> >>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>> There is the list of example hosts = that we were able to reproduce the=20 > > MT> > MT> > MT> > MT> >>>>>>> issue with: > > MT> > MT> > MT> > MT> >>>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>> curl -v http://88.99.60.171:80 > > MT> > MT> > MT> > MT> >>>>>>> curl -v http://163.172.71.252:80 > > MT> > MT> > MT> > MT> >>>>>>> curl -v http://5.9.242.150:80 > > MT> > MT> > MT> > MT> >>>>>>> curl -v https://185.134.205.105:443 > > MT> > MT> > MT> > MT> >>>>>>> curl -v https://136.243.1.231:443 > > MT> > MT> > MT> > MT> >>>>>>> curl -v https://144.76.196.4:443 > > MT> > MT> > MT> > MT> >>>>>>> curl -v http://94.127.191.194:80 > > MT> > MT> > MT> > MT> >>>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>> To reproduce, call curl repeatedly = with a same URL some number of times.=20 > > MT> > MT> > MT> > MT> >>>>>>> You are going to see some of the = requests stuck in=20 > > MT> > MT> > MT> > MT> >>>>>>> `* Trying XXX.XXX.XXX.XXX...` > > MT> > MT> > MT> > MT> >>>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>> For some reason, the easiest way to = reproduce the issue is with nc: > > MT> > MT> > MT> > MT> >>>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>> $ echo "foooooo" | nc -v 88.99.60.171 = 80 > > MT> > MT> > MT> > MT> >>>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>> Only a few such calls are required = until one of them is stuck on connect(): > > MT> > MT> > MT> > MT> >>>>>>> issuing SYN packets with an = exponential backoff. > > MT> > MT> > MT> > MT> >>>>>> Thanks for providing an end-point to = test with. I'll take a look. > > MT> > MT> > MT> > MT> >>>>>> Just to be clear: You are running a = FreeBSD client against one of the above > > MT> > MT> > MT> > MT> >>>>>> servers and experience the problem with = the new timestamp computations. > > MT> > MT> > MT> > MT> >>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>> You are not running arbitrary clients = against a FreeBSD server... > > MT> > MT> > MT> > MT> >>>>>=20 > > MT> > MT> > MT> > MT> >>>>> We are talking about FreeBSD being the = client. Peers that yield this unwanted > > MT> > MT> > MT> > MT> >>>>> behaviour are unknown. Little bit of = tinkering showed that some of them run=20 > > MT> > MT> > MT> > MT> >>>>> Debian: > > MT> > MT> > MT> > MT> >>>>>=20 > > MT> > MT> > MT> > MT> >>>>> telnet 88.99.60.171 22 > > MT> > MT> > MT> > MT> >>>>> Trying 88.99.60.171... > > MT> > MT> > MT> > MT> >>>>> Connected to 88.99.60.171. > > MT> > MT> > MT> > MT> >>>>> Escape character is '^]'. > > MT> > MT> > MT> > MT> >>>>> SSH-2.0-OpenSSH_6.7p1 Debian-5+deb8u3 > > MT> > MT> > MT> > MT> >>>> Also some are hosted by Hetzner, but not = all. I'll will look into > > MT> > MT> > MT> > MT> >>>> this tomorrow, since I'm on a deadline = today (well it is 2am tomorrow > > MT> > MT> > MT> > MT> >>>> morning, to be precise)... > > MT> > MT> > MT> > MT> >>>=20 > > MT> > MT> > MT> > MT> >>> Thanks a lot, I would appreciate that. > > MT> > MT> > MT> > MT> >> Hi Paul, > > MT> > MT> > MT> > MT> >>=20 > > MT> > MT> > MT> > MT> >> I have looked into this. > > MT> > MT> > MT> > MT> >>=20 > > MT> > MT> > MT> > MT> >> * The FreeBSD behaviour is the one which is = specified in the last bullet item > > MT> > MT> > MT> > MT> >> in = https://tools.ietf.org/html/rfc7323#section-5.4 > > MT> > MT> > MT> > MT> >> It is also the one, which is RECOMMENDED = in > > MT> > MT> > MT> > MT> >> = https://tools.ietf.org/html/rfc7323#section-7.1=20 > > MT> > MT> > MT> > MT> >>=20 > > MT> > MT> > MT> > MT> >> * My NAT box (a popular one in Germany) = does NOT rewrite TCP timestamps. > > MT> > MT> > MT> > MT> >>=20 > > MT> > MT> > MT> > MT> >> This means that the host you are referring = to have some sort of protection, > > MT> > MT> > MT> > MT> >> which makes incorrect assumptions. It will = also break multiple hosts behind > > MT> > MT> > MT> > MT> >> a NAT. > > MT> > MT> > MT> > MT> >>=20 > > MT> > MT> > MT> > MT> >> I can run > > MT> > MT> > MT> > MT> >> curl -v http://88.99.60.171:80 > > MT> > MT> > MT> > MT> >> in a loop without any problems from a = FreeBSD head system. I tested 1000 > > MT> > MT> > MT> > MT> >> iterations or so. The TS.val is jumping up = and down as expected. > > MT> > MT> > MT> > MT> >> I'm wondering why you are observing errors = in this case, too. > > MT> > MT> > MT> > MT> >>=20 > > MT> > MT> > MT> > MT> >> However, doing something like > > MT> > MT> > MT> > MT> >> echo "foooooo" | nc -v 88.99.60.171 80 > > MT> > MT> > MT> > MT> >> triggers the problem. > > MT> > MT> > MT> > MT> >>=20 > > MT> > MT> > MT> > MT> >> So I think there is some functionality (in = a middlebox or running on the host), > > MT> > MT> > MT> > MT> >> which incorrectly assume monotonic = timestamps between multiple TCP connections > > MT> > MT> > MT> > MT> >> coming from the same IP address, but only = in case of errors at the application layer. > > MT> > MT> > MT> > MT> >=20 > > MT> > MT> > MT> > MT> > Yeah, exactly, some hosts seem to enable = this only in case of an error in HTTP > > MT> > MT> > MT> > MT> > communication (some smart proxy?). However, = there are some that behave this way > > MT> > MT> > MT> > MT> > regardless of errors, for example these: > > MT> > MT> > MT> > MT> >=20 > > MT> > MT> > MT> > MT> > curl -v https://185.134.205.105:443 > > MT> > MT> > MT> > MT> > curl -v https://136.243.1.231:443 > > MT> > MT> > MT> > MT> Wireshark sees an Encrypted Alert in both = cases. So I guess this is another indication > > MT> > MT> > MT> > MT> of "error at the application layer". > > MT> > MT> > MT> > MT> >=20 > > MT> > MT> > MT> > MT> >>=20 > > MT> > MT> > MT> > MT> >> Do you have any insights whether the hosts = you are listed share something in > > MT> > MT> > MT> > MT> >> common. Some of them are hosted by Hetzner, = but not all. > > MT> > MT> > MT> > MT> >=20 > > MT> > MT> > MT> > MT> > Nope. A whole set of endpoints that we have = detected so far is pretty diverse, > > MT> > MT> > MT> > MT> > containing a lot of different locations = geographically, as well as different > > MT> > MT> > MT> > MT> > hosters. > > MT> > MT> > MT> > MT> OK. Thanks for the clarification. > > MT> > MT> > MT> > MT> >=20 > > MT> > MT> > MT> > MT> >>=20 > > MT> > MT> > MT> > MT> >> I think in general, it is the correct thing = to include the port numbers in > > MT> > MT> > MT> > MT> >> the offset computation. We might add a = sysctl variable to control the inclusion. > > MT> > MT> > MT> > MT> >> This would allow interworking with broken = middleboxes. > > MT> > MT> > MT> > MT> >=20 > > MT> > MT> > MT> > MT> > Yeah, I completely agree that these rare = cases should not dictate the implementation. > > MT> > MT> > MT> > MT> > But an ability to enable a work-around via = sysctl would be greatly appreciated. > > MT> > MT> > MT> > MT> > Currently we are unable to roll-out the = upgrade across all servers because of this > > MT> > MT> > MT> > MT> > issue: even though it happens not so often, = a lot of requests from our users=20 > > MT> > MT> > MT> > MT> > get stuck or fail all together. For example, = a host 185.134.205.105 is a kind of > > MT> > MT> > MT> > MT> > social network that our proxy servers = connect to so securely access to content, > > MT> > MT> > MT> > MT> > such as images, on behalf of our users. > > MT> > MT> > MT> > MT> >=20 > > MT> > MT> > MT> > MT> >>=20 > > MT> > MT> > MT> > MT> >> Please note, this does not fix the case of = multiple clients behind a NAT. > > MT> > MT> > MT> > MT> >=20 > > MT> > MT> > MT> > MT> > Yeah, that's true. Fortunately we don't use = NAT. > > MT> > MT> > MT> > MT> >=20 > > MT> > MT> > MT> > MT> >>=20 > > MT> > MT> > MT> > MT> >> I'm also trying to figure out how and why = Linux and Windows are handling this. > > MT> > MT> > MT> > MT> >=20 > > MT> > MT> > MT> > MT> > Thanks for bothering! > > MT> > MT> > MT> > MT> Will let you know what I figure out. > > MT> > MT> > MT> > MT>=20 > > MT> > MT> > MT> > MT> Best regards > > MT> > MT> > MT> > MT> Michael > > MT> > MT> > MT> > MT> >=20 > > MT> > MT> > MT> > MT> >>=20 > > MT> > MT> > MT> > MT> >> Best regards > > MT> > MT> > MT> > MT> >> Michael > > MT> > MT> > MT> > MT> >>=20 > > MT> > MT> > MT> > MT> >>>=20 > > MT> > MT> > MT> > MT> >>>>=20 > > MT> > MT> > MT> > MT> >>>> Best regards > > MT> > MT> > MT> > MT> >>>> Michael=20 > > MT> > MT> > MT> > MT> >>>>>=20 > > MT> > MT> > MT> > MT> >>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>> Best regards > > MT> > MT> > MT> > MT> >>>>>> Michael > > MT> > MT> > MT> > MT> >>>>>>=20 > > MT> > MT> > MT> > MT> >>>>>>=20 > > MT> > MT> > MT> > MT> >>>>=20 > > MT> > MT> > MT> > MT> >>>>=20 > > MT> > MT> > MT> > MT> >>=20 > > MT> > MT> > MT> > MT> >>=20 > > MT> > MT> > MT> > MT>=20 > > MT> > MT> > MT> > MT> = _______________________________________________ > > MT> > MT> > MT> > MT> freebsd-net@freebsd.org mailing list > > MT> > MT> > MT> > MT> = https://lists.freebsd.org/mailman/listinfo/freebsd-net > > MT> > MT> > MT> > MT> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" > > MT> > MT> > MT>=20 > > MT> > MT> > MT> _______________________________________________ > > MT> > MT> > MT> freebsd-net@freebsd.org mailing list > > MT> > MT> > MT> = https://lists.freebsd.org/mailman/listinfo/freebsd-net > > MT> > MT> > MT> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" > > MT> > MT>=20 > > MT> > _______________________________________________ > > MT> > freebsd-net@freebsd.org mailing list > > MT> > https://lists.freebsd.org/mailman/listinfo/freebsd-net > > MT> > To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" > > MT>=20 > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Wed Jul 17 18:23:32 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B8792B2C00 for ; Wed, 17 Jul 2019 18:23:32 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7EBA48DF90 for ; Wed, 17 Jul 2019 18:23:32 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from [IPv6:2a02:8109:1140:c3d:d878:d920:c09d:25c5] (unknown [IPv6:2a02:8109:1140:c3d:d878:d920:c09d:25c5]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTPSA id AF891721E2822; Wed, 17 Jul 2019 20:23:27 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: Issues with TCP Timestamps allocation From: Michael Tuexen In-Reply-To: <20190717074243.GA65665@hell.ukr.net> Date: Wed, 17 Jul 2019 20:23:26 +0200 Cc: freebsd-net@freebsd.org, Paul Content-Transfer-Encoding: quoted-printable Message-Id: References: <1562579483.67527000.24rw4xi5@frv39.fwdcdn.com> <32FD061B-245C-41D2-81DE-1B4756A7173D@freebsd.org> <1562591379.369129000.gpmxvurq@frv39.fwdcdn.com> <1562599181.734953000.1l9a1d23@frv39.fwdcdn.com> <0C475A01-9BCD-4E4A-9731-09AB919CA9BE@freebsd.org> <1562676414.933145000.z3zteyqp@frv39.fwdcdn.com> <1E9F3F99-C3E9-44DD-AA70-9B11E19D4769@freebsd.org> <20190717074243.GA65665@hell.ukr.net> To: Vitalij Satanivskij X-Mailer: Apple Mail (2.3445.104.11) X-Spam-Status: No, score=-1.7 required=5.0 tests=ALL_TRUSTED,BAYES_00, NORMAL_HTTP_TO_IP,NUMERIC_HTTP_ADDR autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jul 2019 18:23:32 -0000 > On 17. Jul 2019, at 09:42, Vitalij Satanivskij wrote: >=20 >=20 >=20 > Hello.=20 >=20 > Is there any changes about this problem Please find a patch in https://reviews.freebsd.org/D20980 If possible, please test and report. Best regards Michael >=20 >=20 > I'm using FreeBSD 12 on my desktop and can confirm problem occur with = some hosts. >=20 >=20 >=20 > Michael Tuexen wrote: > MT>=20 > MT>=20 > MT> > On 9. Jul 2019, at 14:58, Paul wrote: > MT> >=20 > MT> > Hi Michael, > MT> >=20 > MT> > 9 July 2019, 15:34:29, by "Michael Tuexen" : > MT> >=20 > MT> >>=20 > MT> >>=20 > MT> >>> On 8. Jul 2019, at 17:22, Paul wrote: > MT> >>>=20 > MT> >>>=20 > MT> >>>=20 > MT> >>> 8 July 2019, 17:12:21, by "Michael Tuexen" = : > MT> >>>=20 > MT> >>>>> On 8. Jul 2019, at 15:24, Paul wrote: > MT> >>>>>=20 > MT> >>>>> Hi Michael, > MT> >>>>>=20 > MT> >>>>> 8 July 2019, 15:53:15, by "Michael Tuexen" = : > MT> >>>>>=20 > MT> >>>>>>> On 8. Jul 2019, at 12:37, Paul wrote: > MT> >>>>>>>=20 > MT> >>>>>>> Hi team, > MT> >>>>>>>=20 > MT> >>>>>>> Recently we had an upgrade to 12 Stable. Immediately = after, we have started=20 > MT> >>>>>>> seeing some strange connection establishment timeouts to = some fixed number > MT> >>>>>>> of external (world) hosts. The issue was persistent and = easy to reproduce. > MT> >>>>>>> Thanks to a patience and dedication of our system engineer = we have tracked =20 > MT> >>>>>>> this issue down to a specific commit: > MT> >>>>>>>=20 > MT> >>>>>>> = https://svnweb.freebsd.org/base?view=3Drevision&revision=3D338053 > MT> >>>>>>>=20 > MT> >>>>>>> This patch was also back-ported into 11 Stable: > MT> >>>>>>>=20 > MT> >>>>>>> = https://svnweb.freebsd.org/base?view=3Drevision&revision=3D348435 > MT> >>>>>>>=20 > MT> >>>>>>> Among other things this patch changes the timestamp = allocation strategy, > MT> >>>>>>> by introducing a deterministic randomness via a hash = function that takes > MT> >>>>>>> into account a random key as well as source address, = source port, dest > MT> >>>>>>> address and dest port. As the result, timestamp offsets of = different > MT> >>>>>>> tuples (SA,SP,DA,DP) will be wildly different and will = jump from small=20 > MT> >>>>>>> to large numbers and back, as long as something in the = tuple changes. > MT> >>>>>> Hi Paul, > MT> >>>>>>=20 > MT> >>>>>> this is correct. > MT> >>>>>>=20 > MT> >>>>>> Please note that the same happens with the old method, if = two hosts with > MT> >>>>>> different uptimes are bind a consumer grade NAT. > MT> >>>>>=20 > MT> >>>>> If NAT does not replace timestamps then yes, it should be = the case. > MT> >>>>>=20 > MT> >>>>>>>=20 > MT> >>>>>>> After performing various tests of hosts that produce the = above mentioned=20 > MT> >>>>>>> issue we came to conclusion that there are some = interesting implementations=20 > MT> >>>>>>> that drop SYN packets with timestamps smaller than the = largest timestamp=20 > MT> >>>>>>> value from streams of all recent or current connections = from a specific=20 > MT> >>>>>>> address. This looks as some kind of SYN flood protection. > MT> >>>>>> This also breaks multiple hosts with different uptimes = behind a consumer > MT> >>>>>> level NAT talking to such a server. > MT> >>>>>>>=20 > MT> >>>>>>> To ensure that each external host is not going to see a = wild jumps of=20 > MT> >>>>>>> timestamp values I propose a patch that removes ports from = the equation > MT> >>>>>>> all together, when calculating the timestamp offset: > MT> >>>>>>>=20 > MT> >>>>>>> Index: sys/netinet/tcp_subr.c > MT> >>>>>>> = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > MT> >>>>>>> --- sys/netinet/tcp_subr.c (revision 348435) > MT> >>>>>>> +++ sys/netinet/tcp_subr.c (working copy) > MT> >>>>>>> @@ -2224,7 +2224,22 @@ > MT> >>>>>>> uint32_t > MT> >>>>>>> tcp_new_ts_offset(struct in_conninfo *inc) > MT> >>>>>>> { > MT> >>>>>>> - return (tcp_keyed_hash(inc, V_ts_offset_secret)); > MT> >>>>>>> + /*=20 > MT> >>>>>>> + * Some implementations show a strange behaviour = when a wildly random=20 > MT> >>>>>>> + * timestamps allocated for different streams. It = seems that only the > MT> >>>>>>> + * SYN packets are affected. Observed = implementations drop SYN packets > MT> >>>>>>> + * with timestamps smaller than the largest = timestamp value of all=20 > MT> >>>>>>> + * recent or current connections from specific a = address. To mitigate=20 > MT> >>>>>>> + * this we are going to ensure that each host = will always observe=20 > MT> >>>>>>> + * timestamps as increasing no matter the stream: = by dropping ports > MT> >>>>>>> + * from the equation. > MT> >>>>>>> + */=20 > MT> >>>>>>> + struct in_conninfo inc_copy =3D *inc; > MT> >>>>>>> + > MT> >>>>>>> + inc_copy.inc_fport =3D 0; > MT> >>>>>>> + inc_copy.inc_lport =3D 0; > MT> >>>>>>> + > MT> >>>>>>> + return (tcp_keyed_hash(&inc_copy, V_ts_offset_secret)); > MT> >>>>>>> } > MT> >>>>>>>=20 > MT> >>>>>>> /* > MT> >>>>>>>=20 > MT> >>>>>>> In any case, the solution of the uptime leak, implemented = in rev338053 is=20 > MT> >>>>>>> not going to suffer, because a supposed attacker is = currently able to use=20 > MT> >>>>>>> any fixed values of SP and DP, albeit not 0, anyway, to = remove them out=20 > MT> >>>>>>> of the equation. > MT> >>>>>> Can you describe how a peer can compute the uptime from two = observed timestamps? > MT> >>>>>> I don't see how you can do that... > MT> >>>>>=20 > MT> >>>>> Supposed attacker could run a script that continuously = monitors timestamps, > MT> >>>>> for example via a periodic TCP connection from a fixed local = port (eg 12345)=20 > MT> >>>>> and a fixed local address to the fixed victim's address and = port (eg 80). > MT> >>>>> Whenever large discrepancy is observed, attacker can assume = that reboot has=20 > MT> >>>>> happened (due to V_ts_offset_secret re-generation), hence = the received=20 > MT> >>>>> timestamp is considered an approximate point of reboot from = which the uptime > MT> >>>>> can be calculated, until the next reboot and so on. > MT> >>>> Ahh, I see. The patch we are talking about is not intended to = protect against > MT> >>>> continuous monitoring, which is something you can always do. = You could even > MT> >>>> watch for service availability and detect reboots. A change = of the local key > MT> >>>> would also look similar to a reboot without a temporary loss = of connectivity. > MT> >>>>=20 > MT> >>>> Thanks for the clarification. > MT> >>>>>=20 > MT> >>>>>>>=20 > MT> >>>>>>> There is the list of example hosts that we were able to = reproduce the=20 > MT> >>>>>>> issue with: > MT> >>>>>>>=20 > MT> >>>>>>> curl -v http://88.99.60.171:80 > MT> >>>>>>> curl -v http://163.172.71.252:80 > MT> >>>>>>> curl -v http://5.9.242.150:80 > MT> >>>>>>> curl -v https://185.134.205.105:443 > MT> >>>>>>> curl -v https://136.243.1.231:443 > MT> >>>>>>> curl -v https://144.76.196.4:443 > MT> >>>>>>> curl -v http://94.127.191.194:80 > MT> >>>>>>>=20 > MT> >>>>>>> To reproduce, call curl repeatedly with a same URL some = number of times.=20 > MT> >>>>>>> You are going to see some of the requests stuck in=20 > MT> >>>>>>> `* Trying XXX.XXX.XXX.XXX...` > MT> >>>>>>>=20 > MT> >>>>>>> For some reason, the easiest way to reproduce the issue is = with nc: > MT> >>>>>>>=20 > MT> >>>>>>> $ echo "foooooo" | nc -v 88.99.60.171 80 > MT> >>>>>>>=20 > MT> >>>>>>> Only a few such calls are required until one of them is = stuck on connect(): > MT> >>>>>>> issuing SYN packets with an exponential backoff. > MT> >>>>>> Thanks for providing an end-point to test with. I'll take a = look. > MT> >>>>>> Just to be clear: You are running a FreeBSD client against = one of the above > MT> >>>>>> servers and experience the problem with the new timestamp = computations. > MT> >>>>>>=20 > MT> >>>>>> You are not running arbitrary clients against a FreeBSD = server... > MT> >>>>>=20 > MT> >>>>> We are talking about FreeBSD being the client. Peers that = yield this unwanted > MT> >>>>> behaviour are unknown. Little bit of tinkering showed that = some of them run=20 > MT> >>>>> Debian: > MT> >>>>>=20 > MT> >>>>> telnet 88.99.60.171 22 > MT> >>>>> Trying 88.99.60.171... > MT> >>>>> Connected to 88.99.60.171. > MT> >>>>> Escape character is '^]'. > MT> >>>>> SSH-2.0-OpenSSH_6.7p1 Debian-5+deb8u3 > MT> >>>> Also some are hosted by Hetzner, but not all. I'll will look = into > MT> >>>> this tomorrow, since I'm on a deadline today (well it is 2am = tomorrow > MT> >>>> morning, to be precise)... > MT> >>>=20 > MT> >>> Thanks a lot, I would appreciate that. > MT> >> Hi Paul, > MT> >>=20 > MT> >> I have looked into this. > MT> >>=20 > MT> >> * The FreeBSD behaviour is the one which is specified in the = last bullet item > MT> >> in https://tools.ietf.org/html/rfc7323#section-5.4 > MT> >> It is also the one, which is RECOMMENDED in > MT> >> https://tools.ietf.org/html/rfc7323#section-7.1=20 > MT> >>=20 > MT> >> * My NAT box (a popular one in Germany) does NOT rewrite TCP = timestamps. > MT> >>=20 > MT> >> This means that the host you are referring to have some sort of = protection, > MT> >> which makes incorrect assumptions. It will also break multiple = hosts behind > MT> >> a NAT. > MT> >>=20 > MT> >> I can run > MT> >> curl -v http://88.99.60.171:80 > MT> >> in a loop without any problems from a FreeBSD head system. I = tested 1000 > MT> >> iterations or so. The TS.val is jumping up and down as = expected. > MT> >> I'm wondering why you are observing errors in this case, too. > MT> >>=20 > MT> >> However, doing something like > MT> >> echo "foooooo" | nc -v 88.99.60.171 80 > MT> >> triggers the problem. > MT> >>=20 > MT> >> So I think there is some functionality (in a middlebox or = running on the host), > MT> >> which incorrectly assume monotonic timestamps between multiple = TCP connections > MT> >> coming from the same IP address, but only in case of errors at = the application layer. > MT> >=20 > MT> > Yeah, exactly, some hosts seem to enable this only in case of an = error in HTTP > MT> > communication (some smart proxy?). However, there are some that = behave this way > MT> > regardless of errors, for example these: > MT> >=20 > MT> > curl -v https://185.134.205.105:443 > MT> > curl -v https://136.243.1.231:443 > MT> Wireshark sees an Encrypted Alert in both cases. So I guess this = is another indication > MT> of "error at the application layer". > MT> >=20 > MT> >>=20 > MT> >> Do you have any insights whether the hosts you are listed share = something in > MT> >> common. Some of them are hosted by Hetzner, but not all. > MT> >=20 > MT> > Nope. A whole set of endpoints that we have detected so far is = pretty diverse, > MT> > containing a lot of different locations geographically, as well = as different > MT> > hosters. > MT> OK. Thanks for the clarification. > MT> >=20 > MT> >>=20 > MT> >> I think in general, it is the correct thing to include the port = numbers in > MT> >> the offset computation. We might add a sysctl variable to = control the inclusion. > MT> >> This would allow interworking with broken middleboxes. > MT> >=20 > MT> > Yeah, I completely agree that these rare cases should not = dictate the implementation. > MT> > But an ability to enable a work-around via sysctl would be = greatly appreciated. > MT> > Currently we are unable to roll-out the upgrade across all = servers because of this > MT> > issue: even though it happens not so often, a lot of requests = from our users=20 > MT> > get stuck or fail all together. For example, a host = 185.134.205.105 is a kind of > MT> > social network that our proxy servers connect to so securely = access to content, > MT> > such as images, on behalf of our users. > MT> >=20 > MT> >>=20 > MT> >> Please note, this does not fix the case of multiple clients = behind a NAT. > MT> >=20 > MT> > Yeah, that's true. Fortunately we don't use NAT. > MT> >=20 > MT> >>=20 > MT> >> I'm also trying to figure out how and why Linux and Windows are = handling this. > MT> >=20 > MT> > Thanks for bothering! > MT> Will let you know what I figure out. > MT>=20 > MT> Best regards > MT> Michael > MT> >=20 > MT> >>=20 > MT> >> Best regards > MT> >> Michael > MT> >>=20 > MT> >>>=20 > MT> >>>>=20 > MT> >>>> Best regards > MT> >>>> Michael=20 > MT> >>>>>=20 > MT> >>>>>=20 > MT> >>>>>>=20 > MT> >>>>>> Best regards > MT> >>>>>> Michael > MT> >>>>>>=20 > MT> >>>>>>=20 > MT> >>>>=20 > MT> >>>>=20 > MT> >>=20 > MT> >>=20 > MT>=20 > MT> _______________________________________________ > MT> freebsd-net@freebsd.org mailing list > MT> https://lists.freebsd.org/mailman/listinfo/freebsd-net > MT> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Wed Jul 17 19:58:04 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8BEEFB4AC2 for ; Wed, 17 Jul 2019 19:58:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 2292A91E25 for ; Wed, 17 Jul 2019 19:58:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 19757B4AC1; Wed, 17 Jul 2019 19:58:04 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1829DB4AC0 for ; Wed, 17 Jul 2019 19:58:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E761191E24 for ; Wed, 17 Jul 2019 19:58:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id BE59623867 for ; Wed, 17 Jul 2019 19:58:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6HJw3Fh067858 for ; Wed, 17 Jul 2019 19:58:03 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6HJw3Vt067857 for net@FreeBSD.org; Wed, 17 Jul 2019 19:58:03 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 238796] ipfilter: fix unremovable rules and rules checksum for comparison Date: Wed, 17 Jul 2019 19:58:03 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: cy@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: cy@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.isobsolete attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 2292A91E25 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.985,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jul 2019 19:58:04 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238796 Cy Schubert changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #205808|0 |1 is obsolete| | --- Comment #23 from Cy Schubert --- Created attachment 205851 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D205851&action= =3Dedit This should fix this PR. Can you try this pstch? Make sure your tree is up to date before applying i= t. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Wed Jul 17 20:03:42 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id ACA38B4D56 for ; Wed, 17 Jul 2019 20:03:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 8C43C922B6 for ; Wed, 17 Jul 2019 20:03:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 8BDC2B4D54; Wed, 17 Jul 2019 20:03:42 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8BA02B4D53 for ; Wed, 17 Jul 2019 20:03:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6DE4C922B5 for ; Wed, 17 Jul 2019 20:03:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 48A9923A2D for ; Wed, 17 Jul 2019 20:03:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6HK3gfa087730 for ; Wed, 17 Jul 2019 20:03:42 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6HK3gnj087729 for net@FreeBSD.org; Wed, 17 Jul 2019 20:03:42 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 238796] ipfilter: fix unremovable rules and rules checksum for comparison Date: Wed, 17 Jul 2019 20:03:41 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: cy@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: cy@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 6DE4C922B5 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.982,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jul 2019 20:03:42 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238796 --- Comment #24 from Cy Schubert --- The problem was that the following fix in 2009, ip_fil.h r1.31 and fil.c r1= .53, is incomplete. A number of issues not relating to this PR have already been fixed. The posted patch directly fixes this PR. The upstream fix was incomplete: 2580062 from/to targets should be able to use any interface name --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Thu Jul 18 07:35:59 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1C9E5A1C06 for ; Thu, 18 Jul 2019 07:35:59 +0000 (UTC) (envelope-from satan@ukr.net) Received: from hell.ukr.net (hell.ukr.net [212.42.67.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.ukr.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C2D2976B15; Thu, 18 Jul 2019 07:35:58 +0000 (UTC) (envelope-from satan@ukr.net) Received: from satan by hell.ukr.net with local ID 1ho0xH-00010S-GH ; Thu, 18 Jul 2019 10:35:55 +0300 Date: Thu, 18 Jul 2019 10:35:55 +0300 From: Vitalij Satanivskij To: Michael Tuexen Cc: Vitalij Satanivskij , freebsd-net@freebsd.org, Paul Subject: Re: Issues with TCP Timestamps allocation Message-ID: <20190718073555.GA3770@hell.ukr.net> Reply-To: satan@ukr.net References: <1562579483.67527000.24rw4xi5@frv39.fwdcdn.com> <32FD061B-245C-41D2-81DE-1B4756A7173D@freebsd.org> <1562591379.369129000.gpmxvurq@frv39.fwdcdn.com> <1562599181.734953000.1l9a1d23@frv39.fwdcdn.com> <0C475A01-9BCD-4E4A-9731-09AB919CA9BE@freebsd.org> <1562676414.933145000.z3zteyqp@frv39.fwdcdn.com> <1E9F3F99-C3E9-44DD-AA70-9B11E19D4769@freebsd.org> <20190717074243.GA65665@hell.ukr.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) X-Rspamd-Queue-Id: C2D2976B15 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.90 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.90)[-0.900,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jul 2019 07:35:59 -0000 Yep. Patch work. Michael Tuexen wrote: MT> > On 17. Jul 2019, at 09:42, Vitalij Satanivskij wrote: MT> > MT> > MT> > MT> > Hello. MT> > MT> > Is there any changes about this problem MT> Please find a patch in https://reviews.freebsd.org/D20980 MT> MT> If possible, please test and report. MT> MT> Best regards MT> Michael MT> > MT> > MT> > I'm using FreeBSD 12 on my desktop and can confirm problem occur with some hosts. MT> > MT> > MT> > MT> > Michael Tuexen wrote: MT> > MT> MT> > MT> MT> > MT> > On 9. Jul 2019, at 14:58, Paul wrote: MT> > MT> > MT> > MT> > Hi Michael, MT> > MT> > MT> > MT> > 9 July 2019, 15:34:29, by "Michael Tuexen" : MT> > MT> > MT> > MT> >> MT> > MT> >> MT> > MT> >>> On 8. Jul 2019, at 17:22, Paul wrote: MT> > MT> >>> MT> > MT> >>> MT> > MT> >>> MT> > MT> >>> 8 July 2019, 17:12:21, by "Michael Tuexen" : MT> > MT> >>> MT> > MT> >>>>> On 8. Jul 2019, at 15:24, Paul wrote: MT> > MT> >>>>> MT> > MT> >>>>> Hi Michael, MT> > MT> >>>>> MT> > MT> >>>>> 8 July 2019, 15:53:15, by "Michael Tuexen" : MT> > MT> >>>>> MT> > MT> >>>>>>> On 8. Jul 2019, at 12:37, Paul wrote: MT> > MT> >>>>>>> MT> > MT> >>>>>>> Hi team, MT> > MT> >>>>>>> MT> > MT> >>>>>>> Recently we had an upgrade to 12 Stable. Immediately after, we have started MT> > MT> >>>>>>> seeing some strange connection establishment timeouts to some fixed number MT> > MT> >>>>>>> of external (world) hosts. The issue was persistent and easy to reproduce. MT> > MT> >>>>>>> Thanks to a patience and dedication of our system engineer we have tracked MT> > MT> >>>>>>> this issue down to a specific commit: MT> > MT> >>>>>>> MT> > MT> >>>>>>> https://svnweb.freebsd.org/base?view=revision&revision=338053 MT> > MT> >>>>>>> MT> > MT> >>>>>>> This patch was also back-ported into 11 Stable: MT> > MT> >>>>>>> MT> > MT> >>>>>>> https://svnweb.freebsd.org/base?view=revision&revision=348435 MT> > MT> >>>>>>> MT> > MT> >>>>>>> Among other things this patch changes the timestamp allocation strategy, MT> > MT> >>>>>>> by introducing a deterministic randomness via a hash function that takes MT> > MT> >>>>>>> into account a random key as well as source address, source port, dest MT> > MT> >>>>>>> address and dest port. As the result, timestamp offsets of different MT> > MT> >>>>>>> tuples (SA,SP,DA,DP) will be wildly different and will jump from small MT> > MT> >>>>>>> to large numbers and back, as long as something in the tuple changes. MT> > MT> >>>>>> Hi Paul, MT> > MT> >>>>>> MT> > MT> >>>>>> this is correct. MT> > MT> >>>>>> MT> > MT> >>>>>> Please note that the same happens with the old method, if two hosts with MT> > MT> >>>>>> different uptimes are bind a consumer grade NAT. MT> > MT> >>>>> MT> > MT> >>>>> If NAT does not replace timestamps then yes, it should be the case. MT> > MT> >>>>> MT> > MT> >>>>>>> MT> > MT> >>>>>>> After performing various tests of hosts that produce the above mentioned MT> > MT> >>>>>>> issue we came to conclusion that there are some interesting implementations MT> > MT> >>>>>>> that drop SYN packets with timestamps smaller than the largest timestamp MT> > MT> >>>>>>> value from streams of all recent or current connections from a specific MT> > MT> >>>>>>> address. This looks as some kind of SYN flood protection. MT> > MT> >>>>>> This also breaks multiple hosts with different uptimes behind a consumer MT> > MT> >>>>>> level NAT talking to such a server. MT> > MT> >>>>>>> MT> > MT> >>>>>>> To ensure that each external host is not going to see a wild jumps of MT> > MT> >>>>>>> timestamp values I propose a patch that removes ports from the equation MT> > MT> >>>>>>> all together, when calculating the timestamp offset: MT> > MT> >>>>>>> MT> > MT> >>>>>>> Index: sys/netinet/tcp_subr.c MT> > MT> >>>>>>> =================================================================== MT> > MT> >>>>>>> --- sys/netinet/tcp_subr.c (revision 348435) MT> > MT> >>>>>>> +++ sys/netinet/tcp_subr.c (working copy) MT> > MT> >>>>>>> @@ -2224,7 +2224,22 @@ MT> > MT> >>>>>>> uint32_t MT> > MT> >>>>>>> tcp_new_ts_offset(struct in_conninfo *inc) MT> > MT> >>>>>>> { MT> > MT> >>>>>>> - return (tcp_keyed_hash(inc, V_ts_offset_secret)); MT> > MT> >>>>>>> + /* MT> > MT> >>>>>>> + * Some implementations show a strange behaviour when a wildly random MT> > MT> >>>>>>> + * timestamps allocated for different streams. It seems that only the MT> > MT> >>>>>>> + * SYN packets are affected. Observed implementations drop SYN packets MT> > MT> >>>>>>> + * with timestamps smaller than the largest timestamp value of all MT> > MT> >>>>>>> + * recent or current connections from specific a address. To mitigate MT> > MT> >>>>>>> + * this we are going to ensure that each host will always observe MT> > MT> >>>>>>> + * timestamps as increasing no matter the stream: by dropping ports MT> > MT> >>>>>>> + * from the equation. MT> > MT> >>>>>>> + */ MT> > MT> >>>>>>> + struct in_conninfo inc_copy = *inc; MT> > MT> >>>>>>> + MT> > MT> >>>>>>> + inc_copy.inc_fport = 0; MT> > MT> >>>>>>> + inc_copy.inc_lport = 0; MT> > MT> >>>>>>> + MT> > MT> >>>>>>> + return (tcp_keyed_hash(&inc_copy, V_ts_offset_secret)); MT> > MT> >>>>>>> } MT> > MT> >>>>>>> MT> > MT> >>>>>>> /* MT> > MT> >>>>>>> MT> > MT> >>>>>>> In any case, the solution of the uptime leak, implemented in rev338053 is MT> > MT> >>>>>>> not going to suffer, because a supposed attacker is currently able to use MT> > MT> >>>>>>> any fixed values of SP and DP, albeit not 0, anyway, to remove them out MT> > MT> >>>>>>> of the equation. MT> > MT> >>>>>> Can you describe how a peer can compute the uptime from two observed timestamps? MT> > MT> >>>>>> I don't see how you can do that... MT> > MT> >>>>> MT> > MT> >>>>> Supposed attacker could run a script that continuously monitors timestamps, MT> > MT> >>>>> for example via a periodic TCP connection from a fixed local port (eg 12345) MT> > MT> >>>>> and a fixed local address to the fixed victim's address and port (eg 80). MT> > MT> >>>>> Whenever large discrepancy is observed, attacker can assume that reboot has MT> > MT> >>>>> happened (due to V_ts_offset_secret re-generation), hence the received MT> > MT> >>>>> timestamp is considered an approximate point of reboot from which the uptime MT> > MT> >>>>> can be calculated, until the next reboot and so on. MT> > MT> >>>> Ahh, I see. The patch we are talking about is not intended to protect against MT> > MT> >>>> continuous monitoring, which is something you can always do. You could even MT> > MT> >>>> watch for service availability and detect reboots. A change of the local key MT> > MT> >>>> would also look similar to a reboot without a temporary loss of connectivity. MT> > MT> >>>> MT> > MT> >>>> Thanks for the clarification. MT> > MT> >>>>> MT> > MT> >>>>>>> MT> > MT> >>>>>>> There is the list of example hosts that we were able to reproduce the MT> > MT> >>>>>>> issue with: MT> > MT> >>>>>>> MT> > MT> >>>>>>> curl -v http://88.99.60.171:80 MT> > MT> >>>>>>> curl -v http://163.172.71.252:80 MT> > MT> >>>>>>> curl -v http://5.9.242.150:80 MT> > MT> >>>>>>> curl -v https://185.134.205.105:443 MT> > MT> >>>>>>> curl -v https://136.243.1.231:443 MT> > MT> >>>>>>> curl -v https://144.76.196.4:443 MT> > MT> >>>>>>> curl -v http://94.127.191.194:80 MT> > MT> >>>>>>> MT> > MT> >>>>>>> To reproduce, call curl repeatedly with a same URL some number of times. MT> > MT> >>>>>>> You are going to see some of the requests stuck in MT> > MT> >>>>>>> `* Trying XXX.XXX.XXX.XXX...` MT> > MT> >>>>>>> MT> > MT> >>>>>>> For some reason, the easiest way to reproduce the issue is with nc: MT> > MT> >>>>>>> MT> > MT> >>>>>>> $ echo "foooooo" | nc -v 88.99.60.171 80 MT> > MT> >>>>>>> MT> > MT> >>>>>>> Only a few such calls are required until one of them is stuck on connect(): MT> > MT> >>>>>>> issuing SYN packets with an exponential backoff. MT> > MT> >>>>>> Thanks for providing an end-point to test with. I'll take a look. MT> > MT> >>>>>> Just to be clear: You are running a FreeBSD client against one of the above MT> > MT> >>>>>> servers and experience the problem with the new timestamp computations. MT> > MT> >>>>>> MT> > MT> >>>>>> You are not running arbitrary clients against a FreeBSD server... MT> > MT> >>>>> MT> > MT> >>>>> We are talking about FreeBSD being the client. Peers that yield this unwanted MT> > MT> >>>>> behaviour are unknown. Little bit of tinkering showed that some of them run MT> > MT> >>>>> Debian: MT> > MT> >>>>> MT> > MT> >>>>> telnet 88.99.60.171 22 MT> > MT> >>>>> Trying 88.99.60.171... MT> > MT> >>>>> Connected to 88.99.60.171. MT> > MT> >>>>> Escape character is '^]'. MT> > MT> >>>>> SSH-2.0-OpenSSH_6.7p1 Debian-5+deb8u3 MT> > MT> >>>> Also some are hosted by Hetzner, but not all. I'll will look into MT> > MT> >>>> this tomorrow, since I'm on a deadline today (well it is 2am tomorrow MT> > MT> >>>> morning, to be precise)... MT> > MT> >>> MT> > MT> >>> Thanks a lot, I would appreciate that. MT> > MT> >> Hi Paul, MT> > MT> >> MT> > MT> >> I have looked into this. MT> > MT> >> MT> > MT> >> * The FreeBSD behaviour is the one which is specified in the last bullet item MT> > MT> >> in https://tools.ietf.org/html/rfc7323#section-5.4 MT> > MT> >> It is also the one, which is RECOMMENDED in MT> > MT> >> https://tools.ietf.org/html/rfc7323#section-7.1 MT> > MT> >> MT> > MT> >> * My NAT box (a popular one in Germany) does NOT rewrite TCP timestamps. MT> > MT> >> MT> > MT> >> This means that the host you are referring to have some sort of protection, MT> > MT> >> which makes incorrect assumptions. It will also break multiple hosts behind MT> > MT> >> a NAT. MT> > MT> >> MT> > MT> >> I can run MT> > MT> >> curl -v http://88.99.60.171:80 MT> > MT> >> in a loop without any problems from a FreeBSD head system. I tested 1000 MT> > MT> >> iterations or so. The TS.val is jumping up and down as expected. MT> > MT> >> I'm wondering why you are observing errors in this case, too. MT> > MT> >> MT> > MT> >> However, doing something like MT> > MT> >> echo "foooooo" | nc -v 88.99.60.171 80 MT> > MT> >> triggers the problem. MT> > MT> >> MT> > MT> >> So I think there is some functionality (in a middlebox or running on the host), MT> > MT> >> which incorrectly assume monotonic timestamps between multiple TCP connections MT> > MT> >> coming from the same IP address, but only in case of errors at the application layer. MT> > MT> > MT> > MT> > Yeah, exactly, some hosts seem to enable this only in case of an error in HTTP MT> > MT> > communication (some smart proxy?). However, there are some that behave this way MT> > MT> > regardless of errors, for example these: MT> > MT> > MT> > MT> > curl -v https://185.134.205.105:443 MT> > MT> > curl -v https://136.243.1.231:443 MT> > MT> Wireshark sees an Encrypted Alert in both cases. So I guess this is another indication MT> > MT> of "error at the application layer". MT> > MT> > MT> > MT> >> MT> > MT> >> Do you have any insights whether the hosts you are listed share something in MT> > MT> >> common. Some of them are hosted by Hetzner, but not all. MT> > MT> > MT> > MT> > Nope. A whole set of endpoints that we have detected so far is pretty diverse, MT> > MT> > containing a lot of different locations geographically, as well as different MT> > MT> > hosters. MT> > MT> OK. Thanks for the clarification. MT> > MT> > MT> > MT> >> MT> > MT> >> I think in general, it is the correct thing to include the port numbers in MT> > MT> >> the offset computation. We might add a sysctl variable to control the inclusion. MT> > MT> >> This would allow interworking with broken middleboxes. MT> > MT> > MT> > MT> > Yeah, I completely agree that these rare cases should not dictate the implementation. MT> > MT> > But an ability to enable a work-around via sysctl would be greatly appreciated. MT> > MT> > Currently we are unable to roll-out the upgrade across all servers because of this MT> > MT> > issue: even though it happens not so often, a lot of requests from our users MT> > MT> > get stuck or fail all together. For example, a host 185.134.205.105 is a kind of MT> > MT> > social network that our proxy servers connect to so securely access to content, MT> > MT> > such as images, on behalf of our users. MT> > MT> > MT> > MT> >> MT> > MT> >> Please note, this does not fix the case of multiple clients behind a NAT. MT> > MT> > MT> > MT> > Yeah, that's true. Fortunately we don't use NAT. MT> > MT> > MT> > MT> >> MT> > MT> >> I'm also trying to figure out how and why Linux and Windows are handling this. MT> > MT> > MT> > MT> > Thanks for bothering! MT> > MT> Will let you know what I figure out. MT> > MT> MT> > MT> Best regards MT> > MT> Michael MT> > MT> > MT> > MT> >> MT> > MT> >> Best regards MT> > MT> >> Michael MT> > MT> >> MT> > MT> >>> MT> > MT> >>>> MT> > MT> >>>> Best regards MT> > MT> >>>> Michael MT> > MT> >>>>> MT> > MT> >>>>> MT> > MT> >>>>>> MT> > MT> >>>>>> Best regards MT> > MT> >>>>>> Michael MT> > MT> >>>>>> MT> > MT> >>>>>> MT> > MT> >>>> MT> > MT> >>>> MT> > MT> >> MT> > MT> >> MT> > MT> MT> > MT> _______________________________________________ MT> > MT> freebsd-net@freebsd.org mailing list MT> > MT> https://lists.freebsd.org/mailman/listinfo/freebsd-net MT> > MT> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" MT> > _______________________________________________ MT> > freebsd-net@freebsd.org mailing list MT> > https://lists.freebsd.org/mailman/listinfo/freebsd-net MT> > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" MT> From owner-freebsd-net@freebsd.org Thu Jul 18 07:41:25 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 354ECA1F99 for ; Thu, 18 Jul 2019 07:41:25 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (mail-n.franken.de [193.175.24.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BD0217700A for ; Thu, 18 Jul 2019 07:41:24 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from [IPv6:2a02:8109:1140:c3d:d878:d920:c09d:25c5] (unknown [IPv6:2a02:8109:1140:c3d:d878:d920:c09d:25c5]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTPSA id 9C1BD721E2822; Thu, 18 Jul 2019 09:41:15 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: Issues with TCP Timestamps allocation From: Michael Tuexen In-Reply-To: <20190718073555.GA3770@hell.ukr.net> Date: Thu, 18 Jul 2019 09:41:14 +0200 Cc: Paul , freebsd-net@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <1562579483.67527000.24rw4xi5@frv39.fwdcdn.com> <32FD061B-245C-41D2-81DE-1B4756A7173D@freebsd.org> <1562591379.369129000.gpmxvurq@frv39.fwdcdn.com> <1562599181.734953000.1l9a1d23@frv39.fwdcdn.com> <0C475A01-9BCD-4E4A-9731-09AB919CA9BE@freebsd.org> <1562676414.933145000.z3zteyqp@frv39.fwdcdn.com> <1E9F3F99-C3E9-44DD-AA70-9B11E19D4769@freebsd.org> <20190717074243.GA65665@hell.ukr.net> <20190718073555.GA3770@hell.ukr.net> To: Vitalij Satanivskij X-Mailer: Apple Mail (2.3445.104.11) X-Spam-Status: No, score=-1.7 required=5.0 tests=ALL_TRUSTED,BAYES_00, NORMAL_HTTP_TO_IP,NUMERIC_HTTP_ADDR autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-Rspamd-Queue-Id: BD0217700A X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.95 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.95)[-0.947,0]; ASN(0.00)[asn:680, ipnet:193.174.0.0/15, country:DE]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jul 2019 07:41:25 -0000 > On 18. Jul 2019, at 09:35, Vitalij Satanivskij wrote: >=20 >=20 > Yep. Patch work.=20 Thanks for testing and reporting. Best regards Michael >=20 >=20 > Michael Tuexen wrote: > MT> > On 17. Jul 2019, at 09:42, Vitalij Satanivskij = wrote: > MT> >=20 > MT> >=20 > MT> >=20 > MT> > Hello.=20 > MT> >=20 > MT> > Is there any changes about this problem > MT> Please find a patch in https://reviews.freebsd.org/D20980 > MT>=20 > MT> If possible, please test and report. > MT>=20 > MT> Best regards > MT> Michael > MT> >=20 > MT> >=20 > MT> > I'm using FreeBSD 12 on my desktop and can confirm problem occur = with some hosts. > MT> >=20 > MT> >=20 > MT> >=20 > MT> > Michael Tuexen wrote: > MT> > MT>=20 > MT> > MT>=20 > MT> > MT> > On 9. Jul 2019, at 14:58, Paul wrote: > MT> > MT> >=20 > MT> > MT> > Hi Michael, > MT> > MT> >=20 > MT> > MT> > 9 July 2019, 15:34:29, by "Michael Tuexen" = : > MT> > MT> >=20 > MT> > MT> >>=20 > MT> > MT> >>=20 > MT> > MT> >>> On 8. Jul 2019, at 17:22, Paul wrote: > MT> > MT> >>>=20 > MT> > MT> >>>=20 > MT> > MT> >>>=20 > MT> > MT> >>> 8 July 2019, 17:12:21, by "Michael Tuexen" = : > MT> > MT> >>>=20 > MT> > MT> >>>>> On 8. Jul 2019, at 15:24, Paul wrote: > MT> > MT> >>>>>=20 > MT> > MT> >>>>> Hi Michael, > MT> > MT> >>>>>=20 > MT> > MT> >>>>> 8 July 2019, 15:53:15, by "Michael Tuexen" = : > MT> > MT> >>>>>=20 > MT> > MT> >>>>>>> On 8. Jul 2019, at 12:37, Paul = wrote: > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> Hi team, > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> Recently we had an upgrade to 12 Stable. Immediately = after, we have started=20 > MT> > MT> >>>>>>> seeing some strange connection establishment = timeouts to some fixed number > MT> > MT> >>>>>>> of external (world) hosts. The issue was persistent = and easy to reproduce. > MT> > MT> >>>>>>> Thanks to a patience and dedication of our system = engineer we have tracked =20 > MT> > MT> >>>>>>> this issue down to a specific commit: > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> = https://svnweb.freebsd.org/base?view=3Drevision&revision=3D338053 > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> This patch was also back-ported into 11 Stable: > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> = https://svnweb.freebsd.org/base?view=3Drevision&revision=3D348435 > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> Among other things this patch changes the timestamp = allocation strategy, > MT> > MT> >>>>>>> by introducing a deterministic randomness via a hash = function that takes > MT> > MT> >>>>>>> into account a random key as well as source address, = source port, dest > MT> > MT> >>>>>>> address and dest port. As the result, timestamp = offsets of different > MT> > MT> >>>>>>> tuples (SA,SP,DA,DP) will be wildly different and = will jump from small=20 > MT> > MT> >>>>>>> to large numbers and back, as long as something in = the tuple changes. > MT> > MT> >>>>>> Hi Paul, > MT> > MT> >>>>>>=20 > MT> > MT> >>>>>> this is correct. > MT> > MT> >>>>>>=20 > MT> > MT> >>>>>> Please note that the same happens with the old = method, if two hosts with > MT> > MT> >>>>>> different uptimes are bind a consumer grade NAT. > MT> > MT> >>>>>=20 > MT> > MT> >>>>> If NAT does not replace timestamps then yes, it should = be the case. > MT> > MT> >>>>>=20 > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> After performing various tests of hosts that produce = the above mentioned=20 > MT> > MT> >>>>>>> issue we came to conclusion that there are some = interesting implementations=20 > MT> > MT> >>>>>>> that drop SYN packets with timestamps smaller than = the largest timestamp=20 > MT> > MT> >>>>>>> value from streams of all recent or current = connections from a specific=20 > MT> > MT> >>>>>>> address. This looks as some kind of SYN flood = protection. > MT> > MT> >>>>>> This also breaks multiple hosts with different = uptimes behind a consumer > MT> > MT> >>>>>> level NAT talking to such a server. > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> To ensure that each external host is not going to = see a wild jumps of=20 > MT> > MT> >>>>>>> timestamp values I propose a patch that removes = ports from the equation > MT> > MT> >>>>>>> all together, when calculating the timestamp offset: > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> Index: sys/netinet/tcp_subr.c > MT> > MT> >>>>>>> = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > MT> > MT> >>>>>>> --- sys/netinet/tcp_subr.c (revision 348435) > MT> > MT> >>>>>>> +++ sys/netinet/tcp_subr.c (working copy) > MT> > MT> >>>>>>> @@ -2224,7 +2224,22 @@ > MT> > MT> >>>>>>> uint32_t > MT> > MT> >>>>>>> tcp_new_ts_offset(struct in_conninfo *inc) > MT> > MT> >>>>>>> { > MT> > MT> >>>>>>> - return (tcp_keyed_hash(inc, = V_ts_offset_secret)); > MT> > MT> >>>>>>> + /*=20 > MT> > MT> >>>>>>> + * Some implementations show a strange = behaviour when a wildly random=20 > MT> > MT> >>>>>>> + * timestamps allocated for different = streams. It seems that only the > MT> > MT> >>>>>>> + * SYN packets are affected. Observed = implementations drop SYN packets > MT> > MT> >>>>>>> + * with timestamps smaller than the largest = timestamp value of all=20 > MT> > MT> >>>>>>> + * recent or current connections from = specific a address. To mitigate=20 > MT> > MT> >>>>>>> + * this we are going to ensure that each = host will always observe=20 > MT> > MT> >>>>>>> + * timestamps as increasing no matter the = stream: by dropping ports > MT> > MT> >>>>>>> + * from the equation. > MT> > MT> >>>>>>> + */=20 > MT> > MT> >>>>>>> + struct in_conninfo inc_copy =3D *inc; > MT> > MT> >>>>>>> + > MT> > MT> >>>>>>> + inc_copy.inc_fport =3D 0; > MT> > MT> >>>>>>> + inc_copy.inc_lport =3D 0; > MT> > MT> >>>>>>> + > MT> > MT> >>>>>>> + return (tcp_keyed_hash(&inc_copy, = V_ts_offset_secret)); > MT> > MT> >>>>>>> } > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> /* > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> In any case, the solution of the uptime leak, = implemented in rev338053 is=20 > MT> > MT> >>>>>>> not going to suffer, because a supposed attacker is = currently able to use=20 > MT> > MT> >>>>>>> any fixed values of SP and DP, albeit not 0, anyway, = to remove them out=20 > MT> > MT> >>>>>>> of the equation. > MT> > MT> >>>>>> Can you describe how a peer can compute the uptime = from two observed timestamps? > MT> > MT> >>>>>> I don't see how you can do that... > MT> > MT> >>>>>=20 > MT> > MT> >>>>> Supposed attacker could run a script that continuously = monitors timestamps, > MT> > MT> >>>>> for example via a periodic TCP connection from a fixed = local port (eg 12345)=20 > MT> > MT> >>>>> and a fixed local address to the fixed victim's = address and port (eg 80). > MT> > MT> >>>>> Whenever large discrepancy is observed, attacker can = assume that reboot has=20 > MT> > MT> >>>>> happened (due to V_ts_offset_secret re-generation), = hence the received=20 > MT> > MT> >>>>> timestamp is considered an approximate point of reboot = from which the uptime > MT> > MT> >>>>> can be calculated, until the next reboot and so on. > MT> > MT> >>>> Ahh, I see. The patch we are talking about is not = intended to protect against > MT> > MT> >>>> continuous monitoring, which is something you can = always do. You could even > MT> > MT> >>>> watch for service availability and detect reboots. A = change of the local key > MT> > MT> >>>> would also look similar to a reboot without a temporary = loss of connectivity. > MT> > MT> >>>>=20 > MT> > MT> >>>> Thanks for the clarification. > MT> > MT> >>>>>=20 > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> There is the list of example hosts that we were able = to reproduce the=20 > MT> > MT> >>>>>>> issue with: > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> curl -v http://88.99.60.171:80 > MT> > MT> >>>>>>> curl -v http://163.172.71.252:80 > MT> > MT> >>>>>>> curl -v http://5.9.242.150:80 > MT> > MT> >>>>>>> curl -v https://185.134.205.105:443 > MT> > MT> >>>>>>> curl -v https://136.243.1.231:443 > MT> > MT> >>>>>>> curl -v https://144.76.196.4:443 > MT> > MT> >>>>>>> curl -v http://94.127.191.194:80 > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> To reproduce, call curl repeatedly with a same URL = some number of times.=20 > MT> > MT> >>>>>>> You are going to see some of the requests stuck in=20= > MT> > MT> >>>>>>> `* Trying XXX.XXX.XXX.XXX...` > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> For some reason, the easiest way to reproduce the = issue is with nc: > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> $ echo "foooooo" | nc -v 88.99.60.171 80 > MT> > MT> >>>>>>>=20 > MT> > MT> >>>>>>> Only a few such calls are required until one of them = is stuck on connect(): > MT> > MT> >>>>>>> issuing SYN packets with an exponential backoff. > MT> > MT> >>>>>> Thanks for providing an end-point to test with. I'll = take a look. > MT> > MT> >>>>>> Just to be clear: You are running a FreeBSD client = against one of the above > MT> > MT> >>>>>> servers and experience the problem with the new = timestamp computations. > MT> > MT> >>>>>>=20 > MT> > MT> >>>>>> You are not running arbitrary clients against a = FreeBSD server... > MT> > MT> >>>>>=20 > MT> > MT> >>>>> We are talking about FreeBSD being the client. Peers = that yield this unwanted > MT> > MT> >>>>> behaviour are unknown. Little bit of tinkering showed = that some of them run=20 > MT> > MT> >>>>> Debian: > MT> > MT> >>>>>=20 > MT> > MT> >>>>> telnet 88.99.60.171 22 > MT> > MT> >>>>> Trying 88.99.60.171... > MT> > MT> >>>>> Connected to 88.99.60.171. > MT> > MT> >>>>> Escape character is '^]'. > MT> > MT> >>>>> SSH-2.0-OpenSSH_6.7p1 Debian-5+deb8u3 > MT> > MT> >>>> Also some are hosted by Hetzner, but not all. I'll will = look into > MT> > MT> >>>> this tomorrow, since I'm on a deadline today (well it = is 2am tomorrow > MT> > MT> >>>> morning, to be precise)... > MT> > MT> >>>=20 > MT> > MT> >>> Thanks a lot, I would appreciate that. > MT> > MT> >> Hi Paul, > MT> > MT> >>=20 > MT> > MT> >> I have looked into this. > MT> > MT> >>=20 > MT> > MT> >> * The FreeBSD behaviour is the one which is specified in = the last bullet item > MT> > MT> >> in https://tools.ietf.org/html/rfc7323#section-5.4 > MT> > MT> >> It is also the one, which is RECOMMENDED in > MT> > MT> >> https://tools.ietf.org/html/rfc7323#section-7.1=20 > MT> > MT> >>=20 > MT> > MT> >> * My NAT box (a popular one in Germany) does NOT rewrite = TCP timestamps. > MT> > MT> >>=20 > MT> > MT> >> This means that the host you are referring to have some = sort of protection, > MT> > MT> >> which makes incorrect assumptions. It will also break = multiple hosts behind > MT> > MT> >> a NAT. > MT> > MT> >>=20 > MT> > MT> >> I can run > MT> > MT> >> curl -v http://88.99.60.171:80 > MT> > MT> >> in a loop without any problems from a FreeBSD head = system. I tested 1000 > MT> > MT> >> iterations or so. The TS.val is jumping up and down as = expected. > MT> > MT> >> I'm wondering why you are observing errors in this case, = too. > MT> > MT> >>=20 > MT> > MT> >> However, doing something like > MT> > MT> >> echo "foooooo" | nc -v 88.99.60.171 80 > MT> > MT> >> triggers the problem. > MT> > MT> >>=20 > MT> > MT> >> So I think there is some functionality (in a middlebox or = running on the host), > MT> > MT> >> which incorrectly assume monotonic timestamps between = multiple TCP connections > MT> > MT> >> coming from the same IP address, but only in case of = errors at the application layer. > MT> > MT> >=20 > MT> > MT> > Yeah, exactly, some hosts seem to enable this only in case = of an error in HTTP > MT> > MT> > communication (some smart proxy?). However, there are some = that behave this way > MT> > MT> > regardless of errors, for example these: > MT> > MT> >=20 > MT> > MT> > curl -v https://185.134.205.105:443 > MT> > MT> > curl -v https://136.243.1.231:443 > MT> > MT> Wireshark sees an Encrypted Alert in both cases. So I guess = this is another indication > MT> > MT> of "error at the application layer". > MT> > MT> >=20 > MT> > MT> >>=20 > MT> > MT> >> Do you have any insights whether the hosts you are listed = share something in > MT> > MT> >> common. Some of them are hosted by Hetzner, but not all. > MT> > MT> >=20 > MT> > MT> > Nope. A whole set of endpoints that we have detected so = far is pretty diverse, > MT> > MT> > containing a lot of different locations geographically, as = well as different > MT> > MT> > hosters. > MT> > MT> OK. Thanks for the clarification. > MT> > MT> >=20 > MT> > MT> >>=20 > MT> > MT> >> I think in general, it is the correct thing to include = the port numbers in > MT> > MT> >> the offset computation. We might add a sysctl variable to = control the inclusion. > MT> > MT> >> This would allow interworking with broken middleboxes. > MT> > MT> >=20 > MT> > MT> > Yeah, I completely agree that these rare cases should not = dictate the implementation. > MT> > MT> > But an ability to enable a work-around via sysctl would be = greatly appreciated. > MT> > MT> > Currently we are unable to roll-out the upgrade across all = servers because of this > MT> > MT> > issue: even though it happens not so often, a lot of = requests from our users=20 > MT> > MT> > get stuck or fail all together. For example, a host = 185.134.205.105 is a kind of > MT> > MT> > social network that our proxy servers connect to so = securely access to content, > MT> > MT> > such as images, on behalf of our users. > MT> > MT> >=20 > MT> > MT> >>=20 > MT> > MT> >> Please note, this does not fix the case of multiple = clients behind a NAT. > MT> > MT> >=20 > MT> > MT> > Yeah, that's true. Fortunately we don't use NAT. > MT> > MT> >=20 > MT> > MT> >>=20 > MT> > MT> >> I'm also trying to figure out how and why Linux and = Windows are handling this. > MT> > MT> >=20 > MT> > MT> > Thanks for bothering! > MT> > MT> Will let you know what I figure out. > MT> > MT>=20 > MT> > MT> Best regards > MT> > MT> Michael > MT> > MT> >=20 > MT> > MT> >>=20 > MT> > MT> >> Best regards > MT> > MT> >> Michael > MT> > MT> >>=20 > MT> > MT> >>>=20 > MT> > MT> >>>>=20 > MT> > MT> >>>> Best regards > MT> > MT> >>>> Michael=20 > MT> > MT> >>>>>=20 > MT> > MT> >>>>>=20 > MT> > MT> >>>>>>=20 > MT> > MT> >>>>>> Best regards > MT> > MT> >>>>>> Michael > MT> > MT> >>>>>>=20 > MT> > MT> >>>>>>=20 > MT> > MT> >>>>=20 > MT> > MT> >>>>=20 > MT> > MT> >>=20 > MT> > MT> >>=20 > MT> > MT>=20 > MT> > MT> _______________________________________________ > MT> > MT> freebsd-net@freebsd.org mailing list > MT> > MT> https://lists.freebsd.org/mailman/listinfo/freebsd-net > MT> > MT> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" > MT> > _______________________________________________ > MT> > freebsd-net@freebsd.org mailing list > MT> > https://lists.freebsd.org/mailman/listinfo/freebsd-net > MT> > To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" > MT>=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Thu Jul 18 07:41:55 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 31BA6A2021 for ; Thu, 18 Jul 2019 07:41:55 +0000 (UTC) (envelope-from devgs@ukr.net) Received: from frv190.fwdcdn.com (frv190.fwdcdn.com [212.42.77.190]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.ukr.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DDF1F77112 for ; Thu, 18 Jul 2019 07:41:54 +0000 (UTC) (envelope-from devgs@ukr.net) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Type:MIME-Version:Message-Id:References:In-Reply-To:Cc:To: Subject:From:Date:Sender:Reply-To:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=7H++4pOyr0cZUCHTcNLj911bKB7Dc150d52AenMMRAg=; b=RkRfTBLjJB4RPE8guASnGN0CMO +QqFRlZlf7VYYZO4sSDvRmPBI8N+zSDMgxU/UAsDy+yLiTfa66JwARmtdWWaXZwYj4Tw6IUw0wteY dXHCwXcP6kY/PcYVPR/NQ2Vrxj5vr3eqTtJ00Nao+mQUeMj34lYhxiQ2814n9UNbjUX8=; Received: from [10.10.10.39] (helo=frv39.fwdcdn.com) by frv190.fwdcdn.com with smtp ID 1ho12v-000MTN-Kj for freebsd-net@freebsd.org; Thu, 18 Jul 2019 10:41:45 +0300 Date: Thu, 18 Jul 2019 10:41:45 +0300 From: Paul Subject: Re[2]: Issues with TCP Timestamps allocation To: Michael Tuexen Cc: freebsd-net@freebsd.org Received: from devgs@ukr.net by frv39.fwdcdn.com; Thu, 18 Jul 2019 10:41:45 +0300 In-Reply-To: References: <1562579483.67527000.24rw4xi5@frv39.fwdcdn.com> <32FD061B-245C-41D2-81DE-1B4756A7173D@freebsd.org> <1562591379.369129000.gpmxvurq@frv39.fwdcdn.com> <1562599181.734953000.1l9a1d23@frv39.fwdcdn.com> <0C475A01-9BCD-4E4A-9731-09AB919CA9BE@freebsd.org> <1562676414.933145000.z3zteyqp@frv39.fwdcdn.com> <1E9F3F99-C3E9-44DD-AA70-9B11E19D4769@freebsd.org> <20190717074243.GA65665@hell.ukr.net> X-Reply-Action: reply Message-Id: <1563435560.370832000.fda80wz1@frv39.fwdcdn.com> X-Mailer: mail.ukr.net 5.0 MIME-Version: 1.0 X-Rspamd-Queue-Id: DDF1F77112 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.99)[-0.986,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: binary X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jul 2019 07:41:55 -0000 Thanks for following through and making the patch! Kudos! 17 July 2019, 21:23:33, by "Michael Tuexen" : > > On 17. Jul 2019, at 09:42, Vitalij Satanivskij wrote: > > > > > > > > Hello. > > > > Is there any changes about this problem > Please find a patch in https://reviews.freebsd.org/D20980 > > If possible, please test and report. > > Best regards > Michael > > > > > > I'm using FreeBSD 12 on my desktop and can confirm problem occur with some hosts. > > > > > > > > Michael Tuexen wrote: > > MT> > > MT> > > MT> > On 9. Jul 2019, at 14:58, Paul wrote: > > MT> > > > MT> > Hi Michael, > > MT> > > > MT> > 9 July 2019, 15:34:29, by "Michael Tuexen" : > > MT> > > > MT> >> > > MT> >> > > MT> >>> On 8. Jul 2019, at 17:22, Paul wrote: > > MT> >>> > > MT> >>> > > MT> >>> > > MT> >>> 8 July 2019, 17:12:21, by "Michael Tuexen" : > > MT> >>> > > MT> >>>>> On 8. Jul 2019, at 15:24, Paul wrote: > > MT> >>>>> > > MT> >>>>> Hi Michael, > > MT> >>>>> > > MT> >>>>> 8 July 2019, 15:53:15, by "Michael Tuexen" : > > MT> >>>>> > > MT> >>>>>>> On 8. Jul 2019, at 12:37, Paul wrote: > > MT> >>>>>>> > > MT> >>>>>>> Hi team, > > MT> >>>>>>> > > MT> >>>>>>> Recently we had an upgrade to 12 Stable. Immediately after, we have started > > MT> >>>>>>> seeing some strange connection establishment timeouts to some fixed number > > MT> >>>>>>> of external (world) hosts. The issue was persistent and easy to reproduce. > > MT> >>>>>>> Thanks to a patience and dedication of our system engineer we have tracked > > MT> >>>>>>> this issue down to a specific commit: > > MT> >>>>>>> > > MT> >>>>>>> https://svnweb.freebsd.org/base?view=revision&revision=338053 > > MT> >>>>>>> > > MT> >>>>>>> This patch was also back-ported into 11 Stable: > > MT> >>>>>>> > > MT> >>>>>>> https://svnweb.freebsd.org/base?view=revision&revision=348435 > > MT> >>>>>>> > > MT> >>>>>>> Among other things this patch changes the timestamp allocation strategy, > > MT> >>>>>>> by introducing a deterministic randomness via a hash function that takes > > MT> >>>>>>> into account a random key as well as source address, source port, dest > > MT> >>>>>>> address and dest port. As the result, timestamp offsets of different > > MT> >>>>>>> tuples (SA,SP,DA,DP) will be wildly different and will jump from small > > MT> >>>>>>> to large numbers and back, as long as something in the tuple changes. > > MT> >>>>>> Hi Paul, > > MT> >>>>>> > > MT> >>>>>> this is correct. > > MT> >>>>>> > > MT> >>>>>> Please note that the same happens with the old method, if two hosts with > > MT> >>>>>> different uptimes are bind a consumer grade NAT. > > MT> >>>>> > > MT> >>>>> If NAT does not replace timestamps then yes, it should be the case. > > MT> >>>>> > > MT> >>>>>>> > > MT> >>>>>>> After performing various tests of hosts that produce the above mentioned > > MT> >>>>>>> issue we came to conclusion that there are some interesting implementations > > MT> >>>>>>> that drop SYN packets with timestamps smaller than the largest timestamp > > MT> >>>>>>> value from streams of all recent or current connections from a specific > > MT> >>>>>>> address. This looks as some kind of SYN flood protection. > > MT> >>>>>> This also breaks multiple hosts with different uptimes behind a consumer > > MT> >>>>>> level NAT talking to such a server. > > MT> >>>>>>> > > MT> >>>>>>> To ensure that each external host is not going to see a wild jumps of > > MT> >>>>>>> timestamp values I propose a patch that removes ports from the equation > > MT> >>>>>>> all together, when calculating the timestamp offset: > > MT> >>>>>>> > > MT> >>>>>>> Index: sys/netinet/tcp_subr.c > > MT> >>>>>>> =================================================================== > > MT> >>>>>>> --- sys/netinet/tcp_subr.c (revision 348435) > > MT> >>>>>>> +++ sys/netinet/tcp_subr.c (working copy) > > MT> >>>>>>> @@ -2224,7 +2224,22 @@ > > MT> >>>>>>> uint32_t > > MT> >>>>>>> tcp_new_ts_offset(struct in_conninfo *inc) > > MT> >>>>>>> { > > MT> >>>>>>> - return (tcp_keyed_hash(inc, V_ts_offset_secret)); > > MT> >>>>>>> + /* > > MT> >>>>>>> + * Some implementations show a strange behaviour when a wildly random > > MT> >>>>>>> + * timestamps allocated for different streams. It seems that only the > > MT> >>>>>>> + * SYN packets are affected. Observed implementations drop SYN packets > > MT> >>>>>>> + * with timestamps smaller than the largest timestamp value of all > > MT> >>>>>>> + * recent or current connections from specific a address. To mitigate > > MT> >>>>>>> + * this we are going to ensure that each host will always observe > > MT> >>>>>>> + * timestamps as increasing no matter the stream: by dropping ports > > MT> >>>>>>> + * from the equation. > > MT> >>>>>>> + */ > > MT> >>>>>>> + struct in_conninfo inc_copy = *inc; > > MT> >>>>>>> + > > MT> >>>>>>> + inc_copy.inc_fport = 0; > > MT> >>>>>>> + inc_copy.inc_lport = 0; > > MT> >>>>>>> + > > MT> >>>>>>> + return (tcp_keyed_hash(&inc_copy, V_ts_offset_secret)); > > MT> >>>>>>> } > > MT> >>>>>>> > > MT> >>>>>>> /* > > MT> >>>>>>> > > MT> >>>>>>> In any case, the solution of the uptime leak, implemented in rev338053 is > > MT> >>>>>>> not going to suffer, because a supposed attacker is currently able to use > > MT> >>>>>>> any fixed values of SP and DP, albeit not 0, anyway, to remove them out > > MT> >>>>>>> of the equation. > > MT> >>>>>> Can you describe how a peer can compute the uptime from two observed timestamps? > > MT> >>>>>> I don't see how you can do that... > > MT> >>>>> > > MT> >>>>> Supposed attacker could run a script that continuously monitors timestamps, > > MT> >>>>> for example via a periodic TCP connection from a fixed local port (eg 12345) > > MT> >>>>> and a fixed local address to the fixed victim's address and port (eg 80). > > MT> >>>>> Whenever large discrepancy is observed, attacker can assume that reboot has > > MT> >>>>> happened (due to V_ts_offset_secret re-generation), hence the received > > MT> >>>>> timestamp is considered an approximate point of reboot from which the uptime > > MT> >>>>> can be calculated, until the next reboot and so on. > > MT> >>>> Ahh, I see. The patch we are talking about is not intended to protect against > > MT> >>>> continuous monitoring, which is something you can always do. You could even > > MT> >>>> watch for service availability and detect reboots. A change of the local key > > MT> >>>> would also look similar to a reboot without a temporary loss of connectivity. > > MT> >>>> > > MT> >>>> Thanks for the clarification. > > MT> >>>>> > > MT> >>>>>>> > > MT> >>>>>>> There is the list of example hosts that we were able to reproduce the > > MT> >>>>>>> issue with: > > MT> >>>>>>> > > MT> >>>>>>> curl -v http://88.99.60.171:80 > > MT> >>>>>>> curl -v http://163.172.71.252:80 > > MT> >>>>>>> curl -v http://5.9.242.150:80 > > MT> >>>>>>> curl -v https://185.134.205.105:443 > > MT> >>>>>>> curl -v https://136.243.1.231:443 > > MT> >>>>>>> curl -v https://144.76.196.4:443 > > MT> >>>>>>> curl -v http://94.127.191.194:80 > > MT> >>>>>>> > > MT> >>>>>>> To reproduce, call curl repeatedly with a same URL some number of times. > > MT> >>>>>>> You are going to see some of the requests stuck in > > MT> >>>>>>> `* Trying XXX.XXX.XXX.XXX...` > > MT> >>>>>>> > > MT> >>>>>>> For some reason, the easiest way to reproduce the issue is with nc: > > MT> >>>>>>> > > MT> >>>>>>> $ echo "foooooo" | nc -v 88.99.60.171 80 > > MT> >>>>>>> > > MT> >>>>>>> Only a few such calls are required until one of them is stuck on connect(): > > MT> >>>>>>> issuing SYN packets with an exponential backoff. > > MT> >>>>>> Thanks for providing an end-point to test with. I'll take a look. > > MT> >>>>>> Just to be clear: You are running a FreeBSD client against one of the above > > MT> >>>>>> servers and experience the problem with the new timestamp computations. > > MT> >>>>>> > > MT> >>>>>> You are not running arbitrary clients against a FreeBSD server... > > MT> >>>>> > > MT> >>>>> We are talking about FreeBSD being the client. Peers that yield this unwanted > > MT> >>>>> behaviour are unknown. Little bit of tinkering showed that some of them run > > MT> >>>>> Debian: > > MT> >>>>> > > MT> >>>>> telnet 88.99.60.171 22 > > MT> >>>>> Trying 88.99.60.171... > > MT> >>>>> Connected to 88.99.60.171. > > MT> >>>>> Escape character is '^]'. > > MT> >>>>> SSH-2.0-OpenSSH_6.7p1 Debian-5+deb8u3 > > MT> >>>> Also some are hosted by Hetzner, but not all. I'll will look into > > MT> >>>> this tomorrow, since I'm on a deadline today (well it is 2am tomorrow > > MT> >>>> morning, to be precise)... > > MT> >>> > > MT> >>> Thanks a lot, I would appreciate that. > > MT> >> Hi Paul, > > MT> >> > > MT> >> I have looked into this. > > MT> >> > > MT> >> * The FreeBSD behaviour is the one which is specified in the last bullet item > > MT> >> in https://tools.ietf.org/html/rfc7323#section-5.4 > > MT> >> It is also the one, which is RECOMMENDED in > > MT> >> https://tools.ietf.org/html/rfc7323#section-7.1 > > MT> >> > > MT> >> * My NAT box (a popular one in Germany) does NOT rewrite TCP timestamps. > > MT> >> > > MT> >> This means that the host you are referring to have some sort of protection, > > MT> >> which makes incorrect assumptions. It will also break multiple hosts behind > > MT> >> a NAT. > > MT> >> > > MT> >> I can run > > MT> >> curl -v http://88.99.60.171:80 > > MT> >> in a loop without any problems from a FreeBSD head system. I tested 1000 > > MT> >> iterations or so. The TS.val is jumping up and down as expected. > > MT> >> I'm wondering why you are observing errors in this case, too. > > MT> >> > > MT> >> However, doing something like > > MT> >> echo "foooooo" | nc -v 88.99.60.171 80 > > MT> >> triggers the problem. > > MT> >> > > MT> >> So I think there is some functionality (in a middlebox or running on the host), > > MT> >> which incorrectly assume monotonic timestamps between multiple TCP connections > > MT> >> coming from the same IP address, but only in case of errors at the application layer. > > MT> > > > MT> > Yeah, exactly, some hosts seem to enable this only in case of an error in HTTP > > MT> > communication (some smart proxy?). However, there are some that behave this way > > MT> > regardless of errors, for example these: > > MT> > > > MT> > curl -v https://185.134.205.105:443 > > MT> > curl -v https://136.243.1.231:443 > > MT> Wireshark sees an Encrypted Alert in both cases. So I guess this is another indication > > MT> of "error at the application layer". > > MT> > > > MT> >> > > MT> >> Do you have any insights whether the hosts you are listed share something in > > MT> >> common. Some of them are hosted by Hetzner, but not all. > > MT> > > > MT> > Nope. A whole set of endpoints that we have detected so far is pretty diverse, > > MT> > containing a lot of different locations geographically, as well as different > > MT> > hosters. > > MT> OK. Thanks for the clarification. > > MT> > > > MT> >> > > MT> >> I think in general, it is the correct thing to include the port numbers in > > MT> >> the offset computation. We might add a sysctl variable to control the inclusion. > > MT> >> This would allow interworking with broken middleboxes. > > MT> > > > MT> > Yeah, I completely agree that these rare cases should not dictate the implementation. > > MT> > But an ability to enable a work-around via sysctl would be greatly appreciated. > > MT> > Currently we are unable to roll-out the upgrade across all servers because of this > > MT> > issue: even though it happens not so often, a lot of requests from our users > > MT> > get stuck or fail all together. For example, a host 185.134.205.105 is a kind of > > MT> > social network that our proxy servers connect to so securely access to content, > > MT> > such as images, on behalf of our users. > > MT> > > > MT> >> > > MT> >> Please note, this does not fix the case of multiple clients behind a NAT. > > MT> > > > MT> > Yeah, that's true. Fortunately we don't use NAT. > > MT> > > > MT> >> > > MT> >> I'm also trying to figure out how and why Linux and Windows are handling this. > > MT> > > > MT> > Thanks for bothering! > > MT> Will let you know what I figure out. > > MT> > > MT> Best regards > > MT> Michael > > MT> > > > MT> >> > > MT> >> Best regards > > MT> >> Michael > > MT> >> > > MT> >>> > > MT> >>>> > > MT> >>>> Best regards > > MT> >>>> Michael > > MT> >>>>> > > MT> >>>>> > > MT> >>>>>> > > MT> >>>>>> Best regards > > MT> >>>>>> Michael > > MT> >>>>>> > > MT> >>>>>> > > MT> >>>> > > MT> >>>> > > MT> >> > > MT> >> > > MT> > > MT> _______________________________________________ > > MT> freebsd-net@freebsd.org mailing list > > MT> https://lists.freebsd.org/mailman/listinfo/freebsd-net > > MT> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-net@freebsd.org Thu Jul 18 20:13:27 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8AABEB02FE for ; Thu, 18 Jul 2019 20:13:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 652416FE54 for ; Thu, 18 Jul 2019 20:13:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 64B7AB02FD; Thu, 18 Jul 2019 20:13:27 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 646ADB02FC for ; Thu, 18 Jul 2019 20:13:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 409A56FE52 for ; Thu, 18 Jul 2019 20:13:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 19511D4EB for ; Thu, 18 Jul 2019 20:13:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6IKDQx7006854 for ; Thu, 18 Jul 2019 20:13:26 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6IKDQ7k006853 for net@FreeBSD.org; Thu, 18 Jul 2019 20:13:26 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 238796] ipfilter: fix unremovable rules and rules checksum for comparison Date: Thu, 18 Jul 2019 20:13:26 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: cy@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: cy@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 409A56FE52 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.980,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jul 2019 20:13:27 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238796 --- Comment #25 from Cy Schubert --- Rearranging input arguments breaks checksum. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Fri Jul 19 01:16:24 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 480DCB5FFC for ; Fri, 19 Jul 2019 01:16:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 28E558197B for ; Fri, 19 Jul 2019 01:16:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 28755B5FFB; Fri, 19 Jul 2019 01:16:24 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 283A0B5FFA for ; Fri, 19 Jul 2019 01:16:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0B19A8197A for ; Fri, 19 Jul 2019 01:16:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DDD2E18F79 for ; Fri, 19 Jul 2019 01:16:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6J1GNL9075797 for ; Fri, 19 Jul 2019 01:16:23 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6J1GNhe075795 for net@FreeBSD.org; Fri, 19 Jul 2019 01:16:23 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 238796] ipfilter: failure to detect the same rules when arguments ordered differently Date: Fri, 19 Jul 2019 01:16:23 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: cy@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: cy@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 0B19A8197A X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.981,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jul 2019 01:16:24 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238796 Cy Schubert changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|ipfilter: fix unremovable |ipfilter: failure to detect |rules and rules checksum |the same rules when |for comparison |arguments ordered | |differently --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Fri Jul 19 03:16:17 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6E633B7A8C for ; Fri, 19 Jul 2019 03:16:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4ED0B84556 for ; Fri, 19 Jul 2019 03:16:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 4B682B7A8B; Fri, 19 Jul 2019 03:16:17 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 49A7EB7A8A for ; Fri, 19 Jul 2019 03:16:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 249ED84555 for ; Fri, 19 Jul 2019 03:16:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 155B61A688 for ; Fri, 19 Jul 2019 03:16:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6J3GGJI082955 for ; Fri, 19 Jul 2019 03:16:16 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6J3GGGD082951 for net@FreeBSD.org; Fri, 19 Jul 2019 03:16:16 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 238796] ipfilter: failure to detect the same rules when arguments ordered differently Date: Fri, 19 Jul 2019 03:16:16 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: msl0000023508@gmail.com X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: cy@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 249ED84555 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.984,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jul 2019 03:16:17 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238796 --- Comment #26 from WHR --- (In reply to Cy Schubert from comment #23) This patch seems break adding rules: [root@ipfilter-test /usr/obj]# kldload usr/src/amd64.amd64/sys/modules/ipfilter/ipl.ko=20 [root@ipfilter-test /usr/obj]# kldstat Id Refs Address Size Name 1 7 0xffffffff80200000 24ffe50 kernel 2 1 0xffffffff82819000 2538 intpm.ko 3 1 0xffffffff8281c000 a50 smbus.ko 4 1 0xffffffff8281d000 3b468 ipl.ko [root@ipfilter-test /usr/obj]# echo "pass in quick reply-to tun0:10.1.1.1 on tun0 proto tcp from any to 10.1.1.11 port =3D 22 flags S/FSRPAU keep state"= | ipf -f - 8:1:ioctl(add/insert rule): no data provided with filter rule [root@ipfilter-test /usr/obj]# echo "pass in quick reply-to tun1:10.1.2.1 on tun1 proto tcp from any to 10.1.2.11 port =3D 22 flags S/FSRPAU keep state"= | ipf -f - 8:1:ioctl(add/insert rule): no data provided with filter rule [root@ipfilter-test /usr/obj]# echo "pass in quick reply-to tun0:10.1.1.1 on tun0 proto tcp from any to 10.1.1.11 port =3D 22 flags S/FSRPAU keep state"= | ipf -f - 8:1:ioctl(add/insert rule): no data provided with filter rule [root@ipfilter-test /usr/obj]# ipfstat -Rion # empty list for ipfilter(out) # empty list for ipfilter(in) Kernel version is 13.0-CURRENT r350103. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Fri Jul 19 03:37:03 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BED8EB7E02 for ; Fri, 19 Jul 2019 03:37:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id A020784BD3 for ; Fri, 19 Jul 2019 03:37:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 9DA49B7E01; Fri, 19 Jul 2019 03:37:03 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9C4B8B7E00 for ; Fri, 19 Jul 2019 03:37:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 79DD684BD2 for ; Fri, 19 Jul 2019 03:37:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 518B11AA67 for ; Fri, 19 Jul 2019 03:37:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6J3b3vp000129 for ; Fri, 19 Jul 2019 03:37:03 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6J3b3aL000128 for net@FreeBSD.org; Fri, 19 Jul 2019 03:37:03 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 238796] ipfilter: failure to detect the same rules when arguments ordered differently Date: Fri, 19 Jul 2019 03:37:02 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: cy@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: cy@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: A020784BD3 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.984,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jul 2019 03:37:03 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238796 --- Comment #27 from Cy Schubert --- I'm having no such problems as you are. Do you have INVARIANTS and WITNESS enabled in your kernel? Send me a copy of your ipf.conf and ipnat.conf, please. If you use ippool, = that too, please. Except for rules with arguments out of order I cannot reproduce your proble= m on any of my four firewalls. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Fri Jul 19 04:14:06 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E3C19B85DD for ; Fri, 19 Jul 2019 04:14:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id C69F485BEE for ; Fri, 19 Jul 2019 04:14:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id C62DEB85DC; Fri, 19 Jul 2019 04:14:06 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C5EFFB85DB for ; Fri, 19 Jul 2019 04:14:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A678285BED for ; Fri, 19 Jul 2019 04:14:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 813761B1D7 for ; Fri, 19 Jul 2019 04:14:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6J4E6MQ001270 for ; Fri, 19 Jul 2019 04:14:06 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6J4E6l2001269 for net@FreeBSD.org; Fri, 19 Jul 2019 04:14:06 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 238796] ipfilter: failure to detect the same rules when arguments ordered differently Date: Fri, 19 Jul 2019 04:14:05 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: msl0000023508@gmail.com X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: cy@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: A678285BED X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.984,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jul 2019 04:14:06 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238796 --- Comment #28 from WHR --- This testing OS is installed from the latest 13.0-CURRENT snapshot built im= age that I downloaded today, specifically, http://ftp.freebsd.org/pub/FreeBSD/snapshots/amd64/amd64/ISO-IMAGES/13.0/Fr= eeBSD-13.0-CURRENT-amd64-20190718-r350103-disc1.iso.xz The source code is installed along with other parts from the installer, in /usr/src/ > Do you have INVARIANTS and WITNESS enabled in your kernel? Yes, the prebuilt snapshot images have those enabled. > Send me a copy of your ipf.conf and ipnat.conf, please. Since this is a fresh installation for testing, no rules exist in ipfilter config files. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Fri Jul 19 14:07:10 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0C1A5A369F; Fri, 19 Jul 2019 14:07:10 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 53B626E8FE; Fri, 19 Jul 2019 14:07:09 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-pf1-f177.google.com with SMTP id t16so14220292pfe.11; Fri, 19 Jul 2019 07:07:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:openpgp:message-id:date :user-agent:mime-version:content-language:content-transfer-encoding; bh=T9RrvYvhAhcSLjKsZlSwD2+Er3UKvp78vPRjyYmwq5U=; b=Jg/yTUJkHu8zSkpQ6HsPChV095M3K7MRkPgs6y5bXTifmvQc6GUXcbmgBPX+gG1/5w Ya0Qtfa41xFXRJz0iCN3Vr0NLgEOhCzQc5yyL0wFfWTf5TnO3claWLoVbGbcktPyuQ7j ij5V5m7ZgwvdcFdsI4nV5tLVZbZS7Gl9+eEPuWJTo+MAELeveROc6+50m+CxWOTSaDPR FQtg+h23hyLUbFk5oSwRjtkkLMf7OPqLKBAHueU80s+Szy60gbJrREyq1Sc/fKxGanKg tHKhaUfPp2IQpupjMcN1WeTh7rcZbIJ93fqCj0kYdB2+ILVFQFgvGPlxftvEjfrsRV9f qgRg== X-Gm-Message-State: APjAAAV6MjTAMx6e4DmlkOWrznWu51A1DKRa/mwkLWPLOYPjlfSd8jJ0 S5zHdVk2PsM/eL2j3H5Xpf5D2OAZ X-Google-Smtp-Source: APXvYqwFr/7mP65W/9Iql/cEaX6kZC9IHGcqsZAio1yN5HxgNsojEPiH+Ptn92iCpynlSyul5BzPkg== X-Received: by 2002:a17:90a:d791:: with SMTP id z17mr54288359pju.40.1563533544839; Fri, 19 Jul 2019 03:52:24 -0700 (PDT) Received: from [192.168.1.36] (broadband-82-140-249-80.atc.tvcom.ru. [82.140.249.80]) by smtp.googlemail.com with ESMTPSA id r27sm36237322pgn.25.2019.07.19.03.52.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 19 Jul 2019 03:52:23 -0700 (PDT) To: freebsd-net@freebsd.org, FreeBSD Current From: Andriy Gapon Subject: vmx0: watchdog timeout on queue 2, no interrupts on BSP Openpgp: preference=signencrypt Message-ID: <9c509f7b-8294-d2fe-ea3e-f10fd51f5736@FreeBSD.org> Date: Fri, 19 Jul 2019 13:52:18 +0300 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 53B626E8FE X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of agapon@gmail.com designates 209.85.210.177 as permitted sender) smtp.mailfrom=agapon@gmail.com X-Spamd-Result: default: False [-6.06 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[FreeBSD.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; RCVD_TLS_LAST(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[177.210.85.209.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.90)[-0.903,0]; IP_SCORE(-3.15)[ip: (-9.81), ipnet: 209.85.128.0/17(-3.45), asn: 15169(-2.43), country: US(-0.05)]; FORGED_SENDER(0.30)[avg@FreeBSD.org,agapon@gmail.com]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[avg@FreeBSD.org,agapon@gmail.com]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jul 2019 14:07:10 -0000 Recently we experienced a strange problem. We noticed a lot of these messages in the logs: vmx0: watchdog timeout on queue 2 (always queue 2) Also, we noticed that connections to some end points did not work at all while others worked without problems. I assume that that was because specific flows got assigned to that queue 2. Further investigation has shown that none of interrupts assigned to the BSP has ever fired (since boot, of course). That included vmx0:rx2 and vmx0:tx2. But also interrupts for other drivers as well. Trying to get more information I rebooted the system and the problem disappeared. Has anyone seen anything like that? Any thoughts on possible causes? Any suggestions what to check if/when the problem reoccurs? Thanks! -- Andriy Gapon From owner-freebsd-net@freebsd.org Fri Jul 19 14:43:19 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E534FA4529 for ; Fri, 19 Jul 2019 14:43:19 +0000 (UTC) (envelope-from ellen.santiago@marketmediaprosolutions.com) Received: from n1nlsmtp03.shr.prod.ams1.secureserver.net (n1nlsmtp03.shr.prod.ams1.secureserver.net [188.121.43.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "relay-hosting.secureserver.net", Issuer "Starfield Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5E6D9706BB for ; Fri, 19 Jul 2019 14:43:09 +0000 (UTC) (envelope-from ellen.santiago@marketmediaprosolutions.com) Received: from n3plcpnl0012.prod.ams3.secureserver.net ([160.153.146.157]) by : HOSTING RELAY : with ESMTP id oU5Ahk5bgDN9LoU5AhTTVe; Fri, 19 Jul 2019 07:42:00 -0700 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=marketmediaprosolutions.com; s=default; h=Content-Type:MIME-Version: Message-ID:Date:Subject:To:From:Reply-To:Sender:Cc:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=EdHzZrdKycrO5qkaRdUGRQOwACN/gr5809odIQ2wxj0=; b=fTOVrV0M0WAnRua7CsSEV6Hxi tnJMJ18pbf9Fjr+Lebsj7iga/jGIAmvdVgD0s0PUSy/vnVsCLNYHWfYFbr4IQ/fa5Lo/wYnLje0tH plhi3SSoWKh++mmt0JX/BwKV03TAPFgnIR3OAWeC5sPftkXB4jomBZuWxBS5r28lo2qTRCBIrlOK7 mMt3blWVo6pwypnolEP53/iDgTYtxdNZj3si9xNnVMgW0anEzrIIyVsNTRlwNuQoAK6q6fi5Qfr1a Qyq0vRIYwlhoOoAb4M0Rekfz0OsnEwBavxw7CWvYPsshXjG1V7LjXut4CVQHD9rf68ry8K0XOprDy fMFSyQMVg==; Received: from ec2-54-76-43-17.eu-west-1.compute.amazonaws.com ([54.76.43.17]:54031 helo=WS54) by n3plcpnl0012.prod.ams3.secureserver.net with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.92) (envelope-from ) id 1hoTTU-00EgZV-Bn for freebsd-net@freebsd.org; Fri, 19 Jul 2019 07:03:04 -0700 Reply-To: From: "Ellen Santiago" To: Subject: Contact list of Financial Managers and Treasury Managers Date: Fri, 19 Jul 2019 19:32:19 +0530 Message-ID: MIME-Version: 1.0 X-Mailer: Microsoft Outlook 15.0 Thread-Index: AdU+OlsA1RYdU3TdQoqGayuH8dBALw== Content-Language: en-us X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - n3plcpnl0012.prod.ams3.secureserver.net X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - marketmediaprosolutions.com X-Get-Message-Sender-Via: n3plcpnl0012.prod.ams3.secureserver.net: authenticated_id: ellen.santiago@marketmediaprosolutions.com X-Authenticated-Sender: n3plcpnl0012.prod.ams3.secureserver.net: ellen.santiago@marketmediaprosolutions.com X-Source: X-Source-Args: X-Source-Dir: X-CMAE-Envelope: MS4wfGeQ1Y0JSalb2DOxZdiaXKwjOfuy2PIODXeCWTasV8PRGqyCkymsvtjMPQ95U/2eM+Flq7lb1s+03VtrsmEpRLtbTCGXjHLvXPAxJ8hIlZZ+DW/bBqNF kcaD8tMIwK0RAjysQ6Xtzl2dkww3m8yzZ4VtAFwu0wnZUCNDDY3LUaLJ0404+1b2hm0N7b4YCFi7PaS8LG7RUBpED86Gdms7rdEZZsfXAlx1bcZ8OetqQecU 6+n8cOTsGb6eenSQybZpuQ== X-Rspamd-Queue-Id: 5E6D9706BB X-Spamd-Bar: ++++++ Authentication-Results: mx1.freebsd.org; dkim=none (invalid DKIM record) header.d=marketmediaprosolutions.com header.s=default header.b=fTOVrV0M X-Spamd-Result: default: False [6.60 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_REPLYTO(0.00)[ellen.santiago@marketmediaprosolutions.com]; MX_INVALID(0.50)[greylisted]; HAS_X_SOURCE(0.00)[]; TO_DN_NONE(0.00)[]; REPLYTO_ADDR_EQ_FROM(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[marketmediaprosolutions.com:~]; HAS_X_ANTIABUSE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; IP_SCORE(0.35)[ip: (1.09), ipnet: 188.121.40.0/22(0.51), asn: 26496(0.20), country: US(-0.05)]; ASN(0.00)[asn:26496, ipnet:188.121.40.0/22, country:US]; MID_RHS_MATCH_FROM(0.00)[]; HAS_X_AS(0.00)[ellen.santiago@marketmediaprosolutions.com]; ARC_NA(0.00)[]; RECEIVED_SPAMHAUS_XBL(3.00)[17.43.76.54.zen.spamhaus.org : 127.0.0.4]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(0.91)[0.908,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[marketmediaprosolutions.com]; NEURAL_SPAM_MEDIUM(0.94)[0.938,0]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_SPAM_LONG(1.00)[1.000,0]; RCVD_IN_DNSWL_NONE(0.00)[193.43.121.188.list.dnswl.org : 127.0.5.0]; R_DKIM_PERMFAIL(0.00)[marketmediaprosolutions.com:s=default]; R_SPF_NA(0.00)[]; HAS_X_GMSV(0.00)[ellen.santiago@marketmediaprosolutions.com]; RCVD_TLS_ALL(0.00)[] X-Spam: Yes Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jul 2019 14:43:20 -0000 Hi, Hope you're doing great! We have compiled a comprehensive and specialized email list of "Financial Managers and Treasury Managers." If this is not your target market, kindly let me know your target audience and target geography, we have all types of list available. Let me know if you are interested and I will get back to you with the Counts and Pricing. Please share your thoughts. Awaiting for your response! Regards, Ellen Santiago If you're not interested in this mailing list please Replay as "Leave Out" in subject line From owner-freebsd-net@freebsd.org Fri Jul 19 19:56:02 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 94700AA640 for ; Fri, 19 Jul 2019 19:56:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 75CF883EF3 for ; Fri, 19 Jul 2019 19:56:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 755A3AA63F; Fri, 19 Jul 2019 19:56:02 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 751ABAA63E for ; Fri, 19 Jul 2019 19:56:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 56BE783EF2 for ; Fri, 19 Jul 2019 19:56:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2A2AE26309 for ; Fri, 19 Jul 2019 19:56:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6JJu2K5099210 for ; Fri, 19 Jul 2019 19:56:02 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6JJu2Uw099209 for net@FreeBSD.org; Fri, 19 Jul 2019 19:56:02 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 238796] ipfilter: failure to detect the same rules when arguments ordered differently Date: Fri, 19 Jul 2019 19:56:02 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: cy@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: cy@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 56BE783EF2 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.982,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jul 2019 19:56:02 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238796 --- Comment #29 from Cy Schubert --- I am only able to reproduce this problem when the on argument is moved ahea= d of the reply-to. root@ipftest:~ # echo "pass in quick reply-to tun0:10.1.1.1 on tun0 proto t= cp from any to 10.1.1.11 port =3D 22 flags S/FSRPAU keep state" | ipf -f - root@ipftest:~ # echo "pass in quick reply-to tun1:10.1.2.1 on tun1 proto t= cp from any to 10.1.2.11 port =3D 22 flags S/FSRPAU keep state" | ipf -f - root@ipftest:~ # echo "pass in quick reply-to tun0:10.1.1.1 on tun0 proto t= cp from any to 10.1.1.11 port =3D 22 flags S/FSRPAU keep state" | ipf -f - 32:1:ioctl(add/insert rule): rule already exists root@ipftest:~ # echo "pass in quick on tun0 reply-to tun0:10.1.1.1 proto t= cp from any to 10.1.1.11 port =3D 22 flags S/FSRPAU keep state" | ipf -f - root@ipftest:~ # echo "pass in quick on tun0 reply-to tun0:10.1.1.1 proto t= cp from any to 10.1.1.11 port =3D 22 flags S/FSRPAU keep state" | ipf -f - 32:1:ioctl(add/insert rule): rule already exists root@ipftest:~ #=20 root@ipftest:~ # uname -a FreeBSD ipftest 13.0-CURRENT FreeBSD 13.0-CURRENT r350103 GENERIC amd64 root@ipftest:~ #=20 root@ipftest:~ # kldstat Id Refs Address Size Name 1 9 0xffffffff80200000 24ffe50 kernel 2 1 0xffffffff82819000 2538 intpm.ko 3 1 0xffffffff8281c000 a50 smbus.ko 4 1 0xffffffff8281d000 2498 filemon.ko 5 1 0xffffffff82820000 6baa0 ipl.ko root@ipftest:~ #=20 oot@ipftest:~ # ipfstat -Rion # empty list for ipfilter(out) @1 pass in quick on tun0 reply-to tun0:10.1.1.1 inet proto tcp from any to 10.1.1.11/32 port =3D 22 flags S/FSRPAU keep state @2 pass in quick on tun1 reply-to tun1:10.1.2.1 inet proto tcp from any to 10.1.2.11/32 port =3D 22 flags S/FSRPAU keep state @3 pass in quick on tun0 reply-to tun0:10.1.1.1 inet proto tcp from any to 10.1.1.11/32 port =3D 22 flags S/FSRPAU keep state root@ipftest:~ #=20 root@ipftest:~ # ipf -ZFa root@ipftest:~ # echo "pass in quick reply-to tun0:10.1.1.1 on tun0 proto t= cp from any to 10.1.1.11 port =3D 22 flags S/FSRPAU keep state" | ipf -f - root@ipftest:~ # echo "pass in quick reply-to tun1:10.1.2.1 on tun1 proto t= cp from any to 10.1.2.11 port =3D 22 flags S/FSRPAU keep state" | ipf -f - root@ipftest:~ # echo "pass in quick reply-to tun0:10.1.1.1 on tun0 proto t= cp from any to 10.1.1.11 port =3D 22 flags S/FSRPAU keep state" | ipf -f - 32:1:ioctl(add/insert rule): rule already exists root@ipftest:~ #=20 root@ipftest:~ # ipfstat -Rion=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20 # empty list for ipfilter(out) @1 pass in quick on tun0 reply-to tun0:10.1.1.1 inet proto tcp from any to 10.1.1.11/32 port =3D 22 flags S/FSRPAU keep state @2 pass in quick on tun1 reply-to tun1:10.1.2.1 inet proto tcp from any to 10.1.2.11/32 port =3D 22 flags S/FSRPAU keep state root@ipftest:~ #=20 As you can see it rejects the second attempt to load the same rule, however rearranging the on argument (first example) adds a shadowed rule which it should have rejected. This is probably because the additional interface nam= es appended to frentry_t are out of order, causing fr_ifnames to also be out of order. I have yet to test this hypothesis (yet to decide whether to impleme= nt a new SDT DTrace probe or simply expose ipf_rule_compare to allow FBT probes). The tests above were using the image on ftp.freebsd.org in a virtualbox vm which itself is running on 13.0-CURRENT. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Sat Jul 20 12:02:16 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B158EBC822 for ; Sat, 20 Jul 2019 12:02:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 9096F77BE9 for ; Sat, 20 Jul 2019 12:02:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 902C8BC821; Sat, 20 Jul 2019 12:02:16 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8FDAEBC820 for ; Sat, 20 Jul 2019 12:02:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6A8D177BE7 for ; Sat, 20 Jul 2019 12:02:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 407699C7A for ; Sat, 20 Jul 2019 12:02:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6KC2GeX064734 for ; Sat, 20 Jul 2019 12:02:16 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6KC2GLf064724 for net@FreeBSD.org; Sat, 20 Jul 2019 12:02:16 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 238796] ipfilter: failure to detect the same rules when arguments ordered differently Date: Sat, 20 Jul 2019 12:02:15 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: msl0000023508@gmail.com X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: cy@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 9096F77BE9 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.983,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Jul 2019 12:02:16 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238796 --- Comment #30 from WHR --- (In reply to Cy Schubert from comment #29) I think this because your patch (attachment 205851) only fixed comparing indexes in 'fr_ifnames', but not indexes in 'fr_tifs' and 'fr_dif'. The order of strings in 'fr_names' doesn't necessary be identical between 2 rule objects that representing same rule; for example the argument for keyw= ard 'on' and 'reply-to' both stored in 'fr_names', but the offsets may differ between 2 objects. The correct comparison should at first check the index numbers in 'fr_tifs'= and 'fr_dif', then compare the actual strings referenced by the indexes, in each rule objects, not the indexes itself. And in you last patch, function ipf_ifnames_cmp: > if ((!fr1->fr_ifnames[i] && !fr2->fr_ifnames[i]) || Testing for 0 is incorrect; shouldn't the invalid index be -1? > rc =3D 1; Why not simply 'return 1;' when a difference is already found? --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Sat Jul 20 16:56:30 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B2E59C27F8 for ; Sat, 20 Jul 2019 16:56:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 94FE188A36 for ; Sat, 20 Jul 2019 16:56:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 92A75C27F7; Sat, 20 Jul 2019 16:56:30 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9269FC27F6 for ; Sat, 20 Jul 2019 16:56:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 72D6B88A35 for ; Sat, 20 Jul 2019 16:56:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4B094D5E3 for ; Sat, 20 Jul 2019 16:56:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x6KGuUmp079272 for ; Sat, 20 Jul 2019 16:56:30 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x6KGuUBa079271 for net@FreeBSD.org; Sat, 20 Jul 2019 16:56:30 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 238796] ipfilter: failure to detect the same rules when arguments ordered differently Date: Sat, 20 Jul 2019 16:56:29 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: cy@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: cy@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 72D6B88A35 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.99 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.99)[-0.988,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Jul 2019 16:56:30 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238796 --- Comment #31 from Cy Schubert --- fr_tifs and fr_dif are not indexes. frdest_t fr_tifs[2]; /* "to"/"reply-to" interface */ frdest_t fr_dif; /* duplicate packet interface */ They're a struct with an IP address + an index or a linked list if enclosed= in {}. I don't think the problem is there either. Treating fr_tif, fr_rif and fr_dif simply as indexes is wrong. Agreed. The patch is flawed as it didn't fix anything. I assumed from your description that the problem was any time "to" was used where in fact it is more esoteric than that. Only with "to" when "on" is relocated, and why I am not able reproduce the problem simply with your examples. Taking the same r= ule and relocating on reproduces the problem, but only once as by t= hat time there is a second copy of the shadowed rule. My reply #29 clearly show= s it works in cases where the rule is coded the same but fails to detect the shadowed rule if the on keyword is moved to before the reply-to keyword, but only the first time because the shadowed rule is now added and it subsequen= tly matches. It is clearl that frentry_t is constructed differently when the on keyword is moved. Darren's March 2009 patch resolved one (more serious) issue while breaking this. There is a subtle mistake in the March 2009 patch that needs to be fi= xed. Unfortunately I disagree with his (and many other people's) style of commit= s, batching multiple fixes in one commit. It make it difficult to bisect the patches from each other and it's not clean either, messing up history. What is disconcerting is that this bug is probably symptomatic of a another more serious bug. I've fixed other bugs like this in that it's either hurri= ed development or incomplete features and fixes (ippool is an excellent exampl= e of this where I finally have it working but for IPv4 only) which is why I want= to discover the root cause. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Sat Jul 20 17:08:40 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 37733C2B9B; Sat, 20 Jul 2019 17:08:40 +0000 (UTC) (envelope-from pkelsey@gmail.com) Received: from mail-io1-f49.google.com (mail-io1-f49.google.com [209.85.166.49]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8A4F488F4B; Sat, 20 Jul 2019 17:08:39 +0000 (UTC) (envelope-from pkelsey@gmail.com) Received: by mail-io1-f49.google.com with SMTP id e20so34656278iob.9; Sat, 20 Jul 2019 10:08:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=qwftr7hfAoPOc7UkDypV8VZ/E91Lm1eNDERdVKv3z+s=; b=O8eamPKdMczPcGKP0rdjwb2mBm4icndLorUJmKFLR+h1rPVDhplKCtZITcditw9rVN paV98KUt7u+YdnjG6Z9pDq0B/UmLs+kioukmghBoe0H10BjdQJV5CKJ+X879aYQNvxkM I5dZpij9DIHpmd1ylM1Tc8i50mEG7o5ZxBd1vPiao3XR0sWcqPfryAN2fF8Sgl360kbq j3VmVy+rW6WSGSQzpm+C7ua+kxuzzNcXjt0Xbn4zJLxytVA3NDRBON1v4UDxvcdE2V71 HlWh2Y1i+VP2XWHk5ZPD6VQ3gCaC+GWiNkgwS19pJOFK4qhHaJIeDfCEqwHYNlTCsLbt 5Xfw== X-Gm-Message-State: APjAAAVPJpxPbUTepA7E3ihUvJaJYnFj74zvRV8wDh22Vj5CQx28AoLV +XA/9vQZfXcoKJx26dIj7+jjLCCTu83h9uS2IlAbGEIW X-Google-Smtp-Source: APXvYqz6p5R6RvuZkpPtJ2KRUp57i0OsyvH7L6v4Mo7ls6F0s/B4aw4HArW7jm8c1rzFcLrqMLv3S04tYTytCOTQvyg= X-Received: by 2002:a02:ad15:: with SMTP id s21mr64118056jan.47.1563642512862; Sat, 20 Jul 2019 10:08:32 -0700 (PDT) MIME-Version: 1.0 References: <9c509f7b-8294-d2fe-ea3e-f10fd51f5736@FreeBSD.org> In-Reply-To: <9c509f7b-8294-d2fe-ea3e-f10fd51f5736@FreeBSD.org> From: Patrick Kelsey Date: Sat, 20 Jul 2019 13:08:21 -0400 Message-ID: Subject: Re: vmx0: watchdog timeout on queue 2, no interrupts on BSP To: Andriy Gapon Cc: freebsd-net@freebsd.org, FreeBSD Current X-Rspamd-Queue-Id: 8A4F488F4B X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of pkelsey@gmail.com designates 209.85.166.49 as permitted sender) smtp.mailfrom=pkelsey@gmail.com X-Spamd-Result: default: False [-5.51 / 15.00]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.88)[-0.877,0]; FORGED_SENDER(0.30)[pkelsey@freebsd.org,pkelsey@gmail.com]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[pkelsey@freebsd.org,pkelsey@gmail.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MIME_TRACE(0.00)[0:+,1:+]; DMARC_NA(0.00)[freebsd.org]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[49.166.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.62)[ip: (-7.18), ipnet: 209.85.128.0/17(-3.44), asn: 15169(-2.43), country: US(-0.05)]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Jul 2019 17:08:40 -0000 On Fri, Jul 19, 2019 at 10:07 AM Andriy Gapon wrote: > > Recently we experienced a strange problem. > We noticed a lot of these messages in the logs: > vmx0: watchdog timeout on queue 2 > (always queue 2) > Also, we noticed that connections to some end points did not work at all > while others worked without problems. I assume that that was because > specific flows got assigned to that queue 2. > > Further investigation has shown that none of interrupts assigned to the > BSP has ever fired (since boot, of course). That included vmx0:rx2 and > vmx0:tx2. But also interrupts for other drivers as well. > > Trying to get more information I rebooted the system and the problem > disappeared. > > Has anyone seen anything like that? > Any thoughts on possible causes? > Any suggestions what to check if/when the problem reoccurs? > > Thanks! > > If you are running head at or after r347221 or stable/12 at or after r349112, then this could be due to https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239118 (see Comment 4 - short story is that an iflib change has broken the vmx driver). -Patrick From owner-freebsd-net@freebsd.org Sat Jul 20 19:28:25 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5CE14C5158 for ; Sat, 20 Jul 2019 19:28:25 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from shell1.rawbw.com (shell1.rawbw.com [198.144.192.42]) by mx1.freebsd.org (Postfix) with ESMTP id D1B6C8CC73 for ; Sat, 20 Jul 2019 19:28:18 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from yv.noip.me (c-67-180-169-236.hsd1.ca.comcast.net [67.180.169.236]) (authenticated bits=0) by shell1.rawbw.com (8.15.1/8.15.1) with ESMTPSA id x6KJSB8b050716 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Sat, 20 Jul 2019 12:28:12 -0700 (PDT) (envelope-from yuri@rawbw.com) X-Authentication-Warning: shell1.rawbw.com: Host c-67-180-169-236.hsd1.ca.comcast.net [67.180.169.236] claimed to be yv.noip.me Subject: Re: How to set up ipfw(8) NAT between an alias and the main IP address, when the alias is in another network? To: =?UTF-8?Q?Vin=c3=adcius_Zavam?= Cc: "freebsd-net@freebsd.org" References: <8e388abc-f2ac-b070-cf86-a4d3971ac095@rawbw.com> From: Yuri Message-ID: <8e212c21-1f34-ccf3-3bd2-36d0833c2958@rawbw.com> Date: Sat, 20 Jul 2019 12:28:10 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: D1B6C8CC73 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of yuri@rawbw.com designates 198.144.192.42 as permitted sender) smtp.mailfrom=yuri@rawbw.com X-Spamd-Result: default: False [-4.75 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:198.144.192.32/27]; HAS_XAW(0.00)[]; MX_GOOD(-0.01)[mx.rawbw.net]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.99)[-0.992,0]; FREEMAIL_TO(0.00)[googlemail.com]; RCVD_NO_TLS_LAST(0.10)[]; RECEIVED_SPAMHAUS_PBL(0.00)[236.169.180.67.zen.spamhaus.org : 127.0.0.10]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:7961, ipnet:198.144.192.0/20, country:US]; SUBJECT_ENDS_QUESTION(1.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[rawbw.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[42.192.144.198.list.dnswl.org : 127.0.10.0]; IP_SCORE(-2.54)[ip: (-5.80), ipnet: 198.144.192.0/20(-3.15), asn: 7961(-3.72), country: US(-0.05)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Jul 2019 19:28:25 -0000 On 2019-07-17 01:42, Vinícius Zavam via freebsd-net wrote: > jail ... ip4=inherit ? Thanks, but I need all of them to have unique IPs. Yuri