From owner-freebsd-transport@freebsd.org Wed Jan 17 21:55:41 2018 Return-Path: Delivered-To: freebsd-transport@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B9C5BEBA3AE for ; Wed, 17 Jan 2018 21:55:41 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (mail-n.franken.de [193.175.24.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7F1126DF for ; Wed, 17 Jan 2018 21:55:37 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from [IPv6:2003:cd:6bec:5d00:918e:a13:5702:9225] (p200300CD6BEC5D00918E0A1357029225.dip0.t-ipconnect.de [IPv6:2003:cd:6bec:5d00:918e:a13:5702:9225]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTPSA id 161ED721E282E for ; Wed, 17 Jan 2018 22:55:29 +0100 (CET) From: Michael Tuexen Content-Type: multipart/mixed; boundary="Apple-Mail=_B9794113-F72C-4777-B628-7E55D78DA29C" Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: TCP Fast Open Linux behaviour Message-Id: <2AE83CF2-FF6D-4849-AEB9-7FCCA43613A9@freebsd.org> Date: Wed, 17 Jan 2018 22:55:24 +0100 To: freebsd-transport@freebsd.org X-Mailer: Apple Mail (2.3445.5.20) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-BeenThere: freebsd-transport@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussions of transport level network protocols in FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2018 21:55:41 -0000 --Apple-Mail=_B9794113-F72C-4777-B628-7E55D78DA29C Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii Dear all, at the last Telco the following question was raised: When Linux is acting as TCP client using TCP fast open, does it accept user data sent with the SYN-ACK. I tested it with the Linux kernel 4.9.77, which is the current LTS release and figured out that the data sent with the SYN-ACK is accepted. See the attached .pcapng file. The current Ubuntu 17.10 using the kernel 4.13.0 shows the same behaviour. Best regards Michael --Apple-Mail=_B9794113-F72C-4777-B628-7E55D78DA29C Content-Disposition: attachment; filename=linux.pcapng Content-Type: application/octet-stream; x-unix-mode=0644; name="linux.pcapng" Content-Transfer-Encoding: base64 Cg0NCnAAAABNPCsaAQAAAP//////////AwAQAExpbnV4IDQuOS43Ny12NysEADgARHVtcGNhcCAo V2lyZXNoYXJrKSAyLjIuNiAoR2l0IFJldiBVbmtub3duIGZyb20gdW5rbm93bikAAAAAcAAAAAEA AABMAAAAAQAAAAAABAACAAQAZXRoMAkAAQAJAAAACwAMAAB0Y3AgcG9ydCA4MAwAEABMaW51eCA0 LjkuNzctdjcrAAAAAEwAAAAGAAAAvAAAAAAAAADttQoVEx6aTJkAAACZAAAAuCfrEFtquCfrSMXe CABFAACLoJhAAEAGFZ7AqAF5wKgBba9WAFDFIWIoAAAAANACchCEtAAAAgQFtAQCCAoABcKwAAAA AAEDAwciCv7kfKiRviZLAQFHRVQgL2NnaS1iaW4vaGUgSFRUUC8xLjANClVzZXItYWdlbnQ6IHRj cF9mbw0KQ29ubmVjdGlvbjogY2xvc2UNCg0KAAAAvAAAAAYAAACIAAAAAAAAAO21ChUolIFNZwAA AGcAAAC4J+tIxd64J+sQW2oIAEUAAFkAAEAAQAa2aMCoAW3AqAF5AFCvVjR3Rs/FIWJs0BL//yGb AAACBAW0AQMDBgQCCApiduT0AAXCsCIK/uR8qJG+JksAAEhUVFAvMS4wIDIwMCBPSw0KAIgAAAAG AAAAZAAAAAAAAADttQoVvkeDTUIAAABCAAAAuCfrEFtquCfrSMXeCABFAAA0oJlAAEAGFfTAqAF5 wKgBba9WAFDFIWJsNHdG4YAQAOWEXQAAAQEICgAFwrFiduT0AABkAAAABgAAAHwBAAAAAAAA7bUK FW+aj01cAQAAXAEAALgn60jF3rgn6xBbaggARQABTgAAQABABrVzwKgBbcCoAXkAUK9WNHdG4cUh YmyAGQQCtIkAAAEBCApiduT1AAXCsW9udGVudC10eXBlOiB0ZXh0L2h0bWwNCg0KPCFET0NUWVBF IGh0bWw+CjxodG1sPgo8aGVhZD4KPG1ldGEgY2hhcnNldD0iVVRGLTgiPgo8dGl0bGU+SGFwcHkg RXllYmFsbHM8L3RpdGxlPgo8L2hlYWQ+Cjxib2R5Pgo8cD5UaGlzIHBhZ2Ugd2FzIHJlcXVlc3Rl ZCBmcm9tIDE5Mi4xNjguMS4xMjE6NDQ4ODYgYW5kIHNlcnZlZCBieSBycGkzOjgwIHVzaW5nIFRD UCBhcyB0aGUgdHJhbnNwb3J0IHByb3RvY29sLjwvcD4KPHA+RmFzdCBvcGVuIHdhcyB1c2VkLjwv cD4KPC9ib2R5Pgo8L2h0bWw+CnwBAAAGAAAAZAAAAAAAAADttQoVnnzlTUIAAABCAAAAuCfrEFtq uCfrSMXeCABFAAA0oJpAAEAGFfPAqAF5wKgBba9WAFDFIWJsNHdH/IARAO2EXQAAAQEICgAFwrJi duT1AABkAAAABgAAAGQAAAAAAAAA7bUKFQab8U1CAAAAQgAAALgn60jF3rgn6xBbaggARQAANAAA QABABraNwKgBbcCoAXkAUK9WNHdH/MUhYm2AEAQCkLAAAAEBCApiduT8AAXCsgAAZAAAAA== --Apple-Mail=_B9794113-F72C-4777-B628-7E55D78DA29C-- From owner-freebsd-transport@freebsd.org Fri Jan 19 19:16:20 2018 Return-Path: Delivered-To: freebsd-transport@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7259EEC2D83 for ; Fri, 19 Jan 2018 19:16:20 +0000 (UTC) (envelope-from pkelsey@gmail.com) Received: from mail-pf0-x234.google.com (mail-pf0-x234.google.com [IPv6:2607:f8b0:400e:c00::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 42A9870227 for ; Fri, 19 Jan 2018 19:16:20 +0000 (UTC) (envelope-from pkelsey@gmail.com) Received: by mail-pf0-x234.google.com with SMTP id e76so2066090pfk.1 for ; Fri, 19 Jan 2018 11:16:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to; bh=JFf1DGX7dI4QteRpbEf8uMNSRt9AToQPWi+2rm/g52c=; b=Sy2Gvdxkxgh4RqEJ7F3C6iiri1aJqDhelI5kEVV3diPOlzGddPzdV3im+l7pLsGN9s 7IpLLUwiaU4TqOFsrPRBF7fIxIXK0vuorgvwS8xAhFaZP3gVpumxuUutEanqT8qFLCUa 06D9sNyhx9GxUCFzWXj9YhD/jLA5WgNSjxsagq+jV6w6jRH2uDWrdCIWVOHouHXUb0lK Xbt2zEzT+XMR/VlgWiVa8Hytz85KnJS0VpLUa+yJUYQeHCpkADzOC2j2hNNV0a7xAptZ ZluPRfTWuIM9ifTRtyi3ln9egQOPAh6cIXEeB6lRS6hRzSy6REXCvg1c0jHJ0F7Tqo0s 7gUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to; bh=JFf1DGX7dI4QteRpbEf8uMNSRt9AToQPWi+2rm/g52c=; b=oznw+LhfFI1gjNVJkpIMzZhFEIfS60XYARdLGYFG0/dvSg5efjJGQ26NvGQpRaf1L9 Nt/FlURlK4MRUAH/8Dksd3oxfsSaf5ZpMYbsbOuXyBcATHUqfb3lK/sRznCCIVRXe4+l Rm5L1X8ke1jOglBeXIO7WIrVwKkDby7PyFrKgthbmjc3VMJ61NwPK5+qLqzpEDke67KS 1Nero7NmDFDu2aHgz5iWuYLGmxw3ol2yt3I75gGdq+jXvGSOuBjf79sgflLUfWXK8d7B mmA9V0I5Qx5UrB+7B7cnJlerS7K8FT0d3bsHqfH/K57bIGpI6WkvjNUyhB/xs7PqYp9N Viuw== X-Gm-Message-State: AKwxyte6cso90gNIR4Ip5mGRKAHfvdoAVsezW+97FX6CVTH37/bF8yEB swLsXgskGuW7pXuN6a7R4qjoNw6LSlrc0qvzZHWxdw== X-Google-Smtp-Source: ACJfBosHxrMC8HdzOJ83b+ne8xrH4sVbqpGS9VTb8L15dhI3rHuIEVVJ1b45Pydf/1AT0joxh+VNo5GGzAL8Hx6JM6I= X-Received: by 10.99.113.75 with SMTP id b11mr32992842pgn.271.1516389379337; Fri, 19 Jan 2018 11:16:19 -0800 (PST) MIME-Version: 1.0 Sender: pkelsey@gmail.com Received: by 10.236.156.17 with HTTP; Fri, 19 Jan 2018 11:16:18 -0800 (PST) In-Reply-To: <2AE83CF2-FF6D-4849-AEB9-7FCCA43613A9@freebsd.org> References: <2AE83CF2-FF6D-4849-AEB9-7FCCA43613A9@freebsd.org> From: Patrick Kelsey Date: Fri, 19 Jan 2018 14:16:18 -0500 X-Google-Sender-Auth: hZ9CB-kda7aX65bkqaOy223AAjA Message-ID: Subject: Re: TCP Fast Open Linux behaviour To: freebsd-transport@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-transport@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussions of transport level network protocols in FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jan 2018 19:16:20 -0000 On Wed, Jan 17, 2018 at 4:55 PM, Michael Tuexen wrote: > Dear all, > > at the last Telco the following question was raised: > > When Linux is acting as TCP client using TCP fast open, > does it accept user data sent with the SYN-ACK. > > I tested it with the Linux kernel 4.9.77, which is the current > LTS release and figured out that the data sent with the SYN-ACK > is accepted. See the attached .pcapng file. > > The current Ubuntu 17.10 using the kernel 4.13.0 shows the > same behaviour. > > > Michael, Thank you for performing this test with recent Linux kernels and finding that the issue where the Linux TFO client-side implementation would not ACK data in a SYN-ACK has been resolved. The issue did exist at one time, going back to at least the 4.4 kernel, as highlighted during testing of my TFO server-side implementation by multiple parties. One person involved in that testing elicited an explanation from Yuchung Cheng as to why the flaw in the Linux TFO client-side implementation existed: https://www.ietf.org/mail-archive/web/tcpm/current/msg10175.html I will still need to add a knob to the FreeBSD TFO server-side implementation to enable a workaround mode that doesn't send data in TFO SYN-ACKs when data is available, but instead immediately chases the SYN-ACK with a data segment, so that service providers that want to provide good TFO performance for fielded broken Linux clients can do so. -Patrick From owner-freebsd-transport@freebsd.org Fri Jan 19 21:52:11 2018 Return-Path: Delivered-To: freebsd-transport@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F3755ECAA4E; Fri, 19 Jan 2018 21:52:11 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-qt0-x230.google.com (mail-qt0-x230.google.com [IPv6:2607:f8b0:400d:c0d::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C11E97797B; Fri, 19 Jan 2018 21:52:11 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mail-qt0-x230.google.com with SMTP id e2so7419348qti.0; Fri, 19 Jan 2018 13:52:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=xILgYEqRwmFxrKnvLmu/txJCeJGSgGv4P9Ig8P5ZrIM=; b=ET7Wsd1gsBXc3p9iGcKQZSS/XreGcMZJwfegRCpfdhLey6fZ6hS80Sr+eHALj2NeWz 1rbCcym72if6W0I4Z+oBFwfQEoQfvEiBB/Y9Ikcz0KKPmUSN8m0Uu154DZzoZggo3XzE Ad8eGK3jIZJ4VSSH0tLrUYe193fXuBkkkjfPwHOty7fmnYj+lQyWA2m6HoTE9GRPhVtD Kbkalr4Xa51Ftomi4FesEcQCg+rXjc1X2dHLZdY0ZLmDuIupj6KLCfarrc96V50TAFyj ADzf2Cm0HqBOxSQMP575ml6r9Vu+KKZIhYpJz1SiuzA0vJKDkbizPwJX/vyUgLFnOViN NW5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=xILgYEqRwmFxrKnvLmu/txJCeJGSgGv4P9Ig8P5ZrIM=; b=VHRAfGtWHLAfXTWw71BEM1cxUAnlLKfuz98O4H01d/gBBhLLuSfx1GMVquKqrqIILa KApGAZZWA7rvFVpn6eqk0qF7YkiSmK1JfkK8Q/EttdXkiK2WJNr2kFSIxRvfrbqj9nOk ucV/vMzd+2G4dX5rgQSdWiTqsDHiEo78blsU9/pu7Rs9Q6udXRln8jHlaqRb35qCX8wa Rh4W2cR3BY+OyJlAD8SyHmXJiyk7y09DQCNnli983oHBWTjGhpoELDoX93ISBcFe6b0l vcuNA1YdEEiEYUxz0Bv1qV0T1knndOziY7vVqbh6hvf9UPnECn712TdeGdbvrW7nKhyt Kp0w== X-Gm-Message-State: AKwxytekbJa+o8Q50/hCt9QxkjiaiHuRho5P9zhKneJxiLBa2GlCD71q Qryg6IE2jE/Hd6jz8IGc2sLF1eAEV4pKggDrN9VXGw== X-Google-Smtp-Source: ACJfBotJfGbIcbO80C0s9ODMA4vCBgxgwQFy5F7SaC37tKZYP2Ji4H29u39zlij7BHu3+qh68+gE/2Yi8kD5X5FxdSg= X-Received: by 10.200.64.219 with SMTP id f27mr32863405qtm.227.1516398730240; Fri, 19 Jan 2018 13:52:10 -0800 (PST) MIME-Version: 1.0 Received: by 10.237.54.2 with HTTP; Fri, 19 Jan 2018 13:52:09 -0800 (PST) From: Ryan Stone Date: Fri, 19 Jan 2018 16:52:09 -0500 Message-ID: Subject: 3 ARP cache related reviews To: freebsd-net , "" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-transport@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussions of transport level network protocols in FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jan 2018 21:52:12 -0000 I've opened 3 phabricator reviews fixing issues with the inpcb llentry caching. https://reviews.freebsd.org/D13988 This fix invalidates the llentry cache after the L3 route cache was invalidated by the routing table's generation count incrementing. Without this fix existing connections would continue to use stale L2 headers after the routing table was updated. I tried a simple test where I added a new route that would change the routing of an active TCP connection and confirmed with tcpdump that without this fix, the stack continues to use the old src/dst MAC for the original route. This caused the traffic to be blackholed at the new gateway as the MAC did not match. https://reviews.freebsd.org/D13989 This is a simple cleanup of the inpcb cache invalidation. Rather than copy-and-pasting the separate invalidation of the L2 cache and the L3 cache, I move the invalidation into a single macro and invoke it everywhere that needs to drop the cache. This will hopefully prevent future bugs like D13988 from occurring in new code. https://reviews.freebsd.org/D13990 This fixes an issue where the inpcb L2 cache was not updated following a change to a route's gateway. This led the connection to continue to use the old gateway. If the old gateway stopping routing traffic this would cause the connection to be blackholed. My fix here is more heavy-handed than it needs to be, as I just increment the route table generation count, which will cause all connections to drop their L2 and L3 caches. This could be made more selective by adding a generation count to each route entry, but I don't believe that route table modifications will happen frequently enough for anybody to care. Please speak up if you believe differently. If you're interested in one or more of these changes, please subscribe yourself to the reviews. I don't subscribe mailing lists to reviews to reduce the spam on the lists. Thanks, Ryan