From nobody Fri Oct 21 15:02:59 2022 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Mv74n2Dbwz4fsmC for ; Fri, 21 Oct 2022 15:03:05 +0000 (UTC) (envelope-from Cheng.Cui@netapp.com) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2053.outbound.protection.outlook.com [40.107.92.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Mv74l6kNSz3srs for ; Fri, 21 Oct 2022 15:03:03 +0000 (UTC) (envelope-from Cheng.Cui@netapp.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CVI3TGuoXC3LFA/i/XZ8/x8n3p9BBGiT4pkyMOPEAzO33RE+coBmYLquvPy5ymAmCNUEJUu9g+53drtnSjChw/K/edIVYAMqMb+KgL7KroJZmj8wK3SFIpWt6ZQqeRGLpOg+/q9kv7DY9Tak/N0iPfUxcj8JAEheq7VUqmx7SwZATL3BjbDXw4jhv0mNer7QMbRiSRE8JcIjj4X3JzbivKkeOwWF4M3H2Th0ghfZ00XN2GDJpMgGazl/XkaoxW/VenxhuEKx7VSQCq5+weEOkD/BpB/Pk0eyk3TplAYV421QvdfSon6sCoU1juusRRlnU9hB+6bjV8yxXZ55MGgrFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SQUqdZAxlCBNIp2ZQskv+RNIZsdajI7CoKSaRBPbK9g=; b=dVwLEbOhMLeNJju1IV3PL7qNQDX+s4B79zeiAsDL6LexoEpxI6sSO0+qRLhtGHiS6LauyvC3O1eqx31BERUJ1GecboTSGZr7Nl9F2+ahtDkl82cna7RntaQfzbkGW+IbnS3rntPYVDRbQqo/XHchRhU8SHD9Goqc2xYrzj8OtoBLx778uolYN4Ve5ZpvlDhv8SdP4bE7wZXNsD0lBh1XzvlC3AuTUAD6uc3CxA5KbmGJR+hNsQgDQdVbtyu1+EFL+6qHpDF9pHT4SBB03Me9MS0/QeLPyOkAFMdsnSUjkGUVNt/Vp1z2mUXbQyumZQSuu2c3V/hxBUst1bPnYYIu/w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=netapp.com; dmarc=pass action=none header.from=netapp.com; dkim=pass header.d=netapp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netapp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SQUqdZAxlCBNIp2ZQskv+RNIZsdajI7CoKSaRBPbK9g=; b=hNgcrBEhO3lMTz3br9ZJU8uCDtrEu1LNJ9MoN+XHJnxIVYa5tDcw19o69sUrcERun2Ghtcg0Kan+QB8chS8twoq9Efsgv1CEeBP4jZkvtCd3NandLJEqEnvaCP5Mciu/dx4sHjJA+YL4kq/Vi5fa/QYrqYe4/w4YXwge7mbs2ae7eG2wLqdDas3s0VbZv0wqAanqlYoWTDQjugGrE48UiB8H9Fyti/xDB1BS//f4IhcN3xzKclWAAK0/dXBgqBHSr39b7hJX66TeZaU5D+/slMIrsv6BuWUZffXLjs1SZ4H3QWV2nF26hClr4aPojoOCshihdBeKC6XrtAq9p4SDLQ== Received: from MN2PR06MB6237.namprd06.prod.outlook.com (2603:10b6:208:de::11) by PH0PR06MB8445.namprd06.prod.outlook.com (2603:10b6:510:be::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5723.32; Fri, 21 Oct 2022 15:03:00 +0000 Received: from MN2PR06MB6237.namprd06.prod.outlook.com ([fe80::f584:2124:1967:8a7b]) by MN2PR06MB6237.namprd06.prod.outlook.com ([fe80::f584:2124:1967:8a7b%2]) with mapi id 15.20.5723.033; Fri, 21 Oct 2022 15:02:59 +0000 From: "Cui, Cheng" To: Zhenlei Huang , Michael Tuexen CC: "freebsd-net@freebsd.org" Subject: Re: Too aggressive TCP ACKs Thread-Topic: Too aggressive TCP ACKs Thread-Index: AQHY5Vg7wzLsOqag4UuwXqTbArQ1064Y6aUAgAAHboCAAABWvQ== Date: Fri, 21 Oct 2022 15:02:59 +0000 Message-ID: References: <75D35F36-7759-4168-ADBA-C2414F5B53BC@gmail.com> <712641B3-5196-40CC-9B64-04637F16F649@lurchi.franken.de> <62A0DD30-B3ED-48BE-9C01-146487599092@gmail.com> In-Reply-To: <62A0DD30-B3ED-48BE-9C01-146487599092@gmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: MN2PR06MB6237:EE_|PH0PR06MB8445:EE_ x-ms-office365-filtering-correlation-id: 259f278b-ef09-4a37-eae8-08dab37557e0 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: TPlLr76i2c7erW7jX+WIOc9OdiMtn2pmxel3OTvS0Bzasz7shbN1iqOBi4g1wkbf038YGxZQIVcCfvJtl38TU6W2Q6YDX+l4sXQRZlWkT/PoTPvUTzqwZAWBBSyHvsLmh+HuP+uIuPwsCB3JWI6Gsm4O+Gfi18nQo6bplLOHJvnfjCXz6/UUJnDRe1yo8JXP0VD4BupHFrScRHO0uk2Nipa5zfYCIh58DdZlYg6/gvbNcUJRFPFyg6vJHqra6CuJOyYgFZNtzOHBOHWSV4ATC53YqWCaoFpmaL7YCgD6OMGHoX/JoL62XcDSPk4dOIzf5mgOD+esQNES12KvVyebFeKh5XMBTxpu6ehfqGNQdYzTBDBeVHSjmKgW6EaG4YE2W4YjX6+eF+VlcUkGk5NhVlUDfftAo8wjk6Mo5gBTC98KcAXiGRW70NYFVpjKREULzPgNrjH8zUZp0UldUMXaIT9//laQ6wkzM0e78fmoagypYRMHmmBZV+APFdM1de1B9EDL/Nq+awMMooOh7aHoGPo5vWmwaxYwSp6HuSRl4FryaaH2h6M6vs7CvwS9AEQVDzDjCdGuIPb4NtpQRTDxOJE/FNMvxZZarJXhmlrpd8ldILElx0ljv+oCMfPD45PzSA2hofFsBbrVeR4W9ZP3qoKimQNj2hWTGZ/tRMGpSiWDq5AH+1wXX9AYQHHtFDOMAV5VtiUu7x2AhT1IXPh+U94Iv4Eh7utQ5qNnI7RqvdUsbmJ57bznoqE70o7KFgQO6bS0SjuwEuJC00Dqs4ymz6j5Oj734EKSo1K2/ZG48Ho= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR06MB6237.namprd06.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(4636009)(39860400002)(366004)(346002)(396003)(376002)(136003)(451199015)(8676002)(166002)(478600001)(38070700005)(122000001)(38100700002)(33656002)(966005)(110136005)(76116006)(186003)(66476007)(66946007)(66556008)(64756008)(5660300002)(41300700001)(83380400001)(71200400001)(8936002)(26005)(53546011)(4326008)(9686003)(3480700007)(66446008)(86362001)(316002)(7696005)(52536014)(55016003)(55236004)(2906002)(6506007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?Windows-1252?Q?rEdfBAXtmb68cIgFBVdjxlPbhNLAPIcNew69Uys5L3fzkrGsfAzCiRoa?= =?Windows-1252?Q?yKGeWnvKdzzzUPrfKNNuPRgEptjlMV1Aq0nmDVykOwIjBp5kwX/YfBB2?= =?Windows-1252?Q?LbatO5Is2jbexJjnGls0AA1XDdPff/Vrb1lcP3vUu1NLLxD4rrM/UIdx?= =?Windows-1252?Q?gcwMrSVi1iRnNK3uAOYkqCkzzPa7hwxB6x3r4UvydJev707FfrISou1K?= =?Windows-1252?Q?WZeS4qFeReH3V0zAXG/tx5+mNfBm+hy7VcwvgWUGqAzqRAdPuFd/b+1B?= =?Windows-1252?Q?oxMH+b0wH35CIJHWQsR6NwxR+c3mmMKJsd2oOjRsHYuly5uCP3P6zFIS?= =?Windows-1252?Q?/D2KT8T/WlVoNHwhFb9f5xqClO95Nl4K8xQV08eWWFGDSyuYudaY7pKd?= =?Windows-1252?Q?s6WTEdW4zfgwZXScD3ISFl5YoHVcIo+XL9Sp44g57aHoGvj1qw7blvtQ?= =?Windows-1252?Q?naf8EzD9FbwQ59fA5BPICFgSXWBuNU/2WIqPxg4GSkbH6hPvuVdlwSQ/?= =?Windows-1252?Q?HZXH9Tx6Iwz1jgHA0Pmbnw4uhceuliWJnX2Gh4TF8OgAIRs9I3aZWv2t?= =?Windows-1252?Q?gRbkkVzSJ2HTGa49yKnUOdbjIKXbwpYBmVFgXPWR46i47PqWjl+qcJVR?= =?Windows-1252?Q?iM1Q8dspqFH2IAs2TTYxffpdGG6CnmueyzLA42EMcGctmnQUvWk6T9oX?= =?Windows-1252?Q?5275T8YhFdXjD9z/ddsy7MKM18PJy3dm2i79eo4xGfilGjze+AAVVWQB?= =?Windows-1252?Q?1nglNVOWUV+iG4vXJy7GoMe5zrNA2WzNfLMb8Yn+SphF0NJfGrae5i/G?= =?Windows-1252?Q?TzXqgR6R7Zr3PHsXfiF/46X1DQmjDxe4uDofifIl6YVPg+zqF368M6t9?= =?Windows-1252?Q?iTCAKbzDQwYttvV8sorHuzsLH0VoZf4TnznKucCqhrO5Jdf2zPhfKwdd?= =?Windows-1252?Q?PG3vVI25n/8OzJ8e1Y52Nj+dt8U2a/cc7vJVQuz4ClMNN+MDR0anhhuj?= =?Windows-1252?Q?a5BfCcCZvWYPpz4jDJByzp2ga4Ycm6CYVZ1bfJOO4vAt6Y7Bv4r4QfvJ?= =?Windows-1252?Q?nMO70cw+ckY/ScOEKNDqKr8rT/ZVUvP6NlnKBajc/owUAwkueYSjwGJK?= =?Windows-1252?Q?xxWaNzBp8PZCZsMvbZK2FNynq0LX5zsrmvaiv1lXASb94rDMk4htbXW9?= =?Windows-1252?Q?7Upsdtnje9q/bHq0ZoOUCLM4G+i3VwtRcXwnpVgB1jM5LIL6sfeJVauY?= =?Windows-1252?Q?qjRwanxDixKOSB0qQ/jJpDwaCQDsl1IDe+z8dCesRhRV5UJWdqwxtEc8?= =?Windows-1252?Q?3w/ItD3FgHFSMh389Z2mKr60AQ5qR2qFf1wLVGsOaAkSO6yS9DQO9BKP?= =?Windows-1252?Q?OmPzevNpQ2o550oq7hMPBrGvBnl4Vr57zrb4/VyeE8XXUhyuNhsKg9+3?= =?Windows-1252?Q?UxSFKi8qD9JFsGdpzJ14gkDYfIhXZ85avHLnrzriDuGq9kYXB6aM+aPB?= =?Windows-1252?Q?5oiQ8VbOjE9Fu7s3u96N3CsE07Lw4rfFMmrZNlO89embWRPpAADAPSXb?= =?Windows-1252?Q?xdDB7zQmc0Nc9CK+9SZiHrg+el51GBXVfqoqvcMRsG+T1R2jCtjBUlrA?= =?Windows-1252?Q?pfDi8YNsOafLdQqTqRY/AD5wwrnoARyWuYeMiRSDPY8BgIyzTLEO7RP2?= =?Windows-1252?Q?eFQ29GmAhTc4w0dPWSoj1Dxf9AIA76B0Rok8P87lI4HxiCWWLngmHw?= =?Windows-1252?Q?=3D=3D?= Content-Type: multipart/alternative; boundary="_000_MN2PR06MB6237E90881035621441597989D2D9MN2PR06MB6237namp_" List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 X-OriginatorOrg: netapp.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR06MB6237.namprd06.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 259f278b-ef09-4a37-eae8-08dab37557e0 X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Oct 2022 15:02:59.2937 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4b0911a0-929b-4715-944b-c03745165b3a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 6+yMOo4BXA4KoDzqKb6TYpPY8SAWX3oOcuwcGtnpGqqIlz6h94Af5dHSinZpzVwvPY/kmAmMB0EbSciT7ZbXiA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR06MB8445 X-Rspamd-Queue-Id: 4Mv74l6kNSz3srs X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=netapp.com header.s=selector1 header.b=hNgcrBEh; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=quarantine) header.from=netapp.com; spf=pass (mx1.freebsd.org: domain of Cheng.Cui@netapp.com designates 40.107.92.53 as permitted sender) smtp.mailfrom=Cheng.Cui@netapp.com X-Spamd-Result: default: False [-5.99 / 15.00]; DWL_DNSWL_LOW(-1.00)[netapp.com:dkim]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-0.99)[-0.993]; DMARC_POLICY_ALLOW(-0.50)[netapp.com,quarantine]; R_DKIM_ALLOW(-0.20)[netapp.com:s=selector1]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MLMMJ_DEST(0.00)[freebsd-net@freebsd.org]; FREEMAIL_TO(0.00)[gmail.com,lurchi.franken.de]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[netapp.com:+]; RCPT_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[40.107.92.53:from]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TAGGED_RCPT(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.107.92.53:from] X-ThisMailContainsUnwantedMimeParts: N --_000_MN2PR06MB6237E90881035621441597989D2D9MN2PR06MB6237namp_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable You can also think about MacOS=92s delayed ACK setup in default is conserva= tive. https://developer.apple.com/forums/thread/716394 -- Cheng Cui From: owner-freebsd-net@freebsd.org on beha= lf of Zhenlei Huang Date: Friday, October 21, 2022 at 11:01 AM To: Michael Tuexen Cc: freebsd-net@freebsd.org Subject: Re: Too aggressive TCP ACKs NetApp Security WARNING: This is an external email. Do not click links or o= pen attachments unless you recognize the sender and know the content is saf= e. On Oct 21, 2022, at 10:34 PM, Michael Tuexen > wrote: On 21. Oct 2022, at 16:19, Zhenlei Huang > wrote: Hi, While I was repeating https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D2= 58755, I observed a strange behavior. The TCP ACKs from FreeBSD host are too aggressive. My setup is simple: A B [ MacOS ] <=3D=3D=3D=3D> [ FreeBSD VM ] 192.168.120.1 192.168.12.134 (disable tso and lro) While A <--- B, i.e. A as server and B as client, the packets rate looks go= od. One session on B: root@:~ # iperf3 -c 192.168.120.1 -b 10m Connecting to host 192.168.120.1, port 5201 [ 5] local 192.168.120.134 port 54459 connected to 192.168.120.1 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.25 MBytes 10.5 Mbits/sec 0 257 KBytes [ 5] 1.00-2.00 sec 1.25 MBytes 10.5 Mbits/sec 0 257 KBytes [ 5] 2.00-3.00 sec 1.12 MBytes 9.44 Mbits/sec 0 257 KBytes [ 5] 3.00-4.00 sec 1.25 MBytes 10.5 Mbits/sec 0 257 KBytes [ 5] 4.00-5.00 sec 1.12 MBytes 9.44 Mbits/sec 0 257 KBytes [ 5] 5.00-6.00 sec 1.25 MBytes 10.5 Mbits/sec 0 257 KBytes [ 5] 6.00-7.00 sec 1.12 MBytes 9.44 Mbits/sec 0 257 KBytes [ 5] 7.00-8.00 sec 1.25 MBytes 10.5 Mbits/sec 0 257 KBytes [ 5] 8.00-9.00 sec 1.12 MBytes 9.44 Mbits/sec 0 257 KBytes [ 5] 9.00-10.00 sec 1.25 MBytes 10.5 Mbits/sec 0 257 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 12.0 MBytes 10.1 Mbits/sec 0 sende= r [ 5] 0.00-10.00 sec 12.0 MBytes 10.1 Mbits/sec recei= ver iperf Done. Another session on B: root@:~ # netstat -w 1 -I vmx0 input vmx0 output packets errs idrops bytes packets errs bytes colls 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 342 0 0 22600 526 0 775724 0 150 0 0 9900 851 0 1281454 0 109 0 0 7194 901 0 1357850 0 126 0 0 8316 828 0 1246632 0 122 0 0 8052 910 0 1370780 0 109 0 0 7194 819 0 1233702 0 120 0 0 7920 910 0 1370780 0 110 0 0 7260 819 0 1233702 0 123 0 0 8118 910 0 1370780 0 109 0 0 7194 819 0 1233702 0 73 0 0 5088 465 0 686342 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D While A ---> B, i.e. A as client and B as server, the ACKs sent from B look= s strange. Session on A: % iperf3 -c 192.168.120.134 -b 10m Connecting to host 192.168.120.134, port 5201 [ 5] local 192.168.120.1 port 52370 connected to 192.168.120.134 port 5201 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 1.25 MBytes 10.5 Mbits/sec [ 5] 1.00-2.00 sec 1.25 MBytes 10.5 Mbits/sec [ 5] 2.00-3.00 sec 1.12 MBytes 9.44 Mbits/sec [ 5] 3.00-4.00 sec 1.25 MBytes 10.5 Mbits/sec [ 5] 4.00-5.00 sec 1.12 MBytes 9.44 Mbits/sec [ 5] 5.00-6.00 sec 1.25 MBytes 10.5 Mbits/sec [ 5] 6.00-7.00 sec 1.12 MBytes 9.44 Mbits/sec [ 5] 7.00-8.00 sec 1.25 MBytes 10.5 Mbits/sec [ 5] 8.00-9.00 sec 1.12 MBytes 9.44 Mbits/sec [ 5] 9.00-10.00 sec 1.25 MBytes 10.5 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.00 sec 12.0 MBytes 10.1 Mbits/sec sende= r [ 5] 0.00-10.00 sec 12.0 MBytes 10.1 Mbits/sec recei= ver iperf Done. Session on B: root@:~ # netstat -w 1 -I vmx0 input vmx0 output packets errs idrops bytes packets errs bytes colls 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 649 0 0 960562 330 0 21800 0 819 0 0 1233702 415 0 27390 0 910 0 0 1370780 459 0 30294 0 819 0 0 1233702 415 0 27390 0 910 0 0 1370780 459 0 30294 0 910 0 0 1370780 460 0 30360 0 819 0 0 1233702 414 0 27324 0 910 0 0 1370780 460 0 30360 0 819 0 0 1233702 414 0 27324 0 910 0 0 1370780 460 0 30360 0 285 0 0 412287 147 0 9981 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 The ACK packets replied from B (the FreeBSD VM) are too aggressive. They ar= e about one half of TCP packets received from A. I've tested with different bitrates, from 10m to 300m, all behave the same. Tested with baremetal FreeBSD 13.1 Box as B (with intel em driver), the bitrates is 1g, also behaves the same. Also tried different FreeBSD versions, 11.4, 12.3, stable/13 and current/14= all behave the same. My question is, is that the expected behavior of current default TCP stack? That is what I would expect. TCP (on FreeBSD) is acking every other packet.= This is also what is specified. MacOS, at least newer versions, send less ACKs. Thanks for fast response! My have old memories about SACK which helps TCP performance. This behavior seems odd from my mind. But those memories date back to 2008, that is 14 ye= ars ago. The current implementation of TCP stack in FreeBSD head is too complexed fo= r me. Can you please point me the RFCs specifying this? So I can start over with = a quick glue. Thanks! Best regards Michael Best regards, Zhenlei --_000_MN2PR06MB6237E90881035621441597989D2D9MN2PR06MB6237namp_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable

You can also think = about MacOS=92s delayed ACK setup in default is conservative.

 

https://developer.apple.com/forum= s/thread/716394

 

 

-- =

Cheng Cui

 

From: owner-freebsd-net@f= reebsd.org <owner-freebsd-net@freebsd.org> on behalf of Zhenlei Huang= <zlei.huang@gmail.com>
Date: Friday, October 21, 2022 at 11:01 AM
To: Michael Tuexen <michael.tuexen@lurchi.franken.de>
Cc: freebsd-net@freebsd.org <freebsd-net@freebsd.org>
Subject: Re: Too aggressive TCP ACKs

Ne= tApp Security WARNING: This is an external email. Do not click links = or open attachments unless you recognize the sender and know the content is safe.



 

On Oct 21, 2022, at= 10:34 PM, Michael Tuexen <michael.tuexen@lurchi.franken.de> wrote:

 

On 21. Oct 2022, at 16:19, Zhenlei Huang <zlei.huang@gmail.com> wrote:

Hi,

While I was repeating https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D258755, I observed = a
strange behavior. The TCP ACKs from FreeBSD host are too aggressive.

My setup is simple:
        A     &= nbsp;           &nbs= p;            &= nbsp;  B
  [ MacOS ]  <=3D=3D=3D=3D> [ FreeBSD VM ]
192.168.120.1           &= nbsp;192.168.12.134 (disable tso and lro)
While A <--- B, i.e. A as server and B as client, the packets rate looks= good.

One session on B:

root@:~ # iperf3 -c 192.168.120.1 -b 10m
Connecting to host 192.168.120.1, port 5201
[  5] local 192.168.120.134 port 54459 connected to 192.168.120.1 port= 5201
[ ID] Interval           = Transfer     Bitrate      &nbs= p;  Retr  Cwnd
[  5]   0.00-1.00   sec  1.25 MBytes  10= .5 Mbits/sec    0    257 KBytes   &= nbsp;   
[  5]   1.00-2.00   sec  1.25 MBytes  10= .5 Mbits/sec    0    257 KBytes   &= nbsp;   
[  5]   2.00-3.00   sec  1.12 MBytes  9.= 44 Mbits/sec    0    257 KBytes   &= nbsp;   
[  5]   3.00-4.00   sec  1.25 MBytes  10= .5 Mbits/sec    0    257 KBytes   &= nbsp;   
[  5]   4.00-5.00   sec  1.12 MBytes  9.= 44 Mbits/sec    0    257 KBytes   &= nbsp;   
[  5]   5.00-6.00   sec  1.25 MBytes  10= .5 Mbits/sec    0    257 KBytes   &= nbsp;   
[  5]   6.00-7.00   sec  1.12 MBytes  9.= 44 Mbits/sec    0    257 KBytes   &= nbsp;   
[  5]   7.00-8.00   sec  1.25 MBytes  10= .5 Mbits/sec    0    257 KBytes   &= nbsp;   
[  5]   8.00-9.00   sec  1.12 MBytes  9.= 44 Mbits/sec    0    257 KBytes   &= nbsp;   
[  5]   9.00-10.00  sec  1.25 MBytes  10.5 Mb= its/sec    0    257 KBytes    =    
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           = Transfer     Bitrate      &nbs= p;  Retr
[  5]   0.00-10.00  sec  12.0 MBytes  10.1 Mb= its/sec    0         = ;    sender
[  5]   0.00-10.00  sec  12.0 MBytes  10.1 Mb= its/sec            &= nbsp;     receiver

iperf Done.

Another session on B:

root@:~ # netstat -w 1 -I vmx0
           input &nb= sp;         vmx0   &= nbsp;       output
  packets  errs idrops      bytes &= nbsp;  packets  errs      bytes col= ls
        0     0=     0         =  0          0   = ;  0          0 &nbs= p;   0
        0     0=     0         =  0          0   = ;  0          0 &nbs= p;   0
      342     0  &nb= sp;  0      22600    &nbs= p;   526     0     7= 75724     0
      150     0  &nb= sp;  0       9900    = ;    851     0    12= 81454     0
      109     0  &nb= sp;  0       7194    = ;    901     0    13= 57850     0
      126     0  &nb= sp;  0       8316    = ;    828     0    12= 46632     0
      122     0  &nb= sp;  0       8052    = ;    910     0    13= 70780     0
      109     0  &nb= sp;  0       7194    = ;    819     0    12= 33702     0
      120     0  &nb= sp;  0       7920    = ;    910     0    13= 70780     0
      110     0  &nb= sp;  0       7260    = ;    819     0    12= 33702     0
      123     0  &nb= sp;  0       8118    = ;    910     0    13= 70780     0
      109     0  &nb= sp;  0       7194    = ;    819     0    12= 33702     0
       73     0 &nbs= p;   0       5088   =      465     0   &nb= sp; 686342     0
        0     0=     0         =  0          0   = ;  0          0 &nbs= p;   0
        0     0=     0         =  0          0   = ;  0          0 &nbs= p;   0



=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D


While A ---> B, i.e. A as client and B as server, the ACKs sent from B l= ooks strange.

Session on A:

% iperf3 -c 192.168.120.134 -b 10m
Connecting to host 192.168.120.134, port 5201
[  5] local 192.168.120.1 port 52370 connected to 192.168.120.134 port= 5201
[ ID] Interval           = Transfer     Bitrate
[  5]   0.00-1.00   sec  1.25 MBytes  10= .5 Mbits/sec           &n= bsp;      
[  5]   1.00-2.00   sec  1.25 MBytes  10= .5 Mbits/sec           &n= bsp;      
[  5]   2.00-3.00   sec  1.12 MBytes  9.= 44 Mbits/sec           &n= bsp;      
[  5]   3.00-4.00   sec  1.25 MBytes  10= .5 Mbits/sec           &n= bsp;      
[  5]   4.00-5.00   sec  1.12 MBytes  9.= 44 Mbits/sec           &n= bsp;      
[  5]   5.00-6.00   sec  1.25 MBytes  10= .5 Mbits/sec           &n= bsp;      
[  5]   6.00-7.00   sec  1.12 MBytes  9.= 44 Mbits/sec           &n= bsp;      
[  5]   7.00-8.00   sec  1.25 MBytes  10= .5 Mbits/sec           &n= bsp;      
[  5]   8.00-9.00   sec  1.12 MBytes  9.= 44 Mbits/sec           &n= bsp;      
[  5]   9.00-10.00  sec  1.25 MBytes  10.5 Mb= its/sec            &= nbsp;     
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           = Transfer     Bitrate
[  5]   0.00-10.00  sec  12.0 MBytes  10.1 Mb= its/sec            &= nbsp;     sender
[  5]   0.00-10.00  sec  12.0 MBytes  10.1 Mb= its/sec            &= nbsp;     receiver

iperf Done.

Session on B:

root@:~ # netstat -w 1 -I vmx0
           input &nb= sp;         vmx0   &= nbsp;       output
  packets  errs idrops      bytes &= nbsp;  packets  errs      bytes col= ls
        0     0=     0         =  0          0   = ;  0          0 &nbs= p;   0
        0     0=     0         =  0          0   = ;  0          0 &nbs= p;   0
      649     0  &nb= sp;  0     960562     &nb= sp;  330     0      = 21800     0
      819     0  &nb= sp;  0    1233702      &n= bsp; 415     0      27390=     0
      910     0  &nb= sp;  0    1370780      &n= bsp; 459     0      30294=     0
      819     0  &nb= sp;  0    1233702      &n= bsp; 415     0      27390=     0
      910     0  &nb= sp;  0    1370780      &n= bsp; 459     0      30294=     0
      910     0  &nb= sp;  0    1370780      &n= bsp; 460     0      30360=     0
      819     0  &nb= sp;  0    1233702      &n= bsp; 414     0      27324=     0
      910     0  &nb= sp;  0    1370780      &n= bsp; 460     0      30360=     0
      819     0  &nb= sp;  0    1233702      &n= bsp; 414     0      27324=     0
      910     0  &nb= sp;  0    1370780      &n= bsp; 460     0      30360=     0
      285     0  &nb= sp;  0     412287     &nb= sp;  147     0      =  9981     0
        0     0=     0         =  0          0   = ;  0          0 &nbs= p;   0
        0     0=     0         =  0          0   = ;  0          0 &nbs= p;   0
        0     0=     0         =  0          0   = ;  0          0 &nbs= p;   0


The ACK packets replied from B (the FreeBSD VM) are too aggressive. They ar= e
about one half of TCP packets received from A.

I've tested with different bitrates, from 10m to 300m, all behave the same.=
Tested with baremetal FreeBSD 13.1 Box as B (with intel em driver), the 
bitrates is 1g, also  behaves the same.

Also tried different FreeBSD versions, 11.4, 12.3, stable/13 and current/14= all 
behave the same.


My question is, is that the expected behavior of current default TCP stack?=

That is what I would expect. TCP (on FreeBSD) is acking every other packe= t. This
is also what is specified. MacOS, at least newer versions, send less ACKs.<= /span>

Thanks for fast res= ponse!


My have old memories about SACK which helps TCP performance. This behavior<= o:p>

seems odd from my m= ind. But those memories date back to 2008,= that is 14 years ago.

 

The current impleme= ntation of TCP stack in FreeBSD head is too complexed for me.

Can you please poin= t me the RFCs specifying this? So I can start over with a quick glue.<= /o:p>

 

Thanks!


Best regards
Michael




Best regards,
Zhenlei

 

--_000_MN2PR06MB6237E90881035621441597989D2D9MN2PR06MB6237namp_--