From nobody Thu Jul 1 01:46:10 2021 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 150E611F4EC1 for ; Thu, 1 Jul 2021 01:46:15 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-to1can01on060e.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe5d::60e]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GFgzT6hRtz4pRt for ; Thu, 1 Jul 2021 01:46:13 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Wv8W+/h0fXqA11l71J8ALngqHQU+TB7nzd4XSZf5BErH7qK+CWRt8oBpdKfHhrXChTwHtiawkFqQcAlDLFguxntU8XD/DbbaFPhppRPqhZTppq0TO5JwD4NS5BXFZYIAesUVE1QQ21+icC1/30/wSTKGTA1qYt2A75iWmMWMT+LzwQmbnXxwQWSo+dOqecVehYm98wDwTZv5cTSWaZGXlAOujg9msaeeYDBOyO0oA34ozCi9uUSRXOgRlq/lpUMNUgJgR/QkeQvFexiHkUuA3yDnJO0k5XfOwZmKKEAR5sxopBMUKvt/g5w8kb830F0RRoI1iYzt2MHMYesOEWUHIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gkQV8xcqjnFTrcIBrgF8tCtkOWi8466yCcaRDFmK5yg=; b=GRc9OFgPnliCX9QuDQFtd5NgtIBPiJ15JYPnAFDXJUwdaU4OYuVwWW0d0l57Dk0a1D2AUW4r4xJPvSrn7ZqSeYCM6s2k1Sil9csLn9fkJNfJ1YW4OzpcteUrOZXY4OrKI0hEoJr6DE+hSbd17P5pWOUtzyEXkaKKFcGDdX6T5kes7Avn8Qj5GEfxyyLn33LXBqSWkLFDeVjFm9E9lnsJmAFyvaNok4Opsm+7uDUxqtiOSVApB+WaadPdlcg8cqNIR9FCr9L70mS7FjnjFvTaNVKlcyBmNCVv0xMnAUbmFA0X/7pjHxExaUF65nciN8XAAo1iJuP/hErStAByhjICiw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gkQV8xcqjnFTrcIBrgF8tCtkOWi8466yCcaRDFmK5yg=; b=MDkNztPMlrww2TKBzeBF4d1Vq7he/QL7AT06KlRxK8BL+jPf3hF3So2apVEl0EkN1fiouSVejFHRl7zIkW6WiPdyJwBOgoNsvPCHaCLcYgko4V1eRo739lLFnT82BTe9tFsqWDAznGJmVJSEw0Fufr6Gek/nz2O7UKUerpxHshcxrRfPLH/bi16Hg2aNzIQhAw9bS/3eyxpHvgIusczif/x1y2I2SrGHL2uuWs2e74hoEGuIdvU/MNAY9aBfsiivgsQ892dyBsuW1iFOrPgOl0q2sWpGJENNHHb/eMYV3r/Cyu6bI+zvj98gf6LVADphTtcCMely8e/Xg0tt2ikyzw== Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:19::29) by YQXPR01MB2487.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:51::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.21; Thu, 1 Jul 2021 01:46:10 +0000 Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::583:528b:dbac:37bf]) by YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::583:528b:dbac:37bf%4]) with mapi id 15.20.4264.027; Thu, 1 Jul 2021 01:46:10 +0000 From: Rick Macklem To: Peter Eriksson CC: freebsd-net Subject: Re: RFC: NFS trunking (multiple TCP connections for a mount Thread-Topic: RFC: NFS trunking (multiple TCP connections for a mount Thread-Index: AQHXbHjbAJ3BuaRhHUuq5k+hERhS36sqtG0AgAKnXIY= Date: Thu, 1 Jul 2021 01:46:10 +0000 Message-ID: References: ,<362300CE-30DA-4552-A3E4-0F3DFE385B2A@lysator.liu.se> In-Reply-To: <362300CE-30DA-4552-A3E4-0F3DFE385B2A@lysator.liu.se> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 7f7c96b1-0acc-4dfc-5d6b-08d93c3200aa x-ms-traffictypediagnostic: YQXPR01MB2487: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: I3Dm2NfAgYU7/Xa/kii+of+dtOwGEqfef0E6y5GABiQXC6G56Q8+j4yY8Kk5cio1CSnKYOqJOaMwQH4c2BsyAGCgePeby2TfvcOEaYUcWs8tKpEMnCmw8Lnba1RF6jgaD00OPvbnl3oN6K5mY7tTevBXXLveZxn+rPxjJp3lKI6BEs1tq5cm/Rbh4wS6hoyncO8crRHGRF8HFdJU013xM6iXKOi23RVrYS+Wb81/+wpchXNW9J5TZ8DlHfTs288s7aBn+lQwN+rK+haZIp9f0A2luVBHtzZdzi+bKbewMx5NrGd5u5n4tB8nmM35UcmHOHsZ+3J+s5o/JUrjRLhJJq1Ihyeb9PVU05wgpKcHSH8aL9UHuytJiJOmXWCGevRKBAi4O7QGDI7c33IPapLw4K2TS/kgTNY9n199WunbaXVD7zKiXpOOnUqnSmfBqzZ82EBGqGv+8muan6biTjIxSryotEeuU2kFZjK46+zPfNulbHFpMpAzLbRp87I7QXnJ1fkKzMpDos4LSRGqmUB6iQ4xcCAMb0kceG7lQI61jAssddXuN2BuPs6bEsvhycu/Q8xmaTVnSTEeObu7VopoBrop7oATNqqYpqykqzVEGyjxO1ohpitfnbNa5o2BefBFjoyHPlcu/W+c9Ay7EaQXXq4lQs1OaOji5Rdf91BaqeGswDzb+zjsx2PzA02MjQvOPP1sGCqyG/XxU0Ep/BhyMONye6M7ZrPrgXogm0YfxNTxDTHDLeROpT7y0T/hZp4UePIt6TC5okB+//ndDA6biA== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(396003)(136003)(376002)(39860400002)(366004)(346002)(4326008)(6916009)(66946007)(83380400001)(66446008)(64756008)(55016002)(9686003)(52536014)(66476007)(33656002)(8676002)(316002)(5660300002)(296002)(478600001)(66556008)(786003)(71200400001)(122000001)(86362001)(8936002)(2906002)(38100700002)(76116006)(7696005)(6506007)(966005)(53546011)(91956017)(186003);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?Windows-1252?Q?5++nC7Ly3EivdXnXoGkLm7Nsq/TFcmmt1fQ4AEgF30JPITA6Ixv61+ax?= =?Windows-1252?Q?wTnY++mxSH4RLXcbiM4mdbqjpZpC1n3UAyPKn+G4YrveYXVMsJTpdFTJ?= =?Windows-1252?Q?FxSusLhFAjSfa10UpKby+xImhpEdLCpASeOSOfl+zifLov0QTB1D2JCx?= =?Windows-1252?Q?ps1J3SY8gSYuYiq15QRFEKajLh9FuYbxxkQeqVABLzfS5jMOkwVezIfo?= =?Windows-1252?Q?M4ZupB2o3KpH5hmgbR9zHGui3Y6FnXyLhGQy2ZHTuXC4ikNcOEEnMC2A?= =?Windows-1252?Q?3fx6qYyoSgxNn6eFVT4wQtNh6uH+VKuToAwLqrm05Ge40ZliPEUtAVm8?= =?Windows-1252?Q?8wOJ6YYGK8WMFhNpcRK40nFJ89CyVNLIJ707hQd74/YpxxtDqmVRMQLa?= =?Windows-1252?Q?0l6uthl/HWRrQBvD00flSaT95oULyyL5JeKXbEOyqSrkjb1aaFzKD3Xw?= =?Windows-1252?Q?2lC5VkWg0u/g5Jq5pJSwPBFUwH3ocGM23TmfnZZgO7v23Dk2zNDi47qN?= =?Windows-1252?Q?wyzwvb97GKGohIqYHuf5j9NQLfTqpvwYEvcab3bwhJf4ub0SwV6d2Cqi?= =?Windows-1252?Q?14l6Dymo8lzZkhSmJBjcmUlYruTRbyUwvdJWxMY6jAx8fMpuLa6+oA3m?= =?Windows-1252?Q?U6zDr12wTN6LI8Le1ggysUDVKk4T+00Yv1Iks04H1hGzOfeHNOSjaxu5?= =?Windows-1252?Q?/4VbbINTt4HGRf0TpxJJvAW+JZuF8a4AsQhfTL/omWBA7LTknWVNVo3X?= =?Windows-1252?Q?qrQYso8A3OfSgJ+eeSi0EdxfnYj+Smppggwyg+1qUw1mmc4CBvw2pIRK?= =?Windows-1252?Q?6GaWZ+OmbZhqxZQBwayKpzU1ElJ7S47y074aukaE+aEis7qWFxzq3N+N?= =?Windows-1252?Q?1yWYqfDAl4kjF4Af9i8q6KEgZVCDE53FkxezypUYL2KOMMtxqs3NSvQj?= =?Windows-1252?Q?mJkMlTrWLADLcqaiisLOZ15agoHJpA3pwACFPVDtzFNgjz3FBw4PEMUG?= =?Windows-1252?Q?UzWb6Fl+ZqFuHG7rGfX3v4oh/xNsaY+cUU7YSULLMqn9SilF/gEVM+kR?= =?Windows-1252?Q?saOY2KyoWR1jJutpgz0ENihLD+4i8cbNDtemg+HLpixvXwW3uBM11j3b?= =?Windows-1252?Q?qLUQAIYcGWrjAKK9RuqMREAU9k0NBdRfHrZqS1RzH0mxHKd1BKAy0SHt?= =?Windows-1252?Q?sMbBfMHyyIIf1vqKnpC71yjy6jUBeajqn0cW0LWYhz0t8jQDJEwMtQRU?= =?Windows-1252?Q?no4o9etrSs0Ral4kEJGZrkRq3E/SGjQ5LOj3I0qADpZQwj6zkx9AVXpV?= =?Windows-1252?Q?IBQj7SkOE0pIOZYZd66kVWHDdHXIrPYs2x87QGjSes1tCjW/DNHPjJRS?= =?Windows-1252?Q?s+Z1MjN9G0hSzAplsP0PFdwHVQiag/Xv01vMiJtFRZNtIlfb3h2XcfDV?= =?Windows-1252?Q?hnWXcaL5z8JgTZF9108D7qbBQxhcntwak/36cfNezdE=3D?= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 7f7c96b1-0acc-4dfc-5d6b-08d93c3200aa X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Jul 2021 01:46:10.6412 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: X57NtK619CQGdgW1NRw4hmEbi3jpVIUsJLBxz0kcoE77xdqYXoI2UGsIwjIsE0IIld6KmC93fKQzRB6fqy1sEQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQXPR01MB2487 X-Rspamd-Queue-Id: 4GFgzT6hRtz4pRt X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector1 header.b=MDkNztPM; arc=pass (microsoft.com:s=arcselector9901:i=1); dmarc=pass (policy=none) header.from=uoguelph.ca; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 2a01:111:f400:fe5d::60e as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-5.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a01:111:f400:fe5d::60e:from]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector1]; FREEFALL_USER(0.00)[rmacklem]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a01:111:f400::/48]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[2a01:111:f400:fe5d::60e:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[uoguelph.ca:+]; RCPT_COUNT_TWO(0.00)[2]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; NEURAL_HAM_SHORT(-0.98)[-0.984]; DMARC_POLICY_ALLOW(-0.50)[uoguelph.ca,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:8075, ipnet:2a01:111:f000::/36, country:US]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; MAILMAN_DEST(0.00)[freebsd-net] X-ThisMailContainsUnwantedMimeParts: N In case anyone is interested in testing and/or reviewing the patch, it is at https://reviews.freebsd.org/D30970. Only lightly tested at this point. The NFS mount option is "nconnect=3D", where 2<=3D N <=3D 16, same as Linux. (I haven't done a man page patch yet.) Thanks everyone, for your input, rick ________________________________________ From: Peter Eriksson Sent: Tuesday, June 29, 2021 5:11 AM To: Rick Macklem Cc: freebsd-net Subject: Re: RFC: NFS trunking (multiple TCP connections for a mount CAUTION: This email originated from outside of the University of Guelph. Do= not click links or open attachments unless you recognize the sender and kn= ow the content is safe. If in doubt, forward suspicious emails to IThelp@uo= guelph.ca > I don't understand how multiple TCP connections to the same > server IP address will distribute the load across multiple network > interfaces? > I thought that lagg would have handled this? A lagg typically keeps all data in a TCP stream on a specific lagg member (= depending on how the lagg is set up, unless you select the =93roundrobin=94= option in freebsd - don=92t do that unless you like out-of-order packets= =85) Network equipment with laggs typically hash the IP streams over the lagg me= mbers based on MAC addresses (source&target), IP addresses (source&target) = and port numbers. (We have been diagnosing a fun problem locally where we see packet losses/p= erformance drops over our internal backbone network for certain combination= s of odd/even IP addresses/port numbers when things pass certain SPB =93rou= ters=94 (which typically hash the streams over many =93channels=94 between = routers)=85 Fun fun. :-) I think the multiple NFS TCP streams could make for some nice performance i= mprovements in certain cases. And it would be a more generalisation of havi= ng multiple streams between two hosts - one-or-many over IPv4 and one-or-ma= ny over IPv6 at the same time. Windows SMB has a similar feature. Just avoid the Linux NFS mounting deadlock issue with =93down=94 servers pl= ease :-) - Peter