Date: Tue, 29 Jun 2021 00:23:21 +0000 From: Rick Macklem <rmacklem@uoguelph.ca> To: freebsd-net <freebsd-net@freebsd.org> Subject: RFC: NFS trunking (multiple TCP connections for a mount Message-ID: <YQXPR0101MB0968DC173855A82AAF45F08FDD039@YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM>
next in thread | raw e-mail | index | archive | help
The Linux NFS client now has a mount option "nconnect",=0A=
which specifies that multiple TCP connections be created=0A=
for an NFS mount, where RPCs are done on the connections,=0A=
in a round robin fashion. (Alternating between the two TCP=0A=
connections for the case of nconnect=3D2.)=0A=
=0A=
The Linux man page says:=0A=
nconnect=3Dn=0A=
When using a connection oriented protocol such as TCP, it=0A=
may sometimes be advantageous to set up multiple=0A=
connections between the client and server. For instance,=0A=
if your clients and/or servers are equipped with multiple=0A=
network interface cards (NICs), using multiple connections=0A=
to spread the load may improve overall performance. In=0A=
such cases, the nconnect option allows the user to specify=0A=
the number of connections that should be established=0A=
between the client and server up to a limit of 16.=0A=
=0A=
I don't understand how multiple TCP connections to the same=0A=
server IP address will distribute the load across multiple network=0A=
interfaces?=0A=
I thought that lagg would have handled this?=0A=
=0A=
I could easily implement this, but I only have low end hardware=0A=
to test on, so I doubt that I will see any performance improvement.=0A=
=0A=
However, I do think that having two TCP connections, where the=0A=
RPCs involving large RPC messages (Read/Readdir/Write) are sent=0A=
on one TCP connection and the RPCs that use small RPC messages=0A=
(Lookup/Access/Getattr,...) are sent on the other one.=0A=
--> This would avoid the frequent small RPCs from getting "logjamed"=0A=
behind a bunch of large 1Mbyte Read replies, for example.=0A=
=0A=
So, what do you think?=0A=
- Implement "nconnect" with round robin RPC assignment.=0A=
or=0A=
- Implement two TCP connections where large RPCs are done=0A=
on one and small RPCs on the other.=0A=
or=0A=
???=0A=
=0A=
I will note I see downsides to doing multiple TCP connections/mount.=0A=
1 - Uses up more IP port#s.=0A=
2 - When an NFS server gets overloaded, it will stop receiving RPC requests=
.=0A=
This will eventually apply backpressure through TCP to the client to s=
low=0A=
down RPC requests. Having multiple TCP connections would reduce this=
=0A=
backpressure effect.=0A=
--> To be honest, I suspect the slowdown in RPC replies caused by an=
=0A=
overloaded server, is more effective feedback to the NFS client=
=0A=
than TCP backpressure, but I am not sure.=0A=
=0A=
Comments? rick=0A=
=0A=
=0A=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?YQXPR0101MB0968DC173855A82AAF45F08FDD039>