Date: Thu, 15 Mar 2001 14:17:21 +0100 (CET) From: Jean-Luc.Richier@imag.fr To: Jean-Luc.Richier@imag.fr Subject: bin/25826: nfsd -h option dosen't work in TCP Message-ID: <200103151317.f2FDHLo00698@tuna.imag.fr>
next in thread | raw e-mail | index | archive | help
>Number: 25826
>Category: bin
>Synopsis: nfsd -t -h adr1 -h adr2 doesn't work
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Thu Mar 15 05:30:01 PST 2001
>Closed-Date:
>Last-Modified:
>Originator: Jean-Luc Richier
>Release: FreeBSD 4.2-RELEASE i386
>Organization:
LSR-IMAG
>Environment:
Any FreeBSD >=4.2 (I checked stable and current on March 14, 2001)
>Description:
On a nfs server with >=2 IP addresses, nfsd server being
configured with -h options, as recommended in man nfsd,
mount_nfs accesses over TCP will fail.
The problem is the same when using -a option.
>How-To-Repeat:
Consider a nfs server with two IP addresses, xx and yy.
According to the nsfd man, set the following nfsd options in rc.conf:
nfs_server_enable="YES"
nfs_server_flags="-u -t -n 4 -h xx -h yy
and export some catalog (e.g. /usr/local)
On an other machine, try to mount xx:/usr/local over TCP. The mount
command will fail:
% mount -o -T xx:/usr/local /mnt
nfs server xx:/usr/local: not responding
Also nfsd on xx is now burning CPU:
# ps -ax | grep nfsd
545 ?? Rs 0:23.78 nfsd: master (nfsd)
546 ?? I 0:00.00 nfsd: server (nfsd)
547 ?? I 0:00.00 nfsd: server (nfsd)
548 ?? I 0:00.00 nfsd: server (nfsd)
549 ?? I 0:00.00 nfsd: server (nfsd)
# sleep 4
# ps -ax | grep nfsd
545 ?? Rs 0:27.81 nfsd: master (nfsd)
546 ?? I 0:00.00 nfsd: server (nfsd)
547 ?? I 0:00.00 nfsd: server (nfsd)
548 ?? I 0:00.00 nfsd: server (nfsd)
549 ?? I 0:00.00 nfsd: server (nfsd)
Notes:
Mounting using UDP works.
TCP mount of yy:/usr/local (the last address in the -h list) works
but only if no TCP mount has failed:
% mount -o -Ti yy:/usr/local /mnt
.... mount OK
% umount /mnt
% mount -o -Ti xx:/usr/local /mnt
nfs server xx:/usr/local: not responding
^C
% mount -o -Ti yy:/usr/local /mnt
... no response
>Fix:
>Release-Note:
>Audit-Trail:
>Unformatted:
>Analysis:
The problem is in the nfsd.c code:
with -h option, multiple tcp sockets are created, one for each address
470 /* Now set up the master server socket waiting for tcp connections. */
471 on = 1;
472 FD_ZERO(&sockbits);
473 connect_type_cnt = 0;
474 for (i = 0; tcpflag && i < bindhostc; i++) {
475 if ((tcpsock = socket(AF_INET, SOCK_STREAM, 0)) < 0) {
...
482 setbindhost(&inetaddr, bindhost[i]);
483 if (bind(tcpsock,
484 (struct sockaddr *)&inetaddr, sizeof (inetaddr)) < 0) {
...
488 if (listen(tcpsock, 5) < 0) {
489 syslog(LOG_ERR, "listen failed");
490 exit(1);
491 }
.....
500 }
However in the main loop which accept the tcp connections, the code
tests only one socket (the last one kept in tcpsock):
581 /*
582 * Loop forever accepting connections and passing the sockets
583 * into the kernel for the mounts.
584 */
585 for (;;) {
586 ready = sockbits;
587 if (connect_type_cnt > 1) {
588 if (select(maxsock + 1,
....
593 }
594 if (tcpflag && FD_ISSET(tcpsock, &ready)) {
595 len = sizeof(inetpeer);
596 if ((msgsock = accept(tcpsock,
597 (struct sockaddr *)&inetpeer, &len)) < 0) {
598 syslog(LOG_ERR, "accept failed: %m");
599 exit(1);
There is no loop for the multiple possible values of tcpsock
Fix:
There is different possible fixes
1/ the simplest fix does not consider the tp4 tpip code (which is
not used, being between #ifdef notyet directives.)
Simply loop on all possible values of tpcsock:
*** nfsd.c.DIST Sun Sep 17 00:52:23 2000
--- nfsd.c Thu Mar 15 13:58:58 2001
***************
*** 591,597 ****
exit(1);
}
}
! if (tcpflag && FD_ISSET(tcpsock, &ready)) {
len = sizeof(inetpeer);
if ((msgsock = accept(tcpsock,
(struct sockaddr *)&inetpeer, &len)) < 0) {
--- 591,602 ----
exit(1);
}
}
! for (tcpsock = 0; tcpsock <= maxsock; tcpsock++) {
! if (tcpflag && FD_ISSET(tcpsock, &ready)) {
! #ifdef notyet
! XXX - should test that tcpsock is indeed a tcp socket
! and not the tp4 socket or a tpip socket
! #endif /* notyet */
len = sizeof(inetpeer);
if ((msgsock = accept(tcpsock,
(struct sockaddr *)&inetpeer, &len)) < 0) {
***************
*** 608,613 ****
--- 613,619 ----
nfsdargs.namelen = sizeof(inetpeer);
nfssvc(NFSSVC_ADDSOCK, &nfsdargs);
(void)close(msgsock);
+ }
}
#ifdef notyet
if (tp4flag && FD_ISSET(tp4sock, &ready)) {
2/ If we want to correct the tpip/tp4 code, one has to add 2 fd_set:
fd_set tcpbits, tpipbits;
to set the bits in the fd_set in parallel to the bits in sockbits,
and to use them to test is a selected socket is a tcp socket or a tpip
socket (there is only one tp4socket, so the old code wille work);
3/ And other solution is to have only one tcp socket:
The -h optionhas 2 uses:
- One is to avoid and udp specific problem - cf the man:
If nfsd is to be run on a host with multiple interfaces or interface
aliases, use of the -h option is recommended. If you do not use the op-
tion NFS may not respond to UDP packets from the same IP address they
were sent to.
- The other is to control nfs access - cf the man:
Use of this option is also recommended when securing NFS
exports on a firewalling machine such that the NFS sockets can only be
accessed by the inside interface. Ipfw would then be used to block nfs-
related packets that come in on the outside interface.
But filtering may be succifient.
Therefore multiple tcp sockets are much less useful than udp ones.
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-bugs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200103151317.f2FDHLo00698>