Date: Thu, 2 Jun 2022 23:25:16 +0300 From: Konstantin Belousov <kostikbel@gmail.com> To: Rick Macklem <rmacklem@uoguelph.ca> Cc: John-Mark Gurney <jmg@funkthat.com>, "freebsd-net@FreeBSD.org" <freebsd-net@freebsd.org> Subject: Re: make NFSv3 default now on diskless Message-ID: <YpkcrDPouuCySIEf@kib.kiev.ua> In-Reply-To: <YQBPR0101MB9742FCBB321B58DFC96CB3A3DDDE9@YQBPR0101MB9742.CANPRD01.PROD.OUTLOOK.COM> References: <20220602032456.GY88842@funkthat.com> <YQBPR0101MB97426750C2E6717B06983AC7DDDE9@YQBPR0101MB9742.CANPRD01.PROD.OUTLOOK.COM> <20220602183522.GZ88842@funkthat.com> <YQBPR0101MB9742FCBB321B58DFC96CB3A3DDDE9@YQBPR0101MB9742.CANPRD01.PROD.OUTLOOK.COM>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Jun 02, 2022 at 08:03:05PM +0000, Rick Macklem wrote:
> John-Mark Gurney <jmg@funkthat.com> wrote:
> > Rick Macklem wrote this message on Thu, Jun 02, 2022 at 14:44 +0000:
> > > John-Mark Gurney <jmg@funkthat.com> wrote:
> > > > I just booted FreeBSD-current diskless, using NFS root, and I ended
> > > > up having issues because by default, NFS root is only v2.
> > > >
> > > > One of things that happened was disk space available was listed as
> > > > -138G, or -144830429K. I assume this is because the server is reporting
> > > > TBs instead.
> > > Yes. NFSv2 uses 32bit sizes.
> >
> > Should we make the NFS server clamp various sizes then? instead of reporting
> > negative numbers? (Sorry if this is already done on newer versions of
> > FreeBSD, my server is a bit old.)
> My recollection (from long ago) is that, although the RFC defined them as 32bit signed,
> some implementations choose to mid-interpret them as unsigned, so that 4G was supported
> instead of 2G.
>
> Having said that, I agree with you that you should just use NFSv3 now. (Many servers are
> dropping NFSv2 support entirely.)
>
> > > > If I mount via mount_nfs, the sizes are normal/correct because it mounts
> > > > v3.
> > > I believe most specify "nfsv3" in the "/" mount line of /etc/fstab on the
> > > remote root fs. Then, when the system does a "mount -u" to make it
> > > read/write it gets toggled to NFSv3.
> >
> > well, I tried doing a:
> > mount -u -o nfsv3 /
> It used to work when it was in /etc/fstab, when going from read-only to read-write,
> from what I recall.
>
> > on the system and this didn't work switch over to v3, it was still on
> > v2..
> >
> > I haven't specified nfsv3 in /etc/fstab, but IMO, this should be the
> > default..
> Since NFSv3 is the default, it might not need to be explicit. I can't recall.
>
> >
> > Also, I'm right now booting single user mode, because I'm -mapall to
> > a user, and lots of FreeBSD breaks when files aren't uid 0, and there
> > doesn't appear to be a way to remap the uid to root (that I have found)..
> Yep. It would take effect when going multi-user.
>
> "-mapall=root", although that is not a recommended security setting.
> Why don't you just allow the client to use whatever uid it would normally
> use instead of "-mapall"?
>
> > > > The other issue that I ran into is that NFSv2 can't access >4GB files
> > > > (or create them).
> > > As above, NFSv2 uses 32bit sizes.
> > >
> > > > Anyone object to adding BOOTP_NFSV3 to GENERIC?
> > > Well, that option only works when used with BOOTP_NFSROOT.
> > > The GENERIC configs for amd64, arm64,... use the other way.
> > > (Just to make it confusing, there are two different ways an NFS root
> > > fs is set up.)
> > > See below.
> > >
> > > > Or maybe making it a
> > > > tunable that defaults to set, because it seems a bit crazy to default
> > > > to v2 these days.
> > > I don't think changing the default to NFSv3 will be a problem.
> > > The reason it was NFSv2 was that,
> > > for some non-FreeBSD NFS servers, the NFSv3 file handle is different
> > > than the NFSv2 one.
> > >
> > > I added NFSv3 support to stand/libsa/nfs.c about 15years ago, so every
> > > system should be running the newer NFS code in the loader and be able
> > > to do NFSv3 booting.
> > >
> > > > This option was added in 432aad0e in 1997 so that the nfs_diskless
> > > > structure didn't need to be filled out. Does anything even
> > > > populate/fill it out anymore? I saw code in i386/i386/locore.s that
> > > > does this, but it doesn't appear anywhere else.
> > > Yes. For "options NFS_ROOT" (the other way), the loader uses
> > > "stand/libsa/nfs.c" to acquire the remote file system's root file handle
> > > and fills it in. (See nfs_setup_diskless() in sys/nfs/nfs_diskless.c.)
> > > Looking at it, it appears to enable NFSv3 so long as it finds
> > > "boot.nfsroot.nfshandlelen" set.
> >
> > Well, I'm not seeing that, and this system is booting via
> > pxeboot+loader, so maybe something is broken?
> If it's an up to date (within last 10+ years then, yes, it sounds like it
> might be broken. However, I'm not a loader guy...
>
> You can set it manually at the loader prompt to "28" and then see if it
> boots are is running NFSv3.
>
> I have no way of testing/debugging this. Maybe you can figure out why the
> loader isn't setting "boot.nfsroot.nfshandlelen" to 28?
I have did several (> 5) lab setups with NFS booting, and it was always
like you described:
kernel config:
options NFS_ROOT
/boot/loader.conf
boot.nfsroot.options="nolockd"
nfscl_load="YES"
(yes, my nfscl.ko is a module)
/etc/fstab (an example)
192.168.130.1:/usr/home/netboot/zoo/r-freeb44 / nfs rw,nolockd 0 0
and in dhcpd.conf
host r-freeb44.zoo {
hardware ethernet 28:80:23:a2:37:bc;
fixed-address r-freeb44.zoo;
next-server rott.zoo;
filename "pxeboot";
# filename "loader.efi";
option root-path "192.168.130.1:/usr/home/netboot/zoo/r-freeb44";
}
You need pxeboot (or loader.efi) on the tftp server root (rott.zoo in my
case).
Above demonstrates the loading over the legacy BIOS/PXE. Recomment pxeboot
to loader.efi if you have something not so old so it can pxeboot using UEFI.
Result:
root@r-freeb44:~ # nfsstat -m
192.168.130.1:/usr/home/netboot/zoo/r-freeb44 on /
nfsv3,tcp,resvport,nconnect=1,hard,cto,nolockd,sec=sys,acdirmin=3,acdirmax=60,acregmin=5,acregmax=60,nametimeo=60,negnametimeo=60,rsize=32768,wsize=32768,readdirsize=32768,readahead=1,wcommitsize=16777216,timeout=120,retrans=2
>
> > > > There also appears to possibly be a way via mount options, but I can't
> > > > see where it's documented to set them.
> > > I think you just specify "nfsv3" as a mount option in the root fs
> > > line in /etc/fstab on the root fs on the NFS server.
> >
> > See above, this doesn't appear to work, or doesn't work the way I think
> > it should...
> Well, maybe I just don't recall correctly.
>
> >
> > > I don't think changing the default to NFSv3 will be a problem.
> > > The hassle is testing the various cases, to make sure nothing
> > > breaks. I have no diskless setup to do testing and I don't even know
> > > when installs/upgrades actually replace the loader?
> >
> > Well, this diskless was easier to setup than I expected, partly
> > because I already had most of the infrastructure together (from
> > netbooting another machine). Put pxeboot on a tftp server, configure
> > the dhcp server to send the correct options, extract base.txz to a
> > directory, export it, and it worked. I assume that I'm getting loader
> > from that install since I don't specify it in the dhcp server.
> I didn't think pxeboot used dhcp and I don't know how it figures out
> where to find a file for booting?
>
> > As for testing, we have the CI system for that, right? ;p
> >
> > /me needs to get back to work on the lab.
> >
> > I guess we'd need to list the configurations that we care about, the
> > only ones I can think of, off the top of my head are pxeboot (which I'm
> > testing now), and u-boot..
> > For servers, are there any servers that
> > are NFSv2 only that are in common use today?
> You'd have to look pretty hard. FreeBSD had NFSv3 from day 1,
> since it was in the CSRG stuff.
> > If you're running an
> > ancient server that is NFSv2 only, I think you deserve to have to
> > rebuild kernels or something instead of making 99% of the rest of us
> > do it..
> If you are running a NFSv2 only server, you are running a computer museum.
> (As noted, many servers are dropping NFSv2 support entirely.)
>
> rick
>
>
> --
> John-Mark Gurney Voice: +1 415 225 5579
>
> "All that I will do, has been done, All that I have, has not."
>
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?YpkcrDPouuCySIEf>