Date: Tue, 1 Dec 2015 19:02:42 -0500 (EST) From: Rick Macklem <rmacklem@uoguelph.ca> To: Slawa Olhovchenkov <slw@zxy.spb.ru> Cc: hackers@freebsd.org Subject: Re: NFSv4 details and documentations Message-ID: <2030822608.114589048.1449014562660.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <20151201232450.GI31314@zxy.spb.ru> References: <9BC3EFA2-945F-4C86-89F6-778873B58469@cs.huji.ac.il> <20151130165940.GB31314@zxy.spb.ru> <183609075.112643195.1448924896262.JavaMail.zimbra@uoguelph.ca> <20151201074425.GD31314@zxy.spb.ru> <1745794347.113212991.1448977306722.JavaMail.zimbra@uoguelph.ca> <20151201135315.GH31314@zxy.spb.ru> <1491578361.114386779.1449006984539.JavaMail.zimbra@uoguelph.ca> <20151201232450.GI31314@zxy.spb.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
Slawa Olhovchenkov wrote:
> On Tue, Dec 01, 2015 at 04:56:24PM -0500, Rick Macklem wrote:
>
> > Slawa Olhivchenkov wrote:
> > > On Tue, Dec 01, 2015 at 08:41:46AM -0500, Rick Macklem wrote:
> > >
> > > > > > (Note that "host" here implies that the principal for the
> > > > > > host-based
> > > > > > credential is
> > > > > > "host@<client-host>.<domain>". --> What is after the "=" above is
> > > > > > what
> > > > > > is
> > > > > > before the
> > > > > > "@" in the host based principal name.)
> > > > > > Then system operations are done as nobody, but users are done as
> > > > > > that
> > > > > > user
> > > > > > (they need
> > > > >
> > > > > This is strange. I am mount (by automount) as:
> > > > >
> > > > > /NFS -nfsv4,intr,soft,sec=krb5i,gssname=host storage01:/
> > > > >
> > > > I'd recommend that you never use "intr" or "soft" on NFSv4 mounts.
> > > > (It's somewhere in a man page and basically if you use these and an
> > > > RPC that does locking times out, you break the locking horribly.)
> > >
> > > W/o "intr" and "soft" I can got staled mount and process (till
> > > reboot). This is prodaction servers and this is unacceptable. Correct
> > > locking for me least important, as last resort I am do `umount -f`
> > >
> > When I was a sysadmin, I just made sure my NFS server stayed up and the
> > network
> > fabric was stable. I never recall having a problem using "hard" mounts.
> >
> > But I agree it is a tradeoff. If you don't need file locking visible across
> > multiple clients, use nfsv3,nolockd and intr if it works for you.
>
> Yes, my expirense (with soft, intr) is from nfsv3.
> But (for me) nfsv4 benefits by requrimentsless identical numeric UID
> accross all systems and support file attributes.
> And, perhaps, limitless (by file size), better mapping nobody (nfsv3
> on zfs mapping nobody to 4294967294)
>
NFSv3 is stateless, which means each RPC is independent and the server
doesn't need to "remember anything" about previous RPCs. (A repeat of the
same RPC can result in undesired outcomes, do the DRC was invented to try
and avoid the same RPC being done multiple times on a server, but that was
an implementation issue and not a protocol requirement.)
NFSv4 is stateful. It keeps track of Opens (a form of Windows lock done when
a file is opened) and byte range locks. It requires strick sequencing of these
operations. If you have an RPC fail due to timeout, this sequencing is broken
and the results will be nasty errors from the server like NFS4ERR_BAD_SEQID.
Since the client has no way to fix a bad seqid, it needs to be avoided.
Since every file open is a form of "lock", the problem still exists even
if the client never does byte range locks.
--> Using "intr" or "soft" will break the mount sooner or later and all that
can be done is a umount/mount to fix it. (That's why I recommend "umount -f"
if you have to.)
> > If you need file locking to work and be visible across multiple clients,
> > then
> > the tradeoff is a hung system if your NFS server fails or your network
> > fabric
> > is flakey. (Do you want file locking to work, but only when things are
> > running
> > smoothly?)
>
> I am don't need locking specialy, only indirect (for example: svn and
> etc). I am plan to use it as may (and other people) roaming $HOME.
>
> > You can use "umount -f" but I will admit you can get stuck. If you do a
> > "umount"
> > without "-f" the mount point gets locked and "umount -f" doesn't work.
>
> currenly (w/o using file locks) "umount -f" work on busy (but not
> stucked) file system.
>
> > I, for one, do not believe that NFSv4 is a replacement for NFSv3. It is a
> > very
> > different protocol (some say "NFS in name only") and fills a different
> > somewhat
> > overlapping solution space.
>
> What you mean "replacement"? "drop-in but better"?
> Or better compliance with may requirements?
>
Some would see NFSv4 is the next version of NFS, just like FreeBSD-10 is
the next version of FreeBSD after FreeBSD-9.
I see NFSv3 vs NFSv4 as more a "UFS vs ZFS" situation. Both UFS and ZFS have
their advantages and disadvantages and choosing one vs the other is a tradeoff,
although either can be used for local FreeBSD file storage.
So, I guess I'd say "Or better compliance with my requirements".
> > > > Getting a home directory to work is harder than it should be and I
> > > > don't even know how to make it work, because I haven't done it.
> > > > The login must do a "kinit" so the user has access to the volume
> > > > and I don't know how to set FreeBSD up to do the kinit as a part of
> > > > the login. It also must be done early enough in the login, so that
> > > > it happens before any access to the home dir is attempted.
> > > > (To be honest, unless there is a way to do this in FreeBSD, you
> > > > can forget about Kerberized NFS mounts for home dirs.)
> > >
> > > First access to home directory do as root, not as user.
> > That's going to be a problem. I'm not sure how that will work unless
> > the access as root can be done by "nobody". Normally it is assumed
> > that a user accesses files on the NFS mount as that user.
> > (ie. This just might not be doable unless you get rid of the
> > access as root during the login.)
> >
> > > After root access ticket created in /tmp/krb5cc_UID and home
> > > succesuful accesed.
> > >
> > > > I would start by testing a mount that isn't a home directory, so you
> > > > can log into the machine (home dir not Kerberized NFS mounted) and
> > > > then the user can "kinit" and them "cd /kerberized/mount" and see
> > > > if it works.
> > > > --> Once that works, I don't know how to do the rest.
> > > > (I'm an NFS guy, not a Kerberos one.;-)
> > > >
> > > > Also, I don't know what effect having sshd etc running as root will
> > > > be, since they will then be seen as running by "nobody" on the server.
> > >
> > > As last resort I can export with -maproot=root.
> > >
> > I don't think this works as you'd expect. The server doesn't see "root"
> > as the principal for the host-based credential in the keytab file.
>
> > To be seen as "root" on the server:
> > - I think you need a root@YOUR.REALM user principal in your KDC.
>
> No.
>
> > - The client must "kinit" this to get a TGT in /tmp/krb5cc_0.
>
> I can do by using host principal. Kerberos only accpet ticket by check
> correctly decrypting, i.e. I can got TGT for host/`hostname` and
> because do it from root account place it in /tmp/krb5cc_0.
>
Yes, but the server will see the "principal name" and not a uid of 0.
It will then choose to map that "principal name" to the uid for "nobody".
(Essentially the gssd on the NFS server will get the principal name
"host/'hostname'". Then it will getpwnam() for that, which will fail
and it will assign the uid for "nobody".)
> > - You can create a keytab entry for root@YOUR.REALM and "kinit -k"
> > it, I think?
>
> No, host/`hostname` enough.
>
This will get you a credential, but that credential won't be "root" on the
server as far as I know.
The main use in NFSv4 is for the protocol's "administrative operations" for
things like renewing lock leases. The NFSv4 server simply requires that the
principal name is the same one used when the first lock state was acquired.
(ie. The NFSv4 server uses the principal name like "host" and doesn't turn
it into "root". It does turn it into the uid for "nobody" for a credential
when used to access files, etc. It really isn't intended to be used to
access files etc, but the code does fall back to "nobody" so it works for
"world access".)
If "nobody" is sufficient, then it will work.
> > --> Most would consider putting a keytab entry in /etc/krb5.keytab for
> > "root"
> > too large a security risk, but maybe it's ok for your environment.
> > (Someone could move the keytab entry to another machine and it would
> > still
> > work.)
> > (You mentioned doing something like this in your crontab.)
>
> "kinit -k host/`hostname`" as root create TGT /tmp/krb5cc_0 obtained
> for host/`hostname` principal. Now /tmp/krb5cc_0 may be used gssd for
> using where root (system daemons) try access some files.
> I see this mapped to nobody on server side, but this is acceptably for
> may currently use -- checking sshd .k5login and other files in $HOME.
>
> > Btw, you can use Kerberos with NFSv3. It works about the same and doen't
> > require a host-based credential in /etc/krb5.keytab. It does require that
> > the login do "kinit" before attempting to access the home dir.
>
> I don't see why NFSv3 must be distinct from NFSv4.
> I am also need some ticket for root for accessing files from sshd as
> root, right?
>
> > > > > I am avoid this by "kinit -k host/`hostname`" in crontab and startup
> > > > > script, but may be gssd is best for this functionality?
> > > > >
> > > > Shouldn't matter. "gssd -h" does exactly the same stuff as "kinit -k".
> > > > (I wrote the code essentially cloning what "kinit -k" did.)
> > >
> > > For mount only, not for root access from sshd, as I see.
> > >
> > Yes, for Kerberos root access, I think you will need "root@YOUR.REALM" in
> > your KDC and need to kinit for that.
>
> For root access as root on server side.
> _______________________________________________
> freebsd-hackers@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2030822608.114589048.1449014562660.JavaMail.zimbra>