Date: Sun, 27 Nov 2022 18:11:00 -0800 From: Julian Elischer <julian@freebsd.org> To: Rick Macklem <rick.macklem@gmail.com> Cc: freebsd-current@freebsd.org, "Bjoern A. Zeeb" <bz@freebsd.org>, Marko Zec <zec@fer.hr>, James Gritton <jamie@freebsd.org> Subject: Re: RFC: nfsd in a vnet jail Message-ID: <0798d1c7-96c6-1bd8-fdea-aa456b9ce194@freebsd.org> In-Reply-To: <5244s3o-p9q6-qp97-2623-onso786os643@serrofq.bet> References: <CAM5tNy7CQaBTRWG0m0aN6T0xG2L2zSQJGa%2BatGaH%2BmW%2BwEpdyQ@mail.gmail.com> <d565689e2e61b48bb208cd4ea9f5e392@freebsd.org> <5244s3o-p9q6-qp97-2623-onso786os643@serrofq.bet>
next in thread | previous in thread | raw e-mail | index | archive | help
On 11/27/22 11:13 AM, Bjoern A. Zeeb wrote: > On Sun, 27 Nov 2022, James Gritton wrote: > >> On 2022-11-25 15:17, Rick Macklem wrote: >> >>> Hi, >>> >>> bz@ has encouraged me to fiddle with the nfsd >>> so that it works in a vnet jail. >>> I have now basically done so, specifically for >>> NFSv4, since NFSv3 presents various issues. >>> >>> What I have not yet done is put global variables >>> in the vnet. This needs to be done so that the nfsd >>> can be run in multiple jail instances and/or in and >>> outside of a jail. >>> The problem is that there are 100s of global variables. >>> >>> I can see two approaches: >>> 1 - Move them all into the vnet jail. This would imply >>> that all the sysctls need to somehow be changed, >>> which would seem to be a POLA violation. > > Not a POLA. The sysctl (names) don't change. Just the values are > duplicated per-jail. > > > As Marko and I (mostly Marko) were assigning different variable and sysctls when Vnet first hit, it was generally pretty obvious which were local and which were global. The sysctls values need to mean the same thing in the jail as they do in an unjailed system, so the sysctl names don't change, just the value reported changes. Some systls become read-only or invisible in jails. Sometimes this takes adding some boilerplate virtualization code to each sysctl howeve there is already some inbuilt support in the SYSCTL framework for VNET isolation. You just need to set the CTLFLAG_VNET bit in the sysctl definition. I agree with what Bjorn says below. There is a slight added complication in that it is not just vnet but jailing as well that needs to be take into account because vnet doesn't affect VFS, but jails do. > >>> It also implies a lot of stuff in the vnet. >>> 2 - Just move the global variables that will always >>> differ from one nfsd to another (this would make >>> the sysctls global and apply to all nfsds). >>> This will keep the number of globals in the vnet >>> smaller. >>> >>> I am currently leaning towards #2, put what do others >>> think? >>> >>> rick >>> ps: Personally, I don't know what use there is of >>> running the nfsd inside a vnet jail, but bz@ has >>> some use case. >> >> I would prefer closer to #2, unless you want to support only one >> jail running nfsd (which is admittedly one of the more likely >> scenarios). I imagine it's a case-by-case judgement call, as to >> whether a particular knob should be global or per-jail. > > > I think the call is: everything that if changed in a vnet jail that > could cause the entire system to be DoSed by changing the setting in > the > jail defintitvely stays global. > > Everything which needs to be writeable on a per-instance base probably > needs to be virtualised. > > My main concern with virtualising the variables will be early boot and > and NFSROOT szenarios that will need access to them early on before the > virtual network stacks are properly initialized; I can help sorting > that out if my concerns become real. Most probably was sorted before > for NFSROOT with the IP stack so it's likely an okay job now. > > > Also given I have once done it before for another subsystem; we could > think of a V_FS bit (and I write it that way to not confuse it with > VFS). Now NFS sits somewhere between FS and NET so I am not surt where > it should belong should we ponder that route as yet another option (and > if we think that VNETs will be way too big one day? -- there's probably > a lot other fish still to fry though some of that has been burnt in the > past already). > > I think I have another email or two on the subject (possibly > privately); > sorry Rick that I haven't gotten to them more timely. I'll have a look > later tonight. > > /bz >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?0798d1c7-96c6-1bd8-fdea-aa456b9ce194>