Date: Mon, 19 Dec 2022 08:59:48 -0800 From: Rick Macklem <rick.macklem@gmail.com> To: kib@freebsd.org, James Gritton <jamie@freebsd.org>, "Bjoern A. Zeeb" <bz@freebsd.org> Cc: freebsd-current@freebsd.org Subject: Re: RFC: nfsd in a vnet jail Message-ID: <CAM5tNy73kfEToEEbT9Mz0VAZhNLxoFwEo1FMvoyR1Sg5Xaiv8g@mail.gmail.com> In-Reply-To: <CAM5tNy63yE%2BL0rjfdYSs_WVwh3_gi8fmRVNiTu9BEKzNj_iYgA@mail.gmail.com> References: <CAM5tNy7CQaBTRWG0m0aN6T0xG2L2zSQJGa%2BatGaH%2BmW%2BwEpdyQ@mail.gmail.com> <1955021.aDjkhKmpDe@ravel> <CAM5tNy5a9GYjJcjXLQvsjF77Gsu6yej5XR=mMTAuVKWxoNfR1A@mail.gmail.com> <8351812.Gc231LQI4k@ravel> <CAM5tNy63yE%2BL0rjfdYSs_WVwh3_gi8fmRVNiTu9BEKzNj_iYgA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000e96d9105f0313fba
Content-Type: text/plain; charset="UTF-8"
Hi,
Kostik expressed some concern w.r.t. using a non-default VNET_NFSD kernel
build option and I understand his concern, given that many prefer to use
a GENERIC kernel and binary updates.
Right now there are 29 NFS variables VNET_DEFINED() and several of them
are arrays currently sized at 500. One of the reasons for the non-default
VNET_NFSD kernel option was the bloat this caused to the vnet.
(Chris <bsd-lists@bsdforge.com> expressed concern that adding mountd/nfsd
to the vnet would result in bloat/overhead in a previous post to this
thread.)
Another issue with putting all these variables in the vnet is that the
nfsd.ko
cannot be loaded (it complains the vnet is out of space) and, as such,
options NFSD must be used with VNET_NFSD so that the nfsd is linked into the
kernel.
As such, I am wondering what others think of this alternate plan?
- Pull all the VNET_DEFINE()'d variable (except the 3 manipulated by
sysctls)
into a structure.
- Define a single VNET_DEFINE()'d variable that is a pointer to that
structure
and then malloc() the structure in the function called by VNET_SYSINIT().
This would result in a malloc'd structure for each vnet jail (for kernels
built with VMIMAGE), but would only add 4 variables to the vnet.
If a small C file that only consists of the VNET_DEFINE()s for the 4
variables
is linked into the kernel whenever the VIMAGE option is specified, then I
think nfsd.ko would be loadable.
Unfortunately, this does not deal with vnet'ng the kgssapi, rpcsec_gss for
Kerberized mounts or vnet'ng NFS-over-TLS, but those could be handled in a
similar manner, I think?
So, what do others think of this alternate plan?
rick
ps: Every use of the vnet'd variables is currently wrapped in a macro called
NFSD_VNET(), so the change is pretty easy to do by just re-writing this
macro.
--000000000000e96d9105f0313fba
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-family:monospac=
e">Hi,</div><div class=3D"gmail_default" style=3D"font-family:monospace"><b=
r></div><div class=3D"gmail_default" style=3D"font-family:monospace">Kostik=
expressed some concern w.r.t. using a non-default VNET_NFSD kernel</div><d=
iv class=3D"gmail_default" style=3D"font-family:monospace">build option and=
I understand his concern, given that many prefer to use</div><div class=3D=
"gmail_default" style=3D"font-family:monospace">a GENERIC kernel and binary=
updates.</div><div class=3D"gmail_default" style=3D"font-family:monospace"=
><br></div><div class=3D"gmail_default" style=3D"font-family:monospace">Rig=
ht now there are 29 NFS variables VNET_DEFINED() and several of them</div><=
div class=3D"gmail_default" style=3D"font-family:monospace">are arrays curr=
ently sized at 500. One of the reasons for the non-default</div><div class=
=3D"gmail_default" style=3D"font-family:monospace">VNET_NFSD kernel option =
was the bloat this caused to the vnet.</div><div class=3D"gmail_default" st=
yle=3D"font-family:monospace">(Chris <<a href=3D"mailto:bsd-lists@bsdfor=
ge.com">bsd-lists@bsdforge.com</a>> expressed concern that adding mountd=
/nfsd</div><div class=3D"gmail_default" style=3D"font-family:monospace">=C2=
=A0to the vnet would result in bloat/overhead in a previous post to this</d=
iv><div class=3D"gmail_default" style=3D"font-family:monospace">=C2=A0threa=
d.)</div><div class=3D"gmail_default" style=3D"font-family:monospace"><br><=
/div><div class=3D"gmail_default" style=3D"font-family:monospace">Another i=
ssue with putting all these variables in the vnet is that the nfsd.ko</div>=
<div class=3D"gmail_default" style=3D"font-family:monospace">cannot be load=
ed (it complains the vnet is out of space) and, as such,</div><div class=3D=
"gmail_default" style=3D"font-family:monospace">options NFSD must be used w=
ith VNET_NFSD so that the nfsd is linked into the</div><div class=3D"gmail_=
default" style=3D"font-family:monospace">kernel.</div><div class=3D"gmail_d=
efault" style=3D"font-family:monospace"><br></div><div class=3D"gmail_defau=
lt" style=3D"font-family:monospace">As such, I am wondering what others thi=
nk of this alternate plan?</div><div class=3D"gmail_default" style=3D"font-=
family:monospace">- Pull all the VNET_DEFINE()'d variable (except the 3=
manipulated by sysctls)</div><div class=3D"gmail_default" style=3D"font-fa=
mily:monospace">=C2=A0 into a structure.</div><div class=3D"gmail_default" =
style=3D"font-family:monospace">- Define a single VNET_DEFINE()'d varia=
ble that is a pointer to that structure</div><div class=3D"gmail_default" s=
tyle=3D"font-family:monospace">=C2=A0 and then malloc() the structure in th=
e function called by VNET_SYSINIT().</div><div class=3D"gmail_default" styl=
e=3D"font-family:monospace">This would result in a malloc'd structure f=
or each vnet jail (for kernels</div><div class=3D"gmail_default" style=3D"f=
ont-family:monospace">built with VMIMAGE), but would only add 4 variables t=
o the vnet.</div><div class=3D"gmail_default" style=3D"font-family:monospac=
e"><br></div><div class=3D"gmail_default" style=3D"font-family:monospace">I=
f a small C file that only consists of the VNET_DEFINE()s for the 4 variabl=
es</div><div class=3D"gmail_default" style=3D"font-family:monospace">is lin=
ked into the kernel whenever the VIMAGE option is specified, then I</div><d=
iv class=3D"gmail_default" style=3D"font-family:monospace">think nfsd.ko wo=
uld be loadable.</div><div class=3D"gmail_default" style=3D"font-family:mon=
ospace"><br></div><div class=3D"gmail_default" style=3D"font-family:monospa=
ce">Unfortunately, this does not deal with vnet'ng the kgssapi, rpcsec_=
gss for</div><div class=3D"gmail_default" style=3D"font-family:monospace">K=
erberized mounts or vnet'ng NFS-over-TLS, but those could be handled in=
a</div><div class=3D"gmail_default" style=3D"font-family:monospace">simila=
r manner, I think?</div><div class=3D"gmail_default" style=3D"font-family:m=
onospace"><br></div><div class=3D"gmail_default" style=3D"font-family:monos=
pace">So, what do others think of this alternate plan?</div><div class=3D"g=
mail_default" style=3D"font-family:monospace"><br></div><div class=3D"gmail=
_default" style=3D"font-family:monospace">rick</div><div class=3D"gmail_def=
ault" style=3D"font-family:monospace">ps: Every use of the vnet'd varia=
bles is currently wrapped in a macro called</div><div class=3D"gmail_defaul=
t" style=3D"font-family:monospace">=C2=A0 =C2=A0 NFSD_VNET(), so the change=
is pretty easy to do by just re-writing this macro.</div><div class=3D"gma=
il_default" style=3D"font-family:monospace"><br></div></div>
--000000000000e96d9105f0313fba--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAM5tNy73kfEToEEbT9Mz0VAZhNLxoFwEo1FMvoyR1Sg5Xaiv8g>