Date: Thu, 1 Dec 2022 17:32:26 -0800 From: Rick Macklem <rick.macklem@gmail.com> To: Chris <bsd-lists@bsdforge.com> Cc: Peter Eriksson <pen@lysator.liu.se>, FreeBSD CURRENT <freebsd-current@freebsd.org>, "Bjoern A. Zeeb" <bz@freebsd.org>, Alan Somers <asomers@freebsd.org> Subject: Re: RFC: nfsd in a vnet jail Message-ID: <CAM5tNy7Uc82OqhjVc3W84GAYcJd6P4D4U_criEYTwFmvEU61xg@mail.gmail.com> In-Reply-To: <2980bcbd22f884962d358808f9440d77@bsdforge.com> References: <CAM5tNy7CQaBTRWG0m0aN6T0xG2L2zSQJGa%2BatGaH%2BmW%2BwEpdyQ@mail.gmail.com> <CAOtMX2hxeeNMxxdpma8NJ7ms60eRfuCWoFi7FixdSe83=qibkA@mail.gmail.com> <82103A1E-9D39-47B0-9520-205583C8B680@lysator.liu.se> <CAM5tNy71UAOkCQb9upc_OxhM-y5rp9jMKbKTJr619JFCGsfRkg@mail.gmail.com> <2980bcbd22f884962d358808f9440d77@bsdforge.com>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] On Thu, Dec 1, 2022 at 8:23 AM Chris <bsd-lists@bsdforge.com> wrote: > On 2022-11-29 16:21, Rick Macklem wrote: > > On Sun, Nov 27, 2022 at 10:04 AM Peter Eriksson <pen@lysator.liu.se> > wrote: > > > >> Keep the global variables as defaults that apply to all nfsds and allow > >> (at least some subset) to be overridden inside the net jails if some > things > >> need to be changed from the defaults? > >> > >> This is pretty much a reply to one of the posts selected at random, > > but I thought that better than starting a new email thread. > > > > bz@ and asomers@ have both asked about running mountd within a vnet > prison > > (one via offlist email and the other on phabricator). > > > > I think it is worth discussing here... > > mountd (rightly or wrongly) does two distinctly different things: > > 1 - It pushes the exports into the kernel via nmount() so they > > can be hung off of the "struct mount" for a file system's > > mount point. > > --> This can only work for file system mount points and can > > only be done once for any given file system mount point. > > > > At this time, I have it done once globally outside of the prisons. > > The alternative I can see is doing it within each prison, but I > > think that would require that each prison have its own file > system(s). > > (ie. The prison's root would always be a file system mount point.) > > > > 2 - It handles RPC Mount protocol requests from NFSv3 clients. This one > > is NFSv3 specific, which is why I have done this NFSv4 only at > > this time. To do this, it must be able to register with rpcbind, > > and I have no idea if running rpcbind in a vnet jail is practical. > > > > Enforcing the use for separate file systems for each jail also makes > > things safer, since the exports are enforced by the kernel. Without > > this, a malicious NFSv4 client could "guess" a file handle for a file > > outside the jail and gain access to that file. Put another way, without > > a separate file system, there is no way to stop a malicious client from > > finding files above the Root file handle. (Normal clients will use > > PutRootFH and LookupParent and these won't be able to go above the top > > of the jail.) > > > > So, what do others think of enforcing the requirement that each jail > > have its own file systems for this? > > I don't care for any of it. It looks like additional overhead with the > addition of potential security risks. All for a very limited (and as yet > unknown) use case. > I am thinking that if/when this goes into main, it would be under a new kernel build option called something like NFSD_VIMAGE. I think that would avoid the overhead/security risks for those that do not need/want it. rick > > --chris > > > > rick > > > > > >> - Peter > >> > >> > >> On Fri, Nov 25, 2022, 4:24 PM Rick Macklem <rick.macklem@gmail.com> > wrote: > >> > >>> Hi, > >>> > >>> bz@ has encouraged me to fiddle with the nfsd > >>> so that it works in a vnet jail. > >>> I have now basically done so, specifically for > >>> NFSv4, since NFSv3 presents various issues. > >>> > >>> What I have not yet done is put global variables > >>> in the vnet. This needs to be done so that the nfsd > >>> can be run in multiple jail instances and/or in and > >>> outside of a jail. > >>> The problem is that there are 100s of global variables. > >>> > >>> I can see two approaches: > >>> 1 - Move them all into the vnet jail. This would imply > >>> that all the sysctls need to somehow be changed, > >>> which would seem to be a POLA violation. > >>> It also implies a lot of stuff in the vnet. > >>> 2 - Just move the global variables that will always > >>> differ from one nfsd to another (this would make > >>> the sysctls global and apply to all nfsds). > >>> This will keep the number of globals in the vnet > >>> smaller. > >>> > >>> I am currently leaning towards #2, put what do others > >>> think? > >>> > >>> rick > >>> ps: Personally, I don't know what use there is of > >>> running the nfsd inside a vnet jail, but bz@ has > >>> some use case. > >>> > >> > >> > [-- Attachment #2 --] <div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-family:monospace"><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Dec 1, 2022 at 8:23 AM Chris <<a href="mailto:bsd-lists@bsdforge.com">bsd-lists@bsdforge.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 2022-11-29 16:21, Rick Macklem wrote:<br> > On Sun, Nov 27, 2022 at 10:04 AM Peter Eriksson <<a href="mailto:pen@lysator.liu.se" target="_blank">pen@lysator.liu.se</a>> wrote:<br> > <br> >> Keep the global variables as defaults that apply to all nfsds and allow<br> >> (at least some subset) to be overridden inside the net jails if some things<br> >> need to be changed from the defaults?<br> >> <br> >> This is pretty much a reply to one of the posts selected at random,<br> > but I thought that better than starting a new email thread.<br> > <br> > bz@ and asomers@ have both asked about running mountd within a vnet prison<br> > (one via offlist email and the other on phabricator).<br> > <br> > I think it is worth discussing here...<br> > mountd (rightly or wrongly) does two distinctly different things:<br> > 1 - It pushes the exports into the kernel via nmount() so they<br> > can be hung off of the "struct mount" for a file system's<br> > mount point.<br> > --> This can only work for file system mount points and can<br> > only be done once for any given file system mount point.<br> > <br> > At this time, I have it done once globally outside of the prisons.<br> > The alternative I can see is doing it within each prison, but I<br> > think that would require that each prison have its own file system(s).<br> > (ie. The prison's root would always be a file system mount point.)<br> > <br> > 2 - It handles RPC Mount protocol requests from NFSv3 clients. This one<br> > is NFSv3 specific, which is why I have done this NFSv4 only at<br> > this time. To do this, it must be able to register with rpcbind,<br> > and I have no idea if running rpcbind in a vnet jail is practical.<br> > <br> > Enforcing the use for separate file systems for each jail also makes<br> > things safer, since the exports are enforced by the kernel. Without<br> > this, a malicious NFSv4 client could "guess" a file handle for a file<br> > outside the jail and gain access to that file. Put another way, without<br> > a separate file system, there is no way to stop a malicious client from<br> > finding files above the Root file handle. (Normal clients will use<br> > PutRootFH and LookupParent and these won't be able to go above the top<br> > of the jail.)<br> > <br> > So, what do others think of enforcing the requirement that each jail<br> > have its own file systems for this?<br> <br> I don't care for any of it. It looks like additional overhead with the<br> addition of potential security risks. All for a very limited (and as yet<br> unknown) use case.<br></blockquote><div><span class="gmail_default" style="font-family:monospace">I am thinking that if/when this goes into main, it would be</span></div><div><span class="gmail_default" style="font-family:monospace">under a new kernel build option called something like</span></div><div><span class="gmail_default" style="font-family:monospace">NFSD_VIMAGE. I think that would avoid the overhead/security</span></div><div><span class="gmail_default" style="font-family:monospace">risks for those that do not need/want it.</span></div><div><span class="gmail_default" style="font-family:monospace"><br></span></div><div><span class="gmail_default" style="font-family:monospace">rick</span><span class="gmail_default" style="font-family:monospace"></span> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <br> --chris<br> > <br> > rick<br> > <br> > <br> >> - Peter<br> >> <br> >> <br> >> On Fri, Nov 25, 2022, 4:24 PM Rick Macklem <<a href="mailto:rick.macklem@gmail.com" target="_blank">rick.macklem@gmail.com</a>> wrote:<br> >> <br> >>> Hi,<br> >>> <br> >>> bz@ has encouraged me to fiddle with the nfsd<br> >>> so that it works in a vnet jail.<br> >>> I have now basically done so, specifically for<br> >>> NFSv4, since NFSv3 presents various issues.<br> >>> <br> >>> What I have not yet done is put global variables<br> >>> in the vnet. This needs to be done so that the nfsd<br> >>> can be run in multiple jail instances and/or in and<br> >>> outside of a jail.<br> >>> The problem is that there are 100s of global variables.<br> >>> <br> >>> I can see two approaches:<br> >>> 1 - Move them all into the vnet jail. This would imply<br> >>> that all the sysctls need to somehow be changed,<br> >>> which would seem to be a POLA violation.<br> >>> It also implies a lot of stuff in the vnet.<br> >>> 2 - Just move the global variables that will always<br> >>> differ from one nfsd to another (this would make<br> >>> the sysctls global and apply to all nfsds).<br> >>> This will keep the number of globals in the vnet<br> >>> smaller.<br> >>> <br> >>> I am currently leaning towards #2, put what do others<br> >>> think?<br> >>> <br> >>> rick<br> >>> ps: Personally, I don't know what use there is of<br> >>> running the nfsd inside a vnet jail, but bz@ has<br> >>> some use case.<br> >>> <br> >> <br> >> <br> </blockquote></div></div>home | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAM5tNy7Uc82OqhjVc3W84GAYcJd6P4D4U_criEYTwFmvEU61xg>