Date: Thu, 29 Jun 2023 08:05:43 +0100 From: Alexander Chernikov <melifaro@FreeBSD.org> To: Shivank Garg <shivank@freebsd.org> Cc: freebsd-jail@freebsd.org Subject: Re: Add IP address ioctl (SIOCAIFADDR) from jail is called with host credentials Message-ID: <C63F59DD-9805-4C37-BC7A-C36DA32FDBE2@FreeBSD.org> In-Reply-To: <810c6bd0-261b-4129-bf40-e390be0e8278@app.fastmail.com> References: <CAOVCmzFQjwTaeQZQSD-ep7s=UdDzzczQ6r9wtjK-w3BAwRsKvA@mail.gmail.com> <93d61b80-95cb-4b3e-84dc-1d8b655e66f7@app.fastmail.com> <ab27fc86-e339-420c-8cfa-05c53a3bf4f9@app.fastmail.com> <CAOVCmzFt6NQQzyoHnXeEOagKgn9n_JOex7vs4xOFDZ497qtfKQ@mail.gmail.com> <810c6bd0-261b-4129-bf40-e390be0e8278@app.fastmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_D917104D-A8ED-41BF-9005-E5372A0059A3 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On 28 Jun 2023, at 22:59, Alexander Chernikov <melifaro@freebsd.org> = wrote: >=20 >=20 >=20 > On Wed, 28 Jun 2023, at 6:30 AM, Shivank Garg wrote: >> Hi Alexander, >>=20 >> Thanks for replying. >> I think it would mean struct prison info is lost, when it reaches = ioctl code, Is there some way we can get jail id? > Yes, you should add the hook to the netlink handler. >>=20 >> Another question I have: prison_check_ip4 still relies on checking = struct prison for flags and ip addr.=20 >> = https://github.com/freebsd/freebsd-src/blob/6927176113ee775983952edb3c201f= ed6be318d3/sys/netinet/in_jail.c#L319 >> How do we handle these cases? > I=E2=80=99ll take a look on the weekend. It may indeed be a problem = with nested jails. I looked at the code and after some experiments decided to go with the = simplest approach: https://reviews.freebsd.org/D40793 Netlink now passes proper ucred to the ioctl handler, so your code = should be able to work out-of-the-box after this lands. >>=20 >> It used to work for VNET jails inet calls sometime back when I wrote = mac_ipacl: https://reviews.freebsd.org/D20967 >> - MAC policy to limit jail privilege to set its IP address. We were = planning to merge this code in 14.0. Is there something we can >> do regarding it? > Yep, sure! I=E2=80=99ll try to further decouple ioctl handler and the = actual address modification code so the ioctl hook wont=E2=80=99t get = called in the netlink handler. >> Thanks, >> Shivank >>=20 >> On Wed, 28 Jun 2023 at 04:05, Alexander Chernikov = <melifaro@freebsd.org <mailto:melifaro@freebsd.org>> wrote: >>=20 >>=20 >>=20 >> On Fri, 23 Jun 2023, at 10:27 AM, Alexander Chernikov wrote: >>>=20 >>>=20 >>> On Fri, 23 Jun 2023, at 7:53 AM, Shivank Garg wrote: >>>> Hi, >>>>=20 >>>> I want to check credentials of the thread setting the IP address = with SIOCAIFADDR ioctl. >>>> If the thread is jailed (jailed(td_ucred) =3D=3D 1), I'm applying = some checks on ip address. >>>>=20 >>>> My expectation was that (cred->cr_prison !=3D &prison0) for an = ifconfig call made by the jail. >>> If you=E2=80=99re using -head, it=E2=80=99s a bit more complicated. = ifconfig(8) uses rtnetlink(4) interfaces to communicate with the kernel. = Privilege check is done in Netlink: = https://github.com/freebsd/freebsd-src/blob/764464af49688e74fd6d803df0404c= a4726dd460/sys/netlink/route/iface.c#L1472 . After that, (as of now) = netlink calls ioctl code from its own kernel thread, which may be the = reason of the behavior you=E2=80=99re observing. >> Apparently the previous message was not delivered everywhere. >>>> However, it is showing me some weird behavior. Here are the logs = for a tweaked kernel: >>>>=20 >>>> @@ -339,7 +343,7 @@ in_control(struct socket *so, u_long cmd, void = *data, struct ifnet *ifp, >>>> return (EADDRNOTAVAIL); >>>> struct ucred *cred =3D (td !=3D NULL) ? td->td_ucred : = NULL; >>>> - >>>> + printf("in_control jailed? %d jid %d prison_owns_vnet? = %d\n",jailed(cred),cred->cr_prison->pr_id,prison_owns_vnet(cred)); >>>>=20 >>>> # jexec 1 ifconfig epair0b inet 169.254.123.101/24 = <http://169.254.123.101/24>; up >>>>=20 >>>> Dmesg logs: >>>> [256] in_control jailed? 0 jid 0 prison_owns_vnet? 1 >>>>=20 >>>> Cred value indicates host and jail is 0 but the PR_VNET flag is = set. >>>>=20 >>>> Is this behavior expected? or something going wrong - what's the = next debug step? >>>>=20 >>>> I greatly appreciate your help! >>>>=20 >>>> Thanks, >>>> Shivank >>>=20 >>> /Alexander >>>=20 >>=20 >> /Alexander >=20 > /Alexander --Apple-Mail=_D917104D-A8ED-41BF-9005-E5372A0059A3 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"content-type" content=3D"text/html; = charset=3Dutf-8"></head><body style=3D"overflow-wrap: break-word; = -webkit-nbsp-mode: space; line-break: = after-white-space;"><br><div><br><blockquote type=3D"cite"><div>On 28 = Jun 2023, at 22:59, Alexander Chernikov <melifaro@freebsd.org> = wrote:</div><br class=3D"Apple-interchange-newline"><div><meta = charset=3D"UTF-8"><div style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 12px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><br class=3D"Apple-interchange-newline"><br></div><div = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;">On Wed, 28 Jun = 2023, at 6:30 AM, Shivank Garg wrote:<br></div><blockquote type=3D"cite" = id=3D"qt" style=3D"font-family: Helvetica; font-size: 12px; font-style: = normal; font-variant-caps: normal; font-weight: 400; letter-spacing: = normal; orphans: auto; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; widows: auto; word-spacing: = 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div = dir=3D"ltr"><div>Hi Alexander,<br></div><div><br></div><div>Thanks = for replying.<br></div><div><div>I think it would mean struct prison = info is lost, when it reaches ioctl code, Is there some way we can get = jail id?<br></div></div></div></blockquote><div style=3D"caret-color: = rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: = normal; font-variant-caps: normal; font-weight: 400; letter-spacing: = normal; text-align: start; text-indent: 0px; text-transform: none; = white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;">Yes, you should add the hook to the netlink = handler.</div><blockquote type=3D"cite" id=3D"qt" style=3D"font-family: = Helvetica; font-size: 12px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; orphans: auto; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><div dir=3D"ltr"><div><div><br></div><div>Another = question I have: prison_check_ip4 still relies on checking struct prison = for flags and ip addr. <br></div><div><a = href=3D"https://github.com/freebsd/freebsd-src/blob/6927176113ee775983952e= db3c201fed6be318d3/sys/netinet/in_jail.c#L319">https://github.com/freebsd/= freebsd-src/blob/6927176113ee775983952edb3c201fed6be318d3/sys/netinet/in_j= ail.c#L319</a><br></div></div><div>How do we handle these = cases?<br></div></div></blockquote><div style=3D"caret-color: rgb(0, 0, = 0); font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;">I=E2=80=99ll take a look on the weekend. It may = indeed be a problem with nested jails.</div></div></blockquote>I looked = at the code and after some experiments decided to go with the simplest = approach: <a = href=3D"https://reviews.freebsd.org/D40793">https://reviews.freebsd.org/D4= 0793</a></div><div>Netlink now passes proper ucred to the ioctl handler, = so your code should be able to work out-of-the-box after this = lands.</div><div><br><blockquote type=3D"cite"><div><blockquote = type=3D"cite" id=3D"qt" style=3D"font-family: Helvetica; font-size: = 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; orphans: auto; text-align: start; text-indent: = 0px; text-transform: none; white-space: normal; widows: auto; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><div dir=3D"ltr"><div><br></div><div> It used to work = for VNET jails inet calls sometime back when I wrote mac_ipacl:<span = class=3D"Apple-converted-space"> </span><a = href=3D"https://reviews.freebsd.org/D20967">https://reviews.freebsd.org/D2= 0967</a><br></div><div>- MAC policy to limit jail privilege to set its = IP address. We were planning to merge this code in 14.0. Is there = something we can<br></div><div><div>do regarding = it?<br></div></div></div></blockquote><div style=3D"caret-color: rgb(0, = 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;">Yep, sure! I=E2=80=99ll try to further decouple = ioctl handler and the actual address modification code so the ioctl hook = wont=E2=80=99t get called in the netlink handler.</div><blockquote = type=3D"cite" id=3D"qt" style=3D"font-family: Helvetica; font-size: = 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; orphans: auto; text-align: start; text-indent: = 0px; text-transform: none; white-space: normal; widows: auto; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><div = dir=3D"ltr"><div>Thanks,<br></div><div>Shivank<br></div></div><div><br></d= iv><div class=3D"qt-gmail_quote"><div dir=3D"ltr" = class=3D"qt-gmail_attr">On Wed, 28 Jun 2023 at 04:05, Alexander = Chernikov <<a = href=3D"mailto:melifaro@freebsd.org">melifaro@freebsd.org</a>> = wrote:<br></div><blockquote class=3D"qt-gmail_quote" style=3D"margin: = 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; = border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div = class=3D"qt-msg6359259462117977049"><div><u></u><br></div><div><div><br></= div><div><br></div><div>On Fri, 23 Jun 2023, at 10:27 AM, Alexander = Chernikov wrote:<br></div><blockquote type=3D"cite" = id=3D"qt-m_6359259462117977049qt"><div><br></div><div><br></div><div>On = Fri, 23 Jun 2023, at 7:53 AM, Shivank Garg wrote:<br></div><blockquote = type=3D"cite" id=3D"qt-m_6359259462117977049qt-qt"><div = dir=3D"ltr"><div>Hi,<br></div><div><br></div><div>I want to check = credentials of the thread setting the IP address with SIOCAIFADDR = ioctl.<br></div><div>If the thread is jailed (jailed(td_ucred) =3D=3D = 1), I'm applying some checks on ip = address.<br></div><div><br></div><div>My expectation was that (<span = id=3D"qt-m_6359259462117977049qt-qt-gmail-docs-internal-guid-998c627e-7fff= -437f-e766-ef0b490e856c"><span style=3D"background-color: transparent; = font-variant-numeric: normal; font-variant-east-asian: normal; = font-variant-alternates: normal; vertical-align: baseline;"><span = class=3D"font" style=3D"font-family: Consolas, sans-serif;"><span = class=3D"size" style=3D"font-size: 11pt;">cred->cr_prison !=3D = &prison0)</span></span></span></span> for an ifconfig call made = by the jail.<br></div></div></blockquote><div>If you=E2=80=99re using = -head, it=E2=80=99s a bit more complicated. ifconfig(8) uses = rtnetlink(4) interfaces to communicate with the kernel. Privilege check = is done in Netlink: <a = href=3D"https://github.com/freebsd/freebsd-src/blob/764464af49688e74fd6d80= 3df0404ca4726dd460/sys/netlink/route/iface.c#L1472" = target=3D"_blank">https://github.com/freebsd/freebsd-src/blob/764464af4968= 8e74fd6d803df0404ca4726dd460/sys/netlink/route/iface.c#L1472</a> . = After that, (as of now) netlink calls ioctl code from its own kernel = thread, which may be the reason of the behavior you=E2=80=99re = observing.<br></div></blockquote><div>Apparently the previous message = was not delivered everywhere.<br></div><blockquote type=3D"cite" = id=3D"qt-m_6359259462117977049qt"><blockquote type=3D"cite" = id=3D"qt-m_6359259462117977049qt-qt"><div dir=3D"ltr"><div>However, it = is showing me some weird behavior. Here are the logs for a tweaked = kernel:<br></div><div><br></div><div><div><span class=3D"font" = style=3D"font-family: monospace;">@@ -339,7 +343,7 @@ in_control(struct = socket *so, u_long cmd, void *data, struct ifnet *ifp,<br> = return = (EADDRNOTAVAIL);<br> struct ucred *cred =3D = (td !=3D NULL) ? td->td_ucred : NULL;<br>-<br>+ = printf("in_control jailed? %d jid %d prison_owns_vnet? = %d\n",jailed(cred),cred->cr_prison->pr_id,prison_owns_vnet(cred));</= span></div><div><br></div><div># jexec 1 ifconfig epair0b inet<span = class=3D"Apple-converted-space"> </span><a = href=3D"http://169.254.123.101/24" = target=3D"_blank">169.254.123.101/24</a><span = class=3D"Apple-converted-space"> </span>up<br></div></div><div><div><= br></div><div>Dmesg logs:<br></div><div><span class=3D"font" = style=3D"font-family: monospace;"><b>[256] in_control jailed? 0 jid 0 = prison_owns_vnet? 1</b></span><br></div><div><br></div><div>Cred value = indicates host and jail is 0 but the PR_VNET flag is set.<span = style=3D""><span class=3D"font" style=3D"font-family: Courier, = "Courier New", monospace;"><span class=3D"size" = style=3D"font-size: = 12px;"></span></span></span><br></div></div><div><br></div><div>Is this = behavior expected? or something going wrong - what's the next debug = step?<br></div><div><br></div><div>I greatly appreciate your = help!<br></div><div><br></div><div><div>Thanks,<br></div><div>Shivank<br><= /div></div></div></blockquote><div><br></div><div = id=3D"qt-m_6359259462117977049qt-sig132921232"><div>/Alexander<br></div></= div><div><br></div></blockquote><div><br></div><div = id=3D"qt-m_6359259462117977049sig132921232"><div>/Alexander<br></div></div= ></div></div></blockquote></div></blockquote><div style=3D"caret-color: = rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: = normal; font-variant-caps: normal; font-weight: 400; letter-spacing: = normal; text-align: start; text-indent: 0px; text-transform: none; = white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><br></div><div id=3D"sig132921232" = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;"><div = class=3D"signature">/Alexander</div></div></div></blockquote></div><br></b= ody></html>= --Apple-Mail=_D917104D-A8ED-41BF-9005-E5372A0059A3--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?C63F59DD-9805-4C37-BC7A-C36DA32FDBE2>