From nobody Thu Jun 29 07:05:43 2023 X-Original-To: freebsd-jail@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Qs8cM6xlDz4k1N5 for ; Thu, 29 Jun 2023 07:05:55 +0000 (UTC) (envelope-from melifaro@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Qs8cM6kyhz3J5Z; Thu, 29 Jun 2023 07:05:55 +0000 (UTC) (envelope-from melifaro@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1688022355; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=t0/4cFTg+b68xbcLaJyAycwF1lq5qU1qclD/Ni62OV0=; b=Ee7xGDtEPGBgwmfqUQSYzb2/Qs6j5WDuaGZgwUoKxv7CK/bbx0V0QHyYPdCF291WMzxca3 FIZoHMvq5Hz5ez/WC4NUWVpEoVF+w4EaAUj/4uuJuWVe83V43I3FyyaFKtqm96rJAMegx0 CXVleC/MGiLEUs+80kPTRTeRqN4rmC2dZZTj3QHaiGmZadsKlPp2xu+k9z/QyFuopBQOL1 xjKrpEd9A3z9Qb8ThlJ4DzRNwNMq+qEvJf6OuKN6B9Gkao++9Rp0CiDJxjtEbXP+tmNSaI nsHXeule7FihEyYGQou2/P7hGqi69qqtUUZ3NpfWgA7FtZMFDqY4v93K8Spwbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1688022355; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=t0/4cFTg+b68xbcLaJyAycwF1lq5qU1qclD/Ni62OV0=; b=OywLzqYqNiMTKjSiEoFVQp1zRpL+i/X18wYrzctviyNTJ4Hj6goY6w+etxDbsHbE84r+u7 EURj2Od1ix9biMI1Q6Q3yuB03zWHpnMcOG0+a0So41DHVy6j70oequ32vfrTpo/hRY06X7 nQ7AmdK8hF76yTtXmYXmr8YUeOncmTVICipR2j1fk31WyQDfnWBFPPpVNL2s34V4f8SaWc P06esLtjJK/pdMGzsN1sM0hgyIdLOwhjdn7ZKqOqebucZxWa+IjwnDhdH79VkmD+zsIJgJ 34Bo65kWrWZMha3Uk+dDFckhZauR8BGEHENZQX9MQRDMI3TL8RslKH7No3g8Vw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1688022355; a=rsa-sha256; cv=none; b=AAKNtcaJFAEuHjl1Sd0aIAabtJjmScM8BP0cS4fA+yHKHrwa+DYfQ3uWSCPXOsEWnroYtn GBO7Txv+aM2bvg3uSf73pJniHoHbDpQm0YoECf5kOep/Pq3GVgTCgiGcoq8HIkjxOi45Ou R/gsthJQTe0UTpbTuRcrSBwD6cZbmX9qTep3BmzcGRAouy5eFGwz+6kcXKmXu2L2FMlQAv 1GlBAw0Q3WU9iI60s4wN4Qx6qYcrcLioa0wQ5TX/EGX3uTDDB06xenOdLs+LyWKYCdlqF4 sg9LS0V7xesTwxr2Qn4AofpMIp8uVokVCMgkBvOH7Z0DDF3AaJsWHuX3U8M6ZQ== Received: from smtpclient.apple (unknown [IPv6:2a02:8084:d6bb:510:3954:d8d2:6fb5:8816]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: melifaro/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Qs8cM2tSLz10Fc; Thu, 29 Jun 2023 07:05:55 +0000 (UTC) (envelope-from melifaro@freebsd.org) From: Alexander Chernikov Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_D917104D-A8ED-41BF-9005-E5372A0059A3" List-Id: Discussion about FreeBSD jail(8) List-Archive: https://lists.freebsd.org/archives/freebsd-jail List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-jail@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.600.7\)) Subject: Re: Add IP address ioctl (SIOCAIFADDR) from jail is called with host credentials Date: Thu, 29 Jun 2023 08:05:43 +0100 In-Reply-To: <810c6bd0-261b-4129-bf40-e390be0e8278@app.fastmail.com> Cc: freebsd-jail@freebsd.org To: Shivank Garg References: <93d61b80-95cb-4b3e-84dc-1d8b655e66f7@app.fastmail.com> <810c6bd0-261b-4129-bf40-e390be0e8278@app.fastmail.com> X-Mailer: Apple Mail (2.3731.600.7) X-ThisMailContainsUnwantedMimeParts: N --Apple-Mail=_D917104D-A8ED-41BF-9005-E5372A0059A3 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On 28 Jun 2023, at 22:59, Alexander Chernikov = wrote: >=20 >=20 >=20 > On Wed, 28 Jun 2023, at 6:30 AM, Shivank Garg wrote: >> Hi Alexander, >>=20 >> Thanks for replying. >> I think it would mean struct prison info is lost, when it reaches = ioctl code, Is there some way we can get jail id? > Yes, you should add the hook to the netlink handler. >>=20 >> Another question I have: prison_check_ip4 still relies on checking = struct prison for flags and ip addr.=20 >> = https://github.com/freebsd/freebsd-src/blob/6927176113ee775983952edb3c201f= ed6be318d3/sys/netinet/in_jail.c#L319 >> How do we handle these cases? > I=E2=80=99ll take a look on the weekend. It may indeed be a problem = with nested jails. I looked at the code and after some experiments decided to go with the = simplest approach: https://reviews.freebsd.org/D40793 Netlink now passes proper ucred to the ioctl handler, so your code = should be able to work out-of-the-box after this lands. >>=20 >> It used to work for VNET jails inet calls sometime back when I wrote = mac_ipacl: https://reviews.freebsd.org/D20967 >> - MAC policy to limit jail privilege to set its IP address. We were = planning to merge this code in 14.0. Is there something we can >> do regarding it? > Yep, sure! I=E2=80=99ll try to further decouple ioctl handler and the = actual address modification code so the ioctl hook wont=E2=80=99t get = called in the netlink handler. >> Thanks, >> Shivank >>=20 >> On Wed, 28 Jun 2023 at 04:05, Alexander Chernikov = > wrote: >>=20 >>=20 >>=20 >> On Fri, 23 Jun 2023, at 10:27 AM, Alexander Chernikov wrote: >>>=20 >>>=20 >>> On Fri, 23 Jun 2023, at 7:53 AM, Shivank Garg wrote: >>>> Hi, >>>>=20 >>>> I want to check credentials of the thread setting the IP address = with SIOCAIFADDR ioctl. >>>> If the thread is jailed (jailed(td_ucred) =3D=3D 1), I'm applying = some checks on ip address. >>>>=20 >>>> My expectation was that (cred->cr_prison !=3D &prison0) for an = ifconfig call made by the jail. >>> If you=E2=80=99re using -head, it=E2=80=99s a bit more complicated. = ifconfig(8) uses rtnetlink(4) interfaces to communicate with the kernel. = Privilege check is done in Netlink: = https://github.com/freebsd/freebsd-src/blob/764464af49688e74fd6d803df0404c= a4726dd460/sys/netlink/route/iface.c#L1472 . After that, (as of now) = netlink calls ioctl code from its own kernel thread, which may be the = reason of the behavior you=E2=80=99re observing. >> Apparently the previous message was not delivered everywhere. >>>> However, it is showing me some weird behavior. Here are the logs = for a tweaked kernel: >>>>=20 >>>> @@ -339,7 +343,7 @@ in_control(struct socket *so, u_long cmd, void = *data, struct ifnet *ifp, >>>> return (EADDRNOTAVAIL); >>>> struct ucred *cred =3D (td !=3D NULL) ? td->td_ucred : = NULL; >>>> - >>>> + printf("in_control jailed? %d jid %d prison_owns_vnet? = %d\n",jailed(cred),cred->cr_prison->pr_id,prison_owns_vnet(cred)); >>>>=20 >>>> # jexec 1 ifconfig epair0b inet 169.254.123.101/24 = up >>>>=20 >>>> Dmesg logs: >>>> [256] in_control jailed? 0 jid 0 prison_owns_vnet? 1 >>>>=20 >>>> Cred value indicates host and jail is 0 but the PR_VNET flag is = set. >>>>=20 >>>> Is this behavior expected? or something going wrong - what's the = next debug step? >>>>=20 >>>> I greatly appreciate your help! >>>>=20 >>>> Thanks, >>>> Shivank >>>=20 >>> /Alexander >>>=20 >>=20 >> /Alexander >=20 > /Alexander --Apple-Mail=_D917104D-A8ED-41BF-9005-E5372A0059A3 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8

On 28 = Jun 2023, at 22:59, Alexander Chernikov <melifaro@freebsd.org> = wrote:



On Wed, 28 Jun = 2023, at 6:30 AM, Shivank Garg wrote:
Hi Alexander,

Thanks = for replying.
I think it would mean struct prison = info is lost, when it reaches ioctl code, Is there some way we can get = jail id?
Yes, you should add the hook to the netlink = handler.

Another = question I have: prison_check_ip4 still relies on checking struct prison = for flags and ip addr. 
How do we handle these = cases?
I=E2=80=99ll take a look on the weekend. It may = indeed be a problem with nested jails.
I looked = at the code and after some experiments decided to go with the simplest = approach: https://reviews.freebsd.org/D4= 0793
Netlink now passes proper ucred to the ioctl handler, = so your code should be able to work out-of-the-box after this = lands.


 It used to work = for VNET jails inet calls sometime back when I wrote mac_ipacl: https://reviews.freebsd.org/D2= 0967
- MAC policy to limit jail privilege to set its = IP address. We were planning to merge this code in 14.0. Is there = something we can
do regarding = it?
Yep, sure! I=E2=80=99ll try to further decouple = ioctl handler and the actual address modification code so the ioctl hook = wont=E2=80=99t get called in the netlink handler.
Thanks,
Shivank

On Wed, 28 Jun 2023 at 04:05, Alexander = Chernikov <melifaro@freebsd.org> = wrote:



On Fri, 23 Jun 2023, at 10:27 AM, Alexander = Chernikov wrote:


On = Fri, 23 Jun 2023, at 7:53 AM, Shivank Garg wrote:
Hi,

I want to check = credentials of the thread setting the IP address with SIOCAIFADDR = ioctl.
If the thread is jailed (jailed(td_ucred) =3D=3D = 1), I'm applying some checks on ip = address.

My expectation was that (cred->cr_prison !=3D = &prison0) for an ifconfig call made = by the jail.
If you=E2=80=99re using = -head, it=E2=80=99s a bit more complicated. ifconfig(8) uses = rtnetlink(4) interfaces to communicate with the kernel. Privilege check = is done in Netlink:  https://github.com/freebsd/freebsd-src/blob/764464af4968= 8e74fd6d803df0404ca4726dd460/sys/netlink/route/iface.c#L1472 . = After that, (as of now) netlink calls ioctl code from its own kernel = thread, which may be the reason of the behavior you=E2=80=99re = observing.
Apparently the previous message = was not delivered everywhere.
However, it = is showing me some weird behavior. Here are the logs for a tweaked = kernel:

@@ -339,7 +343,7 @@ in_control(struct = socket *so, u_long cmd, void *data, struct ifnet *ifp,
    =             return = (EADDRNOTAVAIL);
        struct ucred *cred =3D = (td !=3D NULL) ? td->td_ucred : NULL;
-
+       = printf("in_control jailed? %d jid %d prison_owns_vnet? = %d\n",jailed(cred),cred->cr_prison->pr_id,prison_owns_vnet(cred));

# jexec 1 ifconfig epair0b inet 169.254.123.101/24 up
<= br>
Dmesg logs:
[256] in_control jailed? 0 jid 0 = prison_owns_vnet? 1

Cred value = indicates host and jail is 0 but the PR_VNET flag is set.

Is this = behavior expected? or something going wrong - what's the next debug = step?

I greatly appreciate your = help!

Thanks,
Shivank
<= /div>

/Alexander


/Alexander

/Alexander

= --Apple-Mail=_D917104D-A8ED-41BF-9005-E5372A0059A3--