From nobody Wed Feb 9 13:21:52 2022 X-Original-To: hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A2E1E19AF2B1 for ; Wed, 9 Feb 2022 13:22:31 +0000 (UTC) (envelope-from grembo@freebsd.org) Received: from mail.evolve.de (mail.evolve.de [213.239.217.29]) (using TLSv1.3 with cipher TLS_CHACHA20_POLY1305_SHA256 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail.evolve.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jv0sy2Vdfz4VDS; Wed, 9 Feb 2022 13:22:30 +0000 (UTC) (envelope-from grembo@freebsd.org) Received: by mail.evolve.de (OpenSMTPD) with ESMTP id 47cb3433; Wed, 9 Feb 2022 13:22:26 +0000 (UTC) Received: by mail.evolve.de (OpenSMTPD) with ESMTPSA id c78928e7 (TLSv1.3:AEAD-CHACHA20-POLY1305-SHA256:256:NO); Wed, 9 Feb 2022 13:22:25 +0000 (UTC) Date: Wed, 9 Feb 2022 14:21:52 +0100 From: Michael Gmelin To: Alexander Leidinger Cc: Michael Gmelin , hackers@freebsd.org Subject: Re: Behavior of /dev/pts in a jail? Message-ID: <20220209142152.13373548.grembo@freebsd.org> In-Reply-To: <20220209132213.Horde.hjhX_GoM3qNT-7ucnNXd-ae@webmail.leidinger.net> References: <20220209113737.Horde.8QntfZV4xEkYdmHjXMgCpHN@webmail.leidinger.net> <77267259-0758-4C04-867D-77A896D133E4@freebsd.org> <20220209132213.Horde.hjhX_GoM3qNT-7ucnNXd-ae@webmail.leidinger.net> X-Face: $wrgCtfdVw_H9WAY?S&9+/F"!41z'L$uo*WzT8miX?kZ~W~Lr5W7v?j0Sde\mwB&/ypo^}> +a'4xMc^^KroE~+v^&^#[B">soBo1y6(TW6#UZiC]o>C6`ej+i Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAJFBMVEWJBwe5BQDl LASZU0/LTEWEfHbyj0Txi32+sKrp1Mv944X8/fm1rS+cAAAACXBIWXMAAAsTAAAL EwEAmpwYAAAAB3RJTUUH3wESCxwC7OBhbgAAACFpVFh0Q29tbWVudAAAAAAAQ3Jl YXRlZCB3aXRoIFRoZSBHSU1QbbCXAAAAAghJREFUOMu11DFvEzEUAGCfEhBVFzuq AKkLd0O6VrIQsLXVSZXoWE5N1K3DobBBA9fQpRWc8OkWouaIjedWKiyREOKs+3PY fvalCNjgLVHeF7/3bMtBzV8C/VsQ8tecEgCcDgrzjekwKZ7TwsJZd/ywEKwwP+ZM 8P3drTsAwWn2mpWuDDuYiK1bFs6De0KUUFw0tWxm+D4AIhuuvZqtyWYeO7jQ4Aea 7jUqI+ixhQoHex4WshEvSXdood7stlv4oSuFOC4tqGcr0NjEqXgV4mMJO38nld4+ xKNxRDon7khyKVqY7YR4d+Cg0OMrkWXZOM7YDkEfKiilCn1qYv4mighZiynuHHOA Wq9QJq+BIES7lMFUtcikMnkDGHUoncA+uHgrP0ctIEqfwLHzeSo+eUA66AqzwN6n 2ZHJhw6Qh/PoyC/QENyEyC/AyNjq74Bs+3UH0xYwzDUC4B97HgLocg1QLYgDDO1v f3UX9Y307Ew4AHh67YAFFsxEpkXwpXY3eIgMhAAE3R19L919nNnuD2wlPcDE3UeT L2ytEICQib9BXgS2fU8PrD82ToYO1OEmMSnYTjSqSv9wdC0tPYC+rQRQD9ESnldF CyqfmiYW+tlALt8gH2xrMdC/youbjzPXEun+/ReXsMCDyve3dZc09fn2Oas8oXGc Jj6/fOeK5UmSMPmf/jL+GD8BEj0k/Fn6IO4AAAAASUVORK5CYII= List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4Jv0sy2Vdfz4VDS X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=softfail (mx1.freebsd.org: 213.239.217.29 is neither permitted nor denied by domain of grembo@freebsd.org) smtp.mailfrom=grembo@freebsd.org X-Spamd-Result: default: False [-1.07 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEFALL_USER(0.00)[grembo]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-0.97)[-0.974]; R_SPF_SOFTFAIL(0.00)[~all:c]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; MLMMJ_DEST(0.00)[hackers]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:213.239.192.0/18, country:DE]; SUBJECT_ENDS_QUESTION(1.00)[]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On Wed, 09 Feb 2022 13:22:13 +0100 Alexander Leidinger wrote: > Quoting Michael Gmelin (from Wed, 9 Feb 2022 > 12:56:49 +0100): > > > I was able to reproduce the issue locally. > > > > The problem is caused by jexec inheriting the pty from the jail > > host. > > > > If you use a pty that was created inside of the jail, > > gpg-agent/pinentry works as expected. > > > > This can be accomplished, e.g., by running tmux inside of the jail: > > > > jexec gpgtest > > pkg install tmux > > tmux > > gpg --gen-key > > > > Running sshd inside of the jail and connecting to it using ssh has > > the same effect. > > I confirm (with ssh instead of jexec) the behavior. > > What I don't understand is how this works. ls is not build-in to the > shell. So how can it be that the jexec-ed shell can fork ls and it > sees the content of /dev/pts/, and the ls forked from > gpg->gpg-agent->pinentry-wrapper can't? gpg-agent starts as a daemon, so it detaches from the controlling terminal, closing all open fds to the pty. You can test this yourself easily: [root@gpgtest ~]# cat >test.sh<<"EOF" #!/bin/sh sleep .5 echo echo "/dev/pts contains $(ls /dev/pts | wc -l | xargs) files" EOF [root@gpgtest ~]# chmod 755 test.sh [root@gpgtest ~]# ./test.sh /dev/pts contains 1 files [root@gpgtest /]# daemon ./test.sh [root@gpgtest /]# /dev/pts contains 0 files You can also compare the file descriptors opened by your shell vs those opened by gpg-agent using `procstat -f `. > And how could we fix this (or > why wouldn't we want to fix it)? I think that the current behavior is already hacky, but makes jexec more useful in practice (so you *can* actually do ssh from one jail to another without having to create a new pty), thanks to entries in devfs.rules. In the end, jexec just calls jail_attach, it's not starting a whole new user session. Expanding this to daemonized processes would make things a lot more complex (like: is this the pty created from within the jail or one created outside of it, what happens to it if the user session on the jail host ends, etc.). The one thing I could see is adding a parameter to jexec that makes it allocate a new pty within the jail (similar to what ssh would do). This would also workaround the issues I had with bhyve (ttyu* not being allowed to seen inside the jail). Or write a thin wrapper that is called inside of the jail, to run a program in a newly allocated pty. Maybe someone with more insights to how jails work internally could give their input here. In the meantime, tmux is probably the most lightweight way of working around this in your specific use-case, without having to run sshd. Cheers Michael p.s. Once you know what you're looking for, you can find mailing list and forum entries about it going back some 15 years, which include pretty much the same workarounds I came up with *sigh* > > Bye, > Alexander. > -- Michael Gmelin