From owner-freebsd-current@freebsd.org Sun Oct 29 19:14:05 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB5B0E4A270 for ; Sun, 29 Oct 2017 19:14:05 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6137E63795; Sun, 29 Oct 2017 19:14:05 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v9TJDwod015247 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 29 Oct 2017 21:13:58 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v9TJDwod015247 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v9TJDw0e015246; Sun, 29 Oct 2017 21:13:58 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 29 Oct 2017 21:13:58 +0200 From: Konstantin Belousov To: Tijl Coosemans Cc: freebsd-current@FreeBSD.org, gerald@FreeBSD.org Subject: Re: Segfault in _Unwind_* code called from pthread_exit Message-ID: <20171029191358.GU2566@kib.kiev.ua> References: <20170823163707.096f93ab@kalimero.tijl.coosemans.org> <20170824154235.GD1700@kib.kiev.ua> <20170824180830.199885b0@kalimero.tijl.coosemans.org> <20170825173851.09116ddc@kalimero.tijl.coosemans.org> <20170825234442.GO1700@kib.kiev.ua> <20170826202813.1240a1ef@kalimero.tijl.coosemans.org> <20170826184034.GR1700@kib.kiev.ua> <20171029182351.502f53cf@kalimero.tijl.coosemans.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20171029182351.502f53cf@kalimero.tijl.coosemans.org> User-Agent: Mutt/1.9.1 (2017-09-22) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Oct 2017 19:14:05 -0000 On Sun, Oct 29, 2017 at 06:23:51PM +0100, Tijl Coosemans wrote: > On Sat, 26 Aug 2017 21:40:34 +0300 Konstantin Belousov wrote: > > On Sat, Aug 26, 2017 at 08:28:13PM +0200, Tijl Coosemans wrote: > >> I did consider using > >> a CFI directive (see patch below) and it works, but it's architecture > >> specific and it's inserted after the function prologue so there's still > >> a window of a few instructions where a stack unwinder will try to use > >> the return address. > >> > >> Index: lib/libthr/thread/thr_create.c > >> =================================================================== > >> --- lib/libthr/thread/thr_create.c (revision 322802) > >> +++ lib/libthr/thread/thr_create.c (working copy) > >> @@ -251,6 +251,7 @@ create_stack(struct pthread_attr *pattr) > >> static void > >> thread_start(struct pthread *curthread) > >> { > >> + __asm(".cfi_undefined %rip"); > >> sigset_t set; > >> > >> if (curthread->attr.suspend == THR_CREATE_SUSPENDED) > > > > I like this approach much more than the previous patch. What can be > > done is to provide asm trampoline which calls thread_start(). There you > > can add the .cfi_undefined right at the entry. > > > > It is somewhat more work than just setting the return address on the > > kernel-constructed pseudo stack frame, but I believe this is ultimately > > correct way. You still can do it only on some arches, if you do not > > have incentive to code asm for all of them. > > Ok, but then there are two ways to implement the trampoline: > > 1) > movq $0,(%rsp) > jmp thread_start > 2) > subq $8,%rsp > call thread_start > /* NOTREACHED */ > > With 1) you're setting the return address to zero anyway, so you might > as well do that in the kernel like my first patch. With 2) you're > setting up a new call frame, basically redoing what the kernel already > did and on i386 this also means copying the function argument. I do not quite understand the second variant, because the stack is not guaranteed to be zeroed, and it is often not if reused after the previously exited thread. The first variant is what I like, but perhaps we need to emulate the frame as well, i.e. push two zero longs. Currently kernel does not access the usermode stack for the new thread unless dictated by ABI (i.e. it does not touch it for 64bit process on amd64, but have to for 32bit). I like this property. Also, the previous paragraph is indicative: we do not really know in kernel what ABI the userspace follows. It might want frame, may be it does not need it. It could use other register than %rbp as the frame base, etc. > > Do you have any preference (or better alternatives), because I think I > still prefer my first patch. It's the caller's job to set up the call > frame, in this case the kernel. And if the kernel handles it then it > also works with (hypothetical) implementations other than libthr. > > > Also crt1 probably should get the same treatment, despite we already set > > %rbp to zero AFAIR. > > I haven't checked but I imagine the return address of the process entry > point is always zero because the stack is all zeros. Stack is not zero. The environment and argument strings and auxv are copied at top, and at the bottom the ps_strings structure is located, so it is not. If you commit your existing patch as is, I will not resent. But I do think that stuff that can be done in usermode, should be done in usermode, esp. when the amount of efforts is same.