From owner-freebsd-net@freebsd.org Mon Jan 18 04:48:32 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BBF51A86A55 for ; Mon, 18 Jan 2016 04:48:32 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id A0DAD1BFC for ; Mon, 18 Jan 2016 04:48:32 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 9D765A86A52; Mon, 18 Jan 2016 04:48:32 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8428CA86A50; Mon, 18 Jan 2016 04:48:32 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 283401BFA; Mon, 18 Jan 2016 04:48:31 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u0I4mQTo015893 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 18 Jan 2016 06:48:27 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u0I4mQTo015893 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u0I4mQTg015892; Mon, 18 Jan 2016 06:48:26 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 18 Jan 2016 06:48:26 +0200 From: Konstantin Belousov To: Jilles Tjoelker Cc: Boris Astardzhiev , net@freebsd.org, threads@freebsd.org Subject: Re: Does FreeBSD have sendmmsg or recvmmsg system calls? Message-ID: <20160118044826.GS3942@kib.kiev.ua> References: <20160108075815.3243.qmail@f5-external.bushwire.net> <20160108204606.G2420@besplex.bde.org> <20160113080349.GC72455@kib.kiev.ua> <20160116195657.GJ3942@kib.kiev.ua> <20160116202534.GK3942@kib.kiev.ua> <20160117211853.GA37847@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160117211853.GA37847@stack.nl> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Jan 2016 04:48:32 -0000 Added threads@ where this discussion is more appropriate. On Sun, Jan 17, 2016 at 10:18:53PM +0100, Jilles Tjoelker wrote: > This will typically work (if the cancellation occurs while blocked > inside __sys_recvmsg()) but has the usual problem of relying on [EINTR]: > lost wakeups. This is certainly less bad than using the interposable > recvmsg(), though, which would discard the already received data. > > As a slight modification, the first recvmsg could use the interposable > version, since there is no pending data at that point. This avoids > needing to call pthread_testcancel() manually. > > The regular cancellation code closes this race window using the > undocumented thr_wake() system call, on the thread itself, in the signal > handler for the cancellation signal. This causes the next attempt to > sleep(9) to fail with [EINTR]. (On another note, it appears to be > possible for user code (cleanup handlers and pthread_key_create() > destructors) to be called with thr_wake() still active, if the > cancellation signal handler is called immediately after the cancellation > point system call returns.) > > The race in recvmmsg could be removed using this mechanism but it > requires either a separate version of recvmmsg in libthr or a new > interface in libthr. I imagine the new interface as a new cancellation > type which causes cancellation point functions such as recvmsg() to fail > with a new errno when cancelled while leaving cancellation pending. This > seems conceptually possible but adds some code to the common path for > cancellation points. A new version of pthread_testcancel() with a return > value would be needed. Yes, I should have remembered about TDP_WAKEUP. In fact, this discussion and recvmmsg() skeleton structured my understanding of the better split between libc and libthr. I start thinking that libthr should not interpose most of the syscalls. Instead, libthr should interpose (wrappers around) thr_cancel_enter*() and thr_cancel_leave(), the body of the cancellable syscalls should live in libc. The benefits are removal of double implementation of cancellable syscalls, more compact interface between libc and libthr, and potentially we can grow an _np extension which would allow user libraries to correctly implement composable cancellable functions (as needed for sendmmsg).