From owner-freebsd-net@freebsd.org Sat Jan 16 20:25:41 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B40B9A8584B for ; Sat, 16 Jan 2016 20:25:41 +0000 (UTC) (envelope-from kib@kib.kiev.ua) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 962D31988 for ; Sat, 16 Jan 2016 20:25:41 +0000 (UTC) (envelope-from kib@kib.kiev.ua) Received: by mailman.ysv.freebsd.org (Postfix) id 952E9A8584A; Sat, 16 Jan 2016 20:25:41 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7C432A85849 for ; Sat, 16 Jan 2016 20:25:41 +0000 (UTC) (envelope-from kib@kib.kiev.ua) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0A7971987; Sat, 16 Jan 2016 20:25:40 +0000 (UTC) (envelope-from kib@kib.kiev.ua) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u0GKPYEX094943 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sat, 16 Jan 2016 22:25:34 +0200 (EET) (envelope-from kib@kib.kiev.ua) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u0GKPYEX094943 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=kib.kiev.ua; s=tom; t=1452975938; bh=SlzP5hVOa8C+DaBM9r5Kg4Xqxes1G8tBp6KkzEgTuQE=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=B1TdIjV4FSGzZOCxnfLnF7V/5Jf2hn1iisGtu8dQv53vfQJnVMfg08N3YMKplR0rN eRwqCHYgz9MDOJXbvWsx+va0CVZFzhtgUjxwOUMp0B0mYfGZS/ILWs0Z06Govi0WVX OVDKGulzxawL7fyO4PFfzDLcuacafrLKolTUNVc0= Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u0GKPYj2094940; Sat, 16 Jan 2016 22:25:34 +0200 (EET) (envelope-from kib@kib.kiev.ua) X-Authentication-Warning: tom.home: kostik set sender to kib@kib.kiev.ua using -f Date: Sat, 16 Jan 2016 22:25:34 +0200 From: Konstantin Belousov To: Boris Astardzhiev Cc: jilles@freebsd.org, net@freebsd.org Subject: Re: Does FreeBSD have sendmmsg or recvmmsg system calls? Message-ID: <20160116202534.GK3942@kib.kiev.ua> References: <20160107192840.GF3625@kib.kiev.ua> <20160108172323.W1815@besplex.bde.org> <20160108075815.3243.qmail@f5-external.bushwire.net> <20160108204606.G2420@besplex.bde.org> <20160113080349.GC72455@kib.kiev.ua> <20160116195657.GJ3942@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="MW5yreqqjyrRcusr" Content-Disposition: inline In-Reply-To: <20160116195657.GJ3942@kib.kiev.ua> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Jan 2016 20:25:41 -0000 --MW5yreqqjyrRcusr Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jan 16, 2016 at 09:56:57PM +0200, Konstantin Belousov wrote: > I am sorry for delay in answering, you will see the reason for it below. >=20 > On Fri, Jan 15, 2016 at 01:53:14PM +0200, Boris Astardzhiev wrote: > > kb>Big issue with the implementation is the interposing stuff, why do y= ou > > kb> need it at all ? Is it to correctly handle cancellation, to not fa= ll > > kb> into sleepable syscall when previous loop step was cancelled ? > > Yes. I initially thought it was better to use the interposing table. > >=20 > > kb> If yes, you _can_ use pthread_testcancel(3) etc in libc. Libc prov= ides > > kb> stubs for them with trivial implementation, which is reset to the r= eal > > kb> one if libthr is loaded. Then you can simplify your patch > > significantly, > > kb> avoiding the need for interposing and writing the loops both in lib= c and > > kb> libthr. > > Got it. See patch. I think I removed the interposing stuff as > > suggested. I didn't know about the stubs. But how for instance > > pthread_testcancel() will cope with sleeping recvmmsg for example? I'm > > not sure about the cancellation stuff here. Probably my approach is > > not correct? I looked through lib/ for an example and only stumbled on > > lib/libc/gen/sem.c where pthread_testcancel() is used but again I'm > > not sure if I'm tackling it correctly in the calls. > pthread_testcancel() does not need to do anything WRT sleeps in recvmsg. > It adds the cancellation point, which you already have by the call > to recvmsg(). >=20 > > kb> BTW, do you have tests for the cancellation of the new functions ? > > Unfortunately no. Ideas and guidelines how to stress test the calls > > regarding functionality > > and especially cancellation? > Write a test which would do controlled sendmmsg or recvmmsg in one > thread, e.g. over the unix domain socket, and another thread doing > cancellation. You should check both async and deferred modes. >=20 > >=20 > > kb> Again, the patch lacks man page updates. > > I'll try to write some soon. >=20 > So I thought how to implement the desired behaviour of the recvmmsg and > recvmmsg loops WRT cancellation using sendmsg(2) and POSIX pthread API > and realized that it is impossible. In other words, if you want to write > a loop with several calls to say recvmsg and not cancel the loop if > anything was already read by previous recvmsg calls, you cannot. >=20 > I also discussed this with jilles@ who is our POSIX expert, and who > confirmed my conclusion. >=20 > After thinking some more, I believe I managed to construct a possible > way to implement this, in libc, with some libthr extensions. Basically, > the idea is to have function pthread_cancel_is_pending_np(), which > would return the state of pending cancel. For some time I thought that > this cannot work, because cancellation could happen between call to > the cancel_is_pending() and next recvmmsg(). But, libc has a privilege > of having access to the syscalls without libthr interposing, just > call __sys_recvmmsg(), which would give EINTR on the cancel attempt. > This is an implementation detail, but we can rely on it in implementation. > In other words, the structure of the code would be like this > for (i =3D 0; i < vlen; i++) { > if (pthread_cancel_is_pending_np()) > goto out; Right after writing the text and hitting send, I realized that the pthread_cancel_is_pending_np() is not needed at all. You get EINTR =66rom __sys_recvmsg() on the cancel attempt, so everything would just work without the function. The crusial part is to use __sys_recvmsg instead of interposable _recvmsg(). > error =3D __sys_recvmsg(...); > if (error !=3D 0) > goto out; > ... > } > out: > if (any data received) > return (0); > pthread_testcancel(); /* no data received, cancel us if requested */ > handle errors > ... >=20 > Patch to implement pthread_cancel_is_pending_np() is below. >=20 > We have three viable strategies, after all > 1. Ignore cancellation at all for now, since cancellation is not very > popular among apps. This means that your latest, libc only patch > should be finalized. > 2. Implement cancellation with libthr interposing, i.e. finalize your > libc + libthr patch. > 3. Implement cancellation in libc as outlined above. > It is your choice of the approach. >=20 --MW5yreqqjyrRcusr Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJWmqc9AAoJEJDCuSvBvK1BIWoQAJjgKugUrVqUCpQZ0iLcsxhC R5o5x5H1iVbnYtTSUi8Pm1CnFCGjWOMpqg7ZTdkkH55R0UPNpy9jscxoLy5L3KOg xs8OSL+EAvh/40UFJWrHRbC9jIbj9L4ecy50fONR3DeqAKs5NRwn+wCpqJbDFvL2 IQa94pAf7sS+B1wIjCSYrfJF3ENPnKaKi52X2F8OAdYe+VTarrz/4uxEeFmDrakp ck2FhFnuMZYbNuBbehNfJTnz06BrKWggBpTQazybRiL1gjCUQKq/AHCISnj/7+Pd 3DE8UyKi27c5m0pid5lUw+P1BBcK6esi4tMrYpXx3gGTNrS9HZhx0VdXT3wv7Jno aXrp9OLTgCWH88qIT+J7ShAV++1apBJy1FD6VzhDtYI2gSoVQEd/PFpqPPipUkjO bF1MH9jkTpPEN77W89N5Iy+VlhSTqoYq2YF+pEIQuM9IWuycMQwpT2G2gP4MQTm7 htIG6R2C0CjBzOg98U9M0RxcEyjawZ84gRN84z4dP0uK7M16q4VAQhqzRpX9VDa5 YGD69+FR6/wxzNNzpWIPjOuutc8aKZTDB8JCNGFKi+uN8zAs/m2NneVng7MH3p8X Q2DblUyBYZCuW6MvrnDSSc+WU+ll/1gxphUxetPtz6lgT/TmF1q7+dza2T9IbHRa GFMKDFH4KI0HUbpVd+qs =njNG -----END PGP SIGNATURE----- --MW5yreqqjyrRcusr--