From owner-svn-src-head@freebsd.org Thu Feb 2 19:24:18 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B7B21CCD59B; Thu, 2 Feb 2017 19:24:18 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebi.us (glebi.us [96.95.210.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebi.us", Issuer "cell.glebi.us" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 69B2A1AE8; Thu, 2 Feb 2017 19:24:16 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebi.us (localhost [127.0.0.1]) by cell.glebi.us (8.15.2/8.15.2) with ESMTPS id v12JOCk1023436 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 2 Feb 2017 11:24:12 -0800 (PST) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebi.us (8.15.2/8.15.2/Submit) id v12JOBu6023435; Thu, 2 Feb 2017 11:24:11 -0800 (PST) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebi.us: glebius set sender to glebius@FreeBSD.org using -f Date: Thu, 2 Feb 2017 11:24:11 -0800 From: Gleb Smirnoff To: Hartmut.Brandt@dlr.de Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r313043 - head/sys/kern Message-ID: <20170202192411.GK3334@FreeBSD.org> References: <201702011312.v11DC7WJ085025@repo.freebsd.org> <20170201180816.GF3334@FreeBSD.org> <611243783F62AF48AFB07BC25FA4B1061CED9FD9@DLREXMBX01.intra.dlr.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <611243783F62AF48AFB07BC25FA4B1061CED9FD9@DLREXMBX01.intra.dlr.de> User-Agent: Mutt/1.7.2 (2016-11-26) X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Feb 2017 19:24:18 -0000 Hartmut, Thanks for explanation! Is there a test program available to reproduce the problem? I want to try the sequence on my branch. On Thu, Feb 02, 2017 at 08:29:20AM +0000, Hartmut.Brandt@dlr.de wrote: H> To be honest - I feared that when I saw your messages regarding this. Here is my original message from july. Attached is also a small test program. H> H> Hi, H> H> I'm trying to use asio (that's boost::asio without boost) to handle listening sockets asynchronuosly. This appears not to work. There are also some reports on the net about this problem. I was able to reproduce the problem with a small C-programm that does the same steps as asio. The relevant sequence of system calls is: H> H> kqueue() = 3 (0x3) H> socket(PF_INET,SOCK_STREAM,6) = 4 (0x4) H> setsockopt(0x4,0xffff,0x800,0x7fffffffea2c,0x4) = 0 (0x0) H> kevent(3,{ 4,EVFILT_READ,EV_ADD|EV_CLEAR,0x0,0x0,0x0 4,EVFILT_WRITE,EV_ADD|EV_CLEAR,0x0,0x0,0x0 },2,0x0,0,0x0) = 0 (0x0) H> setsockopt(0x4,0xffff,0x4,0x7fffffffea2c,0x4) = 0 (0x0) H> bind(4,{ AF_INET 0.0.0.0:8080 },16) = 0 (0x0) H> listen(0x4,0x80) = 0 (0x0) H> ioctl(4,FIONBIO,0xffffea2c) = 0 (0x0) H> kevent(3,{ 4,EVFILT_READ,EV_ADD|EV_CLEAR,0x0,0x0,0x0 4,EVFILT_WRITE,EV_ADD|EV_CLEAR,0x0,0x0,0x0 },2,0x0,0,0x0) = 0 (0x0) H> kevent(3,0x0,0,0x7fffffffe5a0,32,0x0) ERR#4 'Interrupted system call' H> H> The problem here is that asio registers each file descriptor with EVFILT_READ and EVFILT_WRITE as soon as it is opened (first kevent call). H> After bringing the socket into the listening state and when async_accept() is called it registers the socket a second time. According to the man page this is perfectly legal and can be used to modify the registration. H> H> With this sequence of calls kevent() does not return when a connection is established successfully. H> H> I tracked down the problem and the reason is in soo_kqfilter(). This is called for the first EVFILT_READ registration and decides based on the SO_ACCEPTCONN flag which filter operations to use solisten_filtops or soread_filtops. In this case it chooses soread_filtops. H> H> The second EVFILT_READ registration does not call soo_kqfilter() again, but just updates the filter from the data and fflags field so the listening socket ends up with the wrong filter operations. H> H> H> H> -----Original Message----- H> From: Gleb Smirnoff [mailto:glebius@FreeBSD.org] H> Sent: Wednesday, February 01, 2017 7:08 PM H> To: Hartmut Brandt H> Cc: src-committers@freebsd.org; svn-src-all@freebsd.org; svn-src-head@freebsd.org H> Subject: Re: svn commit: r313043 - head/sys/kern H> H> On Wed, Feb 01, 2017 at 01:12:07PM +0000, Hartmut Brandt wrote: H> H> Author: harti H> H> Date: Wed Feb 1 13:12:07 2017 H> H> New Revision: 313043 H> H> URL: https://svnweb.freebsd.org/changeset/base/313043 H> H> H> H> Log: H> H> Merge filt_soread and filt_solisten and decide what to do when checking H> H> for EVFILT_READ at the point of the check not when the event is registers. H> H> This fixes a problem with asio when accepting a connection. H> H> H> H> Reviewed by: kib@, Scott Mitchell H> H> This goes into opposite direction with what I am doing: H> H> https://reviews.freebsd.org/D9356 H> H> Can you please explain the problem with asio when accepting a connection? H> H> -- H> Totus tuus, Glebius. H> H> #include H> #include H> #include H> #include H> H> #include H> #include H> #include H> #include H> #include H> H> static void H> wait_loop(int kq, int sock) H> { H> struct kevent ev[32]; H> struct sockaddr_in addr; H> socklen_t socklen; H> H> for (;;) { H> int nev = kevent(kq, NULL, 0, ev, 32, NULL); H> if (nev < 1) H> err(1, "kevent"); H> for (int i = 0; i < nev; ++i) { H> if (ev[i].ident == sock) { H> printf("accept\n"); H> int fd = accept(ev[i].ident, H> (struct sockaddr *)&addr, &socklen); H> if (fd == -1) H> err(1, "accept"); H> } H> } H> } H> } H> H> int H> main() H> { H> struct sockaddr_in addr; H> H> /* open a TCP socket */ H> int kq = kqueue(); H> H> int sock = socket(PF_INET, SOCK_STREAM, 0); H> H> struct kevent ev[2]; H> EV_SET(&ev[0], sock, EVFILT_READ, EV_ADD | EV_CLEAR, 0, 0, NULL); H> EV_SET(&ev[1], sock, EVFILT_WRITE, EV_ADD | EV_CLEAR, 0, 0, NULL); H> H> int opt = 1; H> setsockopt(sock, SOL_SOCKET, SO_NOSIGPIPE, &opt, sizeof(opt)); H> H> if (kevent(kq, ev, 2, NULL, 0, NULL) == -1) H> err(1, "kevent"); H> H> setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt)); H> H> memset(&addr, 0, sizeof(addr)); H> addr.sin_port = htons(10000); H> H> bind(sock, (struct sockaddr *)&addr, sizeof(addr)); H> listen(sock, 0x80); H> H> ioctl(sock, FIONBIO, &opt); H> H> if (kevent(kq, ev, 2, NULL, 0, NULL) == -1) H> err(1, "kevent"); H> H> wait_loop(kq, sock); H> } -- Totus tuus, Glebius.