From owner-freebsd-current@freebsd.org Wed Jul 20 05:59:34 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B759CB9FFCA for ; Wed, 20 Jul 2016 05:59:34 +0000 (UTC) (envelope-from sepherosa@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 97819166F for ; Wed, 20 Jul 2016 05:59:34 +0000 (UTC) (envelope-from sepherosa@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 9688CB9FFC9; Wed, 20 Jul 2016 05:59:34 +0000 (UTC) Delivered-To: current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 962ECB9FFC8 for ; Wed, 20 Jul 2016 05:59:34 +0000 (UTC) (envelope-from sepherosa@gmail.com) Received: from mail-vk0-x233.google.com (mail-vk0-x233.google.com [IPv6:2607:f8b0:400c:c05::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FDB7166E for ; Wed, 20 Jul 2016 05:59:34 +0000 (UTC) (envelope-from sepherosa@gmail.com) Received: by mail-vk0-x233.google.com with SMTP id x130so55394962vkc.0 for ; Tue, 19 Jul 2016 22:59:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=HAeKt0LdHz9/qz9QOF63XBUrXZc9yqSOWiHX+7zQsPQ=; b=cQ/7rBBt8p535daL6etgCrv5AYxsVwteHS8cQqALEv1PlbyYKB0toYZ7w5EvWSJHLi hvlPVSIxjaRcpiQwQpMgTq5cMCO6bE1J7EjvNew4HUS2Kd4zZYeEqN8pBzEL4D49Dkie n9mBxaAPBRrM/vHc21lysKOHKjtyrhNIoecOtcKJoglfMcGDVMKl6rqnGPk/hWuIfQJG GSFKG6P3QOnx4Gl1kqxIM2YDzBeAbxUr7VT671MAs5cbrCpLWRMTmXly//OseT0v4WDs Ziy3yE3bkq20SeQXp9ZXvRYsStYkwvjOVe+7VlCrUK1H9199GE4+YHb70ePnOgdAf7dA 038g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=HAeKt0LdHz9/qz9QOF63XBUrXZc9yqSOWiHX+7zQsPQ=; b=cRUZfI4m76aVM1trU7DOM2sF+6Ot2KQtAM0oAh3VCU429WF2z+Zz/eVakoOERTUj8F DNT4VlLJqCzqeiZ7ns4C9d5GrJNteZpjEe6wG0CwlywtLgghQvv/K4GH0HIOWBhjEJ2J x7b0VadgobVcR5M0T66O0LTet7mqfzTsWduoxeDNu3thulaDN4IGaNKvwhevIoaMlDWI u6N8cdzSkO5KEmCV8XPS2a3OjjJgH5bJyXQR2GW+skGjMpVn7UUPDDFsry3MCWSU/UYb S7m0+XJvs0lEqzjNubrfANlAkTQvcjJD4JQ+M7bxzFwRkn30Wwk2P6IlQDq57ZJ+/VD1 GAGg== X-Gm-Message-State: ALyK8tLltCIqDmPiQX/GrAxM4Dez5fKZb/L+V+OJkqfP067BvLvcjqO74LFQtDahuQzbtv1yOOmmtttxDm/QRg== X-Received: by 10.31.92.78 with SMTP id q75mr17653240vkb.112.1468994373284; Tue, 19 Jul 2016 22:59:33 -0700 (PDT) MIME-Version: 1.0 Received: by 10.176.83.101 with HTTP; Tue, 19 Jul 2016 22:59:32 -0700 (PDT) In-Reply-To: <20160720043809.GP38613@kib.kiev.ua> References: <20160720043809.GP38613@kib.kiev.ua> From: Sepherosa Ziehau Date: Wed, 20 Jul 2016 13:59:32 +0800 Message-ID: Subject: Re: (boost::)asio and kqueue problem To: Konstantin Belousov Cc: Hartmut Brandt , "freebsd-current@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Jul 2016 05:59:34 -0000 On Wed, Jul 20, 2016 at 12:38 PM, Konstantin Belousov wrote: > On Tue, Jul 19, 2016 at 05:35:59PM +0200, Hartmut Brandt wrote: >> Hi, >> >> I'm trying to use asio (that's boost::asio without boost) to handle >> listening sockets asynchronuosly. This appears not to work. There are also >> some reports on the net about this problem. I was able to reproduce the >> problem with a small C-programm that does the same steps as asio. The >> relevant sequence of system calls is: >> >> kqueue() = 3 (0x3) >> socket(PF_INET,SOCK_STREAM,6) = 4 (0x4) >> setsockopt(0x4,0xffff,0x800,0x7fffffffea2c,0x4) = 0 (0x0) >> kevent(3,{ 4,EVFILT_READ,EV_ADD|EV_CLEAR,0x0,0x0,0x0 4,EVFILT_WRITE,EV_ADD|EV_CLEAR,0x0,0x0,0x0 },2,0x0,0,0x0) = 0 (0x0) >> setsockopt(0x4,0xffff,0x4,0x7fffffffea2c,0x4) = 0 (0x0) >> bind(4,{ AF_INET 0.0.0.0:8080 },16) = 0 (0x0) >> listen(0x4,0x80) = 0 (0x0) >> ioctl(4,FIONBIO,0xffffea2c) = 0 (0x0) >> kevent(3,{ 4,EVFILT_READ,EV_ADD|EV_CLEAR,0x0,0x0,0x0 4,EVFILT_WRITE,EV_ADD|EV_CLEAR,0x0,0x0,0x0 },2,0x0,0,0x0) = 0 (0x0) >> kevent(3,0x0,0,0x7fffffffe5a0,32,0x0) ERR#4 'Interrupted system call' >> >> The problem here is that asio registers each file descriptor with >> EVFILT_READ and EVFILT_WRITE as soon as it is opened (first kevent call). >> After bringing the socket into the listening state and when async_accept() >> is called it registers the socket a second time. According to the man page >> this is perfectly legal and can be used to modify the registration. >> >> With this sequence of calls kevent() does not return when a connection is >> established successfully. >> >> I tracked down the problem and the reason is in soo_kqfilter(). This is >> called for the first EVFILT_READ registration and decides based on the >> SO_ACCEPTCONN flag which filter operations to use solisten_filtops or >> soread_filtops. In this case it chooses soread_filtops. >> >> The second EVFILT_READ registration does not call soo_kqfilter() again, >> but just updates the filter from the data and fflags field so the >> listening socket ends up with the wrong filter operations. >> >> The attached patch fixes this (kind of) by using the f_touch >> operation (currently used only by the user filter). The filt_sotouch() >> function changes the operation pointer in the knote when the socket is now >> in the listening state. I suppose that the required locking is already >> done in kqueue_register(), but I'm not sure. Asynchronous accepting now works. >> >> A better fix would probably be to change the operation vector on all >> knotes attached to the socket in solisten(), but I fear I don't have the >> necessary understanding of the locking that is required for this. >> >> Could somebody with enough kqueue() knowledge look whether the patch is >> correct lock-wise? > I find it weird that the fix still requires re-registration of the socket > event to get it working after socket is marked as listen. In other words, > until the re-registration is done, the events for the registered filter > are lost. > > IMO more correct solution would be to merge the filt_solisten and > filt_soread, deciding which path to take by testing the SO_ACCEPTCON > flag in the f_event() op. This is reasonable. Thanks, sephe -- Tomorrow Will Never Die