From owner-freebsd-current@freebsd.org Tue Jul 19 17:02:10 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AC64CB9EDF7 for ; Tue, 19 Jul 2016 17:02:10 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 87EC5186F for ; Tue, 19 Jul 2016 17:02:10 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 838BBB9EDF6; Tue, 19 Jul 2016 17:02:10 +0000 (UTC) Delivered-To: current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8338CB9EDF5 for ; Tue, 19 Jul 2016 17:02:10 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-it0-x241.google.com (mail-it0-x241.google.com [IPv6:2607:f8b0:4001:c0b::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4E939186E for ; Tue, 19 Jul 2016 17:02:10 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-it0-x241.google.com with SMTP id j124so1801980ith.3 for ; Tue, 19 Jul 2016 10:02:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=PeSGbd0eXpoePgcmJ11BB03KLajPnscl73PVZl0mSD8=; b=dgQRptZCA0SxqlHZKrzYK4pocwyYaDLNMDdStJzfhtDXQJ9s72QzSIlc1f2BfGh2w8 sVk1a+fVz5CZxnl5FbB575KcLgo8IG3CvY1FO23HAwZsOpnXKassZBSlxr3ofRLgSeMN 3y7+NaB2CIy10saGuVjQ50U4WGBlKOW9pQOWFkzpp/XqrlJTVTx6JIk048OoZMPyLLb2 8gYwTPUl08PChis0aiPahJNfe9gHbGEv/xRqmWSnK1pxPBXTnMRmZUWDGQ74X7CWb2Qe zEKGt5T8/ZPkcxsdVhI/SJly0l2Cxm+3VzsJ300OaQFn+/OvpbRvKLIIz/4RXsq39iLM WFng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=PeSGbd0eXpoePgcmJ11BB03KLajPnscl73PVZl0mSD8=; b=Py9k21aC6jhVooCZpKjF2T9lI4V2bXSmBloEe8umn0K3Emr+MqmoGxS5H+5y8ZiL9y JKS60ZxMdoFSp/JBWfg/cXSZKcelVVWlqluijligjrpcm6yP37a5eMrnBnHZ+4sKP/jY I+oZexHCvRd12nsa3751oYjdWzlH2z1KdvDDRgdgI9MpiMlgg8wjpNgRBoMmN3w8L7SV E3SBdW1tgzfK5Fw02MlypoXErNZfsDadfB5XGwWzO+BKhUanngdYCx8njQKLojVLXrlM /+7lJ9JC6VDkYbN9OKpRjiyzqQ8PvmsDu4GAUwM6QLRWL2eAomaRQv8NOtDJVJE5Z+Mt drjg== X-Gm-Message-State: ALyK8tKLTe00NHNOq7R1F4lGcyofQ2O9j+CwEkeIr4eaUSx6yL+hfiTq26MyqhZglCqmtWjeibXJf7io4ppTFw== X-Received: by 10.36.239.197 with SMTP id i188mr33867782ith.71.1468947729353; Tue, 19 Jul 2016 10:02:09 -0700 (PDT) MIME-Version: 1.0 Received: by 10.36.141.129 with HTTP; Tue, 19 Jul 2016 10:02:08 -0700 (PDT) In-Reply-To: References: From: Adrian Chadd Date: Tue, 19 Jul 2016 10:02:08 -0700 Message-ID: Subject: Re: (boost::)asio and kqueue problem To: Hartmut Brandt Cc: "current@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Jul 2016 17:02:10 -0000 heh, nice catch. Would you please file a PR so we don't forget? Thanks! -a On 19 July 2016 at 08:35, Hartmut Brandt wrote: > Hi, > > I'm trying to use asio (that's boost::asio without boost) to handle > listening sockets asynchronuosly. This appears not to work. There are also > some reports on the net about this problem. I was able to reproduce the > problem with a small C-programm that does the same steps as asio. The > relevant sequence of system calls is: > > kqueue() = 3 (0x3) > socket(PF_INET,SOCK_STREAM,6) = 4 (0x4) > setsockopt(0x4,0xffff,0x800,0x7fffffffea2c,0x4) = 0 (0x0) > kevent(3,{ 4,EVFILT_READ,EV_ADD|EV_CLEAR,0x0,0x0,0x0 > 4,EVFILT_WRITE,EV_ADD|EV_CLEAR,0x0,0x0,0x0 },2,0x0,0,0x0) = 0 (0x0) > setsockopt(0x4,0xffff,0x4,0x7fffffffea2c,0x4) = 0 (0x0) > bind(4,{ AF_INET 0.0.0.0:8080 },16) = 0 (0x0) > listen(0x4,0x80) = 0 (0x0) > ioctl(4,FIONBIO,0xffffea2c) = 0 (0x0) > kevent(3,{ 4,EVFILT_READ,EV_ADD|EV_CLEAR,0x0,0x0,0x0 > 4,EVFILT_WRITE,EV_ADD|EV_CLEAR,0x0,0x0,0x0 },2,0x0,0,0x0) = 0 (0x0) > kevent(3,0x0,0,0x7fffffffe5a0,32,0x0) ERR#4 'Interrupted system > call' > > The problem here is that asio registers each file descriptor with > EVFILT_READ and EVFILT_WRITE as soon as it is opened (first kevent call). > After bringing the socket into the listening state and when async_accept() > is called it registers the socket a second time. According to the man page > this is perfectly legal and can be used to modify the registration. > > With this sequence of calls kevent() does not return when a connection is > established successfully. > > I tracked down the problem and the reason is in soo_kqfilter(). This is > called for the first EVFILT_READ registration and decides based on the > SO_ACCEPTCONN flag which filter operations to use solisten_filtops or > soread_filtops. In this case it chooses soread_filtops. > > The second EVFILT_READ registration does not call soo_kqfilter() again, but > just updates the filter from the data and fflags field so the listening > socket ends up with the wrong filter operations. > > The attached patch fixes this (kind of) by using the f_touch operation > (currently used only by the user filter). The filt_sotouch() function > changes the operation pointer in the knote when the socket is now in the > listening state. I suppose that the required locking is already done in > kqueue_register(), but I'm not sure. Asynchronous accepting now works. > > A better fix would probably be to change the operation vector on all knotes > attached to the socket in solisten(), but I fear I don't have the necessary > understanding of the locking that is required for this. > > Could somebody with enough kqueue() knowledge look whether the patch is > correct lock-wise? > > Regards, > harti > > Index: kern_event.c > =================================================================== > --- kern_event.c (revision 302977) > +++ kern_event.c (working copy) > @@ -1350,8 +1350,8 @@ > KQ_UNLOCK(kq); > knl = kn_list_lock(kn); > kn->kn_kevent.udata = kev->udata; > - if (!fops->f_isfd && fops->f_touch != NULL) { > - fops->f_touch(kn, kev, EVENT_REGISTER); > + if (kn->kn_fop->f_touch != NULL) { > + kn->kn_fop->f_touch(kn, kev, EVENT_REGISTER); > } else { > kn->kn_sfflags = kev->fflags; > kn->kn_sdata = kev->data; > Index: uipc_socket.c > =================================================================== > --- uipc_socket.c (revision 302977) > +++ uipc_socket.c (working copy) > @@ -160,6 +160,7 @@ > static void filt_sowdetach(struct knote *kn); > static int filt_sowrite(struct knote *kn, long hint); > static int filt_solisten(struct knote *kn, long hint); > +static void filt_sotouch(struct knote *kn, struct kevent *kev, u_long > type); > static int inline hhook_run_socket(struct socket *so, void *hctx, int32_t > h_id); > fo_kqfilter_t soo_kqfilter; > > @@ -172,6 +173,7 @@ > .f_isfd = 1, > .f_detach = filt_sordetach, > .f_event = filt_soread, > + .f_touch = filt_sotouch, > }; > static struct filterops sowrite_filtops = { > .f_isfd = 1, > @@ -3091,6 +3093,31 @@ > return (0); > } > > +static void > +filt_sotouch(struct knote *kn, struct kevent *kev, u_long type) > +{ > + struct socket *so = kn->kn_fp->f_data; > + > + switch (type) { > + case EVENT_REGISTER: > + if (kn->kn_fop == &soread_filtops && > + (so->so_options & SO_ACCEPTCONN)) > + kn->kn_fop = &solisten_filtops; > + > + kn->kn_sfflags = kev->fflags; > + kn->kn_sdata = kev->data; > + break; > + > + case EVENT_PROCESS: > + *kev = kn->kn_kevent; > + break; > + > + default: > + panic("filt_sotouch() - invalid type (%ld)", type); > + break; > + } > +} > + > /* > * Some routines that return EOPNOTSUPP for entry points that are not > * supported by a protocol. Fill in as needed. > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"