Date: Fri, 3 Feb 2017 07:51:24 +0000 From: <Hartmut.Brandt@dlr.de> To: <glebius@FreeBSD.org> Cc: <src-committers@freebsd.org>, <svn-src-all@freebsd.org>, <svn-src-head@freebsd.org> Subject: RE: svn commit: r313043 - head/sys/kern Message-ID: <611243783F62AF48AFB07BC25FA4B1061CEDA7DC@DLREXMBX01.intra.dlr.de> In-Reply-To: <20170202192411.GK3334@FreeBSD.org> References: <201702011312.v11DC7WJ085025@repo.freebsd.org> <20170201180816.GF3334@FreeBSD.org> <611243783F62AF48AFB07BC25FA4B1061CED9FD9@DLREXMBX01.intra.dlr.de> <20170202192411.GK3334@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--_002_611243783F62AF48AFB07BC25FA4B1061CEDA7DCDLREXMBX01intra_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
It was attached to my mail, but maybe got removed somewhere. Here it is. It=
does not use asio, but reproduces the same sequence of system calls. You s=
tart it and the try to connect with telnet to port 10000.
harti
#include <sys/socket.h>
#include <sys/types.h>
#include <sys/event.h>
#include <sys/filio.h>
#include <sys/ioctl.h>
#include <netinet/in.h>
#include <stdio.h>
#include <string.h>
#include <err.h>
static void
wait_loop(int kq, int sock)
{
struct kevent ev[32];
struct sockaddr_in addr;
socklen_t socklen;
for (;;) {
int nev =3D kevent(kq, NULL, 0, ev, 32, NULL);
if (nev < 1)
err(1, "kevent");
for (int i =3D 0; i < nev; ++i) {
if (ev[i].ident =3D=3D sock) {
printf("accept\n");
int fd =3D accept(ev[i].ident,
(struct sockaddr *)&addr, &socklen);
if (fd =3D=3D -1)
err(1, "accept");
}
}
}
}
int
main()
{
struct sockaddr_in addr;
/* open a TCP socket */
int kq =3D kqueue();
int sock =3D socket(PF_INET, SOCK_STREAM, 0);
struct kevent ev[2];
EV_SET(&ev[0], sock, EVFILT_READ, EV_ADD | EV_CLEAR, 0, 0, NULL);
EV_SET(&ev[1], sock, EVFILT_WRITE, EV_ADD | EV_CLEAR, 0, 0, NULL);
int opt =3D 1;
setsockopt(sock, SOL_SOCKET, SO_NOSIGPIPE, &opt, sizeof(opt));
if (kevent(kq, ev, 2, NULL, 0, NULL) =3D=3D -1)
err(1, "kevent");
setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));
memset(&addr, 0, sizeof(addr));
addr.sin_port =3D htons(10000);
bind(sock, (struct sockaddr *)&addr, sizeof(addr));
listen(sock, 0x80);
ioctl(sock, FIONBIO, &opt);
if (kevent(kq, ev, 2, NULL, 0, NULL) =3D=3D -1)
err(1, "kevent");
wait_loop(kq, sock);
}
-----Original Message-----
From: Gleb Smirnoff [mailto:glebius@FreeBSD.org]=20
Sent: Thursday, February 02, 2017 8:24 PM
To: Brandt, Hartmut
Cc: src-committers@freebsd.org; svn-src-all@freebsd.org; svn-src-head@freeb=
sd.org
Subject: Re: svn commit: r313043 - head/sys/kern
Hartmut,
Thanks for explanation! Is there a test program available to reproduce th=
e problem? I want to try the sequence on my branch.
On Thu, Feb 02, 2017 at 08:29:20AM +0000, Hartmut.Brandt@dlr.de wrote:
H> To be honest - I feared that when I saw your messages regarding this. He=
re is my original message from july. Attached is also a small test program.
H>=20
H> Hi,
H>=20
H> I'm trying to use asio (that's boost::asio without boost) to handle list=
ening sockets asynchronuosly. This appears not to work. There are also some=
reports on the net about this problem. I was able to reproduce the problem=
with a small C-programm that does the same steps as asio. The relevant seq=
uence of system calls is:
H>=20
H> kqueue() =3D 3 (0x3)
H> socket(PF_INET,SOCK_STREAM,6) =3D 4 (0x4)
H> setsockopt(0x4,0xffff,0x800,0x7fffffffea2c,0x4) =3D 0 (0x0)
H> kevent(3,{ 4,EVFILT_READ,EV_ADD|EV_CLEAR,0x0,0x0,0x0 4,EVFILT_WRITE,EV_A=
DD|EV_CLEAR,0x0,0x0,0x0 },2,0x0,0,0x0) =3D 0 (0x0)
H> setsockopt(0x4,0xffff,0x4,0x7fffffffea2c,0x4) =3D 0 (0x0)
H> bind(4,{ AF_INET 0.0.0.0:8080 },16) =3D 0 (0x0)
H> listen(0x4,0x80) =3D 0 (0x0)
H> ioctl(4,FIONBIO,0xffffea2c) =3D 0 (0x0)
H> kevent(3,{ 4,EVFILT_READ,EV_ADD|EV_CLEAR,0x0,0x0,0x0 4,EVFILT_WRITE,EV_A=
DD|EV_CLEAR,0x0,0x0,0x0 },2,0x0,0,0x0) =3D 0 (0x0)
H> kevent(3,0x0,0,0x7fffffffe5a0,32,0x0) ERR#4 'Interrupted system call'
H>=20
H> The problem here is that asio registers each file descriptor with EVFILT=
_READ and EVFILT_WRITE as soon as it is opened (first kevent call).=20
H> After bringing the socket into the listening state and when async_accept=
() is called it registers the socket a second time. According to the man pa=
ge this is perfectly legal and can be used to modify the registration.
H>=20
H> With this sequence of calls kevent() does not return when a connection i=
s established successfully.
H>=20
H> I tracked down the problem and the reason is in soo_kqfilter(). This is =
called for the first EVFILT_READ registration and decides based on the SO_A=
CCEPTCONN flag which filter operations to use solisten_filtops or soread_fi=
ltops. In this case it chooses soread_filtops.
H>=20
H> The second EVFILT_READ registration does not call soo_kqfilter() again, =
but just updates the filter from the data and fflags field so the listening=
socket ends up with the wrong filter operations.
H>=20
H>=20
H>=20
H> -----Original Message-----
H> From: Gleb Smirnoff [mailto:glebius@FreeBSD.org]
H> Sent: Wednesday, February 01, 2017 7:08 PM
H> To: Hartmut Brandt
H> Cc: src-committers@freebsd.org; svn-src-all@freebsd.org;=20
H> svn-src-head@freebsd.org
H> Subject: Re: svn commit: r313043 - head/sys/kern
H>=20
H> On Wed, Feb 01, 2017 at 01:12:07PM +0000, Hartmut Brandt wrote:
H> H> Author: harti
H> H> Date: Wed Feb 1 13:12:07 2017
H> H> New Revision: 313043
H> H> URL: https://svnweb.freebsd.org/changeset/base/313043
H> H>=20
H> H> Log:
H> H> Merge filt_soread and filt_solisten and decide what to do when chec=
king
H> H> for EVFILT_READ at the point of the check not when the event is reg=
isters.
H> H> This fixes a problem with asio when accepting a connection.
H> H> =20
H> H> Reviewed by: kib@, Scott Mitchell
H>=20
H> This goes into opposite direction with what I am doing:
H>=20
H> https://reviews.freebsd.org/D9356
H>=20
H> Can you please explain the problem with asio when accepting a connection=
?
H>=20
H> --
H> Totus tuus, Glebius.
H>=20
H> #include <sys/socket.h>
H> #include <sys/types.h>
H> #include <sys/event.h>
H> #include <sys/filio.h>
H>=20
H> #include <sys/ioctl.h>
H> #include <netinet/in.h>
H> #include <stdio.h>
H> #include <string.h>
H> #include <err.h>
H>=20
H> static void
H> wait_loop(int kq, int sock)
H> {
H> struct kevent ev[32];
H> struct sockaddr_in addr;
H> socklen_t socklen;
H>=20
H> for (;;) {
H> int nev =3D kevent(kq, NULL, 0, ev, 32, NULL);
H> if (nev < 1)
H> err(1, "kevent");
H> for (int i =3D 0; i < nev; ++i) {
H> if (ev[i].ident =3D=3D sock) {
H> printf("accept\n");
H> int fd =3D accept(ev[i].ident,
H> (struct sockaddr *)&addr, &socklen);
H> if (fd =3D=3D -1)
H> err(1, "accept");
H> }
H> }
H> }
H> }
H>=20
H> int
H> main()
H> {
H> struct sockaddr_in addr;
H>=20
H> /* open a TCP socket */
H> int kq =3D kqueue();
H>=20
H> int sock =3D socket(PF_INET, SOCK_STREAM, 0);
H>=20
H> struct kevent ev[2];
H> EV_SET(&ev[0], sock, EVFILT_READ, EV_ADD | EV_CLEAR, 0, 0, NULL);
H> EV_SET(&ev[1], sock, EVFILT_WRITE, EV_ADD | EV_CLEAR, 0, 0, NULL);
H>=20
H> int opt =3D 1;
H> setsockopt(sock, SOL_SOCKET, SO_NOSIGPIPE, &opt, sizeof(opt));
H>=20
H> if (kevent(kq, ev, 2, NULL, 0, NULL) =3D=3D -1)
H> err(1, "kevent");
H>=20
H> setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));
H>=20
H> memset(&addr, 0, sizeof(addr));
H> addr.sin_port =3D htons(10000);
H>=20
H> bind(sock, (struct sockaddr *)&addr, sizeof(addr));
H> listen(sock, 0x80);
H>=20
H> ioctl(sock, FIONBIO, &opt);
H>=20
H> if (kevent(kq, ev, 2, NULL, 0, NULL) =3D=3D -1)
H> err(1, "kevent");
H>=20
H> wait_loop(kq, sock);
H> }
--=20
Totus tuus, Glebius.
--_002_611243783F62AF48AFB07BC25FA4B1061CEDA7DCDLREXMBX01intra_
Content-Type: text/plain; name="k.c"
Content-Description: k.c
Content-Disposition: attachment; filename="k.c"; size=1459;
creation-date="Fri, 03 Feb 2017 07:47:17 GMT";
modification-date="Fri, 03 Feb 2017 07:47:17 GMT"
Content-Transfer-Encoding: base64
I2luY2x1ZGUgPHN5cy9zb2NrZXQuaD4NCiNpbmNsdWRlIDxzeXMvdHlwZXMuaD4NCiNpbmNsdWRl
IDxzeXMvZXZlbnQuaD4NCiNpbmNsdWRlIDxzeXMvZmlsaW8uaD4NCg0KI2luY2x1ZGUgPHN5cy9p
b2N0bC5oPg0KI2luY2x1ZGUgPG5ldGluZXQvaW4uaD4NCiNpbmNsdWRlIDxzdGRpby5oPg0KI2lu
Y2x1ZGUgPHN0cmluZy5oPg0KI2luY2x1ZGUgPGVyci5oPg0KDQpzdGF0aWMgdm9pZA0Kd2FpdF9s
b29wKGludCBrcSwgaW50IHNvY2spDQp7DQoJc3RydWN0IGtldmVudCBldlszMl07DQoJc3RydWN0
IHNvY2thZGRyX2luIGFkZHI7DQoJc29ja2xlbl90IHNvY2tsZW47DQoNCglmb3IgKDs7KSB7DQoJ
CWludCBuZXYgPSBrZXZlbnQoa3EsIE5VTEwsIDAsIGV2LCAzMiwgTlVMTCk7DQoJCWlmIChuZXYg
PCAxKQ0KCQkJZXJyKDEsICJrZXZlbnQiKTsNCgkJZm9yIChpbnQgaSA9IDA7IGkgPCBuZXY7ICsr
aSkgew0KCQkJaWYgKGV2W2ldLmlkZW50ID09IHNvY2spIHsNCgkJCQlwcmludGYoImFjY2VwdFxu
Iik7DQoJCQkJaW50IGZkID0gYWNjZXB0KGV2W2ldLmlkZW50LA0KCQkJCSAgICAoc3RydWN0IHNv
Y2thZGRyICopJmFkZHIsICZzb2NrbGVuKTsNCgkJCQlpZiAoZmQgPT0gLTEpDQoJCQkJCWVycigx
LCAiYWNjZXB0Iik7DQoJCQl9DQoJCX0NCgl9DQp9DQoNCmludA0KbWFpbigpDQp7DQoJc3RydWN0
IHNvY2thZGRyX2luIGFkZHI7DQoNCgkvKiBvcGVuIGEgVENQIHNvY2tldCAqLw0KCWludCBrcSA9
IGtxdWV1ZSgpOw0KDQoJaW50IHNvY2sgPSBzb2NrZXQoUEZfSU5FVCwgU09DS19TVFJFQU0sIDAp
Ow0KDQoJc3RydWN0IGtldmVudCBldlsyXTsNCglFVl9TRVQoJmV2WzBdLCBzb2NrLCBFVkZJTFRf
UkVBRCwgRVZfQUREIHwgRVZfQ0xFQVIsIDAsIDAsIE5VTEwpOw0KCUVWX1NFVCgmZXZbMV0sIHNv
Y2ssIEVWRklMVF9XUklURSwgRVZfQUREIHwgRVZfQ0xFQVIsIDAsIDAsIE5VTEwpOw0KDQoJaW50
IG9wdCA9IDE7DQoJc2V0c29ja29wdChzb2NrLCBTT0xfU09DS0VULCBTT19OT1NJR1BJUEUsICZv
cHQsIHNpemVvZihvcHQpKTsNCg0KCWlmIChrZXZlbnQoa3EsIGV2LCAyLCBOVUxMLCAwLCBOVUxM
KSA9PSAtMSkNCgkgICAgZXJyKDEsICJrZXZlbnQiKTsNCg0KCXNldHNvY2tvcHQoc29jaywgU09M
X1NPQ0tFVCwgU09fUkVVU0VBRERSLCAmb3B0LCBzaXplb2Yob3B0KSk7DQoNCgltZW1zZXQoJmFk
ZHIsIDAsIHNpemVvZihhZGRyKSk7DQoJYWRkci5zaW5fcG9ydCA9IGh0b25zKDEwMDAwKTsNCg0K
CWJpbmQoc29jaywgKHN0cnVjdCBzb2NrYWRkciAqKSZhZGRyLCBzaXplb2YoYWRkcikpOw0KCWxp
c3Rlbihzb2NrLCAweDgwKTsNCg0KCWlvY3RsKHNvY2ssIEZJT05CSU8sICZvcHQpOw0KDQoJaWYg
KGtldmVudChrcSwgZXYsIDIsIE5VTEwsIDAsIE5VTEwpID09IC0xKQ0KCQllcnIoMSwgImtldmVu
dCIpOw0KDQoJd2FpdF9sb29wKGtxLCBzb2NrKTsNCn0NCg==
--_002_611243783F62AF48AFB07BC25FA4B1061CEDA7DCDLREXMBX01intra_--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?611243783F62AF48AFB07BC25FA4B1061CEDA7DC>