Date: Mon, 2 Jun 2014 10:58:42 -0700 From: Adrian Chadd <adrian@freebsd.org> To: Robert Watson <rwatson@freebsd.org>, Pawel Jakub Dawidek <pjd@freebsd.org>, "freebsd-arch@freebsd.org" <freebsd-arch@freebsd.org> Subject: Seeing ENOTCAPABLE from write FDs in kqueue? Message-ID: <CAJ-VmonJaqg=WV0eTxknOQr51E5qdhDU=MdoCywz-hwZ57jj6w@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Hi guys/all, I've been tinkering with my RSS stuff where I have multiple listen sockets, one per worker thread, all terminating high throughput TCP transactions. It's all HTTP; I'm using libevent, evhttp and a very small amount of glue code to achieve this. libevent craps out from time to time because occasionally one of the events in kqueue returns ENOTCAPABLE. ENOTCAPABLE: ident=3D1686, filter=3D-2, flags=3D0x00004000, fflags=3D0x0000= 0000, data=3D93 ENOTCAPABLE: ident=3D1324, filter=3D-2, flags=3D0x00004000, fflags=3D0x0000= 0000, data=3D93 ENOTCAPABLE: ident=3D1740, filter=3D-2, flags=3D0x00004000, fflags=3D0x0000= 0000, data=3D93 ENOTCAPABLE: ident=3D1628, filter=3D-2, flags=3D0x00004000, fflags=3D0x0000= 0000, data=3D93 ENOTCAPABLE: ident=3D1199, filter=3D-2, flags=3D0x00004000, fflags=3D0x0000= 0000, data=3D93 ENOTCAPABLE: ident=3D818, filter=3D-2, flags=3D0x00004000, fflags=3D0x00000= 000, data=3D93 ENOTCAPABLE: ident=3D389, filter=3D-2, flags=3D0x00004000, fflags=3D0x00000= 000, data=3D93 It's happening on the various data FDs; not on the listen socket. What I'm seeing from ktrace: 27770 rss-http CAP operation requires <CAP_EVENT>, descriptor holds <CAP_MAC_GET,CAP_MAC_SET,CAP_SEM_GETVALUE,CAP_SEM_POST,CAP_SEM_WAIT,CAP_EVE= NT,CAP_KQUEUE_EVENT,CAP_IOCTL,CAP_TTYHOOK,CAP_PDGETPID,CAP_PDWAIT,CAP_PDKIL= L,CAP_EXTATTR_DELETE,CAP_EXTATTR_GET,CAP_EXTATTR_LIST,CAP_EXTATTR_SET,CAP_A= CL_CHECK,CAP_ACL_DELETE,CAP_ACL_GET,CAP_ACL_SET,CAP_KQUEUE_CHANGE> 27770 rss-http CAP operation requires <CAP_EVENT>, descriptor holds <> 27770 rss-http CAP operation requires <CAP_EVENT>, descriptor holds <> 27770 rss-http CAP operation requires <CAP_EVENT>, descriptor holds <> 27770 rss-http CAP operation requires <CAP_EVENT>, descriptor holds <> 27770 rss-http CAP operation requires <CAP_EVENT>, descriptor holds <> .. so, why exactly would I be seeing this? Is this some capability race where we have an FD setup but no capability yet assigned when it's added into the kqueue set? It's happening under high throughput (> 30,000 TCP sessions a second.) Where would I continue debugging this? Thanks! -a
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-VmonJaqg=WV0eTxknOQr51E5qdhDU=MdoCywz-hwZ57jj6w>