Date: Fri, 6 Jun 2014 18:16:19 -0700 From: Adrian Chadd <adrian@freebsd.org> To: Robert Watson <rwatson@freebsd.org>, Pawel Jakub Dawidek <pjd@freebsd.org>, "freebsd-arch@freebsd.org" <freebsd-arch@freebsd.org> Subject: Re: Seeing ENOTCAPABLE from write FDs in kqueue? Message-ID: <CAJ-VmommTpKOzntUbCj6=YjL5PLu03-Yfx49QdKPrB%2BsMVFsKg@mail.gmail.com> In-Reply-To: <CAJ-VmonJaqg=WV0eTxknOQr51E5qdhDU=MdoCywz-hwZ57jj6w@mail.gmail.com> References: <CAJ-VmonJaqg=WV0eTxknOQr51E5qdhDU=MdoCywz-hwZ57jj6w@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
ping? -a On 2 June 2014 10:58, Adrian Chadd <adrian@freebsd.org> wrote: > Hi guys/all, > > I've been tinkering with my RSS stuff where I have multiple listen > sockets, one per worker thread, all terminating high throughput TCP > transactions. It's all HTTP; I'm using libevent, evhttp and a very > small amount of glue code to achieve this. > > libevent craps out from time to time because occasionally one of the > events in kqueue returns ENOTCAPABLE. > > ENOTCAPABLE: ident=3D1686, filter=3D-2, flags=3D0x00004000, fflags=3D0x00= 000000, data=3D93 > > ENOTCAPABLE: ident=3D1324, filter=3D-2, flags=3D0x00004000, fflags=3D0x00= 000000, data=3D93 > > ENOTCAPABLE: ident=3D1740, filter=3D-2, flags=3D0x00004000, fflags=3D0x00= 000000, data=3D93 > > ENOTCAPABLE: ident=3D1628, filter=3D-2, flags=3D0x00004000, fflags=3D0x00= 000000, data=3D93 > > ENOTCAPABLE: ident=3D1199, filter=3D-2, flags=3D0x00004000, fflags=3D0x00= 000000, data=3D93 > > ENOTCAPABLE: ident=3D818, filter=3D-2, flags=3D0x00004000, fflags=3D0x000= 00000, data=3D93 > > ENOTCAPABLE: ident=3D389, filter=3D-2, flags=3D0x00004000, fflags=3D0x000= 00000, data=3D93 > > It's happening on the various data FDs; not on the listen socket. > > What I'm seeing from ktrace: > > 27770 rss-http CAP operation requires <CAP_EVENT>, descriptor holds > <CAP_MAC_GET,CAP_MAC_SET,CAP_SEM_GETVALUE,CAP_SEM_POST,CAP_SEM_WAIT,CAP_E= VENT,CAP_KQUEUE_EVENT,CAP_IOCTL,CAP_TTYHOOK,CAP_PDGETPID,CAP_PDWAIT,CAP_PDK= ILL,CAP_EXTATTR_DELETE,CAP_EXTATTR_GET,CAP_EXTATTR_LIST,CAP_EXTATTR_SET,CAP= _ACL_CHECK,CAP_ACL_DELETE,CAP_ACL_GET,CAP_ACL_SET,CAP_KQUEUE_CHANGE> > > 27770 rss-http CAP operation requires <CAP_EVENT>, descriptor holds <> > > 27770 rss-http CAP operation requires <CAP_EVENT>, descriptor holds <> > > 27770 rss-http CAP operation requires <CAP_EVENT>, descriptor holds <> > > 27770 rss-http CAP operation requires <CAP_EVENT>, descriptor holds <> > > 27770 rss-http CAP operation requires <CAP_EVENT>, descriptor holds <> > > .. so, why exactly would I be seeing this? Is this some capability > race where we have an FD setup but no capability yet assigned when > it's added into the kqueue set? > > It's happening under high throughput (> 30,000 TCP sessions a second.) > > Where would I continue debugging this? > > Thanks! > > > -a
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-VmommTpKOzntUbCj6=YjL5PLu03-Yfx49QdKPrB%2BsMVFsKg>