Date: Sun, 17 Sep 2006 14:04:26 -0700 From: John-Mark Gurney <gurney_j@resnet.uoregon.edu> To: freebsd-arch@FreeBSD.org Cc: freebsd-current@FreeBSD.org Subject: kqueue disable on delivery... Message-ID: <20060917210426.GI9421@funkthat.com>
next in thread | raw e-mail | index | archive | help
I have implemented a couple additional features to kqueue. These allow kqueue to be a multithreaded event delivery system that can guarantee that the event will only be active in one thread at any time. The first is EV_DOD, aka disable on delivery. When the event will be delivered to userland, the knote is marked disabled so we don't have to go through the expense of reallocing the knote each time. (Reallocation of the knote is also lock intensive, and disabling is cheap.) Even though this means that the event will only ever be active in a thread at a time, (when you're done handling the event, you reenable it), removing the event from the queue outside the event handler (say a timeout handler for the connection) poses to be a problem. If you simply close the socket, the event disappears, but then there is a race between another event being created with the same socket, and notification of the handler that you want the event to stop. In order to handle that situation, I have come up w/ EV_FORCEOS, aka FORCE ONE_SHOT. EV_ONESHOT events have the advantage that once queued, they don't care if they have been activated or not, they will be returned the next round. This means that the timeout handler can safely set EV_FORCEOS on the handler, and either if it's _DISABLED (handler running and will reenable it), or it's _ENABLED, it will get dispatched, allowing the handler to detect the EV_FORCEOS flag and teardown the connection. I have expanded a custom web server to make use of these features and so it's been stable. I believe most of the bugs are concurency bugs going from a single threaded kqueue design, to allowing multiple threads access to some of the data structures. The patch is available at: http://people.FreeBSD.org/~jmg/kqueue.dod.patch As part of the patch, I added more strict checking of the flags, so that unknown flags will return an EINVAL. Comments? Suggestions? -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20060917210426.GI9421>