Date: Mon, 15 Nov 2010 19:24:43 +0000 From: Paul LeoNerd Evans <leonerd@leonerd.org.uk> To: John Baldwin <jhb@freebsd.org> Cc: freebsd-hackers@freebsd.org Subject: Re: Managing userland data pointers in kqueue/kevent Message-ID: <20101115192443.GX11110@cel.leo> In-Reply-To: <201011151410.46130.jhb@freebsd.org> References: <20101112184000.GS11110@cel.leo> <201011151125.42697.jhb@freebsd.org> <20101115181211.GV11110@cel.leo> <201011151410.46130.jhb@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--VLyqDKa+9yql3vk0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Mon, Nov 15, 2010 at 02:10:45PM -0500, John Baldwin wrote:
> On Monday, November 15, 2010 1:12:11 pm Paul LeoNerd Evans wrote:
> > On Mon, Nov 15, 2010 at 11:25:42AM -0500, John Baldwin wrote:
> > > I think the assumption is that userland actually maintains a referenc=
e on the=20
> > > specified object (e.g. a file descriptor) and will know to drop the a=
ssociated=20
> > > data when the file descriptor is closed. That is, think of the keven=
t as a=20
> > > member of an eventable object rather than a separate object that has =
a=20
> > > reference to the eventable object. When the eventable object's refer=
ence=20
> > > count drops to zero in userland, then the kevent should be deleted, e=
ither via=20
> > > EV_DELETE, or implicitly (e.g. by closing the associated file descrip=
tor).
> >=20
> > Ah. Well, that could be considered a bit more awkward for the use case I
> > wanted to apply. The idea was that the udata would refer effectively
> > to a closure, to invoke when the event happens. The idea being you can
> > just add an event watcher by, say:
> >=20
> > $ev->EV_SET( $pid, EVFILT_PROC, 0, NOTE_EXIT, 0, sub {
> > print STDERR "The child process $pid has now exited\n";
> > } );
> >=20
> > So, the kernel's udata pointer effectively holds the only reference to
> > this anonymous closure. It's much more flexible this way, especially for
> > oneshot events like that.
> >=20
> > The beauty is also that the kevents() loop can simply know that the
> > udata is always a code reference so just has to invoke it to do whatever
> > the original caller wanted to do.
> >=20
> > Keep in mind my use-case here; I'm not trying to be one specific
> > application, it's a general-purpose kevent-wrapping library.
>=20
> So is GCD (Apple's libdispatch). It also implements closures on top of
> kevent. However, the way it works is that it doesn't expose kevent()
> directly, instead it uses kevent to implement asynchronous I/O on a=20
> socket for example, and since it is logically managing the life cycle
> of a socket, it knows when the socket is closed and cleans up then.
Well, the principle item of work here is a direct API reimplementation
in IO::KQueue on CPAN. I'm trying to simply expose the API of storing an
arbitrary Perl scalar in the udata field. It -could- be a closure, but
of course it doesn't have to. Maybe the using code wants to keep a HASH
ref of some pseudo-structure, or whatever...
> For the above case, if you know an event is one shot, you should either
> use EV_ONESHOT, or use a wrapper around the closure that clears the event
> after the closure runs (or possibly before the closure runs?)
I don't see how passing EV_ONESHOT at all helps here. If it's oneshot by
nature (child process exit), it'll be oneshot whatever happens. I've
already observed that the EV_ONESHOT flag does not get re-emitted by the
kernel anyway, so I'd have to track that one separately somehow.
> Your use case is rare. Almost all consumers of kevent() that I've seen
> use kevent() as one part of a system that maintain the lifecycle of objec=
ts.
> Those objects are only accessed within the system, so the system knows wh=
en
> an object is closed and can release the resources at the same time.
OK. I'm prepared to accept this. It may be that nobody's really tried to
provide a simple kqueue/kevent API wrapping for a high-level language,
that could nicely take advantage of memory management in the language,
rather than at the low C level of explicit malloc+free.
Could we perhaps address the second part of my question for a moment? If
there really isn't a general solution here, could my EV_FREEWATCH flag
be added? It's a single extra flag to export in the API .h file,
completely backward-compatible. Surely quite simple to implement in the
kernel too, because it already knows how to free() its own internal
structures anyway; so just before it does that when it deletes an event
for whatever reason, it could just fire that event back up to userland
to say "I've dropped this, and here have your pointer back". Userland
catches it, SvRECFNT_dec()s or whatever, problem solved.
Is that an API extension anyone would consider accepting? I for one
would use it.
--=20
Paul "LeoNerd" Evans
leonerd@leonerd.org.uk
ICQ# 4135350 | Registered Linux# 179460
http://www.leonerd.org.uk/
--VLyqDKa+9yql3vk0
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
iD8DBQFM4Yj7vLS2TC8cBo0RAoI0AKCvJS0Kd/f/+NEcRm39wbLJEA7J3wCbBLjm
dJVDke7vE7mOeDH9LH0znYk=
=8T/B
-----END PGP SIGNATURE-----
--VLyqDKa+9yql3vk0--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20101115192443.GX11110>