From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 15 19:24:45 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A58AB1065672; Mon, 15 Nov 2010 19:24:45 +0000 (UTC) (envelope-from leonerd@leonerd.org.uk) Received: from cel.leonerd.org.uk (cel.leonerd.org.uk [81.187.167.226]) by mx1.freebsd.org (Postfix) with ESMTP id 479928FC27; Mon, 15 Nov 2010 19:24:45 +0000 (UTC) Received: by cel.leonerd.org.uk (Postfix, from userid 1000) id 0FFF340AA; Mon, 15 Nov 2010 19:24:43 +0000 (GMT) Date: Mon, 15 Nov 2010 19:24:43 +0000 From: Paul LeoNerd Evans To: John Baldwin Message-ID: <20101115192443.GX11110@cel.leo> References: <20101112184000.GS11110@cel.leo> <201011151125.42697.jhb@freebsd.org> <20101115181211.GV11110@cel.leo> <201011151410.46130.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="VLyqDKa+9yql3vk0" Content-Disposition: inline In-Reply-To: <201011151410.46130.jhb@freebsd.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-hackers@freebsd.org Subject: Re: Managing userland data pointers in kqueue/kevent X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Nov 2010 19:24:45 -0000 --VLyqDKa+9yql3vk0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Nov 15, 2010 at 02:10:45PM -0500, John Baldwin wrote: > On Monday, November 15, 2010 1:12:11 pm Paul LeoNerd Evans wrote: > > On Mon, Nov 15, 2010 at 11:25:42AM -0500, John Baldwin wrote: > > > I think the assumption is that userland actually maintains a referenc= e on the=20 > > > specified object (e.g. a file descriptor) and will know to drop the a= ssociated=20 > > > data when the file descriptor is closed. That is, think of the keven= t as a=20 > > > member of an eventable object rather than a separate object that has = a=20 > > > reference to the eventable object. When the eventable object's refer= ence=20 > > > count drops to zero in userland, then the kevent should be deleted, e= ither via=20 > > > EV_DELETE, or implicitly (e.g. by closing the associated file descrip= tor). > >=20 > > Ah. Well, that could be considered a bit more awkward for the use case I > > wanted to apply. The idea was that the udata would refer effectively > > to a closure, to invoke when the event happens. The idea being you can > > just add an event watcher by, say: > >=20 > > $ev->EV_SET( $pid, EVFILT_PROC, 0, NOTE_EXIT, 0, sub { > > print STDERR "The child process $pid has now exited\n"; > > } ); > >=20 > > So, the kernel's udata pointer effectively holds the only reference to > > this anonymous closure. It's much more flexible this way, especially for > > oneshot events like that. > >=20 > > The beauty is also that the kevents() loop can simply know that the > > udata is always a code reference so just has to invoke it to do whatever > > the original caller wanted to do. > >=20 > > Keep in mind my use-case here; I'm not trying to be one specific > > application, it's a general-purpose kevent-wrapping library. >=20 > So is GCD (Apple's libdispatch). It also implements closures on top of > kevent. However, the way it works is that it doesn't expose kevent() > directly, instead it uses kevent to implement asynchronous I/O on a=20 > socket for example, and since it is logically managing the life cycle > of a socket, it knows when the socket is closed and cleans up then. Well, the principle item of work here is a direct API reimplementation in IO::KQueue on CPAN. I'm trying to simply expose the API of storing an arbitrary Perl scalar in the udata field. It -could- be a closure, but of course it doesn't have to. Maybe the using code wants to keep a HASH ref of some pseudo-structure, or whatever... > For the above case, if you know an event is one shot, you should either > use EV_ONESHOT, or use a wrapper around the closure that clears the event > after the closure runs (or possibly before the closure runs?) I don't see how passing EV_ONESHOT at all helps here. If it's oneshot by nature (child process exit), it'll be oneshot whatever happens. I've already observed that the EV_ONESHOT flag does not get re-emitted by the kernel anyway, so I'd have to track that one separately somehow. > Your use case is rare. Almost all consumers of kevent() that I've seen > use kevent() as one part of a system that maintain the lifecycle of objec= ts. > Those objects are only accessed within the system, so the system knows wh= en > an object is closed and can release the resources at the same time. OK. I'm prepared to accept this. It may be that nobody's really tried to provide a simple kqueue/kevent API wrapping for a high-level language, that could nicely take advantage of memory management in the language, rather than at the low C level of explicit malloc+free. Could we perhaps address the second part of my question for a moment? If there really isn't a general solution here, could my EV_FREEWATCH flag be added? It's a single extra flag to export in the API .h file, completely backward-compatible. Surely quite simple to implement in the kernel too, because it already knows how to free() its own internal structures anyway; so just before it does that when it deletes an event for whatever reason, it could just fire that event back up to userland to say "I've dropped this, and here have your pointer back". Userland catches it, SvRECFNT_dec()s or whatever, problem solved. Is that an API extension anyone would consider accepting? I for one would use it. --=20 Paul "LeoNerd" Evans leonerd@leonerd.org.uk ICQ# 4135350 | Registered Linux# 179460 http://www.leonerd.org.uk/ --VLyqDKa+9yql3vk0 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iD8DBQFM4Yj7vLS2TC8cBo0RAoI0AKCvJS0Kd/f/+NEcRm39wbLJEA7J3wCbBLjm dJVDke7vE7mOeDH9LH0znYk= =8T/B -----END PGP SIGNATURE----- --VLyqDKa+9yql3vk0--