Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 13 May 2013 08:47:57 -0700
From:      Adrian Chadd <adrian@freebsd.org>
To:        Eugen-Andrei Gavriloaie <shiretu@gmail.com>
Cc:        freebsd-hackers@freebsd.org
Subject:   Re: Managing userland data pointers in kqueue/kevent
Message-ID:  <CAJ-Vmo=CVeXpf9WNOegD3yG9Q0NwUWaLadVrv1RgeyAaHYADiQ@mail.gmail.com>
In-Reply-To: <84DCA050-99D4-4B22-A031-35E0928709E0@gmail.com>
References:  <CCE4FFC4-F846-4F81-85EE-776B753C63C6@gmail.com> <CAJ-VmokBfBvdLa_Wf2EajF%2BvecVntLDaxdVeNvhAOiPp6HkjNA@mail.gmail.com> <84DCA050-99D4-4B22-A031-35E0928709E0@gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
... holy crap.

On 13 May 2013 08:37, Eugen-Andrei Gavriloaie <shiretu@gmail.com> wrote:
> Hi,
>
> Well, Paul already asked this question like 3-4 times now. Even insisting=
 on it. I will also ask it again:
> If user code is responsible of tracking down the data associated with the=
 signalled entity, what is the point of having user data?
> Is rendered completely useless=85

.. why does everything have to have a well defined purpose that is
also suited for use in _all_ situations?

> Not to mention, that your suggestion with FD index is a definite no-go. T=
he FD values are re-used. Especially in MT environments. Imagine one kqueue=
 call taking place in thread A and another one in thread B. Both threads wa=
iting for events.

.. so don't do that. I mean, you're already having to write your code
to _not_ touch FDs in other threads. I've done this before, it isn't
that hard and it doesn't hurt performance.

> When A does his magic, because of internal business rules, it decides to =
close FD number 123. It closes it and it connects somewhere else by opening=
 a new one. Surprise, we MAY  get the value 123 again as a new socket, we p=
ut it on our index, etc. Now, thread B comes in and it has stale/old events=
 for the old 123 FD. Somethings bad like EOF for the OLD version of FD numb=
er 123 (the one we just closed anyway). Guess what=85 thread B will dealloc=
ate the perfectly good thingy inside the index associated with 123.

So you just ensure that nothing at all calls a close(123); but calls
fd_close(123) which will in turn close(123) and free all the state
associated with it.

You have fd_close() either grab a lock, or you ensure that only the
owning thread can call fd_close(123) and if any other thread calls it,
the behaviour is undefined.

> And regarding the "thread happiness", that is not happiness at all IMHO=
=85

Unless you're writing a high connection throughput web server, the
overhead of grabbing a lock in userland during the fd shutdown process
is trivial. Yes, I've written those. It doesn't hurt you that much.

I'm confused as to why this is still an issue. Sure, fix the kqueue
semantics and do it in a way that doesn't break backwards
compatibility. But please don't claim that it's stopping you from
getting real work done. I've written network apps with kqueue that
scales to 8+ cores and (back in mid-2000's) gigabit + of small HTTP
transactions. This stuff isn't at all problematic.


Adrian



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-Vmo=CVeXpf9WNOegD3yG9Q0NwUWaLadVrv1RgeyAaHYADiQ>