Date: Thu, 19 Jul 2001 00:47:45 -0700 From: Terry Lambert <tlambert2@mindspring.com> To: Louis-Philippe Gagnon <louisphilippe@macadamian.com> Cc: Julian Elischer <julian@elischer.org>, freebsd-hackers@FreeBSD.ORG Subject: Re: flock/pthread bug? Message-ID: <3B5690A1.8EDD0812@mindspring.com> References: <Pine.BSF.4.21.0107181320030.94740-100000@InterJet.elischer.org> <1fbd01c10fba$499834d0$2964a8c0@macadamian.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Louis-Philippe Gagnon wrote:
>
> From: "Julian Elischer" <julian@elischer.org>
> > probably you should try :
> >
> > #define LOCK_NB 0x04 /* don't block when locking */
> >
> But I do want to block; I just don't want the whole process to block.
You can't block just a thread, since there is no "queue a
lock request" interface, only a "try to get the lock, and
return if you can't" or a "block and wait for the lock".
In other words, flock() is one call that can't really be
reasonably implemented by the threads library, since
there is no way to map it safely, and still guarantee
against an order inversion deadlock.
> > Also if you have shared memory, why not use
> >
> > /* Get a spin lock, handle recursion inline (as the less common case) */
Using the "cmpxchgl" instruction is your best approach;
I personally would not use the whole thing, as Julian
suggested, since I think the SMP synchronization ops
are much, much heavier than they should be. You don't
need that much overhead, you don't need reentrancy, and
you don't need it to do counting (neither does SMP).
The easiest way to get shared memory is to use mmap() in
both programs on a file, and to madvise() the caching and
cache writeback off. This is much beeter than SYSV shared
memory (shmat, shmget, etc.), since it is properly resource
tracked, and you will not have a startup order of operation
problem, nor will you have the common problem of trying to
get rid of the segments, should any of your programs exit
abnormally.
> I'd rather have something portable though, and I would
> still like to know if what I was doing should have worked...
The answer is "no". Threads are not processes, they exist
in a single process context, as they currently exist. In
the future, what you are doing will work, but will have a
very high comparative overhead to other approaches.
If you read Stevens, you will see that flock() is much
higher overhead than doing semop() calls.
To wrap either, you would need to:
retry:
st = try_non_blocking_get();
if (st == FAIL) {
yield();
goto retry;
}
Note that the "yield" system call is non-standard, and you
will buzz-loop somewhat, depressing your priority. You
will also have a race window, subject to the "thundering
herd" problem. You could put a "sleep(1);" in place of
the "yield();" call... this would eliminate the buzzing
(somewhat), but would widen the race window.
Another alternative is to use a multiended pipe, where
you write a single character token down the pipe, and
your programs block waiting for a single character read
to complete (you could use a FIFO or socket, in place of
the pipe). The point is that only a single reader can
get the character (the "token") at a time.
Unfortuantely, since file I/O is implemented as a
conversion of a blocking call to a non-blocking call
plus a context switch (normal for a threads library),
you still have the "thundering herd" problem.
Further, if your intent was to have the callers serviced
in the order they called the read, you can't: you will
never have a guarantee about who will be permitted to
complete the read first.
So, for FIFO ordering, there's no way for you to solve
the problem; for semop or flock, you can solve the problem
portably with a "sleep - retry" loop, and not burn too
many cycles (but still lose FIFO ordering).
If you absolutely _must_ have a FIFO, then you need to
either wait, or install the Linux threads port (Linux
threads are hideously expensive, compared to the current
FreeBSD threads implementation, since they effectively
create a new process for each thread, just like Linux
does), and use blocking operations using them.
Probably, if you need FIFO ordering, whatever you are
doing would be better implemented as a finite state
automaton, or a "work-to-do" model, where you don't
really care who wins a race in a "thundering herd",
since all of your programs waiting on the condition
are identical.
Worse comes to worse, you should consider implementing
your threads as real seperate processes (and not fake
ones, like in the Linux threads library).
-- Terry
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3B5690A1.8EDD0812>