Date: Fri, 28 Oct 2011 11:37:14 -0400 From: Ryan Stone <rysto32@gmail.com> To: FreeBSD Current <freebsd-current@freebsd.org> Subject: smp_rendezvous runs with interrupts and preemption enabled on unicore systems Message-ID: <CAFMmRNwWBE6idGJCsNPago4-N1ohAaCD02_D-LCMzDg8pbjE1A@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
I'm seeing issues on a unicore systems running a derivative of FreeBSD
8.2-RELEASE if something calls mem_range_attr_set. It turns out that
the root cause is a bug in smp_rendezvous_cpus. The first part of
smp_rendezvous_cpus attempts to short-circuit the non-SMP case(note
that smp_started is never set to 1 on a unicore system):
if (!smp_started) {
if (setup_func != NULL)
setup_func(arg);
if (action_func != NULL)
action_func(arg);
if (teardown_func != NULL)
teardown_func(arg);
return;
}
The problem is that this runs with interrupts enabled, outside of a
critical section. My system runs with device_polling enabled with hz
set to 2500, so its quite easy to wedge the system by having a thread
run mem_range_attr_set. That has to do a smp_rendezvous, and if a
timer interrupt happens to go off half-way through the action_func and
preempt this thread, the system ends up deadlocked(although once it's
wedged, typing at the serial console stands a good chance of unwedging
the system. Go figure).
I know that smp_rendezvous was reworked substantially on HEAD, but by
inspection it looks like the bug is still present, as the
short-circuit behaviour is still there.
I am not entirely sure of the best way to fix this. Is it as simple
as doing a spinlock_enter before setup_func and a spinlock_exit after
teardown_func? It seems to boot fine, but I'm not at all confident
that I understand the nuances of smp_rendezvous to be sure that there
aren't side effects that I don't know about.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFMmRNwWBE6idGJCsNPago4-N1ohAaCD02_D-LCMzDg8pbjE1A>