Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 28 Oct 2011 11:37:14 -0400
From:      Ryan Stone <rysto32@gmail.com>
To:        FreeBSD Current <freebsd-current@freebsd.org>
Subject:   smp_rendezvous runs with interrupts and preemption enabled on unicore systems
Message-ID:  <CAFMmRNwWBE6idGJCsNPago4-N1ohAaCD02_D-LCMzDg8pbjE1A@mail.gmail.com>

next in thread | raw e-mail | index | archive | help
I'm seeing issues on a unicore systems running a derivative of FreeBSD
8.2-RELEASE if something calls mem_range_attr_set.  It turns out that
the root cause is a bug in smp_rendezvous_cpus.  The first part of
smp_rendezvous_cpus attempts to short-circuit the non-SMP case(note
that smp_started is never set to 1 on a unicore system):

	if (!smp_started) {
		if (setup_func != NULL)
			setup_func(arg);
		if (action_func != NULL)
			action_func(arg);
		if (teardown_func != NULL)
			teardown_func(arg);
		return;
	}

The problem is that this runs with interrupts enabled, outside of a
critical section.  My system runs with device_polling enabled with hz
set to 2500, so its quite easy to wedge the system by having a thread
run mem_range_attr_set.  That has to do a smp_rendezvous, and if a
timer interrupt happens to go off half-way through the action_func and
preempt this thread, the system ends up deadlocked(although once it's
wedged, typing at the serial console stands a good chance of unwedging
the system.  Go figure).

I know that smp_rendezvous was reworked substantially on HEAD, but by
inspection it looks like the bug is still present, as the
short-circuit behaviour is still there.

I am not entirely sure of the best way to fix this.  Is it as simple
as doing a spinlock_enter before setup_func and a spinlock_exit after
teardown_func?  It seems to boot fine, but I'm not at all confident
that I understand the nuances of smp_rendezvous to be sure that there
aren't side effects that I don't know about.



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFMmRNwWBE6idGJCsNPago4-N1ohAaCD02_D-LCMzDg8pbjE1A>