Date: Tue, 14 Apr 2009 22:32:25 -0400 From: Ben Kelly <ben@wanderview.com> To: Artem Belevich <fbsdlist@src.cx> Cc: freebsd-current@freebsd.org Subject: Re: [patch] zfs livelock and thread priorities Message-ID: <AC3C4C3F-40C6-4AF9-BAF3-2C4D1E444839@wanderview.com> In-Reply-To: <08D7DC2A-68BE-47B6-8D5D-5DE6B48F87E5@wanderview.com> References: <DC9F2088-A0AF-467D-8574-F24A045ABD81@wanderview.com> <49C2CFF6.8070608@egr.msu.edu> <BDABA909-C2AE-4A55-869B-CA01BE778A82@wanderview.com> <ed91d4a80904131636u18c90474w7cdaa57bc7000e02@mail.gmail.com> <08D7DC2A-68BE-47B6-8D5D-5DE6B48F87E5@wanderview.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Apr 14, 2009, at 11:50 AM, Ben Kelly wrote:
> On Apr 13, 2009, at 7:36 PM, Artem Belevich wrote:
>> Tried your patch that used PRIBIO+{1,2} for priorities with -current
>> r191008 and the kernel died with "spinlock held too long" panic.
>> Actually, there apparently were two instances of panic on different
>> cores..
>>
>> Here's output of "alltrace" and "ps" after the crash:
>> http://pastebin.com/f140f4596
>>
>> I've reverted the change and kernel booted just fine.
>>
>> The box is quad-core with two ZFS pools -- one single-disk and
>> another
>> one is a two-disk mirror. Freebsd is installed on UFS partitions, ZFS
>> is used for user stuff only.
>
> Thanks for the report!
>
> I don't have a lot of time to look at this today, but it appears
> that there is a race condition on SMP machines when setting the
> priority immediately after the kproc is spawned. As a quick hack I
> tried adding a pause between the kproc_create() and the
> sched_prio(). Can you try this patch?
>
> http://www.wanderview.com/svn/public/misc/zfs_livelock/zfs_thread_priority.diff
>
> I'll try to take a closer look at this later in the week.
Sorry for replying to my own e-mail, but I've updated the patch again
with a less hackish approach. (At the same URL above.) I added a new
kproc_create_priority() function to set the priority of the new thread
before its first scheduled. This should avoid any SMP races with
setting the priority from an external thread.
If you would be willing to try the test again with this new patch I
would appreciate it.
Thanks!
- Ben
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AC3C4C3F-40C6-4AF9-BAF3-2C4D1E444839>