Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 14 Apr 2009 22:32:25 -0400
From:      Ben Kelly <ben@wanderview.com>
To:        Artem Belevich <fbsdlist@src.cx>
Cc:        freebsd-current@freebsd.org
Subject:   Re: [patch] zfs livelock and thread priorities
Message-ID:  <AC3C4C3F-40C6-4AF9-BAF3-2C4D1E444839@wanderview.com>
In-Reply-To: <08D7DC2A-68BE-47B6-8D5D-5DE6B48F87E5@wanderview.com>
References:  <DC9F2088-A0AF-467D-8574-F24A045ABD81@wanderview.com> <49C2CFF6.8070608@egr.msu.edu> <BDABA909-C2AE-4A55-869B-CA01BE778A82@wanderview.com> <ed91d4a80904131636u18c90474w7cdaa57bc7000e02@mail.gmail.com> <08D7DC2A-68BE-47B6-8D5D-5DE6B48F87E5@wanderview.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On Apr 14, 2009, at 11:50 AM, Ben Kelly wrote:
> On Apr 13, 2009, at 7:36 PM, Artem Belevich wrote:
>> Tried your patch that used PRIBIO+{1,2} for priorities with -current
>> r191008 and the kernel died with "spinlock held too long" panic.
>> Actually, there apparently were two instances of panic on different
>> cores..
>>
>> Here's output of "alltrace" and "ps" after the crash:
>> http://pastebin.com/f140f4596
>>
>> I've reverted the change and kernel booted just fine.
>>
>> The box is quad-core with two ZFS pools -- one single-disk and  
>> another
>> one is a two-disk mirror. Freebsd is installed on UFS partitions, ZFS
>> is used for user stuff only.
>
> Thanks for the report!
>
> I don't have a lot of time to look at this today, but it appears  
> that there is a race condition on SMP machines when setting the  
> priority immediately after the kproc is spawned.  As a quick hack I  
> tried adding a pause between the kproc_create() and the  
> sched_prio().  Can you try this patch?
>
>  http://www.wanderview.com/svn/public/misc/zfs_livelock/zfs_thread_priority.diff
>
> I'll try to take a closer look at this later in the week.

Sorry for replying to my own e-mail, but I've updated the patch again  
with a less hackish approach.  (At the same URL above.)  I added a new  
kproc_create_priority() function to set the priority of the new thread  
before its first scheduled.  This should avoid any SMP races with  
setting the priority from an external thread.

If you would be willing to try the test again with this new patch I  
would appreciate it.

Thanks!

- Ben



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AC3C4C3F-40C6-4AF9-BAF3-2C4D1E444839>