From owner-freebsd-current@FreeBSD.ORG  Thu Jan  1 12:35:50 2015
Return-Path: <owner-freebsd-current@FreeBSD.ORG>
Delivered-To: freebsd-current@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id 9A7B3BB2;
 Thu,  1 Jan 2015 12:35:50 +0000 (UTC)
Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 583CA645A9;
 Thu,  1 Jan 2015 12:35:49 +0000 (UTC)
Received: from laptop015.home.selasky.org
 (cm-176.74.213.204.customer.telag.net [176.74.213.204])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (No client certificate requested)
 by mail.turbocat.net (Postfix) with ESMTPSA id 849B81FE022;
 Thu,  1 Jan 2015 13:35:46 +0100 (CET)
Message-ID: <54A53F4F.2000003@selasky.org>
Date: Thu, 01 Jan 2015 13:36:31 +0100
From: Hans Petter Selasky <hps@selasky.org>
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64;
 rv:31.0) Gecko/20100101 Thunderbird/31.3.0
MIME-Version: 1.0
To: Ivan Klymenko <fidaj@ukr.net>
Subject: Re: [RFC] kern/kern_timeout.c rewrite in progress
References: <54A1B38C.1000709@selasky.org>
 <20150101005613.4f788b0c@nonamehost.local> <54A49CA5.2060801@selasky.org>
 <54A4A002.8010802@selasky.org>
In-Reply-To: <54A4A002.8010802@selasky.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Cc: FreeBSD Current <freebsd-current@freebsd.org>, freebsd-arch@freebsd.org
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.18-1
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>,
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 01 Jan 2015 12:35:50 -0000

On 01/01/15 02:16, Hans Petter Selasky wrote:
> On 01/01/15 02:02, Hans Petter Selasky wrote:
>> On 12/31/14 23:56, Ivan Klymenko wrote:
>>> В Mon, 29 Dec 2014 21:03:24 +0100
>>> Hans Petter Selasky <hps@selasky.org> пишет:
>>>
>>
>> Hi,
>>
>> Is your kernel compiled with Witness? Do you see any lock order reversal
>> warnings?
>>
>> Can you do from kgdb:
>>
>> thread apply all bt
>>
>> And send me the result off-list?
>>
>> I'll have a closer look at this tomorrow.
>>
>>>
>>> panic: spin lock held too long
>>> http://paste.org.ru/?acf7io
>>>
>>
>> Thank you!
>>
>

Hi,

I see what is going on. There is an LOR which is not printed because 
MTX_QUIET is passed when locking inside the callback process routine.

It happens because cv_timedwait() is using callouts() to timeout and 
callout_drain() is using cv_wait() to wait for draining. This was not so 
well documented in the old code.

I'll update my patch and send out a new one later today.

I see a room for doing some improvements too:

callout_init(&td->td_slpcallout, CALLOUT_MPSAFE)

Possibly we could use a so-called "DIRECT" callback from the fast IRQ of 
the timer, when waking up other threads from cv_timedwait(), hence only 
spinlocks are involved? This would save waking up the callout SWI only 
to wakeup another thread !

Thank you for testing!

--HPS