From owner-freebsd-current@FreeBSD.ORG Thu Aug 16 00:31:04 2007 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 96A6216A420 for ; Thu, 16 Aug 2007 00:31:04 +0000 (UTC) (envelope-from grafan@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.177]) by mx1.freebsd.org (Postfix) with ESMTP id 519AB13C457 for ; Thu, 16 Aug 2007 00:31:03 +0000 (UTC) (envelope-from grafan@gmail.com) Received: by py-out-1112.google.com with SMTP id u77so155408pyb for ; Wed, 15 Aug 2007 17:31:03 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=R+jkmPgDffeCukQm+LPjUqZAjAE4pnrKBZUIUFaZ2djNAHsTbxIP9QjTV+TQjqAkUDAakBA5pG4I5pfafNlsb4hLOxWzcDo8oX5b0FZSbOpLEbQkRAvKsecJaMrHpmtJ/Qo8erSFUqYsJS73g4cKCT5kWRwxzhtE4zCBDGvHFxM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=Xfj2waaRPG+wWlXpF+bLrWxdVinJnH7QXcWSID4xln6+EoUJOGSBZyrYWVAqMNacgLc0Y2xcbK6jeDNBawEFHs2bPn3hkAk3LBnwEbjQ8O7vsRXSsPFgzFpTVIfoqjntVRuQWP1e1bJrzW1+ZNq74IgdGXLMftac7NrmEWB8jIQ= Received: by 10.65.242.11 with SMTP id u11mr2031291qbr.1187224262291; Wed, 15 Aug 2007 17:31:02 -0700 (PDT) Received: by 10.64.185.10 with HTTP; Wed, 15 Aug 2007 17:31:02 -0700 (PDT) Message-ID: <6eb82e0708151731q103c2a11i65628425014fb39d@mail.gmail.com> Date: Thu, 16 Aug 2007 08:31:02 +0800 From: "Rong-en Fan" To: "Max Laier" In-Reply-To: <200708152052.01275.max@love2party.net> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <6eb82e0708111122g7e36b725h19ede889e4bbc9d6@mail.gmail.com> <200708112129.29751.max@love2party.net> <6eb82e0708150852t354e0a55h674d168ccee1f750@mail.gmail.com> <200708152052.01275.max@love2party.net> Cc: freebsd-current@freebsd.org, Kris Kennaway Subject: Re: panic: sleeping thread owns a non-sleepable lock X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Aug 2007 00:31:04 -0000 On 8/16/07, Max Laier wrote: > On Wednesday 15 August 2007, Rong-en Fan wrote: > > On 8/12/07, Max Laier wrote: > > > On Saturday 11 August 2007, Kris Kennaway wrote: > > > > On Sun, Aug 12, 2007 at 02:22:35AM +0800, Rong-en Fan wrote: > > > > > I'm running 7.0-CURRENT as of yesterday, and it's very easy > > > > > to make it panic: > > > > > > > > > > Sleeping thread (tid 100065, pid 1066) owns a non-sleepable lock > > > > > sched_switch(c50a1600,0,1,1c7a7e4,4217e5,...) at > > > > > sched_switch+0x190 mi_switch(1,0) at mi_switch+0x13f > > > > > sleepq_switch(c50a1600,0,c078a4e2,21b,c07e3820,...) at > > > > > sleepq_switch+0x87 sleepq_wait(c07e3820,0,c0770b7e,3,0,...) at > > > > > sleepq_wait+0x36 _sx_xlock_hard(c07e3820,c50a1600,0,0,0,...) at > > > > > _sx_xlock_hard+0x21d > > > > > fr_checknatout(f9c7a8d0,f9c7a8cc,64,c57ad900,c4de7400,...) at > > > > > fr_checknatout+0x29d > > > > > fr_check(c8cc4644,14,c4de7400,1,f9c7a9b4,...) at fr_check+0x9b1 > > > > > fr_check_wrapper(0,f9c7a9b4,c4de7400,2,c54dab28,...) at > > > > > fr_check_wrapper+0x3f > > > > > pfil_run_hooks(c08057c0,f9c7aa4c,c4de7400,2,c54dab28,...) at > > > > > pfil_run_hooks+0x74 ip_output(c8cc4600,0,f9c7aa10,0,0,...) at > > > > > ip_output+0x913 > > > > > tcp_output(cae322d0,cb277200,0,0,0,...) at tcp_output+0x1106 > > > > > tcp_usr_send(c51e7318,0,cb277200,0,0,...) at tcp_usr_send+0x240 > > > > > kern_sendfile(c50a1600,f9c7acfc,0,0,0,...) at > > > > > kern_sendfile+0x1037 > > > > > sendfile(c50a1600,f9c7acfc,20,16,f9c7ad2c,...) at sendfile+0xa8 > > > > > syscall(f9c7ad38) at syscall+0x315 > > > > > Xint0x80_syscall() at Xint0x80_syscall+0x20 > > > > > --- syscall (393, FreeBSD ELF32, sendfile), eip = 0x28290bff, esp > > > > > = 0xbfbfc6ac, ebp = 0xbfbfe718 --- > > > > > > > > What is the lock it holds, and where is it acquired? > > > > > > My bet is on the pfil rwlock - accquired in pfil_run_hooks and > > > tcbinfo / inp mtxs from tcp_output. Nothing in the transmission path > > > must use sx locks. I keep on telling that. > > > > I compiled kernel with WITNESS and INVARIANTS as in GENERIC. > > After boot, without doing anything (I even set ipnat_enable="NO", i.e. > > only ipfw is running, but ipf/ipnat is compiled in). I got panic. > > I don't have serial console attached, but the console screen is at > > > > http://www.rafan.org/tmp/pfil_panic.jpg > > > > You can find 'show locks', 'show allpcpu', and 'show lock' for each > > lock. Also two trace for thread that are found in 'show allpcpu'. > > > > If necessary, I can setup serial console, but it will take few days. > > The problem is understood and has been reported to the ipfilter > maintainer. One sollution might be to use rwlocks instead, but if there > is a path that sleeps with the lock held another sollution might be > required. Most likely the ioctl path (on copyin/copyout) exercises that > problem. I see. So, only ipfilter has this problem, but not pf and ipfw, right? Thanks. Regards, Rong-En Fan > > -- > /"\ Best regards, | mlaier@freebsd.org > \ / Max Laier | ICQ #67774661 > X http://pf4freebsd.love2party.net/ | mlaier@EFnet > / \ ASCII Ribbon Campaign | Against HTML Mail and News > >