From owner-freebsd-current@FreeBSD.ORG Mon Mar 17 06:17:08 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1C3F41065673 for ; Mon, 17 Mar 2008 06:17:08 +0000 (UTC) (envelope-from pitney.brad@googlemail.com) Received: from rv-out-0910.google.com (rv-out-0910.google.com [209.85.198.188]) by mx1.freebsd.org (Postfix) with ESMTP id E961F8FC1A for ; Mon, 17 Mar 2008 06:17:07 +0000 (UTC) (envelope-from pitney.brad@googlemail.com) Received: by rv-out-0910.google.com with SMTP id g13so3433741rvb.43 for ; Sun, 16 Mar 2008 23:17:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; bh=qQsIJF4+pqJWksE/Q6hamCUBXtatgkH2sEo4wAyX+/A=; b=IpZOmHCu+qQni6hzFmDGOKYHVQBdmtqkDqZOysk4j/5mBkeE2HKRQeSecG2YzFDbyYzyWRN/qqNJw62XIKMb1Dk8l/Ki7QCv566eINS3CT+7SPILjGOLc1Ko/9YT74zCiuJXVyuS7Kmv4ZTX8B8cISejUz+b7vLWrgLhuv0i1M4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=T/sZ0hKzBbw02/TRaBQzjJUYvGbnuL5WP+q6M9DZP81uQXbe+Y+sjTFcrdPckr/fNfYwPCiwtsjWiPRXSDM2bUVurNl3PVIJ2mdXwad3/NwI9WvaRyVDIGDjnl1TefE7Y56D6AqxZNXJ7jFdLi/UHsL2v0oQgAAC3tMM2+dTk64= Received: by 10.141.44.13 with SMTP id w13mr7370168rvj.13.1205734627024; Sun, 16 Mar 2008 23:17:07 -0700 (PDT) Received: by 10.141.41.8 with HTTP; Sun, 16 Mar 2008 23:17:06 -0700 (PDT) Message-ID: <3dd203290803162317u69b76bf2t203be7d863db3831@mail.gmail.com> Date: Mon, 17 Mar 2008 06:17:06 +0000 From: "Brad Pitney" To: freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Mailman-Approved-At: Mon, 17 Mar 2008 11:12:47 +0000 Subject: Another Lock Order Reversal -- sysctl? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Mar 2008 06:17:08 -0000 Hi, this one is different, I also had debugging on and I got more details (I hope) lock order reversal: 1st 0xc474281c process slock (process slock) @ /usr/src/sys/kern/kern_proc.c:710 2nd 0xc07bf5d0 scrlock (scrlock) @ /usr/src/sys/dev/syscons/syscons.c:2526 KDB: stack backtrace: db_trace_self_wrapper(c0746b40,e6fdd3ec,c057644e,c0748ee1,c07bf5d0,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0748ee1,c07bf5d0,c073a70f,c073a70f,c073a69b,...) at kdb_backtrace+0x29 witness_checkorder(c07bf5d0,9,c073a69b,9de,c07bf5d0,...) at witness_checkorder+0x6de _mtx_lock_spin_flags(c07bf5d0,0,c073a69b,9de,c07bf4c0,...) at _mtx_lock_spin_flags+0xc2 sc_puts(c07bf4c0,e6fdd447,1,6380c3a0,c077c9e0,...) at sc_puts+0x7e sc_cnputc(c077c9e0,63,e6fdd5fc,1,a,...) at sc_cnputc+0xc3 cnputc(63,0,c074b14e,28e,e6fdd6fc,...) at cnputc+0x5f cnputs(e6fdd5fc,e6fdd6fc,e6fdd4bc,c05696f1,a,...) at cnputs+0x58 putcons(a,0,1fdd593,c4742a2c,c07439b3,...) at putcons+0x67 putchar(a,e6fdd6fc,0,6,c4742a2c,...) at putchar+0x61 kvprintf(c074396a,c0569690,e6fdd6fc,a,e6fdd744,...) at kvprintf+0x75 printf(c074396a,1ef,0,1b6,0,...) at printf+0x6c calcru1(e6fdda30,4,c0743806,345,c07432db,2c6) at calcru1+0x25e calcru(c4742804,e6fdda28,e6fdda30,2c6,c4864aa8,...) at calcru+0x12d fill_kinfo_proc_only(c4742894,4,c07432db,38a,0,...) at fill_kinfo_proc_only+0x3b7 sysctl_out_proc(c4a35420,c4742804,c07432db,401,0,...) at sysctl_out_proc+0x68 sysctl_kern_proc(c07863e0,0,0,e6fddba8,e6fddba8,...) at sysctl_kern_proc+0x4fd sysctl_root(e6fddba8,0,c0744bd6,574,c4a35420,...) at sysctl_root+0x137 userland_sysctl(c4a35420,e6fddc18,3,0,bfbfe744,...) at userland_sysctl+0x115 __sysctl(c4a35420,e6fddcfc,18,c0749bc1,c07807d0,...) at __sysctl+0xbc syscall(e6fddd38) at syscall+0x253 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (202, FreeBSD ELF32, __sysctl), eip = 0x281bc823, esp = 0xbfbfe66c, ebp = 0xbfbfe698 --- KDB: enter: witness_checkorder db> show pcpu cpuid = 0 curthread = 0xc4a35420: pid 5757 "top" curpcb = 0xe6fddd90 fpcurthread = 0xc4a35420: pid 5757 "top" idlethread = 0xc43e3c60: pid 10 "idle" APIC ID = 0 currentldt = 0x50 spin locks held: exclusive spin mutex process slock r = 0 (0xc474281c) locked @ /usr/src/sys/kern/kern_proc.c:710 db> show alllocks Process 5757 (top) thread 0xc4a35420 (100074) exclusive sleep mutex process lock r = 0 (0xc4742894) locked @ /usr/src/sys/kern/kern_proc.c:1025 shared sx allproc r = 0 (0xc07c16fc) locked @ /usr/src/sys/kern/kern_proc.c:238 exclusive sx sysctl lock r = 0 (0xc07c1bf4) locked @ /usr/src/sys/kern/kern_sysctl.c:1396 exclusive sleep mutex Giant r = 0 (0xc07c11f0) locked @ /usr/src/sys/kern/kern_sysctl.c:1334 exclusive spin mutex process slock r = 0 (0xc474281c) locked @ /usr/src/sys/kern/kern_proc.c:710 exclusive sx so_rcv_sx r = 0 (0xc4b2ab60) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 the actual top command was: top -SCats1 -- Best regards, Brad