From owner-freebsd-current@FreeBSD.ORG Wed Oct 27 13:04:21 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 036EE16A4CE; Wed, 27 Oct 2004 13:04:21 +0000 (GMT) Received: from ptcnat.era.pl (ptcnat.era.pl [213.158.197.100]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1C9E643D49; Wed, 27 Oct 2004 13:04:20 +0000 (GMT) (envelope-from zaks@era.pl) Received: by localhost (Postfix, from userid 1001) id 3BDC811445; Wed, 27 Oct 2004 15:04:18 +0200 (CEST) From: =?iso-8859-2?q?S=B3awek_=AFak?= To: Robert Watson References: <86vfcx7eq9.fsf@thirst.unx.era.pl> Date: Wed, 27 Oct 2004 15:04:18 +0200 In-Reply-To: <86vfcx7eq9.fsf@thirst.unx.era.pl> =?iso-8859-2?q?=28S=B3awek's?= message of "Tue, 26 Oct 2004 16:09:50 +0200") Message-ID: <86k6tc1fe5.fsf@thirst.unx.era.pl> User-Agent: Gnus/5.110003 (No Gnus v0.3) XEmacs/21.4 (Reasonable Discussion, berkeley-unix) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" cc: current@freebsd.org Subject: Re: Hard hangs on AMD64 with mpsafenet enabled X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Oct 2004 13:04:21 -0000 --=-=-= Hi, I've managed to get the crash on serial console. I do break to debugger right after the boot then try to continue and the machine panics. No extra messages from WITNESS or INVARIANTS code. Kernel and core files are available at: http://www.era.pl/w3-1/vendor/kernel.debug.bz2 http://www.era.pl/w3-1/vendor/vmcore.0.bz2 I was unable to reproduce the NFS hangs so far. /S --=-=-= Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline; filename=crash.debugger Content-Transfer-Encoding: 8bit Script started on Wed Oct 27 13:32:57 2004 PROD krab-1(3001)# /tmp ccu -l /dev/cuaa1 Connected FreeBSD/amd64 (krab-4) (ttyd0) login: FreeBSD/amd64 (krab-4) (ttyd0) login: ~KDB: enter: Line break on console [thread 100003] Stopped at kdb_enter+0x2f: nop db> ps pid proc uarea uid ppid pgrp flag stat wmesg wchan cmd 565 ffffff003bcdd000 ffffffffb4799000 0 1 565 0004002 [SLPQ ttyin 0xffffff0000df0810][SLP] getty 546 ffffff007aaae8b8 ffffffffb2435000 0 1 546 0004002 [SLPQ ttyin 0xffffff007a9fcc10][SLP] getty 545 ffffff00609a5ba0 ffffffffb4806000 0 1 545 0004002 [SLPQ ttyin 0xffffff0000a7b810][SLP] getty 544 ffffff0060a84000 ffffffffb47fd000 0 1 544 0004002 [SLPQ ttyin 0xffffff007a9fc010][SLP] getty 543 ffffff007ab095d0 ffffffffb23ee000 0 1 543 0004002 [SLPQ ttyin 0xffffff0000a43410][SLP] getty 542 ffffff006057f000 ffffffffb479e000 0 1 542 0004002 [SLPQ ttyin 0xffffff0000a43810][SLP] getty 541 ffffff007b709ba0 ffffffffb1ac3000 0 1 541 0004002 [SLPQ ttyin 0xffffff0000a43c10][SLP] getty 540 ffffff007b7098b8 ffffffffb1ac2000 0 1 540 0004002 [SLPQ ttyin 0xffffff0000a7b010][SLP] getty 539 ffffff003bcdd8b8 ffffffffb479c000 0 1 539 0004002 [SLPQ ttyin 0xffffff0000a7b410][SLP] getty 511 ffffff003bc58ba0 ffffffffb4798000 125 503 503 0004100 [SLPQ select 0xffffffff80553050][SLP] qmgr 510 ffffff007aaae5d0 ffffffffb2434000 125 503 503 0004100 [SLPQ select 0xffffffff80553050][SLP] pickup 503 ffffff007ab098b8 ffffffffb23ef000 0 1 503 0004100 [SLPQ select 0xffffffff80553050][SLP] master 439 ffffff006057fba0 ffffffffb47a2000 0 1 439 0000000 [SLPQ nanslp 0xffffffff8050e8e0][SLP] cron 426 ffffff006057f2e8 ffffffffb479f000 0 1 426 0000100 [SLPQ select 0xffffffff80553050][SLP] sshd 405 ffffff007ab09ba0 ffffffffb23f0000 0 1 405 0000000 [SLPQ select 0xffffffff80553050][SLP] ntpd 369 ffffff006057f5d0 ffffffffb47a0000 0 363 363 0000100 [SLPQ select 0xffffffff80553050][SLP] rpc.lockd 363 ffffff0060a842e8 ffffffffb47fe000 0 1 363 0000000 [SLPQ select 0xffffffff80553050][SLP] rpc.lockd 358 ffffff003bcdd2e8 ffffffffb479a000 0 1 358 0000000 [SLPQ select 0xffffffff80553050][SLP] rpc.statd 352 ffffff007aaeb2e8 ffffffffb242e000 0 348 348 0000000 [SLPQ - 0xffffff0000de7c00][SLP] nfsd --More-- 351 ffffff007aaeb000 ffffffffb23f1000 0 348 348 0000000 [SLPQ - 0xffffff006027dc00][SLP] nfsd --More-- 350 ffffff0060a848b8 ffffffffb4800000 0 348 348 0000000 [SLPQ - 0xffffff00602fca00][SLP] nfsd 349 ffffff003bc585d0 ffffffffb4796000 0 348 348 0000000 [SLPQ - 0xffffff00602fcc00][SLP] nfsd 348 ffffff007ab092e8 ffffffffb23ed000 0 1 348 0000000 [SLPQ accept 0xffffff0060dd56d6][SLP] nfsd 340 ffffff0060a84ba0 ffffffffb4801000 0 1 340 0000000 [SLPQ select 0xffffffff80553050][SLP] mountd 289 ffffff003bc58000 ffffffffb473a000 0 1 289 0000000 [SLPQ select 0xffffffff80553050][SLP] rpcbind 275 ffffff003bc582e8 ffffffffb4795000 0 1 275 0000000 [SLPQ select 0xffffffff80553050][SLP] syslogd 252 ffffff00609a58b8 ffffffffb4805000 0 1 252 0000000 [SLPQ select 0xffffffff80553050][SLP] devd 66 ffffff007aaeb8b8 ffffffffb2430000 0 0 0 0000204 [SLPQ - 0xffffffffb241ebe4][SLP] schedcpu 65 ffffff007aaebba0 ffffffffb2431000 0 0 0 0000204 [SLPQ - 0xffffffff80557058][SLP] nfsiod 3 64 ffffff007aaae000 ffffffffb2432000 0 0 0 0000204 [SLPQ - 0xffffffff80557050][SLP] nfsiod 2 63 ffffff007b75c2e8 ffffffffb1a7a000 0 0 0 0000204 [SLPQ - 0xffffffff80557048][SLP] nfsiod 1 62 ffffff007b75c5d0 ffffffffb1a7b000 0 0 0 0000204 [SLPQ - 0xffffffff80557040][SLP] nfsiod 0 61 ffffff007b75c8b8 ffffffffb1a7c000 0 0 0 0000204 [SLPQ vlruwt 0xffffff007b75c8b8][SLP] vnlru 60 ffffff007b75cba0 ffffffffb1a7d000 0 0 0 0000204 [SLPQ syncer 0xffffffff8050e4c0][SLP] syncer 59 ffffff007b6e6000 ffffffffb1a7e000 0 0 0 0000204 [SLPQ psleep 0xffffffff8055397c][SLP] bufdaemon 58 ffffff007b6e62e8 ffffffffb1a7f000 0 0 0 000020c [SLPQ pgzero 0xffffffff8055f9f4][SLP] pagezero 57 ffffff007b6e65d0 ffffffffb1a80000 0 0 0 0000204 [SLPQ psleep 0xffffffff8055fa5c][SLP] vmdaemon 56 ffffff007b6e68b8 ffffffffb1abd000 0 0 0 0000204 [SLPQ psleep 0xffffffff8055fa0c][SLP] pagedaemon 55 ffffff007b6e6ba0 ffffffffb1abe000 0 0 0 0000204 [SLPQ - 0xffffff00009d3c48][SLP] fdc0 54 ffffff007b709000 ffffffffb1abf000 0 0 0 0000204 [IWAIT] swi0: sio --More-- 9 ffffff007b7092e8 ffffffffb1ac0000 0 0 0 0000204 [SLPQ actask 0xffffffff80500a20][SLP] acpi_task2 8 ffffff007b7095d0 ffffffffb1ac1000 0 0 0 0000204 [SLPQ actask 0xffffffff80500a20][SLP] acpi_task1 7 ffffff007b777000 ffffffffb1a33000 0 0 0 0000204 [SLPQ actask 0xffffffff80500a20][SLP] acpi_task0 53 ffffff007b7772e8 ffffffffb1a34000 0 0 0 0000204 [IWAIT] swi6: task queue 52 ffffff007b7775d0 ffffffffb1a35000 0 0 0 0000204 [IWAIT] swi6:+ 6 ffffff007b7778b8 ffffffffb1a36000 0 0 0 0000204 [SLPQ - 0xffffff00008ead00][SLP] thread taskq 51 ffffff007b777ba0 ffffffffb1a37000 0 0 0 0000204 [IWAIT] swi6:+ 5 ffffff007b778000 ffffffffb1a38000 0 0 0 0000204 [SLPQ - 0xffffff00008e5000][SLP] kqueue taskq 50 ffffff007b7782e8 ffffffffb1a75000 0 0 0 0000204 [IWAIT] swi3: cambio 49 ffffff007b7785d0 ffffffffb1a76000 0 0 0 0000204 [IWAIT] swi2: camnet 48 ffffff007b7788b8 ffffffffb1a77000 0 0 0 0000204 [IWAIT] swi6: acpitaskq 47 ffffff007b778ba0 ffffffffb1a78000 0 0 0 0000204 [SLPQ - 0xffffffff805011a0][SLP] yarrow 4 ffffff007b75c000 ffffffffb1a79000 0 0 0 0000204 [SLPQ - 0xffffffff80505708][SLP] g_down 3 ffffff007b77d000 ffffffffb19ed000 0 0 0 0000204 [SLPQ - 0xffffffff80505700][SLP] g_up 2 ffffff007b77d2e8 ffffffffb19ee000 0 0 0 0000204 [SLPQ - 0xffffffff805056f0][SLP] g_event 46 ffffff007b77d5d0 ffffffffb19ef000 0 0 0 0000204 [IWAIT] swi1: net 45 ffffff007b77d8b8 ffffffffb19f0000 0 0 0 0000204 [IWAIT] swi4: vm 44 ffffff007b77dba0 ffffffffb19f1000 0 0 0 000020c [RUNQ] swi5: clock sio 43 ffffff007b740000 ffffffffb1a2e000 0 0 0 0000204 [IWAIT] irq31: 42 ffffff007b7402e8 ffffffffb1a2f000 0 0 0 0000204 [IWAIT] irq30: --More-- 41 ffffff007b7405d0 ffffffffb1a30000 0 0 0 0000204 [IWAIT] irq29: 40 ffffff007b7408b8 ffffffffb1a31000 0 0 0 0000204 [IWAIT] irq28: 39 ffffff007b740ba0 ffffffffb1a32000 0 0 0 0000204 [IWAIT] irq27: mpt0 38 ffffff007b7052e8 ffffffffb19a8000 0 0 0 0000204 [IWAIT] irq26: bge1 37 ffffff007b7055d0 ffffffffb19a9000 0 0 0 0000204 [RUNQ] irq25: bge0 mpt2 36 ffffff007b7058b8 ffffffffb19aa000 0 0 0 0000204 [IWAIT] irq24: mpt1 35 ffffff007b705ba0 ffffffffb19ab000 0 0 0 0000204 [IWAIT] irq23: 34 ffffff007b751000 ffffffffb19e8000 0 0 0 0000204 [IWAIT] irq22: 33 ffffff007b7512e8 ffffffffb19e9000 0 0 0 0000204 [IWAIT] irq21: 32 ffffff007b7515d0 ffffffffb19ea000 0 0 0 0000204 [IWAIT] irq20: 31 ffffff007b7518b8 ffffffffb19eb000 0 0 0 0000204 [IWAIT] irq19: 30 ffffff007b751ba0 ffffffffb19ec000 0 0 0 0000204 [IWAIT] irq18: 29 ffffff007b75a8b8 ffffffffb1982000 0 0 0 0000204 [IWAIT] irq17: 28 ffffff007b75aba0 ffffffffb1983000 0 0 0 0000204 [IWAIT] irq16: 27 ffffff007b743000 ffffffffb1984000 0 0 0 0000204 [IWAIT] irq15: ata1 26 ffffff007b7432e8 ffffffffb19a3000 0 0 0 0000204 [IWAIT] irq14: ata0 25 ffffff007b7435d0 ffffffffb19a4000 0 0 0 0000204 [IWAIT] irq13: 24 ffffff007b7438b8 ffffffffb19a5000 0 0 0 0000204 [IWAIT] irq12: psm0 23 ffffff007b743ba0 ffffffffb19a6000 0 0 0 0000204 [IWAIT] irq11: 22 ffffff007b705000 ffffffffb19a7000 0 0 0 0000204 [IWAIT] irq10: --More-- 21 ffffff007b7982e8 ffffffffb195d000 0 0 0 0000204 [IWAIT] irq9: acpi0 20 ffffff007b7985d0 ffffffffb195e000 0 0 0 0000204 [IWAIT] irq8: rtc 19 ffffff007b7988b8 ffffffffb197d000 0 0 0 0000204 [IWAIT] irq7: 18 ffffff007b798ba0 ffffffffb197e000 0 0 0 0000204 [IWAIT] irq6: fdc0 17 ffffff007b75a000 ffffffffb197f000 0 0 0 0000204 [IWAIT] irq5: 16 ffffff007b75a2e8 ffffffffb1980000 0 0 0 0000204 [IWAIT] irq4: sio0 15 ffffff007b75a5d0 ffffffffb1981000 0 0 0 0000204 [IWAIT] irq3: 14 ffffff007b750000 ffffffffb191b000 0 0 0 0000204 [IWAIT] irq0: clk 13 ffffff007b7502e8 ffffffffb1958000 0 0 0 0000204 [IWAIT] irq1: atkbd0 12 ffffff007b7505d0 ffffffffb1959000 0 0 0 000020c [CPU 0] idle: cpu0 11 ffffff007b7508b8 ffffffffb195a000 0 0 0 000020c [CPU 1] idle: cpu1 1 ffffff007b750ba0 ffffffffb195b000 0 0 1 0004200 [SLPQ wait 0xffffff007b750ba0][SLP] init 10 ffffff007b798000 ffffffffb195c000 0 0 0 0000204 [SLPQ ktrace 0xffffffff8050b0f0][SLP] ktrace 0 ffffffff80505880 ffffffff8061d000 0 0 0 0000200 [SLPQ sched 0xffffffff80505880][SLP] swapper db> where kdb_enter() at kdb_enter+0x2f siointr1() at siointr1+0x3f6 siointr() at siointr+0x2e intr_execute_handlers() at intr_execute_handlers+0x112 lapic_handle_intr() at lapic_handle_intr+0x21 Xapic_isr1() at Xapic_isr1+0x7d --- interrupt, rip = 0xffffffff80177f36, rsp = 0xffffffffb192fba0, rbp = 0xffffffffb192fbb0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1a1 idle_proc() at idle_proc+0x7c fork_exit() at fork_exit+0xc3 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffb192fd00, rbp = 0 --- db> c KDB: stack backtrace: mi_switch() at mi_switch+0x2dc idle_proc() at idle_proc+0xb6 fork_exit() at fork_exit+0xc3 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffb1934d00, rbp = 0 --- panic: mi_switch: did not reenter debugger FreeBSD/amd64c (krab-4) (ttyd0p) login: uid = 0 KDB: enter: panic [thread 100004] Stopped at kdb_enter+0x2f: nop db> KD B:ÿüŒøüüüü üŒøŒø ŒøŒø ŒøŒøŒø ÿ Œøü Œø k_ex t f exit+0xc3 fork_trampoline() at or mpoli 0 tð- €t 0, = 0 ff ffb192fd00, rbp = 0 --- dddb db> ~~ [EOT] --=-=-=--