From owner-freebsd-bugs@FreeBSD.ORG Tue Aug 2 21:20:17 2005 Return-Path: X-Original-To: freebsd-bugs@hub.freebsd.org Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 02AE616A41F for ; Tue, 2 Aug 2005 21:20:17 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 68AAC43D45 for ; Tue, 2 Aug 2005 21:20:16 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.3/8.13.3) with ESMTP id j72LKG1K084188 for ; Tue, 2 Aug 2005 21:20:16 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.3/8.13.1/Submit) id j72LKGAE084186; Tue, 2 Aug 2005 21:20:16 GMT (envelope-from gnats) Resent-Date: Tue, 2 Aug 2005 21:20:16 GMT Resent-Message-Id: <200508022120.j72LKGAE084186@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, "Marc G.Fournier" Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BD9CB16A41F for ; Tue, 2 Aug 2005 21:16:15 +0000 (GMT) (envelope-from scrappy@ganymede.hub.org) Received: from hub.org (hub.org [200.46.204.220]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2255D43D48 for ; Tue, 2 Aug 2005 21:16:14 +0000 (GMT) (envelope-from scrappy@ganymede.hub.org) Received: from localhost (unknown [200.46.204.144]) by hub.org (Postfix) with ESMTP id 004D7A246AD for ; Tue, 2 Aug 2005 18:16:10 -0300 (ADT) Received: from hub.org ([200.46.204.220]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 11885-01 for ; Tue, 2 Aug 2005 21:16:10 +0000 (GMT) Received: from ganymede.hub.org (blk-224-176-51.eastlink.ca [24.224.176.51]) by hub.org (Postfix) with ESMTP id AA5ECA246A4 for ; Tue, 2 Aug 2005 18:16:09 -0300 (ADT) Received: by ganymede.hub.org (Postfix, from userid 1000) id 2C4FE482F9; Tue, 2 Aug 2005 18:16:15 -0300 (ADT) Message-Id: <20050802211615.2C4FE482F9@ganymede.hub.org> Date: Tue, 2 Aug 2005 18:16:15 -0300 (ADT) From: "Marc G.Fournier" To: FreeBSD-gnats-submit@FreeBSD.org X-Send-Pr-Version: 3.113 Cc: Subject: kern/84498: file system hangs when jail starts if unionfs is involved X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: "Marc G.Fournier" List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2005 21:20:17 -0000 >Number: 84498 >Category: kern >Synopsis: file system hangs when jail starts if unionfs is involved >Confidential: no >Severity: critical >Priority: high >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Tue Aug 02 21:20:15 GMT 2005 >Closed-Date: >Last-Modified: >Originator: Marc G. Fournier >Release: FreeBSD 6.0-BETA1 i386 >Organization: >Environment: System: FreeBSD ganymede.hub.org 6.0-BETA1 FreeBSD 6.0-BETA1 #2: Sat Jul 30 00:36:35 ADT 2005 root@ganymede.hub.org:/shared/obj/usr/src/sys/kernel i386 Copyright (c) 1992-2005 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 6.0-BETA1 #2: Sat Jul 30 00:36:35 ADT 2005 root@ganymede.hub.org:/shared/obj/usr/src/sys/kernel Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Pentium(R) 4 CPU 2.40GHz (2394.01-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf27 Stepping = 7 Features=0xbfebfbff Features2=0x4400> real memory = 1073676288 (1023 MB) avail memory = 1045864448 (997 MB) mptable_probe: MP Config Table has bad signature: npx0: [FAST] npx0: on motherboard npx0: INT 16 interface acpi0: on motherboard acpi0: Power Button (fixed) pci_link0: irq 3 on acpi0 pci_link1: irq 11 on acpi0 pci_link2: irq 11 on acpi0 pci_link3: irq 7 on acpi0 pci_link4: irq 0 on acpi0 pci_link5: irq 0 on acpi0 pci_link6: irq 10 on acpi0 pci_link7: irq 10 on acpi0 Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0 cpu0: on acpi0 acpi_button0: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 agp0: mem 0xe8000000-0xefffffff at device 0.0 on pci0 pcib1: at device 1.0 on pci0 pci1: on pcib1 pci1: at device 0.0 (no driver attached) pci0: at device 29.0 (no driver attached) pci0: at device 29.1 (no driver attached) pci0: at device 29.2 (no driver attached) pci0: at device 29.3 (no driver attached) pci0: at device 29.7 (no driver attached) pcib2: at device 30.0 on pci0 pci2: on pcib2 rl0: port 0xd000-0xd0ff mem 0xfa030000-0xfa0300ff irq 10 at device 7.0 on pci2 miibus0: on rl0 rlphy0: on miibus0 rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto rl0: Ethernet address: 00:01:80:4f:54:5b fxp0: port 0xd100-0xd13f mem 0xfa031000-0xfa031fff,0xfa000000-0xfa01ffff irq 7 at device 9.0 on pci2 miibus1: on fxp0 inphy0: on miibus1 inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto fxp0: Ethernet address: 00:02:b3:ee:da:3e isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xf000-0xf00f irq 11 at device 31.2 on pci0 atapci0: failed to enable memory mapping! ata0: on atapci0 ata1: on atapci0 pci0: at device 31.3 (no driver attached) pcm0: port 0xea00-0xeaff,0xeb00-0xeb3f mem 0xfa101000-0xfa1011ff,0xfa102000-0xfa1020ff irq 11 at device 31.5 on pci0 pcm0: [GIANT-LOCKED] pcm0: acpi_tz0: on acpi0 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: [FAST] sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A, console atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: model MouseMan+, device ID 0 orm0: at iomem 0xd0000-0xd17ff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled Timecounter "TSC" frequency 2394012756 Hz quality 800 Timecounters tick every 1.000 msec ad0: 38166MB at ata0-master UDMA100 ad1: 76319MB at ata0-slave UDMA100 ad2: 152627MB at ata1-master SATA150 ad3: 190782MB at ata1-slave SATA150 >Description: In preparation of deploying our first non-4.x production server, I've attempted to take a 4.x VM (jail environment) and start it up ... each time it starts, the start up hangs while sendmail is trying to start up, and if you try and cd to the directory where the 'jail root' is located, the file system will subsequently freeze up, and no more keyboard input is available on that termimal ... Of note ... sendmail == postfix, so unless I'm mistaken, that would be the first process to try and access something *on* the unionfs file system ... The following is what Robert has suggested pulling in the way of information from DDB, through a serial console ... as well, I have a core dump, if that can be used to provide further information. ganymede# /usr/sbin/jail /vm/1/test.hub.org test.hub.org 192.168.0.65 4 /bin/sh /etc/rc /etc/rc: cannot create /dev/null: error 45 Skipping disk checks ... /etc/rc: cannot create /dev/null: error 45 cp: /dev/null: Operation not supported Doing initial network setup:. /etc/rc: cannot create /dev/null: error 45 Additional routing options:. Routing daemons:. /etc/rc: cannot create /dev/null: error 45 /etc/rc: cannot create /dev/null: error 45 Clearing /tmp:. Additional daemons: syslogd. Doing additional network setup:. Starting final network daemons: creating ssh1 RSA host key PRNG is not seeded creating ssh2 RSA host key /etc/rc: /usr/bin/ssh-keygen: permission denied creating ssh2 DSA host key /etc/rc: /usr/bin/ssh-keygen: permission denied . ELF ldconfig path: /usr/lib /usr/lib/compat /usr/X11R6/lib /usr/local/lib a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout /usr/X11R6/lib/aout Starting standard daemons: cron sshdPRNG is not seeded sendmail. load: 0.00 cmd: sendmail 1454 [union] 0.00u 0.00s 0% 60k load: 0.00 cmd: sendmail 1454 [union] 0.00u 0.00s 0% 60k KDB: enter: manual escape to debugger [thread pid 13 tid 100003 ] Stopped at kdb_enter+0x2b: nop db> show pcpu cpuid = 0 curthread = 0xc1ec9a80: pid 13 "irq1: atkbd0" curpcb = 0xe3254d90 fpcurthread = none idlethread = 0xc1ec9780: pid 11 "idle: cpu0" APIC ID = 0 currentldt = 0x50 db> show pcpu 0 cpuid = 0 curthread = 0xc1ec9a80: pid 13 "irq1: atkbd0" curpcb = 0xe3254d90 fpcurthread = none idlethread = 0xc1ec9780: pid 11 "idle: cpu0" APIC ID = 0 currentldt = 0x50 db> ps pid proc uid ppid pgrp flag stat wmesg wchan cmd 1454 c23aec48 0 1 1389 4000002 [SLPQ union 0xc4cb6058][SLP] sendmail 1450 c2312000 0 1 1450 0000000 [SLPQ ufs 0xc278e6b8][SLP] cron 1440 c2313000 0 1 1440 0000000 [SLPQ select 0xc0672ee4][SLP] syslogd 1389 c23fc830 0 1385 1389 0004002 [SLPQ union 0xc2291af8][SLP] sh 1385 c2f7c830 0 812 1385 0004002 [SLPQ pause 0xc2f7c864][SLP] csh 1384 c2190c48 1001 627 627 0004100 [SLPQ lockf 0xc27d05c0][SLP] smtpd 1383 c2f7c418 1001 627 627 0004100 [SLPQ lockf 0xc225f680][SLP] smtpd 1382 c2f7c20c 1001 627 627 0004100 [SLPQ lockf 0xc27d0a40][SLP] smtpd 1381 c2665c48 1001 627 627 0004100 [SLPQ lockf 0xc2d15240][SLP] smtpd 1380 c23ae418 60 657 657 0004001 [SLPQ select 0xc0672ee4][SLP] lmtpd 1379 c2f7c000 1001 627 627 0004100 [SLPQ select 0xc0672ee4][SLP] lmtp 1378 c2f7ca3c 0 627 627 0004100 [SLPQ select 0xc0672ee4][SLP] local 1377 c2f79c48 1001 627 627 0004100 [SLPQ select 0xc0672ee4][SLP] cleanup 1376 c23af000 1001 627 627 0004100 [SLPQ select 0xc0672ee4][SLP] trivial-rewrite 1375 c2f7c624 1001 627 627 0004100 [SLPQ select 0xc0672ee4][SLP] proxymap 1374 c2f7cc48 1001 627 627 0004100 [SLPQ select 0xc0672ee4][SLP] smtpd 1352 c2313a3c 0 805 45 0004000 [SLPQ nanslp 0xc066c34c][SLP] fsck_ufs 813 c23fc624 0 809 813 0004002 [SLPQ ufs 0xc278e6b8][SLP] csh 812 c23fcc48 0 1 812 0004102 [SLPQ wait 0xc23fcc48][SLP] login 811 c2664418 0 1 811 0004002 [SLPQ ttyin 0xc2039c10][SLP] getty 810 c218f20c 0 1 810 0004002 [SLPQ ttyin 0xc2039410][SLP] getty 809 c23a8c48 0 1 809 0004102 [SLPQ wait 0xc23a8c48][SLP] login 806 c2664624 0 1 45 0004002 [SLPQ piperd 0xc23a5198][SLP] logger 805 c2664000 0 804 45 0004002 [SLPQ wait 0xc2664000][SLP] fsck 804 c266420c 0 1 45 0000002 [SLPQ wait 0xc266420c][SLP] sh 793 c2664830 0 706 706 0000000 [SLPQ accept 0xc236d892][SLP] perl 792 c2664a3c 0 706 706 0000000 [SLPQ accept 0xc236d892][SLP] perl 791 c2664c48 0 706 706 0000000 [SLPQ accept 0xc236d892][SLP] perl 790 c2665000 0 706 706 0000000 [SLPQ accept 0xc236d892][SLP] perl 789 c266520c 0 706 706 0000000 [SLPQ accept 0xc236d892][SLP] perl 788 c2665418 0 706 706 0000000 [SLPQ accept 0xc236d892][SLP] perl 787 c2665624 0 706 706 0000000 [SLPQ accept 0xc236d892][SLP] perl 786 c2665830 0 706 706 0000000 [SLPQ accept 0xc236d892][SLP] perl 785 c23af418 0 706 706 0000000 [SLPQ accept 0xc236d892][SLP] perl 784 c23fb20c 0 706 706 0000000 [SLPQ accept 0xc236d892][SLP] perl 783 c23af20c 0 706 706 0000000 [SLPQ accept 0xc236d892][SLP] perl 782 c23a8830 0 706 706 0000000 [SLPQ accept 0xc236d892][SLP] perl 781 c23af624 0 706 706 0000000 [SLPQ accept 0xc236d892][SLP] perl 780 c23fb624 0 706 706 0000000 [SLPQ accept 0xc236d892][SLP] perl 779 c23fc000 0 706 706 0000000 [SLPQ accept 0xc236d892][SLP] perl 778 c23a8a3c 0 706 706 0000000 [SLPQ accept 0xc236d892][SLP] perl 777 c2313830 0 706 706 0000000 [SLPQ accept 0xc236d892][SLP] perl 776 c23fbc48 0 706 706 0000000 [SLPQ accept 0xc236d892][SLP] perl 775 c23a820c 0 706 706 0000000 [SLPQ accept 0xc236d892][SLP] perl 774 c23fb830 0 706 706 0000000 [SLPQ accept 0xc236d892][SLP] perl 757 c23afa3c 0 1 757 0000000 [SLPQ select 0xc0672ee4][SLP] moused 752 c23af830 0 726 726 0000101 [SLPQ pause 0xc23af864][SLP] smbd 726 c23afc48 0 1 726 0000101 [SLPQ select 0xc0672ee4][SLP] smbd 725 c23fb000 60 657 657 0004001 [SLPQ select 0xc0672ee4][SLP] imapd 721 c23fba3c 0 1 721 0000001 [SLPQ select 0xc0672ee4][SLP] nmbd 706 c23fc20c 0 1 706 0000000 [SLPQ pause 0xc23fc240][SLP] perl 691 c23fb418 0 1 691 0000101 [SLPQ select 0xc0672ee4][SLP] proftpd 684 c23fc418 80 637 637 0000100 [SLPQ accept 0xc239319e][SLP] httpd 683 c2190830 80 637 637 0000100 [SLPQ accept 0xc239319e][SLP] httpd 682 c2312830 80 637 637 0000100 [SLPQ accept 0xc239319e][SLP] httpd 681 c23a8418 80 637 637 0000100 [SLPQ accept 0xc239319e][SLP] httpd 680 c23a8000 80 637 637 0000100 [SLPQ accept 0xc239319e][SLP] httpd 661 c23a8624 0 1 45 0004002 [SLPQ select 0xc0672ee4][SLP] imspd 657 c23ae000 60 1 657 0000101 [SLPQ select 0xc0672ee4][SLP] master 637 c2313418 0 1 637 0000000 [SLPQ select 0xc0672ee4][SLP] httpd 629 c231320c 1001 627 627 0004100 [SLPQ select 0xc0672ee4][SLP] qmgr 628 c229b418 1001 627 627 0004100 [SLPQ select 0xc0672ee4][SLP] pickup 627 c2312418 0 1 627 0004100 [SLPQ select 0xc0672ee4][SLP] master 553 c229ba3c 0 1 553 0000000 [SLPQ nanslp 0xc066c34c][SLP] cron 536 c1f83c48 0 1 536 0000100 [SLPQ select 0xc0672ee4][SLP] sshd 527 c229b000 1 1 527 0000100 [SLPQ sbwait 0xc227ba64][SLP] rwhod 448 c2190624 0 443 443 0000000 [SLPQ - 0xc22d0a00][SLP] nfsd 447 c2190418 0 443 443 0000000 [SLPQ - 0xc22d0c00][SLP] nfsd 446 c1f83830 0 443 443 0000000 [SLPQ - 0xc22d0e00][SLP] nfsd 445 c219020c 0 443 443 0000000 [SLPQ - 0xc22d0800][SLP] nfsd 443 c2190a3c 0 1 443 0000000 [SLPQ accept 0xc22ce03a][SLP] nfsd 429 c229bc48 0 1 429 0000000 [SLPQ select 0xc0672ee4][SLP] mountd 425 c229b20c 0 1 425 0000000 [SLPQ select 0xc0672ee4][SLP] rpcbind 352 c1f83a3c 53 1 352 0008180 (threaded) named thread 0xc22b6000 ksegrp 0xc1ecb000 [SLPQ kserel 0xc1ecb034][SLP] thread 0xc22b5d80 ksegrp 0xc1ecb000 [SLPQ select 0xc0672ee4][SLP] thread 0xc2191180 ksegrp 0xc1ecc780 [SLPQ ksesigwait 0xc1f83b74][SLP] 291 c229b830 0 1 291 0000000 [SLPQ select 0xc0672ee4][SLP] syslogd 258 c218f000 0 1 258 0000000 [SLPQ select 0xc0672ee4][SLP] devd 44 c218f418 0 0 0 0000204 [SLPQ - 0xe4cd4d08][SLP] schedcpu 43 c218f624 0 0 0 0000204 [SLPQ syncer 0xc066c0bc][SLP] syncer 42 c218f830 0 0 0 0000204 [SLPQ vlruwt 0xc218f830][SLP] vnlru 41 c218fa3c 0 0 0 0000204 [SLPQ psleep 0xc067342c][SLP] bufdaemon 40 c218fc48 0 0 0 000020c [SLPQ pgzero 0xc067bc04][SLP] pagezero 39 c2190000 0 0 0 0000204 [SLPQ psleep 0xc067b754][SLP] vmdaemon 38 c1f1ec48 0 0 0 0000204 [SLPQ psleep 0xc067b710][SLP] pagedaemon 37 c1f80000 0 0 0 0000204 [IWAIT] swi0: sio 36 c1f8020c 0 0 0 0000204 [SLPQ - 0xc1f7f43c][SLP] fdc0 35 c1f80418 0 0 0 0000204 [SLPQ tzpoll 0xc0778674][SLP] acpi_thermal 9 c1f80624 0 0 0 0000204 [SLPQ - 0xc1ec7b80][SLP] thread taskq 34 c1f80830 0 0 0 0000204 [IWAIT] swi6:+ 8 c1f80a3c 0 0 0 0000204 [SLPQ - 0xc1ec7d00][SLP] acpi_task2 7 c1f80c48 0 0 0 0000204 [SLPQ - 0xc1ec7d00][SLP] acpi_task1 6 c1f83000 0 0 0 0000204 [SLPQ - 0xc1ec7d00][SLP] acpi_task0 33 c1f8320c 0 0 0 0000204 [IWAIT] swi6: task queue 32 c1f83418 0 0 0 0000204 [IWAIT] swi2: cambio 5 c1f16624 0 0 0 0000204 [SLPQ - 0xc1f15080][SLP] kqueue taskq 31 c1f16830 0 0 0 0000204 [IWAIT] swi5:+ 30 c1f16a3c 0 0 0 0000204 [SLPQ - 0xc0666ec0][SLP] yarrow 4 c1f16c48 0 0 0 0000204 [SLPQ - 0xc0669868][SLP] g_down 3 c1f1e000 0 0 0 0000204 [SLPQ - 0xc0669864][SLP] g_up 2 c1f1e20c 0 0 0 0000204 [SLPQ - 0xc066985c][SLP] g_event 29 c1f1e418 0 0 0 0000204 [IWAIT] swi3: vm 28 c1f1e624 0 0 0 000020c [RUNQ] swi4: clock sio 27 c1f1e830 0 0 0 0000204 [IWAIT] swi1: net 26 c1f1ea3c 0 0 0 0000204 [IWAIT] irq15: ata1 25 c1ecd20c 0 0 0 0000204 [IWAIT] irq14: ata0 24 c1ecd418 0 0 0 0000204 [IWAIT] irq13: 23 c1ecd624 0 0 0 0000204 [IWAIT] irq12: psm0 22 c1ecd830 0 0 0 0000204 [IWAIT] irq11: pcm0 21 c1ecda3c 0 0 0 0000204 [IWAIT] irq10: rl0 20 c1ecdc48 0 0 0 0000204 [IWAIT] irq9: acpi0 19 c1f16000 0 0 0 0000204 [IWAIT] irq8: rtc 18 c1f1620c 0 0 0 0000204 [IWAIT] irq7: fxp0 17 c1f16418 0 0 0 0000204 [IWAIT] irq6: fdc0 16 c1ec8000 0 0 0 0000204 [IWAIT] irq5: 15 c1ec820c 0 0 0 0000204 [IWAIT] irq4: sio0 14 c1ec8418 0 0 0 0000204 [IWAIT] irq3: 13 c1ec8624 0 0 0 0000204 [CPU 0] irq1: atkbd0 12 c1ec8830 0 0 0 0000204 [IWAIT] irq0: clk 11 c1ec8a3c 0 0 0 000020c [Can run] idle: cpu0 1 c1ec8c48 0 0 1 0004200 [SLPQ wait 0xc1ec8c48][SLP] init 10 c1ecd000 0 0 0 0000204 [SLPQ ktrace 0xc066a2b8][SLP] ktrace 0 c0669960 0 0 0 0000200 [IWAIT] swapper db> show pcpu cpuid = 0 curthread = 0xc1ec9a80: pid 13 "irq1: atkbd0" curpcb = 0xe3254d90 fpcurthread = none idlethread = 0xc1ec9780: pid 11 "idle: cpu0" APIC ID = 0 currentldt = 0x50 db> trace 13 Tracing pid 13 tid 100003 td 0xc1ec9a80 kdb_enter(c062dfe9) at kdb_enter+0x2b scgetc(c0695880,2,1,c2036c00,c067bc20) at scgetc+0x510 sckbdevent(c067bc20,0,c0695880) at sckbdevent+0x1c8 atkbd_intr(c067bc20,0,e3254d10,c04b8589,c067bc20) at atkbd_intr+0x20 atkbdintr(c067bc20) at atkbdintr+0x16 ithread_loop(c1ec6900,e3254d38) at ithread_loop+0x14d fork_exit(c04b843c,c1ec6900,e3254d38) at fork_exit+0x71 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe3254d6c, ebp = 0 --- db> show lockedvnods Locked vnodes 0xc27afbb0: tag ufs, type VDIR usecount 1, writecount 0, refcount 4 mountedhere 0 flags () v_object 0xc0c68318 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xc23a9c00 (pid 813) ino 49701, on dev ad0s1a 0xc278e660: tag ufs, type VDIR usecount 14, writecount 0, refcount 15 mountedhere 0 flags () v_object 0xc374f6b4 ref 0 pages 0 lock type ufs: EXCL (count 1) by thread 0xc22b6600 (pid 1454) with 2 pending ino 49704, on dev ad0s1a 0xc4cb6000: tag union, type VDIR usecount 9, writecount 0, refcount 9 mountedhere 0 flags (VV_ROOT) lock type union: EXCL (count 1) by thread 0xc23a9a80 (pid 1389) with 1 pending vp=0xc4cb6000, uppervp=0xc4cafcc0, lowervp=0xc27d7550 union: upper 0xc4cafcc0: tag ufs, type VDIR usecount 12, writecount 0, refcount 13 mountedhere 0xc201d800 flags () ino 17736, on dev ad0s1a union: lower 0xc27d7550: tag ufs, type VDIR usecount 3, writecount 0, refcount 4 mountedhere 0 flags () v_object 0xc343ee70 ref 0 pages 0 ino 8470704, on dev ad1s1a 0xc2291aa0: tag union, type VDIR usecount 5, writecount 0, refcount 5 mountedhere 0 flags () lock type union: EXCL (count 1) by thread 0xc2f7b600 (pid 1461) with 1 pending vp=0xc2291aa0, uppervp=0xc2291000, lowervp=0xc3445990 union: upper 0xc2291000: tag ufs, type VDIR usecount 5, writecount 0, refcount 6 mountedhere 0 flags () ino 17752, on dev ad0s1a union: lower 0xc3445990: tag ufs, type VDIR usecount 2, writecount 0, refcount 4 mountedhere 0 flags () v_object 0xc4b1f630 ref 0 pages 1 ino 8494974, on dev ad1s1a db> trace 813 Tracing pid 813 tid 100111 td 0xc23a9c00 sched_switch(c23a9c00,0,1) at sched_switch+0x14b mi_switch(1,0,c23a9c00,0,c23a9c00) at mi_switch+0x1ba sleepq_switch(c278e6b8) at sleepq_switch+0x133 sleepq_wait(c278e6b8,0,c05b0514,0,c278e6b8) at sleepq_wait+0xb msleep(c278e6b8,c066a90c,50,c0622b25,0) at msleep+0x356 acquire(ef202974,40,60000,c23a9c00,0) at acquire+0x76 lockmgr(c278e6b8,2002,c278e6dc,c23a9c00,ef202998) at lockmgr+0x3f6 vop_stdlock(ef2029e0,c0656f40,ef2029e0,ef2029a8,c0591a18) at vop_stdlock+0x1e VOP_LOCK_APV(c0657480,ef2029e0,ef2029bc,c0602457,ef2029e0) at VOP_LOCK_APV+0x43 ffs_lock(ef2029e0,2002,c278e660,ef2029fc,c05308bc) at ffs_lock+0x10 VOP_LOCK_APV(c0656f40,ef2029e0) at VOP_LOCK_APV+0x43 vn_lock(c278e660,2002,c23a9c00,c278e660,2002) at vn_lock+0xf4 vget(c278e660,2002,c23a9c00,3002,c21b6c2c) at vget+0xba cache_lookup(c27afbb0,ef202c14,ef202c28) at cache_lookup+0x441 vfs_cache_lookup(ef202b18,c27afbb0,c362b012,ef202b34,c051dd09) at vfs_cache_lookup+0x8f VOP_LOOKUP_APV(c0656f40,ef202b18) at VOP_LOOKUP_APV+0x43 lookup(ef202c00,c27afbb0,0,c23a9c00,ef202c28) at lookup+0x431 namei(ef202c00,0,c064fde0,ef202c24,c21c7a68) at namei+0x396 kern_stat(c23a9c00,80e3008,0,ef202c74) at kern_stat+0x35 stat(c23a9c00,ef202d04,2,0,246) at stat+0x1b syscall(3b,3b,3b,bfbde55c,bfbcd060) at syscall+0x2bb Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (188, FreeBSD ELF32, stat), eip = 0x2814ba47, esp = 0xbfbcbbac, ebp = 0xbfbccc28 --- db> trace 1454 Tracing pid 1454 tid 100093 td 0xc22b6600 sched_switch(c22b6600,0,1) at sched_switch+0x14b mi_switch(1,0,c22b6600,2,c22b6600) at mi_switch+0x1ba sleepq_switch(c4cb6058) at sleepq_switch+0x133 sleepq_wait(c4cb6058,0,ef1be7dc,0,c4cb6058) at sleepq_wait+0xb msleep(c4cb6058,c066b350,50,c4cc1a6f,0) at msleep+0x356 acquire(ef1be830,80,60000,c22b6600,0) at acquire+0x76 lockmgr(c4cb6058,2002,c4cb607c,c22b6600,ef1be854) at lockmgr+0x3f6 vop_stdlock(ef1be878,2,c4cb6000,ef1be894,c05308bc) at vop_stdlock+0x1e VOP_LOCK_APV(c4cc2240,ef1be878) at VOP_LOCK_APV+0x43 vn_lock(c4cb6000,2,c22b6600,1,12) at vn_lock+0xf4 union_allocvp(ef1be948,c201d800,0,0,0) at union_allocvp+0x165 union_root(c201d800,2,ef1be948,c22b6600,1) at union_root+0x3f lookup(c2f53300,8055000,0,c22b6600,c2f53328) at lookup+0x6d7 namei(c2f53300,3,0,0,0) at namei+0x396 elf32_load_file(c23aec48,e335f0f4,ef1bea84,ef1bec00,1000) at elf32_load_file+0xa8 exec_elf32_imgact(ef1bebe8,0,1,0,0) at exec_elf32_imgact+0x52c do_execve(c22b6600,ef1becb4,0,0,ef1becb4) at do_execve+0x23a kern_execve(c22b6600,ef1becb4,0,df1d2000,df1d2000) at kern_execve+0xcc execve(c22b6600,ef1bed04,3,0,293) at execve+0x32 syscall(3b,3b,3b,0,8068da8) at syscall+0x2bb Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (59, FreeBSD ELF32, execve), eip = 0x2817a634, esp = 0xbfbfec44, ebp = 0xbfbfec70 --- db> trace 1389 Tracing pid 1389 tid 100112 td 0xc23a9a80 sched_switch(c23a9a80,0,1) at sched_switch+0x14b mi_switch(1,0,c23a9a80,1,c23a9a80) at mi_switch+0x1ba sleepq_switch(c2291af8) at sleepq_switch+0x133 sleepq_wait(c2291af8,0,c0524382,0,c2291af8) at sleepq_wait+0xb msleep(c2291af8,c066aff0,50,c4cc1a6f,0) at msleep+0x356 acquire(ef1ff92c,80,60000,c23a9a80,0) at acquire+0x76 lockmgr(c2291af8,2002,c2291b1c,c23a9a80,ef1ff950) at lockmgr+0x3f6 vop_stdlock(ef1ff974,2,c2291aa0,ef1ff990,c05308bc) at vop_stdlock+0x1e VOP_LOCK_APV(c4cc2240,ef1ff974) at VOP_LOCK_APV+0x43 vn_lock(c2291aa0,2,c23a9a80,0,9) at vn_lock+0xf4 union_allocvp(ef1ffc14,c201d800,c4cb6000,c4cafcc0,ef1ffc28) at union_allocvp+0x165 union_lookup(ef1ffb18,c4cb6000,c21b8809,ef1ffb34,c051dd09) at union_lookup+0x2cf VOP_LOOKUP_APV(c4cc2240,ef1ffb18) at VOP_LOOKUP_APV+0x43 lookup(ef1ffc00,0,0,c23a9a80,ef1ffc28) at lookup+0x431 namei(ef1ffc00,c05aba97,ef1ffbbc,ef1ffbbc,c37a4948) at namei+0x396 kern_stat(c23a9a80,80b59d8,0,ef1ffc74) at kern_stat+0x35 stat(c23a9a80,ef1ffd04,2,4,297) at stat+0x1b syscall(3b,3b,3b,ffffffff,0) at syscall+0x2bb Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (188, FreeBSD ELF32, stat), eip = 0x8075858, esp = 0xbfbfeab4, ebp = 0xbfbfeb50 --- db> trace 1461 Thread 1461 not found db> panic panic: from debugger cpuid = 0 Uptime: 24m19s Dumping 1023 MB (2 chunks) chunk 0: 1MB (159 pages) ... ok chunk 1: 1023MB (261872 pages) 1007 991 975 959 943 927 911 895 879 863 847 831 815 799 783 767 751 735 719 703 687 671 655 639 623 607 591 575 559 543 527 511 495 479 463 447 431 415 399 383 367 351 335 319 303 287 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15 ... ok Dump complete Automatic reboot in 15 seconds - press a key on the console to abort Rebooting... >How-To-Repeat: This hang happens at the same point every time I do this, so repeating is a simple matter of mounting proc/unionfs for the jail, and then running the jail command to start things up ... >Fix: >Release-Note: >Audit-Trail: >Unformatted: