Date: Tue, 9 Mar 2010 17:22:20 +0100 From: =?ISO-8859-1?Q?Johan_Str=F6m?= <johan@stromnet.se> To: =?ISO-8859-1?Q?Johan_Str=F6m?= <johan@stromnet.se> Cc: freebsd-fs@freebsd.org Subject: Re: ZFS: zpool import hang on "zio->io_cv)" (with DDB output). Help needed! Message-ID: <A6C7D84B-18D8-4614-A98C-4D8DC64A952C@stromnet.se> In-Reply-To: <8BA8475A-424E-489B-B643-8757F375320B@stromnet.se> References: <8BA8475A-424E-489B-B643-8757F375320B@stromnet.se>
next in thread | previous in thread | raw e-mail | index | archive | help
Some followup, pjd tried to help me on IRC, without getting much =20 further on the actual problem. Some more information: Output from DDB PS: http://www.stromnet.se/~johan/back-1-ddb-ps.txt Output from alltrace: = http://www.stromnet.se/~johan/back-1-ddb-alltrace.txt db> show alllocks Process 2738 (sshd) thread 0xffffff0050418000 (100178) Process 12 (intr) thread 0xffffff0002407ab0 (100018) db> show lockedvnods Locked vnodes db > Also tried accessing the disks (to make sure interrupts etc where =20 alive) ,using dd if=3D/dev/adXX of=3D/dev/null count=3D1 on every disk = on =20 the system, works fine. Rebooted the system and tried with zfs debugging enabled too, and =20 doing zpool import tank instead of just zpool import. no difference. =20 Hanging there now. Johan On Mar 9, 2010, at 15:26 , Johan Str=F6m wrote: > Hi List! > > I'm in the tedious process of upgrading my pool on my FreeBSD 8.0 =20 > box (with a pool from 7.x). Yesterday I pulled a few disks from my =20 > mirrored pool (one pool with multiple mirrors), in order to be able =20= > to (free ports) plug in new ones and build my brand new pool. > After removing disks, the pool got into state DEGRADED ofcourse, =20 > since some of the disks where gone, but data was still there. I did =20= > zpool detach on the removed disks, and the pool was ONLINE again, =20 > all fine! > Then I rebooted to make another disk available through my old LSI =20 > MegaRaid card (didnt take time to figure out cryptic syntax of the =20 > megarc CLI util). On boot, I was met by something similar to this: > > pool: tank > state: UNAVAIL > status: One or more devices could not be used because the label is =20 > missing > or invalid. There are insufficient replicas for the pool to =20= > continue > functioning. > action: Destroy and re-create the pool from a backup source. > see: http://www.sun.com/msg/ZFS-8000-5E > scrub: none requested > config: > > NAME STATE READ WRITE CKSUM > tank UNAVAIL 0 0 0 insufficient replicas > ad10s1d ONLINE 0 0 0 > mirror DEGRADED 0 0 0 > ad12 FAULTED 0 0 0 corrupted data > ad16 UNAVAIL 0 0 0 corrupted data > mirror DEGRADED 0 0 0 > ad20 FAULTED 0 0 0 corrupted data > ad18 UNAVAIL 0 0 0 corrupted data > mirror UNAVAIL 0 0 0 insufficient replicas > ad6 UNAVAIL 0 0 0 corrupted data > ad4 FAULTED 0 0 0 corrupted data > mirror ONLINE 0 0 0 > amrd0 ONLINE 0 0 0 > amrd1 ONLINE 0 0 0 > > The reason the disks was still there was due to me haveing an old =20 > zfs.cache file in the boot (it boots from anothre drive, but thats =20 > another story..) > > In this case, a export/import did the trick, after reimporting, the =20= > pool was back online. All fine. I created a new pool and did some =20 > bonnie++ testing on it, and suddenly the box paniced or something (I =20= > didn't have dumpon enabled :/ and didn't see the screen until it =20 > rebooted).. > > Now my problems come. The box came up again, with the above output. =20= > I tried zpool export again, fine. Zpool import however, hang. Waited =20= > an hour, nothing. After rebuilding the kernel with DDB/witness and =20 > did the import again, i've managed to get this output (im not really =20= > sure what is usable here): > > back-1 # zpool import > load: 0.29 cmd: zpool 3193 [zio->io_cv)] 2.54r 0.00u 0.01s 0% 2236k > > > > In DDB: > > tr 3193 > > Tracing pid 3193 tid 100122 td 0xffffff00035ca390 > sched_switch() at sched_switch+0xde > mi_switch() at mi_switch+0x170 > sleepq_wait() at sleepq_wait+0x44 > _cv_wait() at _cv_wait+0x13c > zio_wait() at zio_wait+0x61 > arc_read_nolock() at arc_read_nolock+0x345 > dmu_objset_open_impl() at dmu_objset_open_impl+0xd0 > dsl_pool_open() at dsl_pool_open+0x5a > spa_load() at spa_load+0x31b > spa_tryimport() at spa_tryimport+0xa9 > zfs_ioc_pool_tryimport() at zfs_ioc_pool_tryimport+0x3f > zfsdev_ioctl() at zfsdev_ioctl+0x8d > devfs_ioctl_f() at devfs_ioctl_f+0x76 > kern_ioctl() at kern_ioctl+0xf6 > ioctl() at ioctl+0xfd > syscall() at syscall+0x19e > Xfast_syscall() at Xfast_syscall+0xe1 > --- syscall (54, FreeBSD ELF64, ioctl), rip =3D 0x8010eb86c, rsp =3D =20= > 0x7fffffff8e28, rbp =3D 0x801323300 --- > > > db> show thread 100122 > > Thread 100122 at 0xffffff00035ca390: > proc (pid 3193): 0xffffff000383c460 > name: zpool > stack: 0xffffff805740a000-0xffffff805740dfff > flags: 0x44 pflags: 0x10000 > state: INHIBITED: {SLEEPING} > wmesg: zio->io_cv) wchan: 0xffffff00506e5858 > priority: 131 > container lock: sleepq chain (0xffffffff80c61e68) > db> > > db> show sleepchain 10012 > > db> show sleepchain 3193 > > thread 100122 (pid 3193, zpool) sleeping on 0xffffff00506e5858 "zio-=20= > >io_cv)" > db> show lock 0xffffff00506e5858 > > class: spin mutex > name: zio->io_cv) > flags: {SPIN} > state: {OWNED} > > > > > The box is currenlty in this state, so if you reply to me now I can =20= > continue to do debugging according to instructions. > Since I cannot get anywhere on my own with this, all and any help is =20= > appreciated since I really need this pool to get online.. > > dmesg is posted below (including some LOR?) > > Thanks! > Johan > > > > DMESG: > > Copyright (c) 1992-2009 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, =20= > 1994 > The Regents of the University of California. All rights =20 > reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 8.0-RELEASE-p2 #10: Tue Mar 9 12:44:15 CET 2010 > johan@back-1.stromnet.se:/usr/obj/usr/src/sys/BACK1 > WARNING: WITNESS option enabled, expect reduced performance. > Timecounter "i8254" frequency 1193182 Hz quality 0 > CPU: Intel(R) Core(TM)2 Duo CPU E6750 @ 2.66GHz (2666.68-MHz K8-=20= > class CPU) > Origin =3D "GenuineIntel" Id =3D 0x6fb Stepping =3D 11 > =20 > Features=20 > =3D=20 > 0xbfebfbff=20 > <=20 > FPU=20 > ,VME=20 > ,DE=20 > ,PSE=20 > ,TSC=20 > ,MSR=20 > ,PAE=20 > ,MCE=20 > ,CX8=20 > ,APIC=20 > ,SEP=20 > ,MTRR=20 > ,PGE=20 > ,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> > =20 > Features2=20 > =3D0xe3fd<SSE3,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM> > AMD Features=3D0x20100800<SYSCALL,NX,LM> > AMD Features2=3D0x1<LAHF> > TSC: P-state invariant > real memory =3D 2147483648 (2048 MB) > avail memory =3D 2040631296 (1946 MB) > ACPI APIC Table: <GBT GBTUACPI> > FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs > FreeBSD/SMP: 1 package(s) x 2 core(s) > cpu0 (BSP): APIC ID: 0 > cpu1 (AP): APIC ID: 1 > ioapic0: Changing APIC ID to 2 > ioapic0 <Version 2.0> irqs 0-23 on motherboard > kbd1 at kbdmux0 > cryptosoft0: <software crypto> on motherboard > acpi0: <GBT GBTUACPI> on motherboard > acpi0: [ITHREAD] > acpi0: Power Button (fixed) > acpi0: reservation of 0, a0000 (3) failed > acpi0: reservation of 100000, 7f4e0000 (3) failed > Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 > acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 > acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff =20= > on acpi0 > Timecounter "HPET" frequency 14318180 Hz quality 900 > acpi_button0: <Power Button> on acpi0 > pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 > pci0: <ACPI PCI bus> on pcib0 > vgapci0: <VGA-compatible display> port 0xe000-0xe007 mem =20 > 0xe6300000-0xe637ffff,0xd0000000-0xdfffffff,0xe6000000-0xe60fffff =20 > irq 16 at device 2.0 on pci0 > agp0: <Intel G33 SVGA controller> on vgapci0 > agp0: detected 7164k stolen memory > agp0: aperture size is 256M > uhci0: <Intel 82801I (ICH9) USB controller> port 0xe100-0xe11f irq =20 > 16 at device 26.0 on pci0 > uhci0: [ITHREAD] > uhci0: LegSup =3D 0x2f00 > usbus0: <Intel 82801I (ICH9) USB controller> on uhci0 > uhci1: <Intel 82801I (ICH9) USB controller> port 0xe200-0xe21f irq =20 > 21 at device 26.1 on pci0 > uhci1: [ITHREAD] > uhci1: LegSup =3D 0x2f00 > usbus1: <Intel 82801I (ICH9) USB controller> on uhci1 > uhci2: <Intel 82801I (ICH9) USB controller> port 0xe600-0xe61f irq =20 > 18 at device 26.2 on pci0 > uhci2: [ITHREAD] > uhci2: LegSup =3D 0x2f00 > usbus2: <Intel 82801I (ICH9) USB controller> on uhci2 > ehci0: <Intel 82801I (ICH9) USB 2.0 controller> mem =20 > 0xe6384000-0xe63843ff irq 18 at device 26.7 on pci0 > ehci0: [ITHREAD] > usbus3: EHCI version 1.0 > usbus3: <Intel 82801I (ICH9) USB 2.0 controller> on ehci0 > hdac0: <Intel 82801I High Definition Audio Controller> mem =20 > 0xe6380000-0xe6383fff irq 22 at device 27.0 on pci0 > hdac0: HDA Driver Revision: 20090624_0136 > hdac0: [ITHREAD] > pcib1: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0 > pci1: <ACPI PCI bus> on pcib1 > pcib2: <ACPI PCI-PCI bridge> irq 18 at device 28.2 on pci0 > pci2: <ACPI PCI bus> on pcib2 > em0: <Intel(R) PRO/1000 Network Connection 6.9.14> port =20 > 0xa000-0xa01f mem 0xe1020000-0xe103ffff,0xe1000000-0xe101ffff irq 18 =20= > at device 0.0 on pci2 > em0: Using MSI interrupt > em0: [FILTER] > em0: Ethernet address: 00:1b:21:05:00:b4 > pcib3: <ACPI PCI-PCI bridge> irq 19 at device 28.3 on pci0 > pci3: <ACPI PCI bus> on pcib3 > atapci0: <JMicron JMB363 SATA300 controller> port =20 > 0xb000=20 > -0xb007,0xb100-0xb103,0xb200-0xb207,0xb300-0xb303,0xb400-0xb40f mem =20= > 0xe6100000-0xe6101fff irq 19 at device 0.0 on pci3 > atapci0: [ITHREAD] > atapci0: AHCI called from vendor specific driver > atapci0: AHCI v1.00 controller with 2 3Gbps ports, PM supported > ata2: <ATA channel 0> on atapci0 > ata2: [ITHREAD] > ata3: <ATA channel 1> on atapci0 > ata3: [ITHREAD] > ata4: <ATA channel 2> on atapci0 > ata4: [ITHREAD] > pcib4: <ACPI PCI-PCI bridge> irq 16 at device 28.4 on pci0 > pci4: <ACPI PCI bus> on pcib4 > re0: <RealTek 8168/8168B/8168C/8168CP/8168D/8168DP/8111B/8111C/=20 > 8111CP/8111DP PCIe Gigabit Ethernet> port 0xc000-0xc0ff mem =20 > 0xe3000000-0xe3000fff irq 16 at device 0.0 on pci4 > re0: Using 1 MSI messages > re0: Chip rev. 0x38000000 > re0: MAC rev. 0x00000000 > miibus0: <MII bus> on re0 > rgephy0: <RTL8169S/8110S/8211B media interface> PHY 1 on miibus0 > rgephy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, =20= > 1000baseT-FDX, auto > re0: Ethernet address: 00:1a:4d:5a:97:87 > re0: [FILTER] > uhci3: <Intel 82801I (ICH9) USB controller> port 0xe300-0xe31f irq =20 > 23 at device 29.0 on pci0 > uhci3: [ITHREAD] > uhci3: LegSup =3D 0x2f00 > usbus4: <Intel 82801I (ICH9) USB controller> on uhci3 > uhci4: <Intel 82801I (ICH9) USB controller> port 0xe400-0xe41f irq =20 > 19 at device 29.1 on pci0 > uhci4: [ITHREAD] > uhci4: LegSup =3D 0x2f00 > usbus5: <Intel 82801I (ICH9) USB controller> on uhci4 > uhci5: <Intel 82801I (ICH9) USB controller> port 0xe500-0xe51f irq =20 > 18 at device 29.2 on pci0 > uhci5: [ITHREAD] > uhci5: LegSup =3D 0x2f00 > usbus6: <Intel 82801I (ICH9) USB controller> on uhci5 > ehci1: <Intel 82801I (ICH9) USB 2.0 controller> mem =20 > 0xe6385000-0xe63853ff irq 23 at device 29.7 on pci0 > ehci1: [ITHREAD] > usbus7: EHCI version 1.0 > usbus7: <Intel 82801I (ICH9) USB 2.0 controller> on ehci1 > pcib5: <ACPI PCI-PCI bridge> at device 30.0 on pci0 > pci5: <ACPI PCI bus> on pcib5 > amr0: <LSILogic MegaRAID 1.53> mem 0xe6200000-0xe620ffff irq 20 at =20 > device 0.0 on pci5 > amr0: Using 64-bit DMA > amr0: [ITHREAD] > amr0: delete logical drives supported by controller > amr0: <LSILogic MegaRAID SATA 150-4D> Firmware 713S, BIOS G121, 64MB =20= > RAM > xl0: <3Com 3c905C-TX Fast Etherlink XL> port 0xd000-0xd07f mem =20 > 0xe5004000-0xe500407f irq 19 at device 1.0 on pci5 > miibus1: <MII bus> on xl0 > xlphy0: <3c905C 10/100 internal PHY> PHY 24 on miibus1 > xlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto > xl0: Ethernet address: 00:04:76:ef:c6:36 > xl0: [ITHREAD] > skc0: <SysKonnect Gigabit Ethernet (V1.0)> port 0xd100-0xd1ff mem =20 > 0xe5000000-0xe5003fff irq 18 at device 2.0 on pci5 > pci0:5:2:0: invalid VPD data, remain 0xfc > skc0: SysKonnect SK-NET Gigabit Ethernet Adapter SK-9843 SX rev. (0x0) > sk0: <XaQti Corp. XMAC II> on skc0 > sk0: Ethernet address: 00:00:5a:98:43:68 > miibus2: <MII bus> on sk0 > xmphy0: <XaQti Corp. XMAC II gigabit interface> PHY 0 on miibus2 > xmphy0: 1000baseSX, 1000baseSX-FDX, auto > skc0: [ITHREAD] > isab0: <PCI-ISA bridge> at device 31.0 on pci0 > isa0: <ISA bus> on isab0 > atapci1: <Intel ICH9 SATA300 controller> port =20 > 0xe700=20 > -0xe707,0xe800-0xe803,0xe900-0xe907,0xea00-0xea03,0xeb00-0xeb1f mem =20= > 0xe6386000-0xe63867ff irq 19 at device 31.2 on pci0 > atapci1: [ITHREAD] > atapci1: AHCI called from vendor specific driver > atapci1: AHCI v1.20 controller with 6 3Gbps ports, PM supported > ata5: <ATA channel 0> on atapci1 > ata5: [ITHREAD] > ata6: <ATA channel 1> on atapci1 > ata6: [ITHREAD] > ata7: <ATA channel 2> on atapci1 > ata7: [ITHREAD] > ata8: <ATA channel 3> on atapci1 > ata8: [ITHREAD] > ata9: <ATA channel 4> on atapci1 > ata9: [ITHREAD] > ata10: <ATA channel 5> on atapci1 > ata10: [ITHREAD] > pci0: <serial bus, SMBus> at device 31.3 (no driver attached) > atrtc0: <AT realtime clock> port 0x70-0x73 on acpi0 > fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 =20 > on acpi0 > fdc0: [FILTER] > uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on =20 > acpi0 > uart0: [FILTER] > ppc0: <Parallel port> port 0x378-0x37f irq 7 on acpi0 > ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode > ppc0: [ITHREAD] > ppbus0: <Parallel port bus> on ppc0 > plip0: <PLIP network interface> on ppbus0 > plip0: [ITHREAD] > lpt0: <Printer> on ppbus0 > lpt0: [ITHREAD] > lpt0: Interrupt-driven port > ppi0: <Parallel I/O> on ppbus0 > atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0 > atkbd0: <AT Keyboard> irq 1 on atkbdc0 > kbd0 at atkbd0 > atkbd0: [GIANT-LOCKED] > atkbd0: [ITHREAD] > cpu0: <ACPI CPU> on acpi0 > est0: <Enhanced SpeedStep Frequency Control> on cpu0 > est: CPU supports Enhanced Speedstep, but is not recognized. > est: cpu_vendor GenuineIntel, msr 82a082a0600082a > device_attach: est0 attach returned 6 > p4tcc0: <CPU Frequency Thermal Control> on cpu0 > cpu1: <ACPI CPU> on acpi0 > est1: <Enhanced SpeedStep Frequency Control> on cpu1 > est: CPU supports Enhanced Speedstep, but is not recognized. > est: cpu_vendor GenuineIntel, msr 82a082a0600082a > device_attach: est1 attach returned 6 > p4tcc1: <CPU Frequency Thermal Control> on cpu1 > orm0: <ISA Option ROMs> at iomem 0xcc000-0xcc7ff,0xcd000-0xcefff on =20= > isa0 > sc0: <System console> at flags 0x100 on isa0 > sc0: VGA <16 virtual consoles, flags=3D0x300> > vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on =20= > isa0 > ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM =20= > is present; > to enable, add "vfs.zfs.prefetch_disable=3D0" to /boot/=20 > loader.conf. > ZFS filesystem version 13 > ZFS storage pool version 13 > Timecounters tick every 1.000 msec > IPsec: Initialized Security Association Processing. > usbus0: 12Mbps Full Speed USB v1.0 > usbus1: 12Mbps Full Speed USB v1.0 > usbus2: 12Mbps Full Speed USB v1.0 > usbus3: 480Mbps High Speed USB v2.0 > usbus4: 12Mbps Full Speed USB v1.0 > usbus5: 12Mbps Full Speed USB v1.0 > usbus6: 12Mbps Full Speed USB v1.0 > usbus7: 480Mbps High Speed USB v2.0 > ad4: 476938MB <Seagate ST3500630NS 3.AEG> at ata2-master SATA300 > ugen0.1: <Intel> at usbus0 > uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on =20 > usbus0 > ugen1.1: <Intel> at usbus1 > uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on =20 > usbus1 > ugen2.1: <Intel> at usbus2 > uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on =20 > usbus2 > ugen3.1: <Intel> at usbus3 > uhub3: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on =20 > usbus3 > ugen4.1: <Intel> at usbus4 > uhub4: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on =20 > usbus4 > ugen5.1: <Intel> at usbus5 > uhub5: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on =20 > usbus5 > ugen6.1: <Intel> at usbus6 > uhub6: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on =20 > usbus6 > ugen7.1: <Intel> at usbus7 > uhub7: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on =20 > usbus7 > ad6: 1907729MB <WDC WD20EARS-00S8B1 80.00A80> at ata3-master SATA300 > uhub0: 2 ports with 2 removable, self powered > uhub1: 2 ports with 2 removable, self powered > uhub2: 2 ports with 2 removable, self powered > uhub4: 2 ports with 2 removable, self powered > uhub5: 2 ports with 2 removable, self powered > uhub6: 2 ports with 2 removable, self powered > ad10: 715404MB <WDC WD7500AAKS-00RBA0 30.04G30> at ata5-master SATA300 > ad12: 305245MB <Seagate ST3320620AS 3.AAE> at ata6-master SATA150 > ad14: 715404MB <WDC WD7500AAKS-00RBA0 30.04G30> at ata7-master SATA300 > ad16: 1907729MB <WDC WD20EARS-00S8B1 80.00A80> at ata8-master SATA300 > GEOM_MIRROR: Device mirror/gm1a launched (1/1). > GEOM_MIRROR: Device mirror/gm1b launched (1/1). > ad18: 1907729MB <WDC WD20EARS-00S8B1 80.00A80> at ata9-master SATA300 > GEOM_MIRROR: Device mirror/swap launched (1/1). > ad20: 286187MB <Maxtor 7L300S0 BANC1G10> at ata10-master SATA150 > hdac0: HDA Codec #2: Realtek ALC885 > pcm0: <HDA Realtek ALC885 PCM #0 Analog> at cad 2 nid 1 on hdac0 > pcm1: <HDA Realtek ALC885 PCM #1 Analog> at cad 2 nid 1 on hdac0 > pcm2: <HDA Realtek ALC885 PCM #2 Analog> at cad 2 nid 1 on hdac0 > pcm3: <HDA Realtek ALC885 PCM #3 Analog> at cad 2 nid 1 on hdac0 > pcm4: <HDA Realtek ALC885 PCM #4 Digital> at cad 2 nid 1 on hdac0 > pcm5: <HDA Realtek ALC885 PCM #5 Analog> at cad 2 nid 1 on hdac0 > amr0: delete logical drives supported by controller > amrd0: <LSILogic MegaRAID logical drive> on amr0 > amrd0: 476935MB (976762880 sectors) RAID 0 (optimal) > amrd1: <LSILogic MegaRAID logical drive> on amr0 > amrd1: 476935MB (976762880 sectors) RAID 0 (optimal) > amrd2: <LSILogic MegaRAID logical drive> on amr0 > amrd2: 1907724MB (3907018752 sectors) RAID 0 (optimal) > SMP: AP CPU #1 Launched! > WARNING: WITNESS option enabled, expect reduced performance. > Root mount waiting for: usbus7 usbus3 > uhub3: 6 ports with 6 removable, self powered > uhub7: 6 ports with 6 removable, self powered > Trying to mount root from zfs:zroot > ugen0.2: <Telldus> at usbus0 > ugen1.2: <American Power Conversion> at usbus1 > uma_zalloc_arg: zone "256" with the following non-sleepable locks =20 > held: > exclusive rw ifnet_rw (ifnet_rw) r =3D 0 (0xffffffff80e01f60) locked =20= > @ /usr/src/sys/net/if.c:402 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a > _witness_debugger() at _witness_debugger+0x2c > witness_warn() at witness_warn+0x2c2 > uma_zalloc_arg() at uma_zalloc_arg+0x29d > malloc() at malloc+0x5d > if_grow() at if_grow+0x2f > if_alloc() at if_alloc+0x2b3 > gif_clone_create() at gif_clone_create+0x53 > ifc_simple_create() at ifc_simple_create+0x89 > if_clone_createif() at if_clone_createif+0x64 > ifioctl() at ifioctl+0x6b5 > kern_ioctl() at kern_ioctl+0xf6 > ioctl() at ioctl+0xfd > syscall() at syscall+0x19e > Xfast_syscall() at Xfast_syscall+0xe1 > --- syscall (54, FreeBSD ELF64, ioctl), rip =3D 0x800b8286c, rsp =3D =20= > 0x7fffffffe4a8, rbp =3D 0x7fffffffef6e --- > lock order reversal: > 1st 0xffffffff80c093e0 pf task mtx (pf task mtx) @ /usr/src/sys/=20 > contrib/pf/net/pf_ioctl.c:1393 > 2nd 0xffffffff80e01f60 ifnet_rw (ifnet_rw) @ /usr/src/sys/net/if.c:=20 > 2034 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a > _witness_debugger() at _witness_debugger+0x2c > witness_checkorder() at witness_checkorder+0x66f > _rw_rlock() at _rw_rlock+0x29 > ifunit() at ifunit+0x22 > pfioctl() at pfioctl+0x262a > devfs_ioctl_f() at devfs_ioctl_f+0x76 > kern_ioctl() at kern_ioctl+0xf6 > ioctl() at ioctl+0xfd > syscall() at syscall+0x19e > Xfast_syscall() at Xfast_syscall+0xe1 > --- syscall (54, FreeBSD ELF64, ioctl), rip =3D 0x80099886c, rsp =3D =20= > 0x7fffffffdb68, rbp =3D 0x7fffffffdc20 --- > lock order reversal: > 1st 0xffffff00500c3098 zfs (zfs) @ /usr/src/sys/kern/vfs_mount.c:1054 > 2nd 0xffffff005010f448 devfs (devfs) @ /usr/src/sys/kern/vfs_subr.c:=20= > 2083 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a > _witness_debugger() at _witness_debugger+0x2c > witness_checkorder() at witness_checkorder+0x66f > __lockmgr_args() at __lockmgr_args+0x475 > vop_stdlock() at vop_stdlock+0x39 > VOP_LOCK1_APV() at VOP_LOCK1_APV+0x46 > _vn_lock() at _vn_lock+0x47 > vget() at vget+0x56 > devfs_allocv() at devfs_allocv+0x103 > devfs_root() at devfs_root+0x48 > vfs_donmount() at vfs_donmount+0xf43 > nmount() at nmount+0x63 > syscall() at syscall+0x19e > Xfast_syscall() at Xfast_syscall+0xe1 > --- syscall (378, FreeBSD ELF64, nmount), rip =3D 0x8007b04dc, rsp =3D = =20 > 0x7fffffffdd28, rbp =3D 0x800a04048 --- > n > > r > tun0: link state changed to UP > lock order reversal: > 1st 0xffffff00500c3098 zfs (zfs) @ /usr/src/sys/kern/vfs_mount.c:1200 > 2nd 0xffffff005010f270 syncer (syncer) @ /usr/src/sys/kern/=20 > vfs_subr.c:2188 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a > _witness_debugger() at _witness_debugger+0x2c > witness_checkorder() at witness_checkorder+0x66f > __lockmgr_args() at __lockmgr_args+0x475 > vop_stdlock() at vop_stdlock+0x39 > VOP_LOCK1_APV() at VOP_LOCK1_APV+0x46 > _vn_lock() at _vn_lock+0x47 > vrele() at vrele+0xc3 > dounmount() at dounmount+0x269 > unmount() at unmount+0x27e > syscall() at syscall+0x19e > Xfast_syscall() at Xfast_syscall+0xe1 > --- syscall (22, FreeBSD ELF64, unmount), rip =3D 0x8006a09bc, rsp =3D = =20 > 0x7fffffffde18, rbp =3D 0 --- > KDB: enter: manual escape to debugger > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?A6C7D84B-18D8-4614-A98C-4D8DC64A952C>