Date: Fri, 11 Sep 2009 11:56:49 +1000 From: Duncan Young <duncan.young@pobox.com> To: Boris Kochergin <spawk@acm.poly.edu> Cc: freebsd-current@freebsd.org Subject: Re: zfs woes Message-ID: <4AA9AE61.7060601@pobox.com> In-Reply-To: <4AA98FCA.9090506@acm.poly.edu> References: <4AA981F4.3090607@pobox.com> <4AA98FCA.9090506@acm.poly.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
Boris Kochergin wrote: > Duncan Young wrote: >> Hi all, >> >> Again its happened (my need to restore my main zpool from backups). >> >> I have been having regular crashes (only recently been getting crash >> dumps though), and in the the process >> of upgrading to 7.1-STABLE, upgrading zpools (now 13) and zfs (now 3), I >> now have a problem with >> my main data zpool. >> >> I was having these problems before the upgrade. >> >> After zpool status saying that I had corrupted files (actually a zfs >> filesystem), I tried destroying it (the zfs filesystem), in the hope >> that this >> may make my problems go away). The machine hung. >> >> zpool scrub works, reporting 16 errors. >> zpool status -v crashes. >> starting multiuser hangs (probably on mounting the dud fs). >> I can start single user, and I am currently additional backups of the >> filesytems I can access (most off the corrupted zpool). >> Any accessing the corrupted zfs filesystem crashes the machine. >> >> Now I think it's fair enough that the filesystem is corrupted (I still >> see a need for some type of fsck), but I don't see that the >> machine should crash because of it. Any sugestions? >> >> Script started on Fri Sep 11 17:22:23 2009 >> # uname -a >> FreeBSD 7.2-STABLE FreeBSD 7.2-STABLE #8: Fri Sep 11 04:14:39 EST >> 2009 root@:/usr/obj/usr/src/sys/TRIPLE0 amd64 >> # cd /usr/src/sys/TRIPLE0 >> # kgdb ./kernel.debug /var/crash/vmcore/.25 >> GNU gdb 6.1.1 [FreeBSD] >> Copyright 2004 Free Software Foundation, Inc. >> GDB is free software, covered by the GNU General Public License, and >> you are >> welcome to change it and/or distribute copies of it under certain >> conditions. >> Type "show copying" to see the conditions. >> There is absolutely no warranty for GDB. Type "show warranty" for >> details. >> This GDB was configured as "amd64-marcel-freebsd"... >> >> Unread portion of the kernel message buffer: >> Copyright (c) 1992-2009 The FreeBSD Project. >> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 >> The Regents of the University of California. All rights reserved. >> FreeBSD is a registered trademark of The FreeBSD Foundation. >> FreeBSD 7.2-STABLE #8: Fri Sep 11 04:14:39 EST 2009 >> root@:/usr/obj/usr/src/sys/TRIPLE0 >> module_register: module g_label already exists! >> Module g_label failed to register: 17 >> Timecounter "i8254" frequency 1193182 Hz quality 0 >> CPU: AMD Phenom(tm) II X4 810 Processor (2608.81-MHz K8-class CPU) >> Origin = "AuthenticAMD" Id = 0x100f42 Stepping = 2 >> >> Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT> >> >> Features2=0x802009<SSE3,MON,CX16,POPCNT> >> AMD >> Features=0xee500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM,3DNow!+,3DNow!> >> >> AMD >> Features2=0x37ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,SKINIT,WDT> >> >> TSC: P-state invariant >> Cores per package: 4 >> usable memory = 8441266176 (8050 MB) >> avail memory = 8131702784 (7754 MB) >> ACPI APIC Table: <072709 APIC1045> >> FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs >> cpu0 (BSP): APIC ID: 0 >> cpu1 (AP): APIC ID: 1 >> cpu2 (AP): APIC ID: 2 >> cpu3 (AP): APIC ID: 3 >> This module (opensolaris) contains code covered by the >> Common Development and Distribution License (CDDL) >> see http://opensolaris.org/os/licensing/opensolaris_license/ >> ACPI Warning (tbfadt-0505): Optional field "Pm2ControlBlock" has zero >> address or length: 0 0/1 [20070320] >> ioapic0 <Version 2.1> irqs 0-23 on motherboard >> kbd1 at kbdmux0 >> acpi0: <072709 XSDT1045> on motherboard >> acpi0: [ITHREAD] >> acpi0: Power Button (fixed) >> acpi0: reservation of fee00000, 1000 (3) failed >> acpi0: reservation of ffb80000, 80000 (3) failed >> acpi0: reservation of fec10000, 20 (3) failed >> acpi0: reservation of 0, a0000 (3) failed >> acpi0: reservation of 100000, dff00000 (3) failed >> ACPI HPET table warning: Sequence is non-zero (2) >> Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 >> acpi_timer0: <32-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 >> acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on >> acpi0 >> Timecounter "HPET" frequency 14318180 Hz quality 900 >> pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 >> pci0: <ACPI PCI bus> on pcib0 >> pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0 >> pci1: <ACPI PCI bus> on pcib1 >> vgapci0: <VGA-compatible display> port 0xc000-0xc0ff mem >> 0xf0000000-0xf7ffffff,0xfbce0000-0xfbceffff,0xfbb00000-0xfbbfffff irq 18 >> at device 5.0 on pci1 >> hdac0: <ATI RS780 High Definition Audio Controller> mem >> 0xfbcfc000-0xfbcfffff irq 19 at device 5.1 on pci1 >> hdac0: HDA Driver Revision: 20090624_0136 >> hdac0: [ITHREAD] >> pcib2: <ACPI PCI-PCI bridge> irq 16 at device 4.0 on pci0 >> pci2: <ACPI PCI bus> on pcib2 >> hptrr0: <sx508x> port 0xd800-0xd8ff mem 0xfbe00000-0xfbefffff irq 16 at >> device 0.0 on pci2 >> hptrr: adapter at PCI 2:0:0, IRQ 16 >> pcib3: <ACPI PCI-PCI bridge> irq 17 at device 5.0 on pci0 >> pci3: <ACPI PCI bus> on pcib3 >> em0: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xec00-0xec1f mem >> 0xfbfe0000-0xfbffffff,0xfbf00000-0xfbf7ffff,0xfbfdc000-0xfbfdffff irq 17 >> at device 0.0 on pci3 >> em0: Using MSIX interrupts >> em0: [ITHREAD] >> em0: [ITHREAD] >> em0: [ITHREAD] >> em0: Ethernet address: 00:1b:21:3d:eb:e7 >> atapci0: <ATI AHCI controller> port >> 0xb000-0xb007,0xa000-0xa003,0x9000-0x9007,0x8000-0x8003,0x7000-0x700f >> mem 0xfbaffc00-0xfbafffff irq 22 at device 17.0 on pci0 >> atapci0: [ITHREAD] >> atapci0: AHCI Version 01.10 controller with 6 ports detected >> ata2: <ATA channel 0> on atapci0 >> ata2: [ITHREAD] >> ata3: <ATA channel 1> on atapci0 >> ata3: [ITHREAD] >> ata4: <ATA channel 2> on atapci0 >> ata4: [ITHREAD] >> ata5: <ATA channel 3> on atapci0 >> ata5: [ITHREAD] >> ata6: <ATA channel 4> on atapci0 >> ata6: [ITHREAD] >> ata7: <ATA channel 5> on atapci0 >> ata7: [ITHREAD] >> ohci0: <OHCI (generic) USB controller> mem 0xfbafd000-0xfbafdfff irq 16 >> at device 18.0 on pci0 >> ohci0: [GIANT-LOCKED] >> ohci0: [ITHREAD] >> usb0: OHCI version 1.0, legacy support >> usb0: <OHCI (generic) USB controller> on ohci0 >> usb0: USB revision 1.0 >> uhub0: <ATI OHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0 >> uhub0: 3 ports with 3 removable, self powered >> ohci1: <OHCI (generic) USB controller> mem 0xfbafe000-0xfbafefff irq 16 >> at device 18.1 on pci0 >> ohci1: [GIANT-LOCKED] >> ohci1: [ITHREAD] >> usb1: OHCI version 1.0, legacy support >> usb1: <OHCI (generic) USB controller> on ohci1 >> usb1: USB revision 1.0 >> uhub1: <ATI OHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb1 >> uhub1: 3 ports with 3 removable, self powered >> ehci0: <EHCI (generic) USB 2.0 controller> mem 0xfbaff800-0xfbaff8ff irq >> 17 at device 18.2 on pci0 >> ehci0: [GIANT-LOCKED] >> ehci0: [ITHREAD] >> usb2: EHCI version 1.0 >> usb2: companion controllers, 3 ports each: usb0 usb1 >> usb2: <EHCI (generic) USB 2.0 controller> on ehci0 >> usb2: USB revision 2.0 >> uhub2: <ATI EHCI root hub, class 9/0, rev 2.00/1.00, addr 1> on usb2 >> uhub2: 6 ports with 6 removable, self powered >> ohci2: <OHCI (generic) USB controller> mem 0xfbafb000-0xfbafbfff irq 18 >> at device 19.0 on pci0 >> ohci2: [GIANT-LOCKED] >> ohci2: [ITHREAD] >> usb3: OHCI version 1.0, legacy support >> usb3: <OHCI (generic) USB controller> on ohci2 >> usb3: USB revision 1.0 >> uhub3: <ATI OHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb3 >> uhub3: 3 ports with 3 removable, self powered >> ohci3: <OHCI (generic) USB controller> mem 0xfbafc000-0xfbafcfff irq 18 >> at device 19.1 on pci0 >> ohci3: [GIANT-LOCKED] >> ohci3: [ITHREAD] >> usb4: OHCI version 1.0, legacy support >> usb4: <OHCI (generic) USB controller> on ohci3 >> usb4: USB revision 1.0 >> uhub4: <ATI OHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb4 >> uhub4: 3 ports with 3 removable, self powered >> ehci1: <EHCI (generic) USB 2.0 controller> mem 0xfbaff400-0xfbaff4ff irq >> 19 at device 19.2 on pci0 >> ehci1: [GIANT-LOCKED] >> ehci1: [ITHREAD] >> usb5: EHCI version 1.0 >> usb5: companion controllers, 3 ports each: usb3 usb4 >> usb5: <EHCI (generic) USB 2.0 controller> on ehci1 >> usb5: USB revision 2.0 >> uhub5: <ATI EHCI root hub, class 9/0, rev 2.00/1.00, addr 1> on usb5 >> uhub5: 6 ports with 6 removable, self powered >> pci0: <serial bus, SMBus> at device 20.0 (no driver attached) >> atapci1: <ATI IXP700 UDMA133 controller> port >> 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xff00-0xff0f at device 20.1 on pci0 >> ata0: <ATA channel 0> on atapci1 >> ata0: [ITHREAD] >> hdac1: <ATI SB600 High Definition Audio Controller> mem >> 0xfbaf4000-0xfbaf7fff irq 16 at device 20.2 on pci0 >> hdac1: HDA Driver Revision: 20090624_0136 >> hdac1: [ITHREAD] >> isab0: <PCI-ISA bridge> at device 20.3 on pci0 >> isa0: <ISA bus> on isab0 >> pcib4: <ACPI PCI-PCI bridge> at device 20.4 on pci0 >> pci4: <ACPI PCI bus> on pcib4 >> ohci4: <OHCI (generic) USB controller> mem 0xfbafa000-0xfbafafff irq 18 >> at device 20.5 on pci0 >> ohci4: [GIANT-LOCKED] >> ohci4: [ITHREAD] >> usb6: OHCI version 1.0, legacy support >> usb6: <OHCI (generic) USB controller> on ohci4 >> usb6: USB revision 1.0 >> uhub6: <ATI OHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb6 >> uhub6: 2 ports with 2 removable, self powered >> acpi_button0: <Power Button> on acpi0 >> atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0 >> atkbd0: <AT Keyboard> irq 1 on atkbdc0 >> kbd0 at atkbd0 >> atkbd0: [GIANT-LOCKED] >> atkbd0: [ITHREAD] >> psm0: <PS/2 Mouse> irq 12 on atkbdc0 >> psm0: [GIANT-LOCKED] >> psm0: [ITHREAD] >> psm0: model MouseMan+, device ID 0 >> acpi_aiboost0: <ASUStek AIBOOSTER> on acpi0 >> sio0: configured irq 4 not in bitmap of probed irqs 0 >> sio0: port may not be enabled >> sio0: configured irq 4 not in bitmap of probed irqs 0 >> sio0: port may not be enabled >> sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on >> acpi0 >> sio0: type 16550A >> sio0: [FILTER] >> fdc0: <floppy drive controller (FDE)> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 >> on acpi0 >> fdc0: [FILTER] >> cpu0: <ACPI CPU> on acpi0 >> acpi_throttle0: <ACPI CPU Throttling> on cpu0 >> cpu1: <ACPI CPU> on acpi0 >> cpu2: <ACPI CPU> on acpi0 >> cpu3: <ACPI CPU> on acpi0 >> cryptosoft0: <software crypto> on motherboard >> orm0: <ISA Option ROM> at iomem 0xd7800-0xd87ff on isa0 >> sc0: <System console> at flags 0x100 on isa0 >> sc0: VGA <16 virtual consoles, flags=0x300> >> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on >> isa0 >> ppc0: cannot reserve I/O port range >> sio1: configured irq 3 not in bitmap of probed irqs 0 >> sio1: port may not be enabled >> ulpt0: <Samsung Electronics Co., Ltd. Samsung Laser Printer ML-1200 >> Series, class 0/0, rev 1.10/1.00, addr 2> on uhub0 >> ulpt0: using bi-directional mode >> WARNING: ZFS is considered to be an experimental feature in FreeBSD. >> Timecounters tick every 1.000 msec >> vboxdrv: fAsync=0 offMin=0x12c offMax=0x219 >> hptrr: start channel [0,0] >> hptrr: start channel [0,1] >> hptrr: start channel [0,2] >> hptrr: start channel [0,3] >> hptrr: [0 0] Start channel soft reset. >> hptrr: [0 2] Start channel soft reset. >> hptrr: [0 3] Start channel soft reset. >> hptrr: channel [0,0] started successfully >> hptrr: channel [0,2] started successfully >> hptrr: channel [0,3] started successfully >> hptrr: [0 1] Failed to perform channel hard reset. >> hptrr0: [GIANT-LOCKED] >> hptrr0: [ITHREAD] >> The GEOM class LABEL is already loaded. >> ZFS filesystem version 13 >> ZFS storage pool version 13 >> da0 at hptrr0 bus 0 target 0 lun 0 >> da0: <HPT DISK 0_0 4.00> Fixed Direct Access SCSI-0 device >> da1 at hptrr0 bus 0 target 1 lun 0 >> da1: <HPT DISK 0_1 4.00> Fixed Direct Access SCSI-0 device >> da2 at hptrr0 bus 0 target 2 lun 0 >> da2: <HPT DISK 0_2 4.00> Fixed Direct Access SCSI-0 device >> acd0: DVDR <HL-DT-ST DVDRAM GSA-4163B/A104> at ata0-master PIO4 >> ad4: 476940MB <WDC WD5000AAKS-00C8A0 12.01C02> at ata2-master SATA300 >> acd0: FAILURE - INQUIRY ILLEGAL REQUEST asc=0x24 ascq=0x00 sks=0x40 0x00 >> 0x01 >> (probe0:ata0:0:0:0): TEST UNIT READY. CDB: 0 0 0 0 0 0 >> (probe0:ata0:0:0:0): CAM Status: SCSI Status Error >> (probe0:ata0:0:0:0): SCSI Status: Check Condition >> (probe0:ata0:0:0:0): NOT READY csi:0,0,bb,0 asc:3a,0 >> (probe0:ata0:0:0:0): Medium not present >> (probe0:ata0:0:0:0): Unretryable error >> cd0 at ata0 bus 0 target 0 lun 0 >> cd0: <HL-DT-ST DVDRAM GSA-4163B A104> Removable CD-ROM SCSI-0 device >> cd0: 16.000MB/s transfers >> cd0: Attempt to query device size failed: NOT READY, Medium not present >> ad6: 953869MB <WDC WD10EADS-00L5B1 01.01A01> at ata3-master SATA300 >> ad10: 953869MB <WDC WD10EACS-00D6B0 01.01A01> at ata5-master SATA300 >> ad12: 476940MB <WDC WD5000AACS-00ZUB0 01.01B01> at ata6-master SATA300 >> ad14: 953869MB <Seagate ST31000528AS CC44> at ata7-master SATA300 >> hdac0: HDA Codec #0: ATI RS690/780 HDMI >> pcm0: <HDA ATI RS690/780 HDMI PCM #0 Digital> at cad 0 nid 1 on hdac0 >> hdac1: HDA Codec #0: VIA VT1708S_0 >> pcm1: <HDA VIA VT1708S_0 PCM #0 Analog> at cad 0 nid 1 on hdac1 >> pcm2: <HDA VIA VT1708S_0 PCM #1 Digital> at cad 0 nid 1 on hdac1 >> pcm3: <HDA VIA VT1708S_0 PCM #2 Digital> at cad 0 nid 1 on hdac1 >> SMP: AP CPU #2 Launched! >> SMP: AP CPU #3 Launched! >> SMP: AP CPU #1 Launched! >> Trying to mount root from zfs:rootzfs >> <118>Loading configuration files. >> <118>kernel dumps on /dev/ad4s1b >> <118>Entropy harvesting: >> <118> interrupts >> <118> ethernet >> <118> point_to_point >> <118> kickstart >> <118>. >> GEOM_ELI: Device label/swapB.eli created. >> GEOM_ELI: Encryption: AES-CBC 256 >> GEOM_ELI: Crypto: software >> <118>swapon: adding /dev/label/swapB.eli as swap device >> <118>/etc/rc: WARNING: Use of the early.sh script is deprecated >> <118>/etc/rc: WARNING: Please use a new-style rc.d script instead >> <118>/etc/rc: WARNING: See rc(8) for more information >> <118>gmirror: >> <118>No such device: swap_AB. >> <118> >> <118>Starting file system checks: >> <118>/dev/label/bstrapA: FILE SYSTEM CLEAN; SKIPPING CHECKS >> <118>/dev/label/bstrapA: clean, 303581 free (261 frags, 37915 blocks, >> 0.1% fragmentation) >> <118>/dev/label/bstrapB: FILE SYSTEM CLEAN; SKIPPING CHECKS >> <118>/dev/label/bstrapB: clean, 303502 free (318 frags, 37898 blocks, >> 0.1% fragmentation) >> <118>Setting hostuuid: e039c626-5f54-da11-8e5a-e6b6a8a23cf0. >> <118>Setting hostid: 0xd449202f. >> <118>Mounting local file systems: >> <118>. >> >> >> Fatal trap 12: page fault while in kernel mode >> cpuid = 1; apic id = 01 >> fault virtual address = 0x50 >> fault code = supervisor read data, page not present >> instruction pointer = 0x8:0xffffffff80ad7a5e >> stack pointer = 0x10:0xffffff80a5b2f620 >> frame pointer = 0x10:0xffffff80a5b2f6a0 >> code segment = base 0x0, limit 0xfffff, type 0x1b >> = DPL 0, pres 1, long 1, def32 0, gran 1 >> processor eflags = interrupt enabled, resume, IOPL = 0 >> current process = 314 (txg_thread_enter) >> trap number = 12 >> panic: page fault >> cpuid = 1 >> Uptime: 2m11s >> Physical memory: 8050 MB >> Dumping 1553 MB: 1538 1522 1506 1490 1474 1458 1442 1426 1410 1394 1378 >> 1362 1346 1330 1314 1298 1282 1266 1250 1234 1218 1202 1186 1170 1154 >> 1138 1122 1106 1090 1074 1058 1042 1026 1010 994 978 962 946 930 914 898 >> 882 866 850 834 818 802 786 770 754 738 722 706 690 674 658 642 626 610 >> 594 578 562 546 530 514 498 482 466 450 434 418 402 386 370 354 338 322 >> 306 290 274 258 242 226 210 194 178 162 146 130 114 98 82 66 50 34 18 2 >> >> Reading symbols from /boot/kernel/zfs.ko...Reading symbols from >> /bootdirA/boot/kernel/zfs.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/zfs.ko >> Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from >> /bootdirA/boot/kernel/opensolaris.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/opensolaris.ko >> Reading symbols from /boot/kernel/geom_eli.ko...Reading symbols from >> /bootdirA/boot/kernel/geom_eli.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/geom_eli.ko >> Reading symbols from /boot/kernel/crypto.ko...Reading symbols from >> /bootdirA/boot/kernel/crypto.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/crypto.ko >> Reading symbols from /boot/kernel/zlib.ko...Reading symbols from >> /bootdirA/boot/kernel/zlib.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/zlib.ko >> Reading symbols from /boot/kernel/geom_label.ko...Reading symbols from >> /bootdirA/boot/kernel/geom_label.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/geom_label.ko >> Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from >> /bootdirA/boot/kernel/geom_mirror.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/geom_mirror.ko >> Reading symbols from /boot/kernel/snd_hda.ko...Reading symbols from >> /bootdirA/boot/kernel/snd_hda.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/snd_hda.ko >> Reading symbols from /boot/kernel/sound.ko...Reading symbols from >> /bootdirA/boot/kernel/sound.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/sound.ko >> Reading symbols from /boot/kernel/cd9660_iconv.ko...Reading symbols from >> /bootdirA/boot/kernel/cd9660_iconv.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/cd9660_iconv.ko >> Reading symbols from /boot/kernel/libiconv.ko...Reading symbols from >> /bootdirA/boot/kernel/libiconv.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/libiconv.ko >> Reading symbols from /boot/kernel/aio.ko...Reading symbols from >> /bootdirA/boot/kernel/aio.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/aio.ko >> Reading symbols from /boot/modules/vboxdrv.ko...done. >> Loaded symbols for /boot/modules/vboxdrv.ko >> Reading symbols from /boot/kernel/sem.ko...Reading symbols from >> /bootdirA/boot/kernel/sem.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/sem.ko >> #0 doadump () at pcpu.h:195 >> 195 __asm __volatile("movq %%gs:0,%0" : "=r" (td)); >> (kgdb) backtrace >> #0 doadump () at pcpu.h:195 >> #1 0x0000000000000004 in ?? () >> #2 0xffffffff803aab90 in boot (howto=260) >> at /usr/src/sys/kern/kern_shutdown.c:418 >> #3 0xffffffff803aafb2 in panic (fmt=0x104 <Address 0x104 out of >> bounds>) >> at /usr/src/sys/kern/kern_shutdown.c:574 >> #4 0xffffffff80651d03 in trap_fatal (frame=0xffffff0009572390, >> eva=Variable "eva" is not available. >> ) >> at /usr/src/sys/amd64/amd64/trap.c:770 >> #5 0xffffffff806520d5 in trap_pfault (frame=0xffffff80a5b2f570, >> usermode=0) >> at /usr/src/sys/amd64/amd64/trap.c:686 >> #6 0xffffffff80652a63 in trap (frame=0xffffff80a5b2f570) >> at /usr/src/sys/amd64/amd64/trap.c:457 >> #7 0xffffffff8063cb4e in calltrap () >> at /usr/src/sys/amd64/amd64/exception.S:218 >> #8 0xffffffff80ad7a5e in find_block (th=0xffffff8006930000, >> zseg=0xffffff000dafc580, dnp=0xffffff8006d35600, depth=Variable >> "depth" is not available. >> ) >> at >> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_traverse.c:400 >> >> #9 0xffffffff80ad81a3 in traverse_more (th=0xffffff8006930000) >> at >> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_traverse.c:672 >> >> #10 0xffffffff80ad8368 in traverse_dsl_dataset (ds=0x23, txg_start=1, >> advance=Variable "advance" is not available. >> >> ) >> at >> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/d---Type >> >> <return> to continue, or q <return> to quit--- >> mu_traverse.c:731 >> #11 0xffffffff80ae41c9 in dsl_dataset_destroy_sync () from >> /boot/kernel/zfs.ko >> #12 0xffffffff80ae6d2a in dsl_sync_task_group_sync >> (dstg=0xffffff000d256600, >> tx=0xffffff000d1fca80) >> at >> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_synctask.c:186 >> >> #13 0xffffffff80ae6873 in dsl_pool_sync (dp=0xffffff000d521000, >> txg=3430336) >> at >> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool.c:316 >> >> #14 0xffffffff80af59c3 in spa_sync (spa=0xffffff0006d08000, txg=3430336) >> at >> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:3988 >> >> #15 0xffffffff80afd772 in txg_sync_thread (arg=Variable "arg" is not >> available. >> ) >> at >> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c:352 >> >> #16 0xffffffff80385452 in fork_exit ( >> callout=0xffffffff80afd490 <txg_sync_thread>, arg=0xffffff000d521000, >> frame=0xffffff80a5b2fc80) at /usr/src/sys/kern/kern_fork.c:811 >> #17 0xffffffff8063cf2e in fork_trampoline () >> at /usr/src/sys/amd64/amd64/exception.S:554 >> #18 0x0000000000000000 in ?? () >> #19 0x0000000000000000 in ?? () >> #20 0x0000000000000001 in ?? () >> #21 0x0000000000000000 in ?? () >> #22 0x0000000000000000 in ?? () >> #23 0x0000000000000000 in ?? () >> #24 0x0000000000000000 in ?? () >> #25 0x0000000000000000 in ?? () >> #26 0x0000000000000000 in ?? () >> #27 0x0000000000000000 in ?? () >> #28 0x0000000000000000 in ?? () >> #29 0x0000000000000000 in ?? () >> #30 0x0000000000000000 in ?? () >> #31 0x0000000000000000 in ?? () >> #32 0x0000000000000000 in ?? () >> #33 0x0000000000000000 in ?? () >> #34 0x0000000000000000 in ?? () >> #35 0x0000000000000000 in ?? () >> #36 0x0000000000000000 in ?? () >> #37 0x0000000000000000 in ?? () >> #38 0x0000000000000000 in ?? () >> #39 0x0000000000000000 in ?? () >> #40 0x0000000000000000 in ?? () >> #41 0x0000000000000000 in ?? () >> #42 0x0000000000dde000 in ?? () >> #43 0xffffffff80950d40 in tdq_cpu () >> #44 0xffffffff8095c940 in tdq_groups () >> #45 0xffffffff8095c8c0 in tdq_cpu () >> #46 0xffffff0009572390 in ?? () >> #47 0x0000000000000000 in ?? () >> #48 0xffffff80a5b2f1e8 in ?? () >> #49 0xffffff0009572390 in ?? () >> #50 0xffffffff803ce154 in sched_switch (td=0xffffffff80afd490, >> newtd=0x0, >> flags=0) at /usr/src/sys/kern/sched_ule.c:1938 >> #51 0x0000000000000000 in ?? () >> #52 0x0000000000000000 in ?? () >> #53 0x0000000000000000 in ?? () >> #54 0x0000000000000000 in ?? () >> #55 0x0000000000000000 in ?? () >> #56 0x0000000000000000 in ?? () >> #57 0x0000000000000000 in ?? () >> #58 0x0000000000000000 in ?? () >> #59 0x0000000000000000 in ?? () >> #60 0x0000000000000000 in ?? () >> #61 0x0000000000000000 in ?? () >> #62 0x0000000000000000 in ?? () >> #63 0x0000000000000000 in ?? () >> #64 0x0000000000000000 in ?? () >> #65 0x0000000000000000 in ?? () >> #66 0x0000000000000000 in ?? () >> #67 0x0000000000000000 in ?? () >> #68 0x0000000000000000 in ?? () >> #69 0x0000000000000000 in ?? () >> #70 0x0000000000000000 in ?? () >> #71 0x0000000000000000 in ?? () >> #72 0x0000000000000000 in ?? () >> #73 0x0000000000000000 in ?? () >> #74 0x0000000000000000 in ?? () >> #75 0x0000000000000000 in ?? () >> #76 0x0000000000000000 in ?? () >> #77 0x0000000000000000 in ?? () >> #78 0x0000000000000000 in ?? () >> #79 0x0000000000000000 in ?? () >> #80 0x0000000000000000 in ?? () >> #81 0x0000000000000000 in ?? () >> #82 0x0000000000000000 in ?? () >> #83 0x0000000000000000 in ?? () >> #84 0x0000000000000000 in ?? () >> #85 0x0000000000000000 in ?? () >> #86 0x0000000000000000 in ?? () >> #87 0x0000000000000000 in ?? () >> #88 0x0000000000000000 in ?? () >> #89 0x0000000000000000 in ?? () >> #90 0x0000000000000000 in ?? () >> #91 0x0000000000000000 in ?? () >> #92 0x0000000000000000 in ?? () >> #93 0x0000000000000000 in ?? () >> #94 0x0000000000000000 in ?? () >> #95 0x0000000000000000 in ?? () >> #96 0x0000000000000000 in ?? () >> #97 0x0000000000000000 in ?? () >> #98 0x0000000000000000 in ?? () >> #99 0x0000000000000000 in ?? () >> #100 0x0000000000000000 in ?? () >> #101 0x0000000000000000 in ?? () >> #102 0x0000000000000000 in ?? () >> #103 0x0000000000000000 in ?? () >> #104 0x0000000000000000 in ?? () >> #105 0x0000000000000000 in ?? () >> #106 0x0000000000000000 in ?? () >> #107 0x0000000000000000 in ?? () >> #108 0x0000000000000000 in ?? () >> #109 0x0000000000000000 in ?? () >> #110 0x0000000000000000 in ?? () >> #111 0x0000000000000000 in ?? () >> #112 0x0000000000000000 in ?? () >> #113 0x0000000000000000 in ?? () >> #114 0x0000000000000000 in ?? () >> #115 0x0000000000000000 in ?? () >> #116 0x0000000000000000 in ?? () >> #117 0x0000000000000000 in ?? () >> #118 0x0000000000000000 in ?? () >> Cannot access memory at address 0xffffff80a5b30000 >> (kgdb) q >> # ^D >> Script done on Fri Sep 11 17:26:02 2009 >> >> _______________________________________________ >> freebsd-current@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to >> "freebsd-current-unsubscribe@freebsd.org" > Is the hardware in the machine (CPU, motherboard, and memory) > confirmed to be solid? I ran into a similar situation last month > (http://lists.freebsd.org/pipermail/freebsd-fs/2009-August/006606.html), > the root cause of which turned out to be bad memory. ZFS tends to make > the problem show up more frequently since it uses a lot of memory. > > -Boris I believe the memory to be fine. I ran memtest across it and found no errors. Its also ECC memory (which I have switched on), so much less likely to have errors showing up. Duncan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4AA9AE61.7060601>