Date: Mon, 25 May 2009 00:43:38 -0500 (CDT) From: Larry Rosenman <ler@lerctr.org> To: freebsd-current@freebsd.org Subject: ZFS Crash Message-ID: <alpine.BSF.2.00.0905250040230.1781@borg>
next in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] I'm still seeing crashes with ZFS on -CURRENT. Attached is a textdump, as I couldn't seem to get it to drop a core dump. Ideas? I don't have a core, but will supply whatever other information. This is with a 16G real box. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 512-248-2683 E-Mail: ler@lerctr.org US Mail: 430 Valona Loop, Round Rock, TX 78681-3893 [-- Attachment #2 --] config.txt 0600 0 0 3617 11206424267 7466 ustar root wheel options CONFIG_AUTOGENERATED ident BORG machine amd64 cpu HAMMER makeoptions DEBUG=-g options PRINTF_BUFR_SIZE=32 options SCTP options HWPMC_HOOKS options ATA_STATIC_ID options SMP options STACK options GDB options DDB options KDB_UNATTENDED options KDB_TRACE options KDB options INCLUDE_CONFIG_FILE options AUDIT options KBD_INSTALL_CDEV options _KPOSIX_PRIORITY_SCHEDULING options SYSVSEM options SYSVMSG options SYSVSHM options KTRACE options SCSI_DELAY=5000 options COMPAT_FREEBSD7 options COMPAT_FREEBSD6 options COMPAT_FREEBSD5 options COMPAT_FREEBSD4 options COMPAT_IA32 options COMPAT_43TTY options GEOM_LABEL options GEOM_PART_GPT options PSEUDOFS options PROCFS options CD9660 options MSDOSFS options NTFS options NFSLOCKD options NFS_ROOT options NFSSERVER options NFSCLIENT options MD_ROOT options UFS_GJOURNAL options UFS_DIRHASH options UFS_ACL options SOFTUPDATES options FFS options INET6 options INET options PREEMPTION options SCHED_ULE options FLOWTABLE options GEOM_PART_MBR options GEOM_PART_EBR_COMPAT options GEOM_PART_EBR options GEOM_PART_BSD device isa device mem device io device uart_ns8250 device acpi device pci device fdc device ata device atadisk device atapicd device atapicam device scbus device ch device da device sa device cd device pass device ses device sg device atkbdc device atkbd device psm device kbdmux device vga device splash device sc device agp device uart device ppc device ppbus device lpt device ppi device em device loop device random device ether device tun device pty device md device gif device faith device firmware device bpf device uhci device ehci device usb device uhid device ukbd device ulpt device umass device ums device urio device cpufreq device drm device radeondrm device smbus device intpm device ichsmb device smb device iicbus device iicbb device ic device iic device iicsmb device crypto device cryptodev device rndtest device ipmi device smbios device coretemp msgbuf.txt 0600 0 0 31354 11206424267 7563 ustar root wheel Copyright (c) 1992-2009 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.0-CURRENT #21: Sat May 23 12:04:27 CDT 2009 root@borg.lerctr.org:/usr/obj/usr/src/sys/BORG Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(R) CPU 5120 @ 1.86GHz (1862.01-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x6f6 Stepping = 6 Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Features2=0x4e3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA> AMD Features=0x20100800<SYSCALL,NX,LM> AMD Features2=0x1<LAHF> TSC: P-state invariant real memory = 17179869184 (16384 MB) avail memory = 16618393600 (15848 MB) ACPI APIC Table: <PTLTD APIC > FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 2 package(s) x 2 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 6 cpu3 (AP): APIC ID: 7 This module (opensolaris) contains code covered by the Common Development and Distribution License (CDDL) see http://opensolaris.org/os/licensing/opensolaris_license/ ioapic0 <Version 2.0> irqs 0-23 on motherboard ioapic1 <Version 2.0> irqs 24-47 on motherboard kbd1 at kbdmux0 smbios0: <System Management BIOS> at iomem 0xf60c0-0xf60de on motherboard smbios0: Version: 2.5 cryptosoft0: <software crypto> on motherboard acpi0: <SMCI SMCISLP2> on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 900 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 pci0: <ACPI PCI bus> on pcib0 pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0 pci1: <ACPI PCI bus> on pcib1 pcib2: <ACPI PCI-PCI bridge> irq 16 at device 0.0 on pci1 pci2: <ACPI PCI bus> on pcib2 pcib3: <ACPI PCI-PCI bridge> irq 16 at device 0.0 on pci2 pci3: <ACPI PCI bus> on pcib3 pcib4: <ACPI PCI-PCI bridge> at device 0.0 on pci3 pci4: <ACPI PCI bus> on pcib4 pcib5: <ACPI PCI-PCI bridge> at device 0.2 on pci3 pci5: <ACPI PCI bus> on pcib5 pcib6: <ACPI PCI-PCI bridge> irq 18 at device 2.0 on pci2 pci6: <ACPI PCI bus> on pcib6 em0: <Intel(R) PRO/1000 Network Connection 6.9.9> port 0x2000-0x201f mem 0xd8000000-0xd801ffff irq 18 at device 0.0 on pci6 em0: Using MSI interrupt em0: [FILTER] em0: Ethernet address: 00:30:48:8e:9f:f3 em1: <Intel(R) PRO/1000 Network Connection 6.9.9> port 0x2020-0x203f mem 0xd8020000-0xd803ffff irq 19 at device 0.1 on pci6 em1: Using MSI interrupt em1: [FILTER] em1: Ethernet address: 00:30:48:8e:9f:f2 pcib7: <ACPI PCI-PCI bridge> at device 0.3 on pci1 pci7: <ACPI PCI bus> on pcib7 pcib8: <ACPI PCI-PCI bridge> at device 4.0 on pci0 pci8: <ACPI PCI bus> on pcib8 pcib9: <ACPI PCI-PCI bridge> at device 6.0 on pci0 pci9: <ACPI PCI bus> on pcib9 pci0: <base peripheral> at device 8.0 (no driver attached) pcib10: <ACPI PCI-PCI bridge> irq 17 at device 28.0 on pci0 pci10: <ACPI PCI bus> on pcib10 uhci0: <Intel 631XESB/632XESB/3100 USB controller USB-1> port 0x1800-0x181f irq 17 at device 29.0 on pci0 uhci0: [ITHREAD] uhci0: LegSup = 0x003b usbus0: <Intel 631XESB/632XESB/3100 USB controller USB-1> on uhci0 uhci1: <Intel 631XESB/632XESB/3100 USB controller USB-2> port 0x1820-0x183f irq 19 at device 29.1 on pci0 uhci1: [ITHREAD] uhci1: LegSup = 0x0010 usbus1: <Intel 631XESB/632XESB/3100 USB controller USB-2> on uhci1 uhci2: <Intel 631XESB/632XESB/3100 USB controller USB-3> port 0x1840-0x185f irq 18 at device 29.2 on pci0 uhci2: [ITHREAD] uhci2: LegSup = 0x0010 usbus2: <Intel 631XESB/632XESB/3100 USB controller USB-3> on uhci2 ehci0: <Intel 63XXESB USB 2.0 controller> mem 0xd8500400-0xd85007ff irq 17 at device 29.7 on pci0 ehci0: [ITHREAD] usbus3: EHCI version 1.0 usbus3: <Intel 63XXESB USB 2.0 controller> on ehci0 pcib11: <ACPI PCI-PCI bridge> at device 30.0 on pci0 pci11: <ACPI PCI bus> on pcib11 vgapci0: <VGA-compatible display> port 0x3000-0x30ff mem 0xd0000000-0xd7ffffff,0xd8200000-0xd820ffff irq 18 at device 1.0 on pci11 drm0: <ATI ES1000 RN50> on vgapci0 vgapci0: child drm0 requested pci_enable_busmaster info: [drm] Initialized radeon 1.29.0 20080528 isab0: <PCI-ISA bridge> at device 31.0 on pci0 isa0: <ISA bus> on isab0 atapci0: <Intel 63XXESB2 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1860-0x186f at device 31.1 on pci0 ata0: <ATA channel 0> on atapci0 ata0: [ITHREAD] atapci1: <Intel 63XXESB2 SATA300 controller> port 0x18a0-0x18a7,0x1874-0x1877,0x1878-0x187f,0x1870-0x1873,0x1880-0x189f mem 0xd8500800-0xd8500bff irq 19 at device 31.2 on pci0 atapci1: [ITHREAD] atapci1: AHCI called from vendor specific driver atapci1: AHCI Version 01.10 controller with 6 ports PM supported ata2: <ATA channel 0> on atapci1 ata2: [ITHREAD] ata3: <ATA channel 1> on atapci1 ata3: [ITHREAD] ata4: <ATA channel 2> on atapci1 ata4: [ITHREAD] ata5: <ATA channel 3> on atapci1 ata5: [ITHREAD] ata6: <ATA channel 4> on atapci1 ata6: [ITHREAD] ata7: <ATA channel 5> on atapci1 ata7: [ITHREAD] ichsmb0: <Intel 631xESB/6321ESB (ESB2) SMBus controller> port 0x1100-0x111f irq 19 at device 31.3 on pci0 ichsmb0: [ITHREAD] smbus0: <System Management Bus> on ichsmb0 smb0: <SMBus generic I/O> on smbus0 acpi_button0: <Power Button> on acpi0 atrtc0: <AT realtime clock> port 0x70-0x71 on acpi0 atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: [FILTER] uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 uart1: [FILTER] fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: [FILTER] fd0: <1440-KB 3.5" drive> on fdc0 drive 0 ppc0: <Parallel port> port 0x378-0x37f,0x778-0x77f irq 7 drq 3 on acpi0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/9 bytes threshold ppc0: [ITHREAD] ppbus0: <Parallel port bus> on ppc0 lpt0: <Printer> on ppbus0 lpt0: [ITHREAD] lpt0: Interrupt-driven port ppi0: <Parallel I/O> on ppbus0 cpu0: <ACPI CPU> on acpi0 coretemp0: <CPU On-Die Thermal Sensors> on cpu0 est0: <Enhanced SpeedStep Frequency Control> on cpu0 p4tcc0: <CPU Frequency Thermal Control> on cpu0 cpu1: <ACPI CPU> on acpi0 coretemp1: <CPU On-Die Thermal Sensors> on cpu1 est1: <Enhanced SpeedStep Frequency Control> on cpu1 p4tcc1: <CPU Frequency Thermal Control> on cpu1 cpu2: <ACPI CPU> on acpi0 coretemp2: <CPU On-Die Thermal Sensors> on cpu2 est2: <Enhanced SpeedStep Frequency Control> on cpu2 p4tcc2: <CPU Frequency Thermal Control> on cpu2 cpu3: <ACPI CPU> on acpi0 coretemp3: <CPU On-Die Thermal Sensors> on cpu3 est3: <Enhanced SpeedStep Frequency Control> on cpu3 p4tcc3: <CPU Frequency Thermal Control> on cpu3 ipmi0: <IPMI System Interface> on isa0 ipmi0: KCS mode found at io 0xca2 alignment 0x1 on isa orm0: <ISA Option ROM> at iomem 0xc0000-0xcafff on isa0 sc0: <System console> at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 WARNING: ZFS is considered to be an experimental feature in FreeBSD. Timecounters tick every 1.000 msec usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 480Mbps High Speed USB v2.0 ugen0.1: <Intel> at usbus0 uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0 ugen1.1: <Intel> at usbus1 uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1 ugen2.1: <Intel> at usbus2 uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2 ugen3.1: <Intel> at usbus3 uhub3: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3 ZFS filesystem version 13 ZFS storage pool version 13 acd0: DMA limited to UDMA33, controller found non-ATA66 cable acd0: DVDR <Memorex DVD+-RAM 510L v1/MWS7> at ata0-master UDMA33 ad4: 381554MB <Seagate ST3400620AS 3.AAJ> at ata2-master SATA300 ad6: 381554MB <Seagate ST3400620AS 3.AAJ> at ata3-master SATA300 ad8: 476940MB <Seagate ST3500630AS 3.AAE> at ata4-master SATA300 ad10: 381554MB <Seagate ST3400620AS 3.AAJ> at ata5-master SATA300 ad12: 381554MB <Seagate ST3400620AS 3.AAJ> at ata6-master SATA300 ad14: 381554MB <Seagate ST3400620AS 3.AAJ> at ata7-master SATA300 ipmi0: IPMI device rev. 1, firmware rev. 1.2, version 2.0 ipmi0: Number of channels 8 ipmi0: Attached watchdog SMP: AP CPU #1 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #3 Launched! uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered GEOM: ad4s1: geometry does not match label (255h,63s != 16h,63s). GEOM_LABEL: Label for provider ad4s1a is ufsid/461adb744b3288a4. Root mount waiting for: usbus3 uhub3: 6 ports with 6 removable, self powered Root mount waiting for: usbus3 ugen3.2: <Peppercon AG> at usbus3 ums0: <Peppercon AG Multidevice, class 0/0, rev 2.00/0.01, addr 2> on usbus3 ums0: 3 buttons and [XYZ] coordinates ID=0 ukbd0: <Peppercon AG Multidevice, class 0/0, rev 2.00/0.01, addr 2> on usbus3 kbd2 at ukbd0 Trying to mount root from ufs:/dev/ad4s1a acd0: FAILURE - INQUIRY ILLEGAL REQUEST asc=0x24 ascq=0x00 <118>Entropy harvesting: <118> interrupts <118> ethernet <118> point_to_point (probe0:ata0:0:0:0): TEST UNIT READY. CDB: 0 0 0 0 0 0 (probe0:ata0:0:0:0): CAM Status: SCSI Status Error (probe0:ata0:0:0:0): SCSI Status: Check Condition (probe0:ata0:0:0:0): NOT READY asc:3a,0 (probe0:ata0:0:0:0): Medium not present (probe0:ata0:0:0:0): Unretryable error cd0 at ata0 bus 0 target 0 lun 0 cd0: <Memorex DVD+-RAM 510L v1 MWS7> Removable CD-ROM SCSI-0 device cd0: 33.000MB/s transfers cd0: Attempt to query device size failed: NOT READY, Medium not present <118> kickstart <118>. GEOM_LABEL: Label ufsid/461adb744b3288a4 removed. <118>/dev/ad4s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS <118>/dev/ad4s1a: clean, 32505 free (249 frags, 4032 blocks, 0.1% fragmentation) GEOM_LABEL: Label for provider ad4s1a is ufsid/461adb744b3288a4. GEOM_LABEL: Label ufsid/461adb744b3288a4 removed. <118>Starting Network: lo0 em0. <118>add net default: gateway 192.168.200.10 <118>route: <118>writing to routing socket <118>: <118>File exists <118>add host 192.168.200.4: gateway lo0: route already in table <118>Additional ABI support: <118> linux <118>. <5>em0: link state changed to UP <118>/etc/rc.d/mountd: WARNING: /etc/exports is not readable. <118>Starting mountd. <118>Removing stale Samba tdb files: <118> done <118>May 23 12:26:54 borg postgres[1100]: [1-1] FATAL: the database system is starting up <118>Posting monthly OS statistics to rpt.bsdstats.org <118>Configuring syscons: <118> blanktime <118>. <118>Local package initialization: <118> p4d <118>Perforce Server starting... <118> <118> p4web <118>. <118>moused: <118>unable to open /dev/psm0: No such file or directory <118> <118> <118>Sat May 23 12:26:59 CDT 2009 <118>May 23 12:28:45 borg exim[1402]: gethostby*.gethostanswer: asked for "borg-ipmi.lerctr.org IN AAAA", got type "A" <6>pid 39918 (try), uid 0: exited on signal 10 (core dumped) <118>May 24 14:32:23 borg sshd[94972]: warning: /etc/hosts.allow, line 39: host name/name mismatch: borg.lerctr.org != borg <6>pid 55175 (try), uid 0: exited on signal 10 (core dumped) Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 01 fault virtual address = 0x0 fault code = supervisor write data, page not present instruction pointer = 0x20:0xffffffff8056b9ea stack pointer = 0x28:0xffffff81a9394630 frame pointer = 0x28:0xffffff81a93946f0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 1251 (bacula-sd) trap number = 12 panic: page fault cpuid = 1 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a panic() at panic+0x182 trap_fatal() at trap_fatal+0x2ad trap_pfault() at trap_pfault+0x294 trap() at trap+0x187 calltrap() at calltrap+0x8 --- trap 0xc, rip = 0xffffffff8056b9ea, rsp = 0xffffff81a9394630, rbp = 0xffffff81a93946f0 --- bzero() at bzero+0xa dmu_tx_check_ioerr() at dmu_tx_check_ioerr+0x9a dmu_tx_count_write() at dmu_tx_count_write+0x291 dmu_tx_hold_write() at dmu_tx_hold_write+0x4a zfs_freebsd_write() at zfs_freebsd_write+0x3e1 VOP_WRITE_APV() at VOP_WRITE_APV+0xac vn_write() at vn_write+0x279 dofilewrite() at dofilewrite+0x85 kern_writev() at kern_writev+0x60 write() at write+0x54 syscall() at syscall+0x246 Xfast_syscall() at Xfast_syscall+0xd0 --- syscall (4, FreeBSD ELF64, write), rip = 0x801a2716c, rsp = 0x7fffff3fa758, rbp = 0x801cace28 --- Uptime: 1d11h45m55s panic.txt 0600 0 0 12 11206424267 7135 ustar root wheel page fault version.txt 0600 0 0 151 11206424267 7614 ustar root wheel FreeBSD 8.0-CURRENT #21: Sat May 23 12:04:27 CDT 2009 root@borg.lerctr.org:/usr/obj/usr/src/sys/BORG
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.BSF.2.00.0905250040230.1781>