From owner-freebsd-questions@FreeBSD.ORG Thu Sep 22 01:54:30 2005 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BE8BF16A41F for ; Thu, 22 Sep 2005 01:54:30 +0000 (GMT) (envelope-from snacktime@gmail.com) Received: from zproxy.gmail.com (zproxy.gmail.com [64.233.162.201]) by mx1.FreeBSD.org (Postfix) with ESMTP id 254C143D45 for ; Thu, 22 Sep 2005 01:54:30 +0000 (GMT) (envelope-from snacktime@gmail.com) Received: by zproxy.gmail.com with SMTP id c3so37790nze for ; Wed, 21 Sep 2005 18:54:29 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type; b=A3EQQsR1KtGJD2Xke5ZafmSZVPMCIBJKJqwCdUAwQbBykFq5WgLybH28EzbNY2lWXBq4NdW2Qukov3iK6RJYR5e8PDgB6jjdA3oCzZsoyvl6FKDRkcVflzi3BPf6AqptE+blR3vS3LCExeMKkJO9ycbvyI0qCN/P/1OvSKqo+Jo= Received: by 10.54.70.7 with SMTP id s7mr2854822wra; Wed, 21 Sep 2005 18:54:29 -0700 (PDT) Received: by 10.54.23.29 with HTTP; Wed, 21 Sep 2005 18:54:28 -0700 (PDT) Message-ID: <1f060c4c0509211854c712a8@mail.gmail.com> Date: Wed, 21 Sep 2005 18:54:29 -0700 From: snacktime To: freebsd-questions@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Freebsd 5.4 panic on ffs_blkfree: freeing free frag X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: snacktime List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Sep 2005 01:54:30 -0000 I've had this one server crash about once every 1-2 months now for the last 5 months. The kernel is not compiled with debugging symbols (sorry) but the backtrace is probably better than nothing, at least it tells me where to start looking. I found a couple of vague references to this but nothing really helpful. This last crash happened right when I ran pkgdb -F (althoug= h there were other things happening as well). I have not run fsck in single user mode throughout all the crashes. I'm thinking that might be a good idea and might possibly fix whatever disk problem is present? Dump header from device /dev/da0s1b Architecture: i386 Architecture Version: 16777216 Dump Length: 1073610752B (1023 MB) Blocksize: 512 Dumptime: Wed Sep 21 17:31:14 2005 Hostname: catalog1.paymentonline.net Magic: FreeBSD Kernel Dump Version String: FreeBSD 5.4-STABLE #0: Tue Jul 26 23:54:18 PDT 2005 root@catalog1.paymentonline.net:/usr/obj/usr/src/sys/CAT4 Panic String: ffs_blkfree: freeing free frag Dump Parity: 495827294 Bounds: 5 kgdb /boot/kernel/kernel vmcore.5 [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you ar= e welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". (no debugging symbols found)...#0 0xc068790c in doadump () (kgdb) bt #0 0xc068790c in doadump () #1 0xc0688076 in boot () #2 0xc0688464 in panic () #3 0xc07f73a8 in ffs_blkfree () #4 0xc0808538 in handle_workitem_freeblocks () #5 0xc080538f in process_worklist_item () #6 0xc08050a0 in softdep_process_worklist () #7 0xc06f2816 in sched_sync () #8 0xc066ccd0 in fork_exit () #9 0xc08863dc in fork_trampoline () (kgdb) Dump Status: good Top of kernel config file (the rest is straight from the GENERIC config): machine i386 cpu I686_CPU ident CAT4 # To statically compile in device wiring instead of /boot/device.hints #hints "GENERIC.hints" # Default places to look for devices. options SCHED_4BSD # 4BSD scheduler options INET # InterNETworking options INET6 # IPv6 communications protocols options FFS # Berkeley Fast Filesystem options SOFTUPDATES # Enable FFS soft updates support options UFS_ACL # Support for access control lists options UFS_DIRHASH # Improve performance on big directories options MD_ROOT # MD is a potential root device options NFSCLIENT # Network Filesystem Client options NFSSERVER # Network Filesystem Server options NFS_ROOT # NFS usable as /, requires NFSCLIENT options MSDOSFS # MSDOS Filesystem options CD9660 # ISO 9660 Filesystem options PROCFS # Process filesystem (requires PSEUDOFS) options PSEUDOFS # Pseudo-filesystem framework options GEOM_GPT # GUID Partition Tables. options COMPAT_43 # Compatible with BSD 4.3 [KEEP THIS!] options COMPAT_FREEBSD4 # Compatible with FreeBSD4 options SCSI_DELAY=3D15000 # Delay (in ms) before probing SCSI options KTRACE # ktrace(1) support options SYSVSHM # SYSV-style shared memory options SYSVMSG # SYSV-style message queues options SYSVSEM # SYSV-style semaphores options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions options KBD_INSTALL_CDEV # install a CDEV entry in /dev options AHC_REG_PRETTY_PRINT # Print register bitfields in debug # output. Adds ~128k to driver. options AHD_REG_PRETTY_PRINT # Print register bitfields in debug # output. Adds ~215k to driver. options ADAPTIVE_GIANT # Giant mutex is adaptive. options HZ=3D1000 # To make an SMP kernel, the next line is needed options SMP # Symmetric MultiProcessor Kernel device apic # I/O APIC #options MAC ## Up limits for postgres options SHMMAXPGS=3D131072 options SEMMNI=3D128 options SEMMNS=3D512 options SEMUME=3D100 options SEMMNU=3D256 dmesg output: Copyright (c) 1992-2005 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 5.4-STABLE #0: Tue Jul 26 23:54:18 PDT 2005 root@catalog1.paymentonline.net:/usr/obj/usr/src/sys/CAT4 ACPI APIC Table: Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(TM) CPU 3.06GHz (3056.51-MHz 686-class CPU) Origin =3D "GenuineIntel" Id =3D 0xf29 Stepping =3D 9 Features=3D0xbfebfbff Hyperthreading: 2 logical CPUs real memory =3D 1073610752 (1023 MB) avail memory =3D 1041051648 (992 MB) FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 ioapic0: Changing APIC ID to 8 ioapic1: Changing APIC ID to 9 ioapic2: Changing APIC ID to 10 MADT: Forcing active-low polarity and level trigger for SCI ioapic0 irqs 0-15 on motherboard ioapic1 irqs 16-31 on motherboard ioapic2 irqs 32-47 on motherboard npx0: on motherboard npx0: INT 16 interface acpi0: on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-safe" frequency 3579545 Hz quality 1000 acpi_timer0: <32-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 cpu0: on acpi0 cpu1: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pci0: at device 4.0 (no driver attached) pci0: at device 4.1 (no driver attached) pci0: at device 4.2 (no driver attached) pci0: at device 14.0 (no driver attached) atapci0: port 0x8b0-0x8bf,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 at device 15.1 on pci0 ata0: channel #0 on atapci0 ata1: channel #1 on atapci0 ohci0: mem 0xfe100000-0xfe100fff irq 5 at device 15.2 on pci0 usb0: OHCI version 1.0, legacy support usb0: SMM does not respond, resetting usb0: on ohci0 usb0: USB revision 1.0 uhub0: (0x1166) OHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 4 ports with 4 removable, self powered isab0: at device 15.3 on pci0 isa0: on isab0 pcib1: on acpi0 pci4: on pcib1 pcib2: at device 8.0 on pci4 pci5: on pcib2 ahc0: port 0xcc00-0xccff mem 0xfccff000-0xfccfffff irq 30 at device 6.0 on pci5 aic7899: Ultra160 Wide Channel A, SCSI Id=3D7, 32/253 SCBs ahc1: port 0xc800-0xc8ff mem 0xfccfe000-0xfccfefff irq 31 at device 6.1 on pci5 aic7899: Ultra160 Wide Channel B, SCSI Id=3D7, 32/253 SCBs pcib3: on acpi0 pci3: on pcib3 bge0: mem 0xfcf10000-0xfcf1ffff irq 28 at device 6.0 on pci3 miibus0: on bge0 brgphy0: on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto bge0: Ethernet address: 00:0f:1f:6e:f2:84 bge1: mem 0xfcf00000-0xfcf0ffff irq 29 at device 8.0 on pci3 miibus1: on bge1 brgphy1: on miibus1 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto bge1: Ethernet address: 00:0f:1f:6e:f2:85 pcib4: on acpi0 pci2: on pcib4 pcib5: on acpi0 pci1: on pcib5 fdc0: port 0x3f7,0x3f0-0x3f5 irq 6 drq 2 on acpi0 fd0: <1440-KB 3.5" drive> on fdc0 drive 0 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0 sio1: type 16550A orm0: at iomem 0xec000-0xeffff,0xc8000-0xcdfff,0xc0000-0xc7fff on isa0 pmtimer0 on isa0 atkbdc0: at port 0x64,0x60 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 ppc0: parallel port not found. sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=3D0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounters tick every 1.000 msec acd0: CDROM at ata0-master PIO4 Waiting 15 seconds for SCSI devices to settle ses0 at ahc0 bus 0 target 6 lun 0 ses0: Fixed Processor SCSI-2 device ses0: 3.300MB/s transfers ses0: SAF-TE Compliant Device da0 at ahc0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-3 device da0: 160.000MB/s transfers (80.000MHz, offset 127, 16bit), Tagged Queueing Enabled da0: 34732MB (71132959 512 byte sectors: 255H 63S/T 4427C) SMP: AP CPU #1 Launched! Mounting root from ufs:/dev/da0s1a WARNING: / was not properly dismounted WARNING: /: multilabel flag on fs but no MAC support WARNING: /tmp was not properly dismounted WARNING: /tmp: multilabel flag on fs but no MAC support WARNING: /usr was not properly dismounted /usr: mount pending error: blocks 28 files 1 WARNING: /usr: multilabel flag on fs but no MAC support WARNING: /var was not properly dismounted WARNING: /var: multilabel flag on fs but no MAC support Accounting enabled