From owner-freebsd-current Wed Jan 15 9:13:16 2003 Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A08C837B401 for ; Wed, 15 Jan 2003 09:13:11 -0800 (PST) Received: from redux.oneway.com (redux.oneway.com [216.22.46.103]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9003A43ED8 for ; Wed, 15 Jan 2003 09:13:10 -0800 (PST) (envelope-from jay@oneway.com) Received: from oneway.com (office3.tmcs.net [209.104.62.3]) by redux.oneway.com (8.12.5/8.12.5) with ESMTP id h0FHCUCj035434 for ; Wed, 15 Jan 2003 12:12:31 -0500 (EST) Date: Wed, 15 Jan 2003 11:12:57 -0600 Subject: Panic and Crash 5.0- CVSup from Jan 14 Content-Type: text/plain; delsp=yes; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jason Kuri To: current@FreeBSD.ORG Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <97E11E9C-28AC-11D7-B6F3-00039398BAE0@oneway.com> X-Mailer: Apple Mail (2.551) Sender: owner-freebsd-current@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi everyone, Looks like late last night my server took a nose dive. I was able to get a trace... which I included below along with my dmesg. I'm wondering if anyone can shed some light on what happenned? It's an extremely un-loaded server... I've not moved any services onto it yet (waiting for 5.0 to be 'blessed' as it were) but the machine has been up for weeks on an earlier 5.0-current (december 12) I updated to 5.0-CURRENT as of yesterday and after a few hours: Bang.. kernel panic. The server is running apache 2, but I just built that yesterday and turned it on, there is no content that is accessible. I'm guessing, based on the trace, that it happenned when the nightly process on another server tried to back up a bunch of files to the machine. (using Rsync) This is the only time it would have had any decent amount of disk activity. all filesystems are ufs + softupdates. it would have been writing to /opt (/dev/ad0s1g) Below are the general specs, trace, and dmesg. Any thoughts would be greatly appreciated. Jay specs: Tyan 230t (model 2507t ) VIA Apollo Pro133A Chipset Dual proc 1.13 P3 processors w 1G ECC ram - trace - Debugger(c04d1abb,1000000,c04e0372,df0f1a94,1) at Debugger+0x55 panic(c04e0372,c051846c,c0563ac0,c67cbe00,c63e3580) at panic+0x11f initiate_write_inodeblock_ufs1(c6b69680,d29d9a30,d29d9a30,c6b93800,c21af 620) at initiate_write_inodeblock_ufs1+0x3f5 softdep_disk_io_initiation(d29d9a30,c1326730,3,d29d9a30,df0f1b90) at softdep_disk_io_initiation+0x73 spec_xstrategy(c64b6c34,d29d9a30,df0f1b90,c02a6568,df0f1bbc) at spec_xstrategy+0x102 spec_specstrategy(df0f1bbc,df0f1be4,c0328e28,df0f1bbc,1) at spec_specstrategy+0x1b spec_vnoperate(df0f1bbc,1,0,10,df0f1bd4) at spec_vnoperate+0x18 bwrite(d29d9a30,2,28,0,cdb4edac) at bwrite+0x358 vfs_bio_awrite(d29d9a30,12,df0f1c58,c0347397,df0f1c60) at vfs_bio_awrite+0x306 spec_fsync(df0f1cc4,df0f1d08,c0339bbb,df0f1cc4,20002) at spec_fsync+0x1a3 spec_vnoperate(df0f1cc4,20002,c21af620,c04d5e88,0) at spec_vnoperate+0x18 sched_sync(0,df0f1d48,0,0,0) at sched_sync+0x1db fork_exit(c03399e0,0,df0f1d48) at fork_exit+0xb3 fork_trampoline() at fork_trampoline+0x1a - dmesg - Copyright (c) 1992-2003 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 5.0-CURRENT #4: Tue Jan 14 03:52:54 GMT 2003 root@doppel.oneway.com:/usr/src/sys/i386/compile/DOPPEL Preloaded elf kernel "/boot/kernel/kernel" at 0xc0684000. Preloaded elf module "/boot/kernel/acpi.ko" at 0xc06840a8. Timecounter "i8254" frequency 1193182 Hz CPU: Intel(R) Pentium(R) III CPU family 1133MHz (1132.96-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x6b1 Stepping = 1 Features=0x383fbff real memory = 1073676288 (1023 MB) avail memory = 1036435456 (988 MB) Programming 24 pins in IOAPIC #0 IOAPIC #0 intpin 2 -> irq 0 FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): apic id: 0, version: 0x00040011, at 0xfee00000 cpu1 (AP): apic id: 1, version: 0x00040011, at 0xfee00000 io0 (APIC): apic id: 2, version: 0x00178011, at 0xfec00000 Initializing GEOMetry subsystem Pentium Pro MTRR support enabled npx0: on motherboard npx0: INT 16 interface acpi0: on motherboard ACPI-0625: *** Info: GPE Block0 defined as GPE0 to GPE15 Using $PIR table, 8 entries at 0xc00fdd20 acpi0: power button is handled as a fixed feature programming model. Timecounter "ACPI-fast" frequency 3579545 Hz acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0 acpi_cpu0: on acpi0 acpi_cpu1: on acpi0 acpi_button0: on acpi0 pcib0: port 0x6000-0x607f,0x5000-0x500f,0x4080-0x40ff,0x4000-0x407f,0xcf8-0xcff on acpi0 pci0: on pcib0 IOAPIC #0 intpin 12 -> irq 2 agp0: mem 0xf0000000-0xf3ffffff at device 0.0 on pci0 pcib1: at device 1.0 on pci0 pci1: on pcib1 isab0: at device 7.0 on pci0 isa0: on isab0 atapci0: port 0xe000-0xe00f at device 7.1 on pci0 ata0: at 0x1f0 irq 14 on atapci0 ata1: at 0x170 irq 15 on atapci0 uhci0: port 0xe400-0xe41f irq 2 at device 7.2 on pci0 usb0: on uhci0 usb0: USB revision 1.0 uhub0: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered uhci1: port 0xe800-0xe81f irq 2 at device 7.3 on pci0 usb1: on uhci1 usb1: USB revision 1.0 uhub1: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub1: 2 ports with 2 removable, self powered pci0: at device 7.4 (no driver attached) pci0: at device 8.0 (no driver attached) fxp0: port 0xec00-0xec3f mem 0xf9000000-0xf90fffff,0xf9100000-0xf9100fff irq 11 at device 9.0 on pci0 fxp0: Ethernet address 00:03:47:09:d3:51 inphy0: on miibus0 inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto fdc0: port 0x3f7,0x3f0-0x3f5 irq 6 drq 2 on acpi0 fdc0: FIFO enabled, 8 bytes threshold fd0: <1440-KB 3.5" drive> on fdc0 drive 0 sio0 port 0x3f8-0x3ff irq 4 on acpi0 sio0: type 16550A, console sio1 port 0x2f8-0x2ff irq 3 on acpi0 sio1: type 16550A ppc0 port 0x378-0x37f irq 7 on acpi0 ppc0: Generic chipset (EPP/NIBBLE) in COMPATIBLE mode plip0: on ppbus0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 atkbdc0: port 0x64,0x60 irq 1 on acpi0 atkbd0: flags 0x1 irq 1 on atkbdc0 kbd0 at atkbd0 orm0: