From owner-freebsd-hackers Sun Mar 28 16:18:20 1999 Delivered-To: freebsd-hackers@freebsd.org Received: from luke.pmr.com (luke.pmr.com [207.170.114.132]) by hub.freebsd.org (Postfix) with ESMTP id 4F665156E5 for ; Sun, 28 Mar 1999 16:17:59 -0800 (PST) (envelope-from bob@luke.pmr.com) Received: (from bob@localhost) by luke.pmr.com (8.9.2/8.9.2) id SAA08932 for freebsd-hackers@freebsd.org; Sun, 28 Mar 1999 18:17:39 -0600 (CST) (envelope-from bob) Date: Sun, 28 Mar 1999 18:17:39 -0600 From: Bob Willcox To: hackers list Subject: Panic in 3.1-stable running amanda, need help debugging Message-ID: <19990328181739.A8801@luke.pmr.com> Reply-To: Bob Willcox Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.1i Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I have just recently started getting the below (very repeatable) panic on my backup system when it runs amanda. This same configuration had been running for weeks w/o problems and now it consistently fails. Note that this is under very heavy network traffic conditions. I am backing up 9 systems and have the dump max (or whatever its called) set to 5. Also, most of the systems are on my full-duplex switching 100TX hub (a few are on a 10BaseT segment, one is across an ISDN link). I upgraded the backup system to yesterday's level of 3.1-stable and added the kernel debugger (and built with symbols). It still fails (not sure whether I should be happy about that or not :-). The hardware is a PPro 200 w/64MB of memory. As I am not very familiar with FreeBSD internals I thought someone out there could give me some tips on how to go about further debugging this problem. I thank you for any help you can provide. Bob Gdb trace: (kgdb) where #0 boot (howto=260) at ../../kern/kern_shutdown.c:285 #1 0xf014e705 in panic (fmt=0xf0233f4c "from debugger") at ../../kern/kern_shutdown.c:446 #2 0xf012aab1 in db_panic (addr=-266261713, have_addr=0, count=-1, modif=0xf4812d5c "") at ../../ddb/db_command.c:432 #3 0xf012aa51 in db_command (last_cmdp=0xf0251e64, cmd_table=0xf0251cc4, aux_cmd_tablep=0xf0267acc) at ../../ddb/db_command.c:332 #4 0xf012ab16 in db_command_loop () at ../../ddb/db_command.c:454 #5 0xf012ce67 in db_trap (type=3, code=0) at ../../ddb/db_trap.c:71 #6 0xf021290a in kdb_trap (type=3, code=0, regs=0xf4812e4c) at ../../i386/i386/db_interface.c:157 #7 0xf021c0b4 in trap (frame={tf_es = 16, tf_ds = 16, tf_edi = -196231424, tf_esi = 256, tf_ebp = -192860528, tf_isp = -192860556, tf_ebx = -266105266, tf_edx = -266043248, tf_ecx = -267680992, tf_eax = 18, tf_trapno = 3, tf_err = 0, tf_eip = -266261713, tf_cs = 8, tf_eflags = 598, tf_esp = -266043264, tf_ss = -266111117}) at ../../i386/i386/trap.c:548 #8 0xf0212b2f in Debugger (msg=0xf0237773 "panic") at ../../i386/i386/db_interface.c:317 #9 0xf014e6fc in panic (fmt=0xf0238e4e "receive 1") at ../../kern/kern_shutdown.c:444 #10 0xf01667d3 in soreceive (so=0xf44dbf00, psa=0x0, uio=0xf4812f40, mp0=0x0, controlp=0x0, flagsp=0x0) at ../../kern/uipc_socket.c:659 #11 0xf015c6d4 in soo_read (fp=0xf0c54e40, uio=0xf4812f40, cred=0xf0c60000) at ../../kern/sys_socket.c:69 #12 0xf01591ed in read (p=0xf4763ce0, uap=0xf4812f94) at ../../kern/sys_generic.c:121 #13 0xf021c8c3 in syscall (frame={tf_es = -272695257, tf_ds = -272695257, tf_edi = -272638492, tf_esi = 64, tf_ebp = -272638364, tf_isp = -192860188, tf_ebx = 0, tf_edx = 82768, tf_ecx = 6, tf_eax = 3, tf_trapno = 7, tf_err = 7, tf_eip = 537674705, tf_cs = 31, tf_eflags = 514, tf_esp = -272638820, tf_ss = 39}) at ../../i386/i386/trap.c:1100 #14 0x200c43d1 in ?? () #15 0x1f64 in ?? () #16 0x1099 in ?? () Dmesg output: Copyright (c) 1992-1999 FreeBSD Inc. Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. FreeBSD 3.1-STABLE #1: Sun Mar 28 09:35:01 CST 1999 bob@deathstar.pmr.com:/usr/src/sys/compile/DEATHSTAR Timecounter "i8254" frequency 1193182 Hz Timecounter "TSC" frequency 199310420 Hz CPU: Pentium Pro (199.31-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x616 Stepping=6 Features=0xf9ff real memory = 33554432 (32768K bytes) avail memory = 29814784 (29116K bytes) Preloaded elf kernel "kernel" at 0xf02cd000. Probing for devices on PCI bus 0: chip0: rev 0x02 on pci0.0.0 chip1: rev 0x01 on pci0.1.0 ahc0: rev 0x00 int a irq 12 on pci0.10.0 ahc0: aic7870 Single Channel A, SCSI Id=7, 16/255 SCBs fxp0: rev 0x01 int a irq 10 on pci0.11.0 fxp0: Ethernet address 00:a0:c9:31:e6:21 ncr0: rev 0x01 int a irq 11 on pci0.12.0 ncr1: rev 0x03 int a irq 9 on pci0.13.0 Probing for devices on the ISA bus: sc0 on isa sc0: VGA color <16 virtual consoles, flags=0x0> atkbdc0 at 0x60-0x6f on motherboard atkbd0 irq 1 on isa psm0 not found sio0 at 0x3f8-0x3ff irq 4 flags 0x10 on isa sio0: type 16550A sio1 at 0x2f8-0x2ff irq 3 on isa sio1: type 16550A fdc0 at 0x3f0-0x3f7 irq 6 drq 2 on isa fdc0: FIFO enabled, 8 bytes threshold fd0: 1.44MB 3.5in ppc0 at 0x378 irq 7 on isa ppc0: W83877F chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/16 bytes threshold nlpt0: on ppbus 0 nlpt0: Interrupt-driven port ppi0: on ppbus 0 plip0: on ppbus 0 vga0 at 0x3b0-0x3df maddr 0xa0000 msize 131072 on isa npx0 on motherboard npx0: INT 16 interface Waiting 10 seconds for SCSI devices to settle sa0 at ahc0 bus 0 target 1 lun 0 sa0: Removable Sequential Access SCSI-2 device sa0: 10.000MB/s transfers (10.000MHz, offset 15) sa1 at ncr0 bus 0 target 5 lun 0 sa1: Removable Sequential Access SCSI-2 device sa1: 4.807MB/s transfers (4.807MHz, offset 8) changing root device to da0s1a cd0 at ncr0 bus 0 target 4 lun 0 cd0: Removable CD-ROM SCSI-2 device cd0: 4.237MB/s transfers (4.237MHz, offset 8) cd0: Attempt to query device size failed: NOT READY, Medium not present da1 at ncr1 bus 0 target 1 lun 0 da1: Fixed Direct Access SCSI-2 device da1: 40.000MB/s transfers (20.000MHz, offset 15, 16bit) da1: 4134MB (8467200 512 byte sectors: 255H 63S/T 527C) da2 at ncr1 bus 0 target 2 lun 0 da2: Fixed Direct Access SCSI-2 device da2: 40.000MB/s transfers (20.000MHz, offset 15, 16bit), Tagged Queueing Enabled da2: 8715MB (17850000 512 byte sectors: 255H 63S/T 1111C) ch0 at ahc0 bus 0 target 0 lun 0 ch0: Removable Changer SCSI-2 device ch0: 3.300MB/s transfers ch0: 11 slots, 1 drive, 1 picker, 0 portals da0 at ncr1 bus 0 target 0 lun 0 da0: < DFRSS2W 4B4B> Fixed Direct Access SCSI-2 device da0: 20.000MB/s transfers (10.000MHz, offset 15, 16bit), Tagged Queueing Enabled da0: 2150MB (4404489 512 byte sectors: 255H 63S/T 274C) WARNING: / was not properly dismounted ffs_mountfs: superblock updated for soft updates ffs_mountfs: superblock updated for soft updates ffs_mountfs: superblock updated for soft updates ffs_mountfs: superblock updated for soft updates ffs_mountfs: superblock updated for soft updates ffs_mountfs: superblock updated for soft updates ffs_mountfs: superblock updated for soft updates link_elf: symbol splash_register undefined Kernel config file: # # DEATHSTAR -- Configure file of the DEATHSTAR system # # For more information read the handbook part System Administration -> # Configuring the FreeBSD Kernel -> The Configuration File. # The handbook is available in /usr/share/doc/handbook or online as # latest version from the FreeBSD World Wide Web server # # # An exhaustive list of options and more detailed explanations of the # device lines is present in the ./LINT configuration file. If you are # in doubt as to the purpose or necessity of a line, check first in LINT. # # $Id$ machine "i386" cpu "I686_CPU" ident DEATHSTAR maxusers 128 options INET #InterNETworking options FFS #Berkeley Fast Filesystem options FFS_ROOT #FFS usable as root device [keep this!] options MFS #Memory Filesystem options NFS #Network Filesystem options MSDOSFS #MSDOS Filesystem options "CD9660" #ISO 9660 Filesystem options "CD9660_ROOT" #CD-ROM usable as root. "CD9660" req'ed options PROCFS #Process filesystem options "COMPAT_43" #Compatible with BSD 4.3 [KEEP THIS!] options SCSI_DELAY=10000 #Be pessimistic about Joe SCSI device options UCONSOLE #Allow users to grab the console options FAILSAFE #Be conservative options USERCONFIG #boot -c editor options VISUAL_USERCONFIG #visual boot -c editor options SOFTUPDATES #enable soft updates support config kernel root on da0 controller isa0 controller pci0 controller fdc0 at isa? port "IO_FD1" bio irq 6 drq 2 disk fd0 at fdc0 drive 0 # A single entry for any of these controllers (ncr, ahb, ahc) is # sufficient for any number of installed devices. controller ncr0 controller ahc0 controller scbus0 device da0 device sa0 device pass0 device cd0 device ch0 # atkbdc0 controlls both the keyboard and the PS/2 mouse controller atkbdc0 at isa? port IO_KBD tty device atkbd0 at isa? tty irq 1 device psm0 at isa? tty irq 12 device vga0 at isa? port ? conflicts # splash screen/screen saver #pseudo-device splash # syscons is the default console driver, resembling an SCO console device sc0 at isa? tty device npx0 at isa? port IO_NPX irq 13 # Serial ports device sio0 at isa? port "IO_COM1" flags 0x10 tty irq 4 device sio1 at isa? port "IO_COM2" tty irq 3 # Parallel port device ppc0 at isa? port? net irq 7 controller ppbus0 device nlpt0 at ppbus? device plip0 at ppbus? device ppi0 at ppbus? #controller vpo0 at ppbus? # Order is important here due to intrusive probes, do *not* alphabetize # this list of network interfaces until the probes have been fixed. # Right now it appears that the ie0 must be probed before ep0. See # revision 1.20 of this file. device de0 device fxp0 pseudo-device loop pseudo-device ether pseudo-device sl 2 pseudo-device ppp 2 pseudo-device tun 2 pseudo-device pty 64 pseudo-device gzip # Exec gzipped a.out's # # Enable debug support # options KTRACE #kernel tracing options DDB #kernel debugger options INVARIANTS #extra sanity checks options INVARIANT_SUPPORT #needed for INVARIANTS # # These three options provide support for System V Interface # Definition-style interprocess communication, in the form of shared # memory, semaphores, and message queues, respectively. # options SYSVSHM options SYSVSEM options SYSVMSG # The `bpfilter' pseudo-device enables the Berkeley Packet Filter. Be # aware of the legal and administrative consequences of enabling this # option. The number of devices determines the maximum number of # simultaneous BPF clients programs runnable. pseudo-device bpfilter 4 #Berkeley packet filter -- Bob Willcox The man who follows the crowd will usually get no bob@luke.pmr.com further than the crowd. The man who walks alone is Austin, TX likely to find himself in places no one has ever been. -- Alan Ashley-Pitt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message