From owner-freebsd-bugs Tue Mar 30 9:20:18 1999 Delivered-To: freebsd-bugs@freebsd.org Received: from freefall.freebsd.org (freefall.FreeBSD.ORG [204.216.27.21]) by hub.freebsd.org (Postfix) with ESMTP id 7ED3914CCA for ; Tue, 30 Mar 1999 09:20:17 -0800 (PST) (envelope-from gnats@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.9.2/8.9.2) id JAA21188; Tue, 30 Mar 1999 09:20:00 -0800 (PST) (envelope-from gnats@FreeBSD.org) Received: from luke.pmr.com (luke.pmr.com [207.170.114.132]) by hub.freebsd.org (Postfix) with ESMTP id 6E00314EA6 for ; Tue, 30 Mar 1999 09:10:03 -0800 (PST) (envelope-from bob@luke.pmr.com) Received: (from bob@localhost) by luke.pmr.com (8.9.2/8.9.2) id LAA33066; Tue, 30 Mar 1999 11:09:42 -0600 (CST) (envelope-from bob) Message-Id: <199903301709.LAA33066@luke.pmr.com> Date: Tue, 30 Mar 1999 11:09:42 -0600 (CST) From: bob@pmr.com Reply-To: bob@pmr.com To: FreeBSD-gnats-submit@freebsd.org X-Send-Pr-Version: 3.2 Subject: kern/10872: Panic in soreceive() in 3.1-stable running amanda Sender: owner-freebsd-bugs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >Number: 10872 >Category: kern >Synopsis: Panic in sorecieve() due to NULL mbuf pointer >Confidential: no >Severity: critical >Priority: high >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Tue Mar 30 09:20:00 PST 1999 >Closed-Date: >Last-Modified: >Originator: Bob Willcox >Release: FreeBSD 3.1-STABLE i386 >Organization: Power Micro Research >Environment: FreeBSD deathstar.pmr.com 3.1-STABLE FreeBSD 3.1-STABLE #4: Tue Mar 30 08:59:32 CST 1999 bob@deathstar.pmr.com:/usr/src/sys/compile/DEATHSTAR i386 >Description: A panic occurs on this system during my nightly amanda backups (this is my amanda backup server). The panic is the result of the sb_mb pointer being NULL in soreceive when loaded into m at line 642 in uipc_socket.c. At the time of the panic amanda is loading the system pretty well with 5 dumps running (from 5 different systems on the network) and writing to the Mammoth tape drive. Note that this problem suddenly started happening (last Friday morning). Prior to that I had not changed this system (deathstar) for several weeks, though the client systems had changed (I don't have a precise record of those changes). I have since changed deathstar (upgraded to more recent 3.1-stable and modified the kernel configuration) in a (so far) futile attempt to work-arround the problem. Some (hopefully helpful) info from the crash dump: #0 boot (howto=260) at ../../kern/kern_shutdown.c:285 285 dumppcb.pcb_cr3 = rcr3(); (kgdb) where #0 boot (howto=260) at ../../kern/kern_shutdown.c:285 #1 0xf014e705 in panic (fmt=0xf0233f4c "from debugger") at ../../kern/kern_shutdown.c:446 #2 0xf012aab1 in db_panic (addr=-266261713, have_addr=0, count=-1, modif=0xf4224d5c "") at ../../ddb/db_command.c:432 #3 0xf012aa51 in db_command (last_cmdp=0xf0251e64, cmd_table=0xf0251cc4, aux_cmd_tablep=0xf0267acc) at ../../ddb/db_command.c:332 #4 0xf012ab16 in db_command_loop () at ../../ddb/db_command.c:454 #5 0xf012ce67 in db_trap (type=3, code=0) at ../../ddb/db_trap.c:71 #6 0xf021290a in kdb_trap (type=3, code=0, regs=0xf4224e4c) at ../../i386/i386/db_interface.c:157 #7 0xf021c0b4 in trap (frame={tf_es = 16, tf_ds = 16, tf_edi = -202329632, tf_esi = 256, tf_ebp = -199078256, tf_isp = -199078284, tf_ebx = -266105266, tf_edx = -266043248, tf_ecx = -267680032, tf_eax = 18, tf_trapno = 3, tf_err = 0, tf_eip = -266261713, tf_cs = 8, tf_eflags = 598, tf_esp = -266043264, tf_ss = -266111117}) at ../../i386/i386/trap.c:548 #8 0xf0212b2f in Debugger (msg=0xf0237773 "panic") at ../../i386/i386/db_interface.c:317 #9 0xf014e6fc in panic (fmt=0xf0238e4e "receive 1") at ../../kern/kern_shutdown.c:444 #10 0xf01667d3 in soreceive (so=0xf3f0b1e0, psa=0x0, uio=0xf4224f40, mp0=0x0, controlp=0x0, flagsp=0x0) at ../../kern/uipc_socket.c:659 #11 0xf015c6d4 in soo_read (fp=0xf1026540, uio=0xf4224f40, cred=0xf0f2a180) at ../../kern/sys_socket.c:69 #12 0xf01591ed in read (p=0xf418f3c0, uap=0xf4224f94) at ../../kern/sys_generic.c:121 #13 0xf021c8c3 in syscall (frame={tf_es = -272695257, tf_ds = -272695257, tf_edi = -272638492, tf_esi = 64, tf_ebp = -272638364, tf_isp = -199077916, tf_ebx = 0, tf_edx = 82768, tf_ecx = 6, tf_eax = 3, tf_trapno = 7, tf_err = 7, tf_eip = 537674705, tf_cs = 31, tf_eflags = 514, tf_esp = -272638820, tf_ss = 39}) at ../../i386/i386/trap.c:1100 #14 0x200c43d1 in ?? () #15 0x1f64 in ?? () #16 0x1099 in ?? () (kgdb) up 10 #10 0xf01667d3 in soreceive (so=0xf3f0b1e0, psa=0x0, uio=0xf4224f40, mp0=0x0, controlp=0x0, flagsp=0x0) at ../../kern/uipc_socket.c:659 Source file is more recent than executable. 659 KASSERT(m != 0 || !so->so_rcv.sb_cc, ("receive 1")); (kgdb) list 654 if (m == 0 || (((flags & MSG_DONTWAIT) == 0 && 655 so->so_rcv.sb_cc < uio->uio_resid) && 656 (so->so_rcv.sb_cc < so->so_rcv.sb_lowat || 657 ((flags & MSG_WAITALL) && uio->uio_resid <= so->so_rcv.sb_hiwat)) && 658 m->m_nextpkt == 0 && (pr->pr_flags & PR_ATOMIC) == 0)) { 659 KASSERT(m != 0 || !so->so_rcv.sb_cc, ("receive 1")); 660 if (so->so_error) { 661 if (m) 662 goto dontblock; 663 error = so->so_error; (kgdb) print *so $1 = {so_zone = 0xf0f0ef00, so_type = 1, so_options = 0, so_linger = 0, so_state = 2, so_pcb = 0xf400bea0 "", so_proto = 0xf0259294, so_head = 0x0, so_incomp = {tqh_first = 0x0, tqh_last = 0xf3f0b1f8}, so_comp = { tqh_first = 0x0, tqh_last = 0xf3f0b200}, so_list = {tqe_next = 0x0, tqe_prev = 0x0}, so_qlen = 0, so_incqlen = 0, so_qlimit = 0, so_timeo = 0, so_error = 0, so_sigio = 0x0, so_oobmark = 0, so_rcv = {sb_cc = 4380, sb_hiwat = 17520, sb_mbcnt = 6528, sb_mbmax = 140160, sb_lowat = 1, sb_mb = 0x0, sb_sel = {si_pid = 0, si_flags = 0}, sb_flags = 1, sb_timeo = 0}, so_snd = {sb_cc = 0, sb_hiwat = 17520, sb_mbcnt = 0, sb_mbmax = 140160, sb_lowat = 2048, sb_mb = 0x0, sb_sel = {si_pid = 0, si_flags = 0}, sb_flags = 0, sb_timeo = 0}, so_upcall = 0, so_upcallarg = 0x0, so_uid = 90, so_gencnt = 3716} (kgdb) print m $2 = (struct mbuf *) 0x0 (kgdb) print *uio $3 = {uio_iov = 0xf4224f38, uio_iovcnt = 1, uio_offset = 0xffffffffffffffff, uio_resid = 820, uio_segflg = UIO_USERSPACE, uio_rw = UIO_READ, uio_procp = 0xf418f3c0} Dmesg output: Copyright (c) 1992-1999 FreeBSD Inc. Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. FreeBSD 3.1-STABLE #4: Tue Mar 30 08:59:32 CST 1999 bob@deathstar.pmr.com:/usr/src/sys/compile/DEATHSTAR Timecounter "i8254" frequency 1193182 Hz Timecounter "TSC" frequency 199309847 Hz CPU: Pentium Pro (199.31-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x616 Stepping=6 Features=0xf9ff real memory = 33554432 (32768K bytes) avail memory = 29958144 (29256K bytes) Preloaded elf kernel "kernel" at 0xf02cd000. Probing for devices on PCI bus 0: chip0: rev 0x02 on pci0.0.0 chip1: rev 0x01 on pci0.1.0 ahc0: rev 0x00 int a irq 12 on pci0.10.0 ahc0: aic7870 Single Channel A, SCSI Id=7, 16/255 SCBs fxp0: rev 0x01 int a irq 10 on pci0.11.0 fxp0: Ethernet address 00:a0:c9:31:e6:21 ncr0: rev 0x01 int a irq 11 on pci0.12.0 ncr1: rev 0x03 int a irq 9 on pci0.13.0 Probing for devices on the ISA bus: sc0 on isa sc0: VGA color <16 virtual consoles, flags=0x0> atkbdc0 at 0x60-0x6f on motherboard atkbd0 irq 1 on isa psm0 not found sio0 at 0x3f8-0x3ff irq 4 flags 0x10 on isa sio0: type 16550A sio1 at 0x2f8-0x2ff irq 3 on isa sio1: type 16550A fdc0 at 0x3f0-0x3f7 irq 6 drq 2 on isa fdc0: FIFO enabled, 8 bytes threshold fd0: 1.44MB 3.5in ppc0 at 0x378 irq 7 on isa ppc0: W83877F chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/16 bytes threshold nlpt0: on ppbus 0 nlpt0: Interrupt-driven port ppi0: on ppbus 0 plip0: on ppbus 0 vga0 at 0x3b0-0x3df maddr 0xa0000 msize 131072 on isa npx0 on motherboard npx0: INT 16 interface Waiting 10 seconds for SCSI devices to settle sa0 at ahc0 bus 0 target 1 lun 0 sa0: Removable Sequential Access SCSI-2 device sa0: 10.000MB/s transfers (10.000MHz, offset 15) sa1 at ncr0 bus 0 target 5 lun 0 sa1: Removable Sequential Access SCSI-2 device sa1: 4.807MB/s transfers (4.807MHz, offset 8) changing root device to da0s1a cd0 at ncr0 bus 0 target 4 lun 0 cd0: Removable CD-ROM SCSI-2 device cd0: 4.237MB/s transfers (4.237MHz, offset 8) cd0: Attempt to query device size failed: NOT READY, Medium not present da1 at ncr1 bus 0 target 1 lun 0 da1: Fixed Direct Access SCSI-2 device da1: 40.000MB/s transfers (20.000MHz, offset 15, 16bit) da1: 4134MB (8467200 512 byte sectors: 255H 63S/T 527C) da2 at ncr1 bus 0 target 2 lun 0 da2: Fixed Direct Access SCSI-2 device da2: 40.000MB/s transfers (20.000MHz, offset 15, 16bit), Tagged Queueing Enabled da2: 8715MB (17850000 512 byte sectors: 255H 63S/T 1111C) da0 at ncr1 bus 0 target 0 lun 0 da0: < DFRSS2W 4B4B> Fixed Direct Access SCSI-2 device da0: 20.000MB/s transfers (10.000MHz, offset 15, 16bit), Tagged Queueing Enabled da0: 2150MB (4404489 512 byte sectors: 255H 63S/T 274C) ch0 at ahc0 bus 0 target 0 lun 0 ch0: Removable Changer SCSI-2 device ch0: 3.300MB/s transfers ch0: 11 slots, 1 drive, 1 picker, 0 portals WARNING: / was not properly dismounted ffs_mountfs: superblock updated for soft updates ffs_mountfs: superblock updated for soft updates ffs_mountfs: superblock updated for soft updates ffs_mountfs: superblock updated for soft updates ffs_mountfs: superblock updated for soft updates ffs_mountfs: superblock updated for soft updates ffs_mountfs: superblock updated for soft updates link_elf: symbol splash_register undefined Kernel config file: # # DEATHSTAR -- Configure file of the DEATHSTAR system # # For more information read the handbook part System Administration -> # Configuring the FreeBSD Kernel -> The Configuration File. # The handbook is available in /usr/share/doc/handbook or online as # latest version from the FreeBSD World Wide Web server # # # An exhaustive list of options and more detailed explanations of the # device lines is present in the ./LINT configuration file. If you are # in doubt as to the purpose or necessity of a line, check first in LINT. # # $Id$ machine "i386" cpu "I686_CPU" ident DEATHSTAR maxusers 40 options INET #InterNETworking options FFS #Berkeley Fast Filesystem options FFS_ROOT #FFS usable as root device [keep this!] options MFS #Memory Filesystem options NFS #Network Filesystem options MSDOSFS #MSDOS Filesystem options "CD9660" #ISO 9660 Filesystem options "CD9660_ROOT" #CD-ROM usable as root. "CD9660" req'ed options PROCFS #Process filesystem options "COMPAT_43" #Compatible with BSD 4.3 [KEEP THIS!] options SCSI_DELAY=10000 #Be pessimistic about Joe SCSI device options UCONSOLE #Allow users to grab the console options FAILSAFE #Be conservative options USERCONFIG #boot -c editor options VISUAL_USERCONFIG #visual boot -c editor options SOFTUPDATES #enable soft updates support #options "NMBCLUSTERS=4096" config kernel root on da0 controller isa0 controller pci0 controller fdc0 at isa? port "IO_FD1" bio irq 6 drq 2 disk fd0 at fdc0 drive 0 # A single entry for any of these controllers (ncr, ahb, ahc) is # sufficient for any number of installed devices. controller ncr0 controller ahc0 controller scbus0 device da0 device sa0 device pass0 device cd0 device ch0 # atkbdc0 controlls both the keyboard and the PS/2 mouse controller atkbdc0 at isa? port IO_KBD tty device atkbd0 at isa? tty irq 1 device psm0 at isa? tty irq 12 device vga0 at isa? port ? conflicts # splash screen/screen saver #pseudo-device splash # syscons is the default console driver, resembling an SCO console device sc0 at isa? tty device npx0 at isa? port IO_NPX irq 13 # Serial ports device sio0 at isa? port "IO_COM1" flags 0x10 tty irq 4 device sio1 at isa? port "IO_COM2" tty irq 3 # Parallel port device ppc0 at isa? port? net irq 7 controller ppbus0 device nlpt0 at ppbus? device plip0 at ppbus? device ppi0 at ppbus? #controller vpo0 at ppbus? # Order is important here due to intrusive probes, do *not* alphabetize # this list of network interfaces until the probes have been fixed. # Right now it appears that the ie0 must be probed before ep0. See # revision 1.20 of this file. device de0 device fxp0 pseudo-device loop pseudo-device ether pseudo-device sl 2 pseudo-device ppp 2 pseudo-device tun 2 pseudo-device pty 64 pseudo-device gzip # Exec gzipped a.out's # # Enable debug support # options KTRACE #kernel tracing options DDB #kernel debugger options INVARIANTS #extra sanity checks options INVARIANT_SUPPORT #needed for INVARIANTS # # These three options provide support for System V Interface # Definition-style interprocess communication, in the form of shared # memory, semaphores, and message queues, respectively. # options SYSVSHM options SYSVSEM options SYSVMSG # The `bpfilter' pseudo-device enables the Berkeley Packet Filter. Be # aware of the legal and administrative consequences of enabling this # option. The number of devices determines the maximum number of # simultaneous BPF clients programs runnable. pseudo-device bpfilter 4 #Berkeley packet filter >How-To-Repeat: All I have to do is run amanda and wait for about an hour and a half (that's how long it takes to fail). >Fix: Wish I had one to offer. >Release-Note: >Audit-Trail: >Unformatted: To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-bugs" in the body of the message