Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 10 Jun 1999 22:39:25 -0700 (PDT)
From:      reggie@lambdawerks.org
To:        freebsd-gnats-submit@freebsd.org
Subject:   kern/12127: persistent crash on idle SMP system
Message-ID:  <19990611053925.7DB9915332@hub.freebsd.org>

next in thread | raw e-mail | index | archive | help

>Number:         12127
>Category:       kern
>Synopsis:       persistent crash on idle SMP system
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Jun 10 22:40:01 PDT 1999
>Closed-Date:
>Last-Modified:
>Originator:     Reginald Perry
>Release:        4.0 CURRENT compiled June 4, 1999
>Organization:
>Environment:
FreeBSD trane.lambdawerks.org 4.0-CURRENT FreeBSD 4.0-CURRENT #0: Fri Jun  4 20:01:39 PDT 1999     root@:/usr/src/sys/compile/TRANE  i386

>Description:
After telnetting into the system this afternoon, I come home to find it had crashed.
This has been pretty consistent, but this is the first time that I have lots
of details. This is a dual pentium 166MMX system on a Tyan Tomcat III motherboard. the 
boot messages from the reboot before the crash is attached. I have the debug
kernel and the crash dump if you want it. Ill keep it for a time in case you want it.
Here is the backtrace from the crash dump:

trane# gdb -k /usr/src/sys/compile/TRANE/kernel.debug ./vmcore.0 
GNU gdb 4.18
Copyright 1998 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-unknown-freebsd"...
SMP 2 cpus
IdlePTD 3325952
initial pcb at 2aee40
panicstr: from debugger
panic messages:
---
Fatal trap 12: page fault while in kernel mode
mp_lock = 00000002; cpuid = 0; lapic.id = 00000000
fault virtual address	= 0xc6c
fault code		= supervisor read, page not present
instruction pointer	= 0x8:0xc0221506
stack pointer	        = 0x10:0xc649de3c
frame pointer	        = 0x10:0xc649de44
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, def32 1, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 4 (syncer)
interrupt mask		= net tty bio cam  <- SMP: XXX
panic: from debugger
mp_lock = 00000002; cpuid = 0; lapic.id = 00000000
panic: from debugger
mp_lock = 00000003; cpuid = 0; lapic.id = 00000000
boot() called on cpu#0

dumping to dev (4,131081), offset 280576
dump 128 127 126 125 124 123 122 121 120 119 118 117 116 115 114 113 112 111 110 109 108 107 106 105 104 103 102 101 100 99 98 97 96 95 94 93 92 91 90 89 88 87 86 85 84 83 82 81 80 79 78 77 76 75 74 73 72 71 70 69 68 67 66 65 64 63 62 61 60 59 58 57 56 55 54 53 52 51 50 49 48 47 46 45 44 43 42 41 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 
---
#0  boot (howto=260) at ../../kern/kern_shutdown.c:288
288			dumppcb.pcb_cr3 = rcr3();
(kgdb) where
#0  boot (howto=260) at ../../kern/kern_shutdown.c:288
#1  0xc014c7d5 in panic (fmt=0xc025fdf4 "from debugger")
    at ../../kern/kern_shutdown.c:451
#2  0xc012e8ed in db_panic (addr=-1071508218, have_addr=0, count=1, 
    modif=0xc649dcb8 "") at ../../ddb/db_command.c:434
#3  0xc012e88d in db_command (last_cmdp=0xc028c050, cmd_table=0xc028beb0, 
    aux_cmd_tablep=0xc02ac554) at ../../ddb/db_command.c:334
#4  0xc012e952 in db_command_loop () at ../../ddb/db_command.c:456
#5  0xc0130ab3 in db_trap (type=12, code=0) at ../../ddb/db_trap.c:71
#6  0xc0212fbe in kdb_trap (type=12, code=0, regs=0xc649ddfc)
    at ../../i386/i386/db_interface.c:157
#7  0xc0225c8e in trap_fatal (frame=0xc649ddfc, eva=3180)
    at ../../i386/i386/trap.c:903
#8  0xc0225925 in trap_pfault (frame=0xc649ddfc, usermode=0, eva=3180)
    at ../../i386/i386/trap.c:801
#9  0xc022556f in trap (frame={tf_fs = 24, tf_es = 16, tf_ds = 16, 
      tf_edi = -1066754304, tf_esi = -954306560, tf_ebp = -968237500, 
      tf_isp = -968237528, tf_ebx = -957843996, tf_edx = 0, tf_ecx = 1, 
      tf_eax = 795, tf_trapno = 12, tf_err = 0, tf_eip = -1071508218, 
      tf_cs = 8, tf_eflags = 66071, tf_esp = -956480028, tf_ss = 0})
    at ../../i386/i386/trap.c:427
#10 0xc0221506 in pmap_pte_quick (pmap=0xc71e7000, va=3337123300)
    at ../../i386/i386/pmap.c:708
#11 0xc022263f in pmap_remove_all (pa=240062464) at ../../i386/i386/pmap.c:1882
#12 0xc02239ee in pmap_page_protect (phys=240062464, prot=0)
    at ../../i386/i386/pmap.c:3149
#13 0xc016d33d in vfs_busy_pages (bp=0xc34de1b0, clear_modify=1)
    at ../../vm/vm_page.h:460
#14 0xc016b075 in bwrite (bp=0xc34de1b0) at ../../kern/vfs_bio.c:522
#15 0xc016f62e in vop_stdbwrite (ap=0xc649df1c) at ../../kern/vfs_default.c:309
#16 0xc016f47d in vop_defaultop (ap=0xc649df1c) at ../../kern/vfs_default.c:131
#17 0xc01f5bf1 in ufs_vnoperate (ap=0xc649df1c)
    at ../../ufs/ufs/ufs_vnops.c:2309
#18 0xc016bac8 in vfs_bio_awrite (bp=0xc34de1b0) at vnode_if.h:1145
#19 0xc01ef09e in ffs_fsync (ap=0xc649df74) at ../../ufs/ffs/ffs_vnops.c:210
#20 0xc01719ce in sched_sync () at vnode_if.h:499
#21 0xc013f043 in kproc_start (udata=0xc0266fa8) at ../../kern/init_main.c:294
#22 0xc0213a28 in fork_trampoline ()
Cannot access memory at address 0x336000.



Jun  7 11:21:37 trane /kernel: Copyright (c) 1992-1999 The FreeBSD Project.
Jun  7 11:21:37 trane /kernel: Copyright (c) 1982, 1986, 1989, 1991, 1993
Jun  7 11:21:37 trane /kernel: The Regents of the University of California. All rights reserved.
Jun  7 11:21:37 trane /kernel: FreeBSD 4.0-CURRENT #0: Fri Jun  4 20:01:39 PDT 1999
Jun  7 11:21:37 trane /kernel: root@:/usr/src/sys/compile/TRANE
Jun  7 11:21:37 trane /kernel: Timecounter "i8254"  frequency 1193182 Hz
Jun  7 11:21:37 trane /kernel: CPU: Pentium/P55C (586-class CPU)
Jun  7 11:21:37 trane /kernel: Origin = "GenuineIntel"  Id = 0x543  Stepping=3
Jun  7 11:21:37 trane /kernel: Features=0x8003bf<FPU,VME,DE,PSE,TSC,MSR,MCE,CX8,APIC,MMX>
Jun  7 11:21:37 trane /kernel: real memory  = 134217728 (131072K bytes)
Jun  7 11:21:37 trane /kernel: config> pnp 1 0 os enable irq0 5 drq0 1 drq1 5 port0 0x220 port1 0x330 port2 0x388
Jun  7 11:21:37 trane /kernel: config> pnp 1 1 os disable
Jun  7 11:21:37 trane /kernel: config> pnp 1 2 os enable port0 0x620 port1 0xa20 port2 0xe20
Jun  7 11:21:37 trane /kernel: config> pnp 1 3 os disable
Jun  7 11:21:37 trane /kernel: config> quit
Jun  7 11:21:37 trane /kernel: sio0: system console
Jun  7 11:21:37 trane /kernel: avail memory = 127332352 (124348K bytes)
Jun  7 11:21:37 trane /kernel: Programming 24 pins in IOAPIC #0
Jun  7 11:21:37 trane /kernel: FreeBSD/SMP: Multiprocessor motherboard
Jun  7 11:21:37 trane /kernel: cpu0 (BSP): apic id:  0, version: 0x00030010, at 0xfee00000
Jun  7 11:21:37 trane /kernel: cpu1 (AP):  apic id:  1, version: 0x00030010, at 0xfee00000
Jun  7 11:21:37 trane /kernel: io0 (APIC): apic id:  2, version: 0x00170011, at 0xfec00000
Jun  7 11:21:37 trane /kernel: Preloaded elf kernel "kernel" at 0xc031a000.
Jun  7 11:21:37 trane /kernel: Preloaded userconfig_script "/boot/kernel.conf" at 0xc031a0a8.
Jun  7 11:21:37 trane /kernel: Intel Pentium detected, installing workaround for F00F bug
Jun  7 11:21:37 trane /kernel: Probing for PnP devices:
Jun  7 11:21:37 trane /kernel: CSN 1 Vendor ID: CTL009e [0x9e008c0e] Serial 0x0d9191f1 Comp ID: PNPb02f [0x2fb0d041]
Jun  7 11:21:37 trane /kernel: npx0: <math processor> on motherboard
Jun  7 11:21:37 trane /kernel: npx0: INT 16 interface
Jun  7 11:21:37 trane /kernel: pcib0: <PCI host bus adapter> on motherboard
Jun  7 11:21:37 trane /kernel: pci0: <PCI bus> on pcib0
Jun  7 11:21:37 trane /kernel: chip0: <Intel 82439HX PCI cache memory controller> at device 0.0 on pci0
Jun  7 11:21:37 trane /kernel: isab0: <Intel 82371SB PCI to ISA bridge> at device 7.0 on pci0
Jun  7 11:21:37 trane /kernel: ide_pci0: <Intel PIIX3 Bus-master IDE controller> at device 7.1 on pci0
Jun  7 11:21:37 trane /kernel: vga-pci0: <Matrox MGA 2164W graphics accelerator> irq 18 at device 18.0 on pci0
Jun  7 11:21:37 trane /kernel: bt0: <Buslogic Multi-Master SCSI Host Adapter> irq 17 at device 19.0 on pci0
Jun  7 11:21:37 trane /kernel: bt0: BT-958 FW Rev. 5.06I Ultra Wide SCSI Host Adapter, SCSI ID 7, 192 CCBs
Jun  7 11:21:37 trane /kernel: fxp0: <Intel EtherExpress Pro 10/100B Ethernet> irq 16 at device 20.0 on pci0
Jun  7 11:21:37 trane /kernel: fxp0: Ethernet address 00:a0:c9:90:bb:52
Jun  7 11:21:37 trane /kernel: devclass_alloc_unit: bt0 already exists, using next available unit number
Jun  7 11:21:37 trane /kernel: isa0: <ISA bus> on motherboard
Jun  7 11:21:37 trane /kernel: fdc0: <NEC 72065B or clone> at port 0x3f0-0x3f7 irq 6 drq 2 on isa0
Jun  7 11:21:37 trane /kernel: fdc0: FIFO enabled, 8 bytes threshold
Jun  7 11:21:37 trane /kernel: fd0: <1440-KB 3.5" drive> at fdc0 drive 0
Jun  7 11:21:37 trane /kernel: atkbdc0: <keyboard controller (i8042)> at port 0x60-0x6f on isa0
Jun  7 11:21:37 trane /kernel: atkbd0: <AT Keyboard> irq 1 on atkbdc0
Jun  7 11:21:37 trane /kernel: psm0: <PS/2 Mouse> irq 12 on atkbdc0
Jun  7 11:21:37 trane /kernel: psm0: model Generic PS/2 mouse, device ID 0
Jun  7 11:21:37 trane /kernel: vga0: <Generic ISA VGA> on isa0
Jun  7 11:21:37 trane /kernel: sc0: <System console> on isa0
Jun  7 11:21:37 trane /kernel: sc0: VGA color <16 virtual consoles, flags=0x0>
Jun  7 11:21:37 trane /kernel: sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
Jun  7 11:21:37 trane /kernel: sio0: type 16550A, console
Jun  7 11:21:37 trane /kernel: sio1 at port 0x2f8-0x2ff irq 3 on isa0
Jun  7 11:21:37 trane /kernel: sio1: type 16550A
Jun  7 11:21:37 trane /kernel: sio2: not probed (disabled)
Jun  7 11:21:37 trane /kernel: sio3: not probed (disabled)
Jun  7 11:21:37 trane /kernel: ppc0 at port 0x378-0x37f irq 7 flags 0x40 on isa0
Jun  7 11:21:37 trane /kernel: ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode
Jun  7 11:21:37 trane /kernel: ppc0: FIFO with 16/16/15 bytes threshold
Jun  7 11:21:37 trane /kernel: ppb0: IEEE1284 device found /NIBBLE/ECP
Jun  7 11:21:38 trane /kernel: Probing for PnP devices on ppbus0:
Jun  7 11:21:38 trane /kernel: ppbus0: <HEWLETT-PACKARD DESKJET 600> MLC,PCL,PML
Jun  7 11:21:38 trane /kernel: plip0: <PLIP network interface> on ppbus 0
Jun  7 11:21:38 trane /kernel: lpt0: <generic printer> on ppbus 0
Jun  7 11:21:38 trane /kernel: lpt0: Interrupt-driven port
Jun  7 11:21:38 trane /kernel: ppi0: <generic parallel i/o> on ppbus 0
Jun  7 11:21:38 trane /kernel: sb0 at port 0x220 irq 5 drq 1 on isa0
Jun  7 11:21:38 trane /kernel: snd0: <SoundBlaster 16 4.16> 
Jun  7 11:21:38 trane /kernel: sbxvi0 at port 0xffffffff drq 5 on isa0
Jun  7 11:21:38 trane /kernel: isa_compat: didn't get ports for sbxvi
Jun  7 11:21:38 trane /kernel: snd0: <SoundBlaster 16 4.16> 
Jun  7 11:21:38 trane /kernel: sbmidi0 at port 0x330 on isa0
Jun  7 11:21:38 trane /kernel: snd0: <SoundBlaster MPU-401> 
Jun  7 11:21:38 trane /kernel: opl0 at port 0x388 on isa0
Jun  7 11:21:38 trane /kernel: snd0: <Yamaha OPL3 FM> 
Jun  7 11:21:38 trane /kernel: awe0 at port 0x620 on isa0
Jun  7 11:21:38 trane /kernel: awe0: <SoundBlaster EMU8000 MIDI (RAM4096k)>
Jun  7 11:21:38 trane /kernel: APIC_IO: Testing 8254 interrupt delivery
Jun  7 11:21:38 trane /kernel: APIC_IO: routing 8254 via pin 2
Jun  7 11:21:38 trane /kernel: Waiting 2 seconds for SCSI devices to settle
Jun  7 11:21:38 trane /kernel: SMP: AP CPU #1 Launched!
Jun  7 11:21:38 trane /kernel: sa0 at bt0 bus 0 target 5 lun 0
Jun  7 11:21:38 trane /kernel: sa0: <HP C1533A 9608> Removable Sequential Access SCSI-2 device 
Jun  7 11:21:38 trane /kernel: sa0: 10.000MB/s transfers (10.000MHz, offset 8)
Jun  7 11:21:38 trane /kernel: da0 at bt0 bus 0 target 0 lun 0
Jun  7 11:21:38 trane /kernel: da0: <IBM DCAS-34330W   !# S65A> Fixed Direct Access SCSI-2 device 
Jun  7 11:21:38 trane /kernel: da0: 40.000MB/s transfers (20.000MHz, offset 15, 16bit)
Jun  7 11:21:38 trane /kernel: da0: 4134MB (8467200 512 byte sectors: 255H 63S/T 527C)
Jun  7 11:21:38 trane /kernel: da1 at bt0 bus 0 target 4 lun 0
Jun  7 11:21:38 trane /kernel: da1: <IBM DCAS-34330W   !# S65A> Fixed Direct Access SCSI-2 device 
Jun  7 11:21:38 trane /kernel: da1: 40.000MB/s transfers (20.000MHz, offset 15, 16bit)
Jun  7 11:21:38 trane /kernel: da1: 4134MB (8467200 512 byte sectors: 255H 63S/T 527C)
Jun  7 11:21:38 trane /kernel: changing root device to da0s1a
Jun  7 11:21:38 trane /kernel: cd0 at bt0 bus 0 target 1 lun 0
Jun  7 11:21:38 trane /kernel: cd0: <TOSHIBA CD-ROM XM-5701TA 0167> Removable CD-ROM SCSI-2 device 
Jun  7 11:21:38 trane /kernel: cd0: 10.000MB/s transfers (10.000MHz, offset 8)
Jun  7 11:21:38 trane /kernel: cd0: cd present [251805 x 2048 byte records]
Jun  7 11:21:38 trane savecore: no core dump
Jun  7 11:21:40 trane lpd[165]: restarted

#
# TRANE -- Config file for Dual P5 166MMX w/ BT UW SCSI
#
# For more information read the handbook part System Administration -> 
# Configuring the FreeBSD Kernel -> The Configuration File. 
# The handbook is available in /usr/share/doc/handbook or online as
# latest version from the FreeBSD World Wide Web server 
# <URL:http://www.FreeBSD.ORG/>;
#
# An exhaustive list of options and more detailed explanations of the 
# device lines is present in the ./LINT configuration file. If you are 
# in doubt as to the purpose or necessity of a line, check first in LINT.
#
#	$Id: GENERIC,v 1.172 1999/05/21 04:37:35 wpaul Exp $

machine		i386
cpu		I586_CPU
cpu		I686_CPU
ident		TRANE
maxusers	64

makeoptions	DEBUG=-g		#Build kernel with gdb(1) debug symbols

options		INET			#InterNETworking
options		FFS			#Berkeley Fast Filesystem
options		FFS_ROOT		#FFS usable as root device [keep this!]
options         SOFTUPDATES
options		MFS			#Memory Filesystem
options		MFS_ROOT		#MFS usable as root device, "MFS" req'ed
options		NFS			#Network Filesystem
options		NFS_ROOT		#NFS usable as root device, "NFS" req'ed
options		MSDOSFS			#MSDOS Filesystem
options		CD9660			#ISO 9660 Filesystem
options		CD9660_ROOT		#CD-ROM usable as root. "CD9660" req'ed
options		PROCFS			#Process filesystem
options		COMPAT_43		#Compatible with BSD 4.3 [KEEP THIS!]
options		UCONSOLE		#Allow users to grab the console
options		USERCONFIG		#boot -c editor
options		VISUAL_USERCONFIG	#visual boot -c editor

# To make an SMP kernel, the next two are needed
options	SMP			# Symmetric MultiProcessor Kernel
options	APIC_IO			# Symmetric (APIC) I/O
# Optionally these may need tweaked, (defaults shown):
#options	NCPU=2			# number of CPUs
#options	NBUS=4			# number of busses
#options	NAPIC=1			# number of IO APICs
#options	NINTR=24		# number of INTs

controller	isa0
controller	pnp0			# PnP support for ISA
controller	eisa0
controller	pci0

controller	fdc0	at isa? port IO_FD1 irq 6 drq 2
disk		fd0	at fdc0 drive 0
disk		fd1	at fdc0 drive 1

controller	wdc0	at isa? port IO_WD1 irq 14
disk		wd0	at wdc0 drive 0
disk		wd1	at wdc0 drive 1

controller	wdc1	at isa? port IO_WD2 irq 15
disk		wd2	at wdc1 drive 0
disk		wd3	at wdc1 drive 1

# ATAPI devices on wdc?
device		wcd0		#IDE CD-ROM
device		wfd0		#IDE Floppy (e.g. LS-120)
device		wst0		#IDE Tape (e.g. Travan)

controller	bt0	at isa? port ? irq ?

controller	scbus0

device		da0	#Only need one of these, the code dynamically grows
device		sa0
device		pass0
device		cd0


# atkbdc0 controls both the keyboard and the PS/2 mouse
controller	atkbdc0	at isa? port IO_KBD
device		atkbd0	at atkbdc? irq 1
device		psm0	at atkbdc? irq 12

device		vga0	at isa? port ? conflicts

# splash screen/screen saver
pseudo-device	splash

# syscons is the default console driver, resembling an SCO console
device		sc0	at isa?

# Enable this and PCVT_FREEBSD for pcvt vt220 compatible console driver
#device		vt0	at isa?
#options		XSERVER			# support for X server
#options		FAT_CURSOR		# start with block cursor
# If you have a ThinkPAD, uncomment this along with the rest of the PCVT lines
#options		PCVT_SCANSET=2		# IBM keyboards are non-std

device		npx0	at nexus? port IO_NPX irq 13

device		sio0	at isa? port IO_COM1 flags 0x10 irq 4
device		sio1	at isa? port IO_COM2 irq 3
device		sio2	at isa? disable port IO_COM3 irq 5
device		sio3	at isa? disable port IO_COM4 irq 9

# Parallel port
device		ppc0	at isa? port? flags 0x40 irq 7
controller	ppbus0
device		lpt0	at ppbus?
device		plip0	at ppbus?
device		ppi0	at ppbus?
#controller	vpo0	at ppbus?

# Controls all "VOXWARE" driver sound devices.  See Luigi's driver
# below for an alternate which may work better for some cards.
#
controller	snd0
device sb0      at isa? port 0x220 irq 5 drq 1
device sbxvi0   at isa? drq 5
device sbmidi0  at isa? port 0x330
device opl0     at isa? port 0x388
device awe0     at isa? port 0x620

#
# The following Ethernet NICs are all PCI devices.
#
device de0		# DEC/Intel DC21x4x (``Tulip'')
device fxp0		# Intel EtherExpress PRO/100B (82557, 82558)
device xl0		# 3Com 3c90x (``Boomerang'', ``Cyclone'')

#device ed0 at isa? port 0x280 net irq 10 iomem 0xc0000

pseudo-device	loop
pseudo-device	ether
pseudo-device	sl	1
pseudo-device	ppp	1
pseudo-device	tun	1
pseudo-device	pty	16
pseudo-device	gzip		# Exec gzipped a.out's

# KTRACE enables the system-call tracing facility ktrace(2).
# This adds 4 KB bloat to your kernel, and slightly increases
# the costs of each syscall.
options		KTRACE		#kernel tracing

# This provides support for System V shared memory and message queues.
#
options         SYSVSHM
options         SYSVMSG
options         SYSVSEM

#  The `bpfilter' pseudo-device enables the Berkeley Packet Filter.  Be
#  aware of the legal and administrative consequences of enabling this
#  option.  The number of devices determines the maximum number of
#  simultaneous BPF clients programs runnable.
pseudo-device	bpfilter 4	#Berkeley packet filter

# USB support
#controller	uhci0
#controller	ohci0
#controller	usb0
#
#device		ugen0
#device		uhid0
#device		ukbd0
#device		ulpt0
#controller	umass0
#device		ums0

#####################################################################
# DEBUGGING OPTIONS

#
# Enable the kernel debugger.
#
options		DDB
options		BREAK_TO_DEBUGGER	#a BREAK on a comconsole goes to 
					#DDB, if available.
# POSIX P1003.1B

# Real time extensions added int the 1993 Posix
# P1003_1B: Infrastructure
# _KPOSIX_PRIORITY_SCHEDULING: Build in _POSIX_PRIORITY_SCHEDULING
# _KPOSIX_VERSION:             Version kernel is built for

options		P1003_1B
options		_KPOSIX_PRIORITY_SCHEDULING
options		_KPOSIX_VERSION=199309L



>How-To-Repeat:
This happened on an idle system.
>Fix:


>Release-Note:
>Audit-Trail:
>Unformatted:


To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-bugs" in the body of the message




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?19990611053925.7DB9915332>