Date: Sat, 15 Mar 2014 17:45:23 -0700 From: Doug Hardie <bc979@lafn.org> To: "questions@freebsd.org FreeBSD" <questions@freebsd.org> Subject: Frequent Page Faults Message-ID: <F0344C79-D919-43B5-88B2-DCDE590741B1@lafn.org>
next in thread | raw e-mail | index | archive | help
I have a system running:
FreeBSD zoon.lafn.org 7.2-RELEASE-p5 FreeBSD 7.2-RELEASE-p5 #3: Thu Aug =
19 20:09:11 PDT 2010
This morning it started crashing frequently. The system had no issues =
prior to today. Sometimes it auto reboots, others it just hangs. The =
console messages are always very similar. I do have core dumps for 3 of =
them. At first it appeared that the problem was being caused by an =
attack on port 110. While the attack was in process, the system would =
stay up for only a few minutes. After discovering and blocking the =
attack, the system remained up a couple hours but crashed while I was =
watching it. I am beginning to suspect a HW issue that was worsened by =
the load of the attack, but not caused directly by the attack.
I have been in the process of building a new set of disks for this =
system using 9.2, but thats not complete yet. In addition it will take =
a couple days to get the disks on site. Its a remote facility. I found =
several references to issues with this problem and 7.2 that have =
apparently been fixed. However, I have been running 7.2 since it first =
came out on this system without any similar issues. Actually, I don't =
recall any issues at all with 7.2. The source for this system no longer =
exists. My development system has been upgraded to 9.2. Its a modified =
kernel with some of the older processors commented out and includes =
QUOTA and ALTQ. I don't recall any other changes.
I can hurry up the setup of the 9.2 system, but it would be at least =
until Wed before it could be installed at tried. If that would correct =
the problem that would be great. However, I have a concern that there =
is also a HW issue here and am not sure how to identify such. My review =
of the dumps shows that acpi is always involved. Don't know for sure =
what that implies though. I don't believe the os has degraded and it =
has not been touched since 2010. That pretty much leaves me with a HW =
issue. Any ideas where the problem is will be appreciated.
Here are a couple of the dumps. The first one was while the attack was =
in progress. The second after it was terminated.
--------------------------------------------------------------
zoon# kgdb /boot/kernel/kernel vmcore.0
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you =
are
welcome to change it and/or distribute copies of it under certain =
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for =
details.
This GDB was configured as "i386-marcel-freebsd"...
Unread portion of the kernel message buffer:
3)
trap number =3D 12
panic: page fault
cpuid =3D 3
Uptime: 3m48s
Physical memory: 1993 MB
Dumping 187 MB: 172 156 140 124
Fatal trap 12: page fault while in kernel mode
cpuid =3D 1; apic id =3D 01
fault virtual address =3D 0x4
fault code =3D supervisor write, page not present
instruction pointer =3D 0x20:0xc0c72c00
stack pointer =3D 0x28:0xe566faf4
frame pointer =3D 0x28:0xe566fb14
code segment =3D base 0x0, limit 0xfffff, type 0x1b
=3D DPL 0, pres 1, def32 1, gran 1
processor eflags =3D interrupt enabled, resume, IOPL =3D 0
current process =3D 4 (g_down)
trap number =3D 12
108 92 76 60 44 28 12
Reading symbols from /boot/kernel/fdescfs.ko...Reading symbols from =
/boot/kernel/fdescfs.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/fdescfs.ko
Reading symbols from /boot/kernel/pflog.ko...Reading symbols from =
/boot/kernel/pflog.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/pflog.ko
Reading symbols from /boot/kernel/pf.ko...Reading symbols from =
/boot/kernel/pf.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/pf.ko
Reading symbols from /boot/kernel/acpi.ko...Reading symbols from =
/boot/kernel/acpi.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/acpi.ko
#0 doadump () at pcpu.h:196
196 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) where
#0 doadump () at pcpu.h:196
#1 0xc07a5e27 in boot (howto=3D260) at =
/usr2/src/sys/kern/kern_shutdown.c:418
#2 0xc07a60f9 in panic (fmt=3DVariable "fmt" is not available.
) at /usr2/src/sys/kern/kern_shutdown.c:574
#3 0xc0aa792c in trap_fatal (frame=3D0xc4ff8c48, eva=3D1361334589)
at /usr2/src/sys/i386/i386/trap.c:939
#4 0xc0aa7b90 in trap_pfault (frame=3D0xc4ff8c48, usermode=3D0, =
eva=3D1361334589)
at /usr2/src/sys/i386/i386/trap.c:852
#5 0xc0aa8512 in trap (frame=3D0xc4ff8c48) at =
/usr2/src/sys/i386/i386/trap.c:530
#6 0xc0a8d62b in calltrap () at /usr2/src/sys/i386/i386/exception.s:159
#7 0xc0e21715 in acpi_cpu_c1 ()
at =
/usr2/src/sys/modules/acpi/acpi/../../../i386/acpica/acpi_machdep.c:550
#8 0xc0e1a594 in acpi_cpu_idle ()
at =
/usr2/src/sys/modules/acpi/acpi/../../../dev/acpica/acpi_cpu.c:943
#9 0xc0a97f78 in cpu_idle () at /usr2/src/sys/i386/i386/machdep.c:1183
#10 0xc07c7904 in sched_idletd (dummy=3D0x0)
at /usr2/src/sys/kern/sched_ule.c:2681
#11 0xc07808d9 in fork_exit (callout=3D0xc07c7640 <sched_idletd>, =
arg=3D0x0,=20
frame=3D0xc4ff8d38) at /usr2/src/sys/kern/kern_fork.c:810
#12 0xc0a8d6a0 in fork_trampoline () at =
/usr2/src/sys/i386/i386/exception.s:264
--------------------------------------------------------------
zoon# kgdb /boot/kernel/kernel vmcore.2
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you =
are
welcome to change it and/or distribute copies of it under certain =
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for =
details.
This GDB was configured as "i386-marcel-freebsd"...
Unread portion of the kernel message buffer:
Fatal double fault:
eip =3D 0xc0e21715
esp =3D 0xc4ff8d80
ebp =3D 0xc4ff8c88
cpuid =3D 3; apic id =3D 03
panic: double fault
cpuid =3D 3
Uptime: 2h19m49s
Physical memory: 1993 MB
Dumping 187 MB: 172 156 140 124 108 92 76 60 44 28 12
Reading symbols from /boot/kernel/fdescfs.ko...Reading symbols from =
/boot/kernel/fdescfs.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/fdescfs.ko
Reading symbols from /boot/kernel/pflog.ko...Reading symbols from =
/boot/kernel/pflog.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/pflog.ko
Reading symbols from /boot/kernel/pf.ko...Reading symbols from =
/boot/kernel/pf.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/pf.ko
Reading symbols from /boot/kernel/acpi.ko...Reading symbols from =
/boot/kernel/acpi.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/acpi.ko
#0 doadump () at pcpu.h:196
196 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) up
#1 0xc07a5e27 in boot (howto=3D260) at =
/usr2/src/sys/kern/kern_shutdown.c:418
418 /usr2/src/sys/kern/kern_shutdown.c: No such file or directory.
in /usr2/src/sys/kern/kern_shutdown.c
(kgdb) down
#0 doadump () at pcpu.h:196
196 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) where
#0 doadump () at pcpu.h:196
#1 0xc07a5e27 in boot (howto=3D260) at =
/usr2/src/sys/kern/kern_shutdown.c:418
#2 0xc07a60f9 in panic (fmt=3DVariable "fmt" is not available.
) at /usr2/src/sys/kern/kern_shutdown.c:574
#3 0xc0aa763b in dblfault_handler () at =
/usr2/src/sys/i386/i386/trap.c:972
#4 0xc0e21715 in acpi_cpu_c1 ()
at =
/usr2/src/sys/modules/acpi/acpi/../../../i386/acpica/acpi_machdep.c:550
#5 0xc0e1a594 in acpi_cpu_idle ()
at =
/usr2/src/sys/modules/acpi/acpi/../../../dev/acpica/acpi_cpu.c:943
#6 0xc0a97f78 in cpu_idle () at /usr2/src/sys/i386/i386/machdep.c:1183
#7 0xc07c7904 in sched_idletd (dummy=3D0x0)
at /usr2/src/sys/kern/sched_ule.c:2681
#8 0xc07808d9 in fork_exit (callout=3D0xc07c7640 <sched_idletd>, =
arg=3D0x0,=20
frame=3D0xc4ff8d38) at /usr2/src/sys/kern/kern_fork.c:810
#9 0xc0a8d6a0 in fork_trampoline () at =
/usr2/src/sys/i386/i386/exception.s:264
(kgdb)=20
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?F0344C79-D919-43B5-88B2-DCDE590741B1>