Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 10 Dec 2013 15:15:34 +0400
From:      Eugene Mitrofanov <eugene@imedia.ru>
To:        Peter Holm <peter@holm.cc>
Cc:        FreeBSD Stable Mailing List <freebsd-stable@freebsd.org>
Subject:   Re: 9.2-RELEASE + memcached = page fault
Message-ID:  <201312101515.34305@badger>
In-Reply-To: <20131209180532.GA24975@x2.osted.lan>
References:  <201312091924.11590@badger> <20131209180532.GA24975@x2.osted.lan>

next in thread | previous in thread | raw e-mail | index | archive | help
Hello

On Monday 09 December 2013, Peter Holm wrote:
> On Mon, Dec 09, 2013 at 07:24:11PM +0400, Eugene Mitrofanov wrote:
> > Hello
> > 
> > after upgrading to 9.2-release i can see periodical page faults. advises welcome.
> > 
> > examples:
> > 
> > 1.
> > 
> > Fatal trap 12: page fault while in kernel mode
> > cpuid = 5; apic id = 22
> > fault virtual address   = 0x18
> > fault code              = supervisor read data, page not present
> > instruction pointer     = 0x20:0xffffffff8051095c
> > stack pointer           = 0x28:0xffffff88a9d77800
> > frame pointer           = 0x28:0xffffff88a9d77810
> > code segment            = base 0x0, limit 0xfffff, type 0x1b
> >                         = DPL 0, pres 1, long 1, def32 0, gran 1
> > processor eflags        = interrupt enabled, resume, IOPL = 0
> > current process         = 1704 (memcached)
> > trap number             = 12
> > panic: page fault
> > cpuid = 5
> > Uptime: 2d23h33m33s
> > 
> > 2.
> > 
> > Unread portion of the kernel message buffer:
> > stack pointer           = 0x28:0xffffff88a9b70800
> > frame pointer           = 0x28:0xffffff88a9b70810
> > code segment            = base 0x0, limit 0xfffff, type 0x1b
> >                         = DPL 0, pres 1, long 1, def32 0, gran 1
> > processor eflags        = interrupt enabled, resume, IOPL = 0
> > current process         = 2023 (memcached)
> > trap number             = 12
> > panic: page fault
> > cpuid = 4
> > Uptime: 1d10h59m7s
> > 
> > good luck
> 
> It would be nice with a stack trace. Check out:
> http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/kerneldebug-online-ddb.html
> 

Yes of course. You may find it below.

1st:

#0  doadump (textdump=<value optimized out>) at pcpu.h:234
234	pcpu.h: No such file or directory.
	in pcpu.h
(kgdb) #0  doadump (textdump=<value optimized out>) at pcpu.h:234
#1  0xffffffff80523556 in kern_reboot (howto=260)
    at /usr/src/sys/kern/kern_shutdown.c:449
#2  0xffffffff80523a3c in panic (fmt=0x1 <Address 0x1 out of bounds>)
    at /usr/src/sys/kern/kern_shutdown.c:637
#3  0xffffffff80792b9d in trap_fatal (frame=0xfffffe003e20d000, 
    eva=<value optimized out>) at /usr/src/sys/amd64/amd64/trap.c:879
#4  0xffffffff80792eb8 in trap_pfault (frame=0xffffff88a9d77750, usermode=0)
    at /usr/src/sys/amd64/amd64/trap.c:795
#5  0xffffffff8079323c in trap (frame=0xffffff88a9d77750)
    at /usr/src/sys/amd64/amd64/trap.c:463
#6  0xffffffff8077cb5f in calltrap ()
    at /usr/src/sys/amd64/amd64/exception.S:232
#7  0xffffffff8051095c in _mtx_trylock (m=0x0, opts=0, 
    file=<value optimized out>, line=<value optimized out>)
    at /usr/src/sys/kern/kern_mutex.c:295
#8  0xffffffff8076cbae in vm_page_cowsetup (m=0xfffffe08529ecc08)
    at /usr/src/sys/vm/vm_page.c:2849
#9  0xffffffff8058465c in socow_setup (m0=0xfffffe02997b6100, 
    uio=0xffffff88a9d779e0) at /usr/src/sys/kern/uipc_cow.c:130
#10 0xffffffff80592465 in sosend_copyin (uio=0xffffff88a9d779e0, 
    retmp=0xffffff88a9d77960, atomic=0, space=0xffffff88a9d77970, flags=0)
    at /usr/src/sys/kern/uipc_socket.c:991
#11 0xffffffff80592798 in sosend_generic (so=0xfffffe00575abd48, 
    addr=0xfffffe013f50d950, uio=0xffffff88a9d779e0, top=0x0, control=0x0, 
    flags=0, td=0xfffffe003e238920) at /usr/src/sys/kern/uipc_socket.c:1329
#12 0xffffffff805946e3 in kern_sendit (td=0xfffffe003e238920, s=32, 
    mp=0xffffff88a9d77ab0, flags=0, control=0x0, segflg=<value optimized out>)
    at /usr/src/sys/kern/uipc_syscalls.c:811
#13 0xffffffff8059499c in sendit (td=0xfffffe003e238920, s=32, 
    mp=0xffffff88a9d77ab0, flags=0) at /usr/src/sys/kern/uipc_syscalls.c:739
#14 0xffffffff80594a27 in sys_sendmsg (td=0xfffffe003e238920, 
    uap=0xffffff88a9d77bb0) at /usr/src/sys/kern/uipc_syscalls.c:944
#15 0xffffffff80792490 in amd64_syscall (td=0xfffffe003e238920, traced=0)
    at subr_syscall.c:135
#16 0xffffffff8077ce47 in Xfast_syscall ()
    at /usr/src/sys/amd64/amd64/exception.S:391
#17 0x0000000800d3fe2c in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb) 

2nd: 

#0  doadump (textdump=<value optimized out>) at pcpu.h:234
234	pcpu.h: No such file or directory.
	in pcpu.h
(kgdb) #0  doadump (textdump=<value optimized out>) at pcpu.h:234
#1  0xffffffff80523556 in kern_reboot (howto=260)
    at /usr/src/sys/kern/kern_shutdown.c:449
#2  0xffffffff80523a3c in panic (fmt=0x1 <Address 0x1 out of bounds>)
    at /usr/src/sys/kern/kern_shutdown.c:637
#3  0xffffffff80792b9d in trap_fatal (frame=0xfffffe0010664950, 
    eva=<value optimized out>) at /usr/src/sys/amd64/amd64/trap.c:879
#4  0xffffffff80792eb8 in trap_pfault (frame=0xffffff88a9b70750, usermode=0)
    at /usr/src/sys/amd64/amd64/trap.c:795
#5  0xffffffff8079323c in trap (frame=0xffffff88a9b70750)
    at /usr/src/sys/amd64/amd64/trap.c:463
#6  0xffffffff8077cb5f in calltrap ()
    at /usr/src/sys/amd64/amd64/exception.S:232
#7  0xffffffff8051095c in _mtx_trylock (m=0x0, opts=0, 
    file=<value optimized out>, line=<value optimized out>)
    at /usr/src/sys/kern/kern_mutex.c:295
#8  0xffffffff8076cbae in vm_page_cowsetup (m=0xfffffe0850e13900)
    at /usr/src/sys/vm/vm_page.c:2849
#9  0xffffffff8058465c in socow_setup (m0=0xfffffe00543fe300, 
    uio=0xffffff88a9b709e0) at /usr/src/sys/kern/uipc_cow.c:130
#10 0xffffffff80592465 in sosend_copyin (uio=0xffffff88a9b709e0, 
    retmp=0xffffff88a9b70960, atomic=0, space=0xffffff88a9b70970, flags=0)
    at /usr/src/sys/kern/uipc_socket.c:991
#11 0xffffffff80592798 in sosend_generic (so=0xfffffe02ad39caa0, 
    addr=0xfffffe02a3fda520, uio=0xffffff88a9b709e0, top=0x0, control=0x0, 
    flags=0, td=0xfffffe013301a000) at /usr/src/sys/kern/uipc_socket.c:1329
#12 0xffffffff805946e3 in kern_sendit (td=0xfffffe013301a000, s=21, 
    mp=0xffffff88a9b70ab0, flags=0, control=0x0, segflg=<value optimized out>)
    at /usr/src/sys/kern/uipc_syscalls.c:811
#13 0xffffffff8059499c in sendit (td=0xfffffe013301a000, s=21, 
    mp=0xffffff88a9b70ab0, flags=0) at /usr/src/sys/kern/uipc_syscalls.c:739
#14 0xffffffff80594a27 in sys_sendmsg (td=0xfffffe013301a000, 
    uap=0xffffff88a9b70bb0) at /usr/src/sys/kern/uipc_syscalls.c:944
#15 0xffffffff80792490 in amd64_syscall (td=0xfffffe013301a000, traced=0)
    at subr_syscall.c:135
#16 0xffffffff8077ce47 in Xfast_syscall ()
    at /usr/src/sys/amd64/amd64/exception.S:391
#17 0x0000000800d3fe2c in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb) 


> Does this look similar?
> http://lists.freebsd.org/pipermail/freebsd-stable/2012-March/066995.html

Hm. The problem description looks like very similar to mine. I disabled mentioned sysctl variables.
Will wait.

> 
> - Peter
> 
> 

-- 
EVM7-RIPE



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201312101515.34305>