From owner-freebsd-questions@FreeBSD.ORG Wed Dec 14 21:47:04 2005 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 62FDB16A41F for ; Wed, 14 Dec 2005 21:47:04 +0000 (GMT) (envelope-from mags@oniduo.pt) Received: from smtp.net4b.pt (relay-out17861.net4b.pt [195.245.178.61]) by mx1.FreeBSD.org (Postfix) with ESMTP id DA44743D99 for ; Wed, 14 Dec 2005 21:46:44 +0000 (GMT) (envelope-from mags@oniduo.pt) Received: (qmail 19706 invoked from network); 14 Dec 2005 21:46:40 +0000 Received: from unknown (HELO [192.168.1.75]) ([194.38.156.250]) (envelope-sender ) by smtp.net4b.pt (qmail-ldap-1.03) with SMTP for ; 14 Dec 2005 21:46:39 +0000 From: Miguel Saturnino To: freebsd-questions@freebsd.org In-Reply-To: <1134387350.788.21.camel@localhost> References: <1134387350.788.21.camel@localhost> Content-Type: text/plain Date: Wed, 14 Dec 2005 21:46:36 +0000 Message-Id: <1134596796.788.9.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.4.2 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Subject: Re: FreeBSD 6.0-RELEASE - panic: page fault X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2005 21:47:04 -0000 On Mon, 2005-12-12 at 11:35 +0000, Miguel Saturnino wrote: > My server reboots sporadically (once a day, maximum) and I hope someone > can shed some light on why this is happening. > > I've upgraded to 6.0-RELEASE yesterday but the symptoms remain... > Attached is the backtrace of the crash dump and the relevant portion > of /var/log/messages. > > My custom kernel is the generic kernel plus support for quotas. > > Also, during last buildworld I got a "segmentation fault" (only thing I > have in /etc/make.conf is "CFLAGS= -O -pipe" and "NO_PROFILE= true"). I > then tried it again (in the exact same conditions), and it worked. Don't > know if that might indicate bad RAM (the only memory test I'm able to > run is memtest but it can only lock (to test) about 15% of the total > system memory...). > Since I got no answers, I'll reply to myself ;) I have now replaced the memory, but the problem persists. So I guess it's highly unlikely that the problem is with the RAM. I started saving the output of top, vmstat and iostat at each minute, and this is from the last records I got before the last crash (the machine is a web server but has only a few low traffic sites): top: ==== last pid: 29978; load averages: 0.00, 0.00, 0.00 up 0+16:10:46 19:41:00 123 processes: 1 running, 122 sleeping Mem: 186M Active, 115M Inact, 64M Wired, 14M Cache, 60M Buf, 114M Free Swap: 2048M Total, 2048M Free vmstat: ======= procs memory page disk faults cpu r b w avm fre flt re pi po fr sr ad0 in sy cs us sy id 0 2 0 517256 130988 84 0 0 0 90 4 0 214 42726 495 7 4 90 iostat: ======= tty ad0 cpu tin tout KB/t tps MB/s us ni sy in id 0 12 13.06 6 0.07 7 0 3 0 90 latest backtrace: ================= Unread portion of the kernel message buffer: kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode fault virtual address = 0x24 fault code = supervisor read, page not present instruction pointer = 0x20:0xc0657a48 stack pointer = 0x28:0xd565eafc frame pointer = 0x28:0xd565eb10 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = resume, IOPL = 0 current process = 37 (swi1: net) trap number = 12 panic: page fault Uptime: 16h11m16s Dumping 510 MB (2 chunks) chunk 0: 1MB (159 pages) ... ok chunk 1: 510MB (130544 pages) 494 478 462 446 430 414 398 382 366 350 334 318 302 286 270 254 238 222 206 190 174 158 142 126 110 94 78 62 46 30 14 #0 doadump () at pcpu.h:165 165 __asm __volatile("movl %%fs:0,%0" : "=r" (td)); (kgdb) bt #0 doadump () at pcpu.h:165 #1 0xc0638202 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:399 #2 0xc0638498 in panic (fmt=0xc084ece2 "%s") at /usr/src/sys/kern/kern_shutdown.c:555 #3 0xc0808370 in trap_fatal (frame=0xd565eabc, eva=36) at /usr/src/sys/i386/i386/trap.c:831 #4 0xc0807b12 in trap (frame= {tf_fs = -1065418744, tf_es = -1046544344, tf_ds = -714801112, tf_edi = 1, tf_esi = -1032972416, tf_ebp = -714740976, tf_isp = -714741016, tf_ebx = -1046540928, tf_edx = -1032972416, tf_ecx = -1064083040, tf_eax = 0, tf_trapno = 12, tf_err = 0, tf_eip = -1067091384, tf_cs = 32, tf_eflags = 589954, tf_esp = 40, tf_ss = 0}) at /usr/src/sys/i386/i386/trap.c:267 #5 0xc07f750a in calltrap () at /usr/src/sys/i386/i386/exception.s:139 #6 0xc0657a48 in propagate_priority (td=0xc26e1780) at /usr/src/sys/kern/subr_turnstile.c:233 #7 0xc06581f6 in turnstile_wait (lock=0xc09361a0, owner=0xc26e1780) at /usr/src/sys/kern/subr_turnstile.c:628 #8 0xc062fa6c in _mtx_lock_sleep (m=0xc09361a0, tid=3248426368, opts=0, file=0x0, line=0) at /usr/src/sys/kern/kern_mutex.c:553 #9 0xc06772bd in sf_buf_mext (addr=0xd4623000, args=0x0) at /usr/src/sys/kern/uipc_syscalls.c:1690 #10 0xc066d6bd in mb_free_ext (m=0xc2162b00) at /usr/src/sys/kern/uipc_mbuf.c:258 #11 0xc06748a4 in sbdrop_locked (sb=0xc1deb0c8, len=0) at mbuf.h:427 #12 0xc06d7938 in tcp_input (m=0xc1be3d00, off0=20) at /usr/src/sys/netinet/tcp_input.c:2121 #13 0xc06cf1c1 in ip_input (m=0xc1be3d00) at /usr/src/sys/netinet/ip_input.c:778 #14 0xc06ac483 in netisr_processqueue (ni=0xc09292d8) at /usr/src/sys/net/netisr.c:236 #15 0xc06ac67e in swi_net (dummy=0x0) at /usr/src/sys/net/netisr.c:349 #16 0xc0623df9 in ithread_loop (arg=0xc19e4180) at /usr/src/sys/kern/kern_intr.c:547 #17 0xc0623080 in fork_exit (callout=0xc0623ca0 , arg=0xc19e4180, frame=0xd565ed38) at /usr/src/sys/kern/kern_fork.c:789 #18 0xc07f756c in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:208 (kgdb) f 5 #5 0xc07f750a in calltrap () at /usr/src/sys/i386/i386/exception.s:139 139 call trap Current language: auto; currently asm Hope someone can help me debug this... Thanks, Miguel