From owner-freebsd-stable@FreeBSD.ORG Wed Dec 13 07:04:04 2006 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5CFEA16A403 for ; Wed, 13 Dec 2006 07:04:04 +0000 (UTC) (envelope-from tarkhil@webmail.sub.ru) Received: from mail.sub.ru (host-205-2.netflow.ru [88.212.205.2]) by mx1.FreeBSD.org (Postfix) with SMTP id 1BC6E43CD9 for ; Wed, 13 Dec 2006 07:02:35 +0000 (GMT) (envelope-from tarkhil@webmail.sub.ru) Received: (qmail 14310 invoked by uid 0); 13 Dec 2006 10:08:18 +0300 Received: from unknown (HELO ?85.192.19.9?) (tarkhil%sub.ru@85.192.19.9) by techno.sub.ru with SMTP; 13 Dec 2006 07:08:18 -0000 Message-ID: <457FA5D7.9060502@webmail.sub.ru> Date: Wed, 13 Dec 2006 10:03:51 +0300 From: Alex Povolotsky User-Agent: Thunderbird 1.5.0.8 (X11/20061128) MIME-Version: 1.0 To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 7bit Subject: Strange panics X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Dec 2006 07:04:04 -0000 Hello! I'm getting lots of panics on quite lightly loaded box under 6.1-RELEASE. The box runned two jails without any flaws for several weeks, after adding third one it begins to panics at least once a day. I have no idea on what could cause this. Here is my dmesg.boot (partially, I can send it all) Copyright (c) 1992-2006 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 6.1-RELEASE #0: Tue Dec 12 08:44:28 MSK 2006 tarkhil@box2s.sub.ru:/usr/obj/usr/src/sys/SMP Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(TM) CPU 2.80GHz (2793.01-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf43 Stepping = 3 Features=0xbfebfbff Features2=0x641d> AMD Features=0x20100000 Logical CPUs per core: 2 real memory = 2147352576 (2047 MB) avail memory = 2096328704 (1999 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 6 cpu3 (AP): APIC ID: 7 and here is kgbd output Fatal trap 12: page fault while in kernel mode cpuid = 2; apic id = 06 fault virtual address = 0xac fault code = supervisor write, page not present instruction pointer = 0x20:0xc06ee9d0 stack pointer = 0x28:0xe91e2b18 frame pointer = 0x28:0xe91e2b34 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 16260 (tcpserver) trap number = 12 panic: page fault cpuid = 2 Uptime: 19h33m1s (kgdb) bt #0 doadump () at pcpu.h:165 #1 0xc06507c9 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:402 #2 0xc0650af1 in panic (fmt=0xc08958ae "%s") at /usr/src/sys/kern/kern_shutdown.c:558 #3 0xc084a2cc in trap_fatal (frame=0xe91e2ad8, eva=172) at /usr/src/sys/i386/i386/trap.c:836 #4 0xc084a00b in trap_pfault (frame=0xe91e2ad8, usermode=0, eva=172) at /usr/src/sys/i386/i386/trap.c:744 #5 0xc0849c45 in trap (frame= {tf_fs = -1065156600, tf_es = -1056636888, tf_ds = 40, tf_edi = 55, tf_esi = 0, tf_ebp = -383898828, tf_isp = -383898876, tf_ebx = -383898480, tf_edx = -944133120, tf_ecx = 0, tf_eax = 4, tf_trapno = 12, tf_err = 2, tf_eip = -1066473008, tf_cs = 32, tf_eflags = 66182, tf_esp = -880992256, tf_ss = 0}) at /usr/src/sys/i386/i386/trap.c:434 #6 0xc0836c4a in calltrap () at /usr/src/sys/i386/i386/exception.s:139 #7 0xc06ee9d0 in ip_ctloutput (so=0x4, sopt=0xe91e2c90) at atomic.h:146 #8 0xc06feb73 in tcp_ctloutput (so=0xc811b6f4, sopt=0xe91e2c90) at /usr/src/sys/netinet/tcp_usrreq.c:1038 #9 0xc068bdf8 in sosetopt (so=0xc811b6f4, sopt=0xe91e2c90) at /usr/src/sys/kern/uipc_socket.c:1560 #10 0xc0691115 in kern_setsockopt (td=0xc7b9ac00, s=0, level=4, name=4, val=0xc7b9ac00, valseg=UIO_USERSPACE, valsize=0) at /usr/src/sys/kern/uipc_syscalls.c:1351 #11 0xc0691046 in setsockopt (td=0xc7b9ac00, uap=0x4) at /usr/src/sys/kern/uipc_syscalls.c:1307 #12 0xc084a613 in syscall (frame= {tf_fs = -1078001605, tf_es = 59, tf_ds = -1078001605, tf_edi = -1077942204, tf_esi = 3, tf_ebp = -1077942408, tf_isp = -383898268, tf_ebx = 0, tf_edx = 2, tf_ecx = 134545464, tf_eax = 105, tf_trapno = 12, tf_err = 2, tf_eip = 186394799, tf_cs = 51, tf_eflags = 642, tf_esp = -1077942452, tf_ss = 59}) at /usr/src/sys/i386/i386/trap.c:981 #13 0xc0836c9f in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:200 #14 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) === cut === Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0xac fault code = supervisor write, page not present instruction pointer = 0x20:0xc06ee9d0 stack pointer = 0x28:0xe8f77b18 frame pointer = 0x28:0xe8f77b34 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 86881 (tcpserver) trap number = 12 panic: page fault cpuid = 0 Uptime: 1h55m49s Dumping 2047 MB (2 chunks) chunk 0: 1MB (158 pages) ... ok chunk 1: 2047MB (524000 pages) 2031 2015 1999 1983 1967 1951 1935 1919 1903 1887 1871 1855 1839 1823 1807 1791 1775 1759 1743 1727 1711 1695 1679 1663 1647 1631 1615 1599 1583 1567 1551 1535 1519 1503 1487 1471 1455 1439 1423 1407 1391 1375 1359 1343 1327 1311 1295 1279 1263 1247 1231 1215 1199 1183 1167 1151 1135 1119 1103 1087 1071 1055 1039 1023 1007 991 975 959 943 927 911 895 879 863 847 831 815 799 783 767 751 735 719 703 687 671 655 639 623 607 591 575 559 543 527 511 495 479 463 447 431 415 399 383 367 351 335 319 303 287 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15 #0 doadump () at pcpu.h:165 165 pcpu.h: No such file or directory. in pcpu.h (kgdb) bt #0 doadump () at pcpu.h:165 #1 0xc06507c9 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:402 #2 0xc0650af1 in panic (fmt=0xc08958ae "%s") at /usr/src/sys/kern/kern_shutdown.c:558 #3 0xc084a2cc in trap_fatal (frame=0xe8f77ad8, eva=172) at /usr/src/sys/i386/i386/trap.c:836 #4 0xc084a00b in trap_pfault (frame=0xe8f77ad8, usermode=0, eva=172) at /usr/src/sys/i386/i386/trap.c:744 #5 0xc0849c45 in trap (frame= {tf_fs = -939130872, tf_es = -1056636888, tf_ds = 40, tf_edi = 55, tf_esi = 0, tf_ebp = -386434252, tf_isp = -386434300, tf_ebx = -386433904, tf_edx = -951618816, tf_ecx = 0, tf_eax = 4, tf_trapno = 12, tf_err = 2, tf_eip = -1066473008, tf_cs = 32, tf_eflags = 66182, tf_esp = -953071872, tf_ss = 0}) at /usr/src/sys/i386/i386/trap.c:434 #6 0xc0836c4a in calltrap () at /usr/src/sys/i386/i386/exception.s:139 #7 0xc06ee9d0 in ip_ctloutput (so=0x4, sopt=0xe8f77c90) at atomic.h:146 #8 0xc06feb73 in tcp_ctloutput (so=0xc8c0142c, sopt=0xe8f77c90) at /usr/src/sys/netinet/tcp_usrreq.c:1038 #9 0xc068bdf8 in sosetopt (so=0xc8c0142c, sopt=0xe8f77c90) at /usr/src/sys/kern/uipc_socket.c:1560 #10 0xc0691115 in kern_setsockopt (td=0xc7477300, s=0, level=4, name=4, val=0xc7477300, valseg=UIO_USERSPACE, valsize=0) at /usr/src/sys/kern/uipc_syscalls.c:1351 #11 0xc0691046 in setsockopt (td=0xc7477300, uap=0x4) at /usr/src/sys/kern/uipc_syscalls.c:1307 #12 0xc084a613 in syscall (frame= {tf_fs = -1078001605, tf_es = 59, tf_ds = -1078001605, tf_edi = -1077942088, tf_esi = 3, tf_ebp = -1077942296, tf_isp = -386433692, tf_ebx = 0, tf_edx = 2, tf_ecx = 134545464, tf_eax = 105, tf_trapno = 12, tf_err = 2, tf_eip = 186394799, tf_cs = 51, tf_eflags = 646, tf_esp = -1077942340, tf_ss = 59}) at /usr/src/sys/i386/i386/trap.c:981 #13 0xc0836c9f in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:200 #14 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) === cut === NIC is Intel (em). Can anyone provide any help with this? Alex.