From owner-freebsd-stable@FreeBSD.ORG Mon Aug 16 18:48:24 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3CBA9106564A for ; Mon, 16 Aug 2010 18:48:24 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 0EB4A8FC1D for ; Mon, 16 Aug 2010 18:48:22 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o7GImIIC009390 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 16 Aug 2010 21:48:18 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o7GImIOC020553; Mon, 16 Aug 2010 21:48:18 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o7GImIC4020551; Mon, 16 Aug 2010 21:48:18 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 16 Aug 2010 21:48:18 +0300 From: Kostik Belousov To: Alexey Tarasov Message-ID: <20100816184818.GS2396@deviant.kiev.zoral.com.ua> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Ian02MSQ7xDzAZO2" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-1.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_50, DNS_FROM_OPENWHOIS,URIBL_SBL autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-stable@freebsd.org Subject: Re: STABLE kernel panic: privileged instruction fault X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Aug 2010 18:48:24 -0000 --Ian02MSQ7xDzAZO2 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Aug 16, 2010 at 07:15:16PM +0400, Alexey Tarasov wrote: > Hello. >=20 > I have a couple of Supermicro servers which got the similar kernel panic = with all FreeBSD versions I tried since 6.4. > Now I want to investigate into the problem. > The servers get into panic with similar workload: file server with a lot = of files and connections. Web server software is nginx. File system is UFS+= GJOURNAL. Outgoing traffic on each server is ~10 MB/s. > I think it is not software problem, because when I've installed Linux wit= h such configuration there were no kernel panics. >=20 > Here is the short overview of the hardware: >=20 > CPU: Intel(R) Pentium(R) 4 CPU 3.00GHz (2992.51-MHz K8-class CPU) > Origin =3D "GenuineIntel" Id =3D 0xf65 Family =3D f Model =3D 6 Step= ping =3D 5 > Features=3D0xbfebfbff > Features2=3D0xe59d > AMD Features=3D0x20100800 > AMD Features2=3D0x1 > TSC: P-state invariant > real memory =3D 2147483648 (2048 MB) > avail memory =3D 2054619136 (1959 MB) >=20 > DMESG: http://lexasoft.ru/m/dmesg.txt >=20 > CORE: http://lexasoft.ru/m/core.txt >=20 > Fatal trap 1: privileged instruction fault while in kernel mode > cpuid =3D 1; apic id =3D 01 > instruction pointer =3D 0x20:0xffffff8040d2cc83 > stack pointer =3D 0x28:0xffffff8040d2ca80 > frame pointer =3D 0x28:0xffffff0060c0b740 > code segment =3D base 0x0, limit 0xfffff, type 0x1b > =3D DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags =3D interrupt enabled, resume, IOPL =3D 0 > current process =3D 9388 (nginx) > trap number =3D 1 > panic: privileged instruction fault > cpuid =3D 1 > Uptime: 17d15h48m49s > Physical memory: 2032 MB > Dumping 1485 MB: 1470 1454 1438 1422 1406 1390 1374 1358 1342 1326 1310 1= 294 1278 1262 1246 1230 1214 1198 1182 1166 1150 1134 1118 1102 1086 1070 1= 054 1038 1022 1006 990 974 958 942 926 910 894 878 862 846 830 814 798 782 = 766 750 734 718 702 686 670 654 638 622 606 590 574 558 542 526 510 494 478= 462 446 430 414 398 382 366 350 334 318 302 286 270 254 238 222 206 190 17= 4 158 142 126 110 94 78 62 46 30 14 >=20 >=20 > (kgdb) #0 doadump () at pcpu.h:223 > #1 0xffffffff80590c59 in boot (howto=3D260) > at /usr/src/sys/kern/kern_shutdown.c:416 > #2 0xffffffff8059108c in panic (fmt=3D0xffffffff80951fc4 "%s") > at /usr/src/sys/kern/kern_shutdown.c:579 > #3 0xffffffff80878fd8 in trap_fatal (frame=3D0xffffff0060c0b740, eva=3DV= ariable "eva" is not available. > ) > at /usr/src/sys/amd64/amd64/trap.c:857 > #4 0xffffffff808799ea in trap (frame=3D0xffffff8040d2c9d0) > at /usr/src/sys/amd64/amd64/trap.c:644 > #5 0xffffffff8085f983 in calltrap () > at /usr/src/sys/amd64/amd64/exception.S:224 > #6 0xffffff8040d2cc83 in ?? () > #7 0xffffff8040d2cb50 in ?? () > #8 0xffffff8040d2caf0 in ?? () > #9 0xffffff8040d2cbf0 in ?? () > #10 0xffffff0060c0b740 in ?? () > #11 0xffffffff80b83c60 in sysent () > #12 0xffffff8040d2cc80 in ?? () > #13 0xffffff8040d2cae0 in ?? () > #14 0xffffffff8059c431 in bintime (bt=3D0xffffffff80ad3140) > at /usr/src/sys/kern/kern_tc.c:200 > Previous frame inner to this frame (corrupt stack?) > (kgdb)=20 The backtrace make absolutely no sense. I would not trust kgdb anyway. Compile ddb in and do backtrace in console on the panic. Also, disassemble the kernel at the fault address. I am very curious which instruction causes this. This is stock GENERIC on the bare metal booted, right ? --Ian02MSQ7xDzAZO2 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkxph/EACgkQC3+MBN1Mb4iCtwCgkruZk3ZiAn21vgCC5UM4EduC tJYAoOML5tJ4Tz01wOkcv21fzT96AmXS =u7lT -----END PGP SIGNATURE----- --Ian02MSQ7xDzAZO2--