From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 14 19:32:42 2007 Return-Path: X-Original-To: hackers@freebsd.org Delivered-To: freebsd-hackers@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2903516A402 for ; Wed, 14 Feb 2007 19:32:42 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from gnome.kiev.sovam.com (gnome.kiev.sovam.com [212.109.32.24]) by mx1.freebsd.org (Postfix) with ESMTP id B62E213C494 for ; Wed, 14 Feb 2007 19:32:41 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from relay01.kiev.sovam.com ([62.64.120.200]) by gnome.kiev.sovam.com with esmtp (Exim 4.60) (envelope-from ) id 1HHPrs-000CDU-G0 for hackers@freebsd.org; Wed, 14 Feb 2007 21:32:40 +0200 Received: from [212.82.216.227] (helo=fw.zoral.com.ua) by relay01.kiev.sovam.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.60) (envelope-from ) id 1HHOQ5-0007pK-90 for hackers@freebsd.org; Wed, 14 Feb 2007 20:00:01 +0200 Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by fw.zoral.com.ua (8.13.4/8.13.4) with ESMTP id l1EHxWR5071728 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 14 Feb 2007 19:59:32 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.13.8/8.13.8) with ESMTP id l1EHxWCh065068; Wed, 14 Feb 2007 19:59:32 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.13.8/8.13.8/Submit) id l1EHxUOB064971; Wed, 14 Feb 2007 19:59:30 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 14 Feb 2007 19:59:30 +0200 From: Kostik Belousov To: Geoff Garside Message-ID: <20070214175930.GN25802@deviant.kiev.zoral.com.ua> References: <000301c75052$90d02950$4b00000a@Enki> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="qYrsQHciA3Wqs7Iv" Content-Disposition: inline In-Reply-To: <000301c75052$90d02950$4b00000a@Enki> User-Agent: Mutt/1.4.2.2i X-Virus-Scanned: ClamAV version 0.88.7, clamav-milter version 0.88.7 on fw.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-0.1 required=5.0 tests=ALL_TRUSTED,SPF_NEUTRAL autolearn=failed version=3.1.7 X-Spam-Checker-Version: SpamAssassin 3.1.7 (2006-10-05) on fw.zoral.com.ua X-Scanner-Signature: d5fca1c6a35a350575bd8b7885f5ad02 X-DrWeb-checked: yes X-SpamTest-Envelope-From: kostikbel@gmail.com X-SpamTest-Group-ID: 00000000 X-SpamTest-Info: Profiles 773 [Feb 14 2007] X-SpamTest-Info: helo_type=3 X-SpamTest-Info: {received from trusted relay: not dialup} X-SpamTest-Method: none X-SpamTest-Method: Local Lists X-SpamTest-Rate: 0 X-SpamTest-Status: Not detected X-SpamTest-Status-Extended: not_detected X-SpamTest-Version: SMTP-Filter Version 3.0.0 [0255], KAS30/Release X-Delayed: more then 1h on relay01.kiev.sovam.com Cc: hackers@freebsd.org Subject: Re: FreeBSD 5.5 persistent crashing X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Feb 2007 19:32:42 -0000 --qYrsQHciA3Wqs7Iv Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Feb 14, 2007 at 04:09:54PM -0000, Geoff Garside wrote: > Hi, > I?m trying to get to the bottom of some issues we have been experiencing > with a server of ours. We have so far tried replacing the memory in the > server and we are still experiencing the crashes. >=20 > If anyone has any ideas as to what could be causing this, or possible kgdb > tricks to try. >=20 > Server details > * Dual Xeon 3GHz > * 1GB DDR2 400MHz > * 3ware 8006/2LP RAID > * 2x 160GB SATA drives >=20 > Uname Output > # uname -a > FreeBSD xxx 5.5-RELEASE-p11 FreeBSD 5.5-RELEASE-p11 #0: Sun Feb 11 17:08:= 57 > GMT 2007 geoff@xxx:/usr/obj/usr/src/sys/xxx i386 >=20 >=20 > Kernel Debugger output > # kgdb kernel.debug /usr/crash/vmcore.7 > [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.s= o: > Undefined symbol "ps_pglobal_lookup"] > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you = are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for detail= s. > This GDB was configured as "i386-marcel-freebsd". >=20 > Unread portion of the kernel message buffer: > Cannot access memory at address 0xc0c3c3a1 > (kgdb) where > #0 doadump () at pcpu.h:160 > #1 0xc04e09f5 in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c= :412 > #2 0xc04e0d19 in panic (fmt=3D0xc0623851 "%s") at > /usr/src/sys/kern/kern_shutdown.c:568 > #3 0xc0601b14 in trap_fatal (frame=3D0xe7231740, eva=3D28) at > /usr/src/sys/i386/i386/trap.c:822 > #4 0xc0601853 in trap_pfault (frame=3D0xe7231740, usermode=3D0, eva=3D28= ) at > /usr/src/sys/i386/i386/trap.c:737 > #5 0xc06014ad in trap (frame=3D > {tf_fs =3D -1068564456, tf_es =3D -1067319280, tf_ds =3D -106856446= 4, tf_edi > =3D 4, tf_esi =3D 0, tf_ebp =3D -417130596, tf_isp =3D -417130644, tf_ebx= =3D 131074, > tf_edx =3D -1013448320, tf_ecx =3D 0, tf_eax =3D 4, tf_trapno =3D 12, tf_= err =3D 2, > tf_eip =3D -1068229475, tf_cs =3D 8, tf_eflags =3D 66118, tf_esp =3D 7, t= f_ss =3D > -417129328}) at /usr/src/sys/i386/i386/trap.c:427 > #6 0xc05ef8aa in calltrap () at /usr/src/sys/i386/i386/exception.s:140 > #7 0xc04f0018 in MD5Update (context=3D0x4, input=3D0x20002
of bounds>, inputLen=3D3281518976) at /usr/src/sys/kern/md5c.c:172 > #8 0xc049fc21 in procfs_doprocfile (td=3D0xc3980180, p=3D0xc4951a98, > pn=3D0xc1fe7c00, sb=3D0xe72317f0, uio=3D0x0) at /usr/src/sys/fs/procfs/pr= ocfs.c:73 > #9 0xc04a3e90 in pfs_readlink (va=3D0x0) at pcpu.h:157 > #10 0xc053cbb8 in kern_readlink (td=3D0xc3980180, path=3D0x0, > pathseg=3DUIO_USERSPACE, buf=3D0x0, bufseg=3DUIO_USERSPACE, count=3D1024)= at > vnode_if.h:925 > #11 0xc053cade in readlink (td=3D0xc3980180, uap=3D0x0) at > /usr/src/sys/kern/vfs_syscalls.c:2197 > #12 0xc0601e4f in syscall (frame=3D > {tf_fs =3D 47, tf_es =3D 47, tf_ds =3D 47, tf_edi =3D 135512892, tf= _esi =3D > 135663632, tf_ebp =3D -1077940936, tf_isp =3D -417129116, tf_ebx =3D 6741= 01364, > tf_edx =3D -1077941960, tf_ecx =3D 0, tf_eax =3D 58, tf_trapno =3D 135517= 392, tf_err > =3D 2, tf_eip =3D 672575044, tf_cs =3D 31, tf_eflags =3D 647, tf_esp =3D = -1077942020, > tf_ss =3D 47}) at /usr/src/sys/i386/i386/trap.c:1014 > #13 0xc05ef8ff in Xint0x80_syscall () at > /usr/src/sys/i386/i386/exception.s:201 > #14 0x0000002f in ?? () > #15 0x0000002f in ?? () > #16 0x0000002f in ?? () > #17 0x0813c33c in ?? () > #18 0x08161010 in ?? () > #19 0xbfbfed38 in ?? () > #20 0xe7231d64 in ?? () > #21 0x282df874 in ?? () > #22 0xbfbfe938 in ?? () > #23 0x00000000 in ?? () > #24 0x0000003a in ?? () > #25 0x0813d4d0 in ?? () > #26 0x00000002 in ?? () > #27 0x2816ae44 in ?? () > #28 0x0000001f in ?? () > #29 0x00000287 in ?? () > #30 0xbfbfe8fc in ?? () > #31 0x0000002f in ?? () > #32 0x00000000 in ?? () > #33 0x00000000 in ?? () > #34 0x00000000 in ?? () > #35 0x00000000 in ?? () > #36 0x0bf63000 in ?? () > #37 0xc3984a98 in ?? () > #38 0xc3980180 in ?? () > #39 0xe7231600 in ?? () > #40 0xe72315e8 in ?? () > #41 0xc1efc780 in ?? () > #42 0xc04f105f in sched_switch (td=3D0x8161010, newtd=3D0x282df874, flags= =3DCannot > access memory at address 0xbfbfed48 > ) at /usr/src/sys/kern/sched_4bsd.c:881 > Previous frame inner to this frame (corrupt stack?) > (kgdb) >=20 > Regards, > Geoff Garside You may try the following patch (this seems to be the issue I fixed recently in HEAD and RELENG_6). On the other hand, I do not know right locking proto= col for RELENG_5. Index: fs/procfs/procfs.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /usr/local/arch/ncvs/src/sys/fs/procfs/procfs.c,v retrieving revision 1.11.2.1 diff -u -r1.11.2.1 procfs.c --- fs/procfs/procfs.c 31 Jan 2005 23:25:58 -0000 1.11.2.1 +++ fs/procfs/procfs.c 14 Feb 2007 17:59:12 -0000 @@ -69,10 +69,12 @@ { char *fullpath =3D "unknown"; char *freepath =3D NULL; + struct vnode *textvp; =20 - vn_lock(p->p_textvp, LK_EXCLUSIVE | LK_RETRY, td); - vn_fullpath(td, p->p_textvp, &fullpath, &freepath); - VOP_UNLOCK(p->p_textvp, 0, td); + textvp =3D p->p_textvp; + vn_lock(textvp, LK_EXCLUSIVE | LK_RETRY, td); + vn_fullpath(td, textvp, &fullpath, &freepath); + VOP_UNLOCK(textvp, 0, td); sbuf_printf(sb, "%s", fullpath); if (freepath) free(freepath, M_TEMP); --qYrsQHciA3Wqs7Iv Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFF004BC3+MBN1Mb4gRArR4AJ9R2zmSTNb9cjC11Jrr4margJKXmACfdBpw 90dk+dDWxAAzQa2I+Lq4D5I= =KRDn -----END PGP SIGNATURE----- --qYrsQHciA3Wqs7Iv--