Date: Wed, 14 Feb 2007 19:59:30 +0200 From: Kostik Belousov <kostikbel@gmail.com> To: Geoff Garside <geoff.garside@openhosting.co.uk> Cc: hackers@freebsd.org Subject: Re: FreeBSD 5.5 persistent crashing Message-ID: <20070214175930.GN25802@deviant.kiev.zoral.com.ua> In-Reply-To: <000301c75052$90d02950$4b00000a@Enki> References: <000301c75052$90d02950$4b00000a@Enki>
next in thread | previous in thread | raw e-mail | index | archive | help
--qYrsQHciA3Wqs7Iv
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Wed, Feb 14, 2007 at 04:09:54PM -0000, Geoff Garside wrote:
> Hi,
> I?m trying to get to the bottom of some issues we have been experiencing
> with a server of ours. We have so far tried replacing the memory in the
> server and we are still experiencing the crashes.
>=20
> If anyone has any ideas as to what could be causing this, or possible kgdb
> tricks to try.
>=20
> Server details
> * Dual Xeon 3GHz
> * 1GB DDR2 400MHz
> * 3ware 8006/2LP RAID
> * 2x 160GB SATA drives
>=20
> Uname Output
> # uname -a
> FreeBSD xxx 5.5-RELEASE-p11 FreeBSD 5.5-RELEASE-p11 #0: Sun Feb 11 17:08:=
57
> GMT 2007 geoff@xxx:/usr/obj/usr/src/sys/xxx i386
>=20
>=20
> Kernel Debugger output
> # kgdb kernel.debug /usr/crash/vmcore.7
> [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.s=
o:
> Undefined symbol "ps_pglobal_lookup"]
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you =
are
> welcome to change it and/or distribute copies of it under certain
> conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB. Type "show warranty" for detail=
s.
> This GDB was configured as "i386-marcel-freebsd".
>=20
> Unread portion of the kernel message buffer:
> Cannot access memory at address 0xc0c3c3a1
> (kgdb) where
> #0 doadump () at pcpu.h:160
> #1 0xc04e09f5 in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c=
:412
> #2 0xc04e0d19 in panic (fmt=3D0xc0623851 "%s") at
> /usr/src/sys/kern/kern_shutdown.c:568
> #3 0xc0601b14 in trap_fatal (frame=3D0xe7231740, eva=3D28) at
> /usr/src/sys/i386/i386/trap.c:822
> #4 0xc0601853 in trap_pfault (frame=3D0xe7231740, usermode=3D0, eva=3D28=
) at
> /usr/src/sys/i386/i386/trap.c:737
> #5 0xc06014ad in trap (frame=3D
> {tf_fs =3D -1068564456, tf_es =3D -1067319280, tf_ds =3D -106856446=
4, tf_edi
> =3D 4, tf_esi =3D 0, tf_ebp =3D -417130596, tf_isp =3D -417130644, tf_ebx=
=3D 131074,
> tf_edx =3D -1013448320, tf_ecx =3D 0, tf_eax =3D 4, tf_trapno =3D 12, tf_=
err =3D 2,
> tf_eip =3D -1068229475, tf_cs =3D 8, tf_eflags =3D 66118, tf_esp =3D 7, t=
f_ss =3D
> -417129328}) at /usr/src/sys/i386/i386/trap.c:427
> #6 0xc05ef8aa in calltrap () at /usr/src/sys/i386/i386/exception.s:140
> #7 0xc04f0018 in MD5Update (context=3D0x4, input=3D0x20002 <Address 0x20=
002 out
> of bounds>, inputLen=3D3281518976) at /usr/src/sys/kern/md5c.c:172
> #8 0xc049fc21 in procfs_doprocfile (td=3D0xc3980180, p=3D0xc4951a98,
> pn=3D0xc1fe7c00, sb=3D0xe72317f0, uio=3D0x0) at /usr/src/sys/fs/procfs/pr=
ocfs.c:73
> #9 0xc04a3e90 in pfs_readlink (va=3D0x0) at pcpu.h:157
> #10 0xc053cbb8 in kern_readlink (td=3D0xc3980180, path=3D0x0,
> pathseg=3DUIO_USERSPACE, buf=3D0x0, bufseg=3DUIO_USERSPACE, count=3D1024)=
at
> vnode_if.h:925
> #11 0xc053cade in readlink (td=3D0xc3980180, uap=3D0x0) at
> /usr/src/sys/kern/vfs_syscalls.c:2197
> #12 0xc0601e4f in syscall (frame=3D
> {tf_fs =3D 47, tf_es =3D 47, tf_ds =3D 47, tf_edi =3D 135512892, tf=
_esi =3D
> 135663632, tf_ebp =3D -1077940936, tf_isp =3D -417129116, tf_ebx =3D 6741=
01364,
> tf_edx =3D -1077941960, tf_ecx =3D 0, tf_eax =3D 58, tf_trapno =3D 135517=
392, tf_err
> =3D 2, tf_eip =3D 672575044, tf_cs =3D 31, tf_eflags =3D 647, tf_esp =3D =
-1077942020,
> tf_ss =3D 47}) at /usr/src/sys/i386/i386/trap.c:1014
> #13 0xc05ef8ff in Xint0x80_syscall () at
> /usr/src/sys/i386/i386/exception.s:201
> #14 0x0000002f in ?? ()
> #15 0x0000002f in ?? ()
> #16 0x0000002f in ?? ()
> #17 0x0813c33c in ?? ()
> #18 0x08161010 in ?? ()
> #19 0xbfbfed38 in ?? ()
> #20 0xe7231d64 in ?? ()
> #21 0x282df874 in ?? ()
> #22 0xbfbfe938 in ?? ()
> #23 0x00000000 in ?? ()
> #24 0x0000003a in ?? ()
> #25 0x0813d4d0 in ?? ()
> #26 0x00000002 in ?? ()
> #27 0x2816ae44 in ?? ()
> #28 0x0000001f in ?? ()
> #29 0x00000287 in ?? ()
> #30 0xbfbfe8fc in ?? ()
> #31 0x0000002f in ?? ()
> #32 0x00000000 in ?? ()
> #33 0x00000000 in ?? ()
> #34 0x00000000 in ?? ()
> #35 0x00000000 in ?? ()
> #36 0x0bf63000 in ?? ()
> #37 0xc3984a98 in ?? ()
> #38 0xc3980180 in ?? ()
> #39 0xe7231600 in ?? ()
> #40 0xe72315e8 in ?? ()
> #41 0xc1efc780 in ?? ()
> #42 0xc04f105f in sched_switch (td=3D0x8161010, newtd=3D0x282df874, flags=
=3DCannot
> access memory at address 0xbfbfed48
> ) at /usr/src/sys/kern/sched_4bsd.c:881
> Previous frame inner to this frame (corrupt stack?)
> (kgdb)
>=20
> Regards,
> Geoff Garside
You may try the following patch (this seems to be the issue I fixed recently
in HEAD and RELENG_6). On the other hand, I do not know right locking proto=
col
for RELENG_5.
Index: fs/procfs/procfs.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /usr/local/arch/ncvs/src/sys/fs/procfs/procfs.c,v
retrieving revision 1.11.2.1
diff -u -r1.11.2.1 procfs.c
--- fs/procfs/procfs.c 31 Jan 2005 23:25:58 -0000 1.11.2.1
+++ fs/procfs/procfs.c 14 Feb 2007 17:59:12 -0000
@@ -69,10 +69,12 @@
{
char *fullpath =3D "unknown";
char *freepath =3D NULL;
+ struct vnode *textvp;
=20
- vn_lock(p->p_textvp, LK_EXCLUSIVE | LK_RETRY, td);
- vn_fullpath(td, p->p_textvp, &fullpath, &freepath);
- VOP_UNLOCK(p->p_textvp, 0, td);
+ textvp =3D p->p_textvp;
+ vn_lock(textvp, LK_EXCLUSIVE | LK_RETRY, td);
+ vn_fullpath(td, textvp, &fullpath, &freepath);
+ VOP_UNLOCK(textvp, 0, td);
sbuf_printf(sb, "%s", fullpath);
if (freepath)
free(freepath, M_TEMP);
--qYrsQHciA3Wqs7Iv
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (FreeBSD)
iD8DBQFF004BC3+MBN1Mb4gRArR4AJ9R2zmSTNb9cjC11Jrr4margJKXmACfdBpw
90dk+dDWxAAzQa2I+Lq4D5I=
=KRDn
-----END PGP SIGNATURE-----
--qYrsQHciA3Wqs7Iv--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070214175930.GN25802>