From owner-freebsd-hackers@FreeBSD.ORG Wed May 25 17:37:23 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 39FF91065674 for ; Wed, 25 May 2011 17:37:23 +0000 (UTC) (envelope-from aduane@juniper.net) Received: from exprod7og124.obsmtp.com (exprod7og124.obsmtp.com [64.18.2.26]) by mx1.freebsd.org (Postfix) with ESMTP id CDEAC8FC16 for ; Wed, 25 May 2011 17:37:21 +0000 (UTC) Received: from P-EMHUB01-HQ.jnpr.net ([66.129.224.36]) (using TLSv1) by exprod7ob124.postini.com ([64.18.6.12]) with SMTP ID DSNKTd0+Ud4xSPQMwsltIpfAdNHRb1rEsYPG@postini.com; Wed, 25 May 2011 10:37:22 PDT Received: from p-emfe02-wf.jnpr.net (172.28.145.25) by P-EMHUB01-HQ.jnpr.net (172.24.192.35) with Microsoft SMTP Server (TLS) id 8.2.254.0; Wed, 25 May 2011 10:20:50 -0700 Received: from EMBX01-WF.jnpr.net ([fe80::1914:3299:33d9:e43b]) by p-emfe02-wf.jnpr.net ([fe80::c126:c633:d2dc:8090%11]) with mapi; Wed, 25 May 2011 13:20:32 -0400 From: Andrew Duane To: Damien Fleuriot , "freebsd-hackers@freebsd.org" Date: Wed, 25 May 2011 13:20:31 -0400 Thread-Topic: DEBUG - analysing core dumps Thread-Index: Acwa/kwIf9affM/zTs+DD5wIW9BhEQAASQFg Message-ID: References: <4DDD3021.1000109@my.gd> In-Reply-To: <4DDD3021.1000109@my.gd> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Subject: RE: DEBUG - analysing core dumps X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 May 2011 17:37:23 -0000 Damien Fleuriot wrote: > Hello list, >=20 >=20 >=20 > We've got these boxes at work running FreeBSD 8.1-STABLE amd64 and > serving as firewalls and openvpn gateways. >=20 > We use CARP interfaces to provide an active-passive fault tolerant > system.=20 >=20 >=20 > Today, we received a nagios alert from the master box saying it's > rsyslogd process had crashed. >=20 > I logged on to it and tried to relaunch it, to no avail: > pid 2303 (rsyslogd), uid 0: exited on signal 11 (core dumped) >=20 >=20 >=20 >=20 > I would like advice on how to debug the output from the core dump. >=20 > This is what I get from gdb: >=20 > # gdb > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and > you are welcome to change it and/or distribute copies of it under > certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for > details. This GDB was configured as "amd64-marcel-freebsd". > (gdb) core rsyslogd.core > Core was generated by `rsyslogd'. > Program terminated with signal 11, Segmentation fault. > #0 0x00000000004258ec in ?? () >=20 >=20 > Sadly, getting a backtrace with "bt" gives me more lines with "??", > which is totally not helpful: > [SNIP] > #13 0x00007fffff1f9d70 in ?? () > #14 0x0000000000000000 in ?? () > #15 0x6f70732f7261762f in ?? () > #16 0x6c737973722f6c6f in ?? () > #17 0x5f6e70766f2f676f in ?? () > #18 0x746174732e676f6c in ?? () > #19 0x0000000000000065 in ?? () > #20 0x0000000000000000 in ?? () > [SNIP] >=20 > I am not sure what steps I should follow to get more information ? >=20 >=20 >=20 > Also, I believe that often, core dumps with signal 11 =3D RAM problems > and I would like a confirmation here. >=20 > I am concerned because rsyslogd is the only process that crashes in > this way, even after I rebooted the firewall. >=20 > Thanks for your input :) For what it's worth, the addresses shown in frames 15, 16, 17, and 18 are A= SCII: ops/rav/ lsysr/lo _npvo/go tats.gol /Andrew