From owner-freebsd-hackers@FreeBSD.ORG Wed May 25 17:06:46 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 56F4B106566B for ; Wed, 25 May 2011 17:06:46 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id E74488FC0A for ; Wed, 25 May 2011 17:06:43 +0000 (UTC) Received: by wwc33 with SMTP id 33so8379815wwc.31 for ; Wed, 25 May 2011 10:06:42 -0700 (PDT) Received: by 10.227.19.130 with SMTP id a2mr4840836wbb.8.1306341413113; Wed, 25 May 2011 09:36:53 -0700 (PDT) Received: from dfleuriot-at-hi-media.com ([83.167.62.196]) by mx.google.com with ESMTPS id b16sm480502wbh.24.2011.05.25.09.36.51 (version=SSLv3 cipher=OTHER); Wed, 25 May 2011 09:36:52 -0700 (PDT) Message-ID: <4DDD3021.1000109@my.gd> Date: Wed, 25 May 2011 18:36:49 +0200 From: Damien Fleuriot User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.17) Gecko/20110414 Thunderbird/3.1.10 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: DEBUG - analysing core dumps X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 May 2011 17:06:46 -0000 Hello list, We've got these boxes at work running FreeBSD 8.1-STABLE amd64 and serving as firewalls and openvpn gateways. We use CARP interfaces to provide an active-passive fault tolerant system. Today, we received a nagios alert from the master box saying it's rsyslogd process had crashed. I logged on to it and tried to relaunch it, to no avail: pid 2303 (rsyslogd), uid 0: exited on signal 11 (core dumped) I would like advice on how to debug the output from the core dump. This is what I get from gdb: # gdb GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd". (gdb) core rsyslogd.core Core was generated by `rsyslogd'. Program terminated with signal 11, Segmentation fault. #0 0x00000000004258ec in ?? () Sadly, getting a backtrace with "bt" gives me more lines with "??", which is totally not helpful: [SNIP] #13 0x00007fffff1f9d70 in ?? () #14 0x0000000000000000 in ?? () #15 0x6f70732f7261762f in ?? () #16 0x6c737973722f6c6f in ?? () #17 0x5f6e70766f2f676f in ?? () #18 0x746174732e676f6c in ?? () #19 0x0000000000000065 in ?? () #20 0x0000000000000000 in ?? () [SNIP] I am not sure what steps I should follow to get more information ? Also, I believe that often, core dumps with signal 11 = RAM problems and I would like a confirmation here. I am concerned because rsyslogd is the only process that crashes in this way, even after I rebooted the firewall. Thanks for your input :) -- dfl