Date: Fri, 30 Jun 2017 22:16:11 -0700 From: Mark Millard <markmi@dsl-only.net> To: "Michael W. Lucas" <mwlucas@michaelwlucas.com> Cc: hackers@freebsd.org Subject: Re: extract panic message & debugging from vmcore.0 ? Message-ID: <A88AB4DB-FF35-48E0-AD5C-1E627426DB88@dsl-only.net> In-Reply-To: <DC97F55F-7672-4429-9E3F-665CDA98D3F0@dsl-only.net> References: <20170630021205.GA43579@mail.michaelwlucas.com> <DC97F55F-7672-4429-9E3F-665CDA98D3F0@dsl-only.net>
next in thread | previous in thread | raw e-mail | index | archive | help
[Some more notes prompted by your reply.]
On 2017-Jun-29, at 7:35 PM, Mark Millard <markmi at dsl-only.net> wrote:
> On 2017-Jun-29, at 7:12 PM, Michael W. Lucas <mwlucas at =
michaelwlucas.com> wrote:
>=20
>>=20
>> # cd /usr/obj/usr/src/sys/GENERIC
>> # make gdbinit
>> # gdb kernel.debug
>=20
> Did you miss a /var/crash/vmcore.0 above?
I should have noted that gdbinit is likely
for setting up to use gdb remotely against
a live system for remote kernel debugging.
(Not something that I've ever done.)
It likely does nothing useful for vmcore.*
inspection. gdb itself does not deal with
vmcore.* files to my knowledge.
So this likely was instead a false direction
for your purposes.
> Separately. . .
>=20
> I'm no expert but you might need to pick
> between the default minidump vs. a full
> memory dump depending on purposes and what
> information is to be extracted:
>=20
> debug.minidump: 1
> vs.
> debug.minidump: 0
>=20
> in the sysctl debug.minidump output. I
> doubt that they are fully equivalent.
>=20
> Of course a full memory dump implies
> needing space ready to hold that full
> copy.
I'll also note that virtual addresses
vs. physical addresses are likely an
issue for at least debug.minidump=3D0 .
As for the crashinfo that you mention
in your reply: On head -320482 . . .
# grep -r crashinfo /etc
/etc/rc.d/savecore: if checkyesno crashinfo_enable; then
/etc/rc.d/savecore: ${crashinfo_program} -b -d =
${dumpdir}
/etc/defaults/rc.conf:crashinfo_enable=3D"YES" # Automatically generate =
crash dump summary.
/etc/defaults/rc.conf:crashinfo_program=3D"/usr/sbin/crashinfo" # Script =
to generate crash dump summary.
Normally the boot sequence produces a crashinfo report
after creating the vmcore.* file, at least if crashinfo
is enabled.
The context for the below is a powerpc with
debug.minidump=3D0 (disabled) and a production
kernel for the failures. (The debug kernel
does not get the problems involved for
head -r320482 .) vmcore.*'s, core.txt.*'s,
info.*'s and *.last's were all produced
towards the end of the boot sequence before
the login prompt each time.
# ls -lT /var/crash/
total 2685292
-rw-r--r-- 1 root wheel 2 Jun 29 23:17:52 2017 bounds
-rw------- 1 root wheel 0 Jun 29 22:47:14 2017 core.txt.5
-rw------- 1 root wheel 26573 Jun 29 23:18:51 2017 core.txt.6
-rw------- 1 root wheel 351 Jun 29 22:46:15 2017 info.5
-rw------- 1 root wheel 378 Jun 29 23:17:52 2017 info.6
lrwxr-xr-x 1 root wheel 0 Jun 29 22:47:06 2017 info.last ->=20=
-rw-r--r-- 1 root wheel 5 Feb 22 02:37:33 2016 minfree
-rw------- 1 root wheel 2147487744 Jun 29 22:47:06 2017 vmcore.5
-rw------- 1 root wheel 2147487744 Jun 29 23:18:43 2017 vmcore.6
lrwxr-xr-x 1 root wheel 0 Jun 29 22:47:06 2017 vmcore.last =
->=20
(There are oddities for the system that
have made the links and core.txt.5 empty.)
# more /var/crash/info.5
Dump header from device: /dev/label/FBSDG4Sswap
Architecture: powerpc
Architecture Version: 1
Dump Length: 2147487744
Blocksize: 512
Dumptime: Thu Jun 29 22:34:22 2017
Hostname: FBSDG4S
Magic: FreeBSD Kernel Dump
Version String: FreeBSD 12.0-CURRENT r320482M
Panic String:=20
Dump Parity: 4013015019
Bounds: 5
Dump Status: good
core.txt.6 attempted to have sections for:
ps -axlww
vmstat -s
vmstat -m
vmstat -z
vmstat -i
pstat -T
pstat -s
iostat
ipcs -a
ipcs -T
nfsstat
netstat -s
netstat -m
netstat -anA
netstat -aL
fstat
dmesg
kernel config
ddb capture buffer
(That does not mean that they all were
useful/accurate.)
I'll not list the text here.
Notes:
powerpc is not actually supported for kgdb
and the like and I had to change some things
to get this much working as well as it does
for me now. (It is limited but has been
useful to me.)
=3D=3D=3D
Mark Millard
markmi at dsl-only.net
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?A88AB4DB-FF35-48E0-AD5C-1E627426DB88>