Date: Mon, 22 Aug 2016 16:54:22 +0100 From: Matthew Seaman <matthew@FreeBSD.org> To: freebsd-hackers@freebsd.org Subject: How do I debug this? Message-ID: <a6b6b604-80ad-926a-5d85-871e9dd7ebbe@freebsd.org>
next in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --Me6iBDatIGq3QgwvnhRJpKmV5vCBhNWCN Content-Type: multipart/mixed; boundary="NhVCCx3Kx3lcXvukKeO7a5CEGcqkHe8si" From: Matthew Seaman <matthew@freebsd.org> To: freebsd-hackers@freebsd.org Message-ID: <a6b6b604-80ad-926a-5d85-871e9dd7ebbe@freebsd.org> Subject: How do I debug this? --NhVCCx3Kx3lcXvukKeO7a5CEGcqkHe8si Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable We've got an intermittent crash bug with postgresql92-server-9.2.17_2 which seems to be occurring in process startup: # gdb -c /usr/local/pgsql/data/postgres.core /usr/local/bin/postgres GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you = are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for detail= s. This GDB was configured as "amd64-marcel-freebsd"... Core was generated by `postgres'. Program terminated with signal 11, Segmentation fault. Reading symbols from /lib/libthr.so.3...done. Loaded symbols for /lib/libthr.so.3 Reading symbols from /usr/local/lib/libxml2.so.2...done. Loaded symbols for /usr/local/lib/libxml2.so.2 Reading symbols from /usr/local/lib/libicuuc.so.55...done. Loaded symbols for /usr/local/lib/libicuuc.so.55 Reading symbols from /usr/local/lib/libicui18n.so.55...done. Loaded symbols for /usr/local/lib/libicui18n.so.55 Reading symbols from /usr/local/lib/libssl.so.8...done. Loaded symbols for /usr/local/lib/libssl.so.8 Reading symbols from /usr/local/lib/libcrypto.so.8...done. Loaded symbols for /usr/local/lib/libcrypto.so.8 Reading symbols from /lib/libm.so.5...done. Loaded symbols for /lib/libm.so.5 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /lib/libz.so.6...done. Loaded symbols for /lib/libz.so.6 Reading symbols from /usr/lib/liblzma.so.5...done. Loaded symbols for /usr/lib/liblzma.so.5 Reading symbols from /usr/local/lib/libicudata.so.55...done. Loaded symbols for /usr/local/lib/libicudata.so.55 Reading symbols from /usr/lib/libc++.so.1...done. Loaded symbols for /usr/lib/libc++.so.1 Reading symbols from /lib/libcxxrt.so.1...done. Loaded symbols for /lib/libcxxrt.so.1 Reading symbols from /lib/libgcc_s.so.1...done. Loaded symbols for /lib/libgcc_s.so.1 Reading symbols from /usr/local/lib/postgresql/auto_explain.so...done. Loaded symbols for /usr/local/lib/postgresql/auto_explain.so Reading symbols from /usr/local/lib/postgresql/plpgsql.so...done. Loaded symbols for /usr/local/lib/postgresql/plpgsql.so Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x0000000800d1cd0a in _r_debug_postinit () from /libexec/ld-elf.so.1 [New Thread 805806400 (LWP 100362/)] (gdb) bt full #0 0x0000000800d1cd0a in _r_debug_postinit () from /libexec/ld-elf.so.1 No symbol table info available. #1 0x0000000800d1a46d in .text () from /libexec/ld-elf.so.1 No symbol table info available. #2 0x0000000804ffe174 in ?? () from /lib/libgcc_s.so.1 No symbol table info available. #3 0x00007fffffffe410 in ?? () No symbol table info available. #4 0x00000008050078f1 in ?? () from /lib/libgcc_s.so.1 No symbol table info available. #5 0x00007fffffffe410 in ?? () No symbol table info available. #6 0x0000000800d1d9a8 in dlclose () from /libexec/ld-elf.so.1 No symbol table info available. Previous frame inner to this frame (corrupt stack?) Application is: postgresql92-server-9.2.17_2 On: % freebsd-version -ku 10.3-RELEASE-p4 10.3-RELEASE-p5 This is apparently a postgres child process crashing on startup somewhere before where main() gets called. Doesn't happen often -- only when the server is reasonably busy and then it will take somewhere between a day and a week between incidents. We can't correlate it with any particular SQL being run. This was a production server so we've had to move the customers assigned to it back to some slightly older but very hardware but running the same version of postgresql (same package compiled on our own pkgrepo) and same 10.3-RELEASE-p4/p5 -- where they are running quite happily. Matthew --NhVCCx3Kx3lcXvukKeO7a5CEGcqkHe8si-- --Me6iBDatIGq3QgwvnhRJpKmV5vCBhNWCN Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJXuyA0AAoJEABRPxDgqeTnNGoP/0b+v0jyRHvUGKn6tPtlL41i k+xOuyqt/qqyUrGUJguHwOYh60HvxeWw6bWWd7xTJHZiPA2YTGja876orBiZvqPF OQ29YBD5TCvh8MPYBotAPiiHjmqLXXoLm/JyLmRsq2ExlECmAb2Pf6ybbHgZ55yT WwH5TKCiFSFRGCIyQRsKlj2Eq2rIV66PWh8f5PmgZVCPdjdFJwPWqH2D3X5Zcd0G HZJW+2T3G1ijrayNKC6uQ5Fil9YaE0InjjBnxtyj+D1f0r+0gwuyR25kMF2wSJMd mTu/VisReEkof8pCaRYtlNsge5cgMX3nlaDrT/Ww12iY+L8MiTC1NiOZYovO4jcu oMehegoafD+F7xnbtghAjpzDqPR2LZPYVZRYItiUXBkGfoXNU+fC6OO+0f/Ge1zi OxZgWnIYZjrxMKW5GbJK4NmLxHQckkphm9zhP1Ej4HoffwIi9szjTljxVGu4ot2A s0It4DqvsQu3VNBwKtdXdXMDaNTP5h7B01GaMGX7j+Q7TjkJFn+Di/FNylPiPH9h 1PPfNFdDfPpxJXMojey/g7Mlc5uGzTeB1cEC/FGXRll7ktJcLn8H8FA1mPaLjMRg 0375+QXrzMCeEGzUjr86ag/+ncS5c2p1lWs1yPoQGtyizAh6Ypo7h92qvLbSd5mW 9CvXwqiAoVzfRm7upa4B =eNzk -----END PGP SIGNATURE----- --Me6iBDatIGq3QgwvnhRJpKmV5vCBhNWCN--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?a6b6b604-80ad-926a-5d85-871e9dd7ebbe>