From owner-freebsd-current@freebsd.org Wed May 8 20:52:47 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C3C5C159373D for ; Wed, 8 May 2019 20:52:46 +0000 (UTC) (envelope-from ler@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 673468E586 for ; Wed, 8 May 2019 20:52:46 +0000 (UTC) (envelope-from ler@FreeBSD.org) Received: from ler-imac.local (unknown [IPv6:2600:1700:210:b180:2d2f:26c8:8dbb:59f7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: ler/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 1CB96B746 for ; Wed, 8 May 2019 20:52:46 +0000 (UTC) (envelope-from ler@FreeBSD.org) Date: Wed, 8 May 2019 15:52:45 -0500 From: Larry Rosenman To: freebsd-current@FreeBSD.org Subject: Crash loading dtraceall Message-ID: <20190508205245.ulbo6fusk3b4py7t@ler-imac.local> Mail-Followup-To: freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="swragf7kp7z3ilk7" Content-Disposition: inline User-Agent: NeoMutt/20180716 X-Rspamd-Queue-Id: 673468E586 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.99 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.99)[-0.988,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 May 2019 20:52:47 -0000 --swragf7kp7z3ilk7 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Greetings, Somewhere between r346483 and r347241 loading dtraceall causes a crash. I have the cores and kernels.=20 It's hard for me to bisect more than this, as the box is remote.=20 What more do you need? (this dump is fropm r347355).=20 [ler@oldtbh2.lerctr.org:/var/crash] $ more core.txt.5 oldtbh2.lerctr.org dumped core - see /var/crash/vmcore.5 Wed May 8 15:46:58 CDT 2019 FreeBSD oldtbh2.lerctr.org 13.0-CURRENT FreeBSD 13.0-CURRENT r347355 LER-MI= NIMAL amd64 panic: page fault GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain condition= s. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid =3D 3; apic id =3D 03 fault virtual address =3D 0x0 fault code =3D supervisor read data , page not present instruction pointer =3D 0x20:0xffffffff825cb5ea stack pointer =3D 0x28:0xfffffe00d74d7d90 frame pointer =3D 0x28:0xfffffe00d74d7df0 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 1531 (kldload) trap number =3D 12 panic: page fault cpuid =3D 3 time =3D 1557348131 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00d74d7= a40 vpanic() at vpanic+0x19d/frame 0xfffffe00d74d7a90 panic() at panic+0x43/frame 0xfffffe00d74d7af0 trap_fatal() at trap_fatal+0x394/frame 0xfffffe00d74d7b50 trap_pfault() at trap_pfault+0x49/frame 0xfffffe00d74d7bb0 trap() at trap+0x2b4/frame 0xfffffe00d74d7cc0 calltrap() at calltrap+0x8/frame 0xfffffe00d74d7cc0 --- trap 0xc, rip =3D 0xffffffff825cb5ea, rsp =3D 0xfffffe00d74d7d90, rbp = =3D 0xfffffe00d74d7df0 --- fbt_provide_module_function() at fbt_provide_module_function+0x7a/frame 0xf= ffffe00d74d7df0 link_elf_each_function_nameval() at link_elf_each_function_nameval+0xf7/fra= me 0xfffffe00d74d7e40 fbt_provide_module() at fbt_provide_module+0xde/frame 0xfffffe00d74d8270 fbt_linker_file_cb() at fbt_linker_file_cb+0x12/frame 0xfffffe00d74d8280 linker_file_foreach() at linker_file_foreach+0x52/frame 0xfffffe00d74d82b0 linker_load_module() at linker_load_module+0xbd8/frame 0xfffffe00d74d85e0 linker_load_dependencies() at linker_load_dependencies+0x2fd/frame 0xfffffe= 00d74d8630 link_elf_load_file() at link_elf_load_file+0x105e/frame 0xfffffe00d74d86f0 linker_load_module() at linker_load_module+0x9ef/frame 0xfffffe00d74d8a20 kern_kldload() at kern_kldload+0xa7/frame 0xfffffe00d74d8a60 sys_kldload() at sys_kldload+0x5b/frame 0xfffffe00d74d8a90 amd64_syscall() at amd64_syscall+0x25c/frame 0xfffffe00d74d8bb0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00d74d8bb0 --- syscall (304, FreeBSD ELF64, sys_kldload), rip =3D 0x8002de43a, rsp =3D= 0x7fffffffe658, rbp =3D 0x7fffffffebd0 --- Uptime: 2m40s Dumping 2258 out of 64482 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..= 91% Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /usr/lib/de= bug//boot/kernel/zfs.ko.debug...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/krpc.ko...Reading symbols from /usr/lib/d= ebug//boot/kernel/krpc.ko.debug...done. done. Loaded symbols for /boot/kernel/krpc.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /us= r/lib/debug//boot/kernel/opensolaris.ko.debug...done. done. Loaded symbols for /boot/kernel/opensolaris.ko Reading symbols from /boot/kernel/if_em.ko...Reading symbols from /usr/lib/= debug//boot/kernel/if_em.ko.debug...done. done. Loaded symbols for /boot/kernel/if_em.ko Reading symbols from /boot/kernel/iflib.ko...Reading symbols from /usr/lib/= debug//boot/kernel/iflib.ko.debug...done. done. Loaded symbols for /boot/kernel/iflib.ko Reading symbols from /boot/kernel/if_bce.ko...Reading symbols from /usr/lib= /debug//boot/kernel/if_bce.ko.debug...done. done. Loaded symbols for /boot/kernel/if_bce.ko Reading symbols from /boot/kernel/miibus.ko...Reading symbols from /usr/lib= /debug//boot/kernel/miibus.ko.debug...done. done. Loaded symbols for /boot/kernel/miibus.ko Reading symbols from /boot/kernel/coretemp.ko...Reading symbols from /usr/l= ib/debug//boot/kernel/coretemp.ko.debug...done. done. Loaded symbols for /boot/kernel/coretemp.ko Reading symbols from /boot/kernel/filemon.ko...Reading symbols from /usr/li= b/debug//boot/kernel/filemon.ko.debug...done. done. Loaded symbols for /boot/kernel/filemon.ko Reading symbols from /boot/kernel/fuse.ko...Reading symbols from /usr/lib/d= ebug//boot/kernel/fusefs.ko.debug...done. done. Loaded symbols for /boot/kernel/fuse.ko Reading symbols from /boot/kernel/linux.ko...Reading symbols from /usr/lib/= debug//boot/kernel/linux.ko.debug...done. done. Loaded symbols for /boot/kernel/linux.ko Reading symbols from /boot/kernel/linux_common.ko...Reading symbols from /u= sr/lib/debug//boot/kernel/linux_common.ko.debug...done. done. Loaded symbols for /boot/kernel/linux_common.ko Reading symbols from /boot/kernel/ichsmb.ko...Reading symbols from /usr/lib= /debug//boot/kernel/ichsmb.ko.debug...done. done. Loaded symbols for /boot/kernel/ichsmb.ko Reading symbols from /boot/kernel/smbus.ko...Reading symbols from /usr/lib/= debug//boot/kernel/smbus.ko.debug...done. done. Loaded symbols for /boot/kernel/smbus.ko Reading symbols from /boot/kernel/ichwd.ko...Reading symbols from /usr/lib/= debug//boot/kernel/ichwd.ko.debug...done. done. Loaded symbols for /boot/kernel/ichwd.ko Reading symbols from /boot/kernel/cpuctl.ko...Reading symbols from /usr/lib= /debug//boot/kernel/cpuctl.ko.debug...done. done. Loaded symbols for /boot/kernel/cpuctl.ko Reading symbols from /boot/kernel/crypto.ko...Reading symbols from /usr/lib= /debug//boot/kernel/crypto.ko.debug...done. done. Loaded symbols for /boot/kernel/crypto.ko Reading symbols from /boot/kernel/cryptodev.ko...Reading symbols from /usr/= lib/debug//boot/kernel/cryptodev.ko.debug...done. done. Loaded symbols for /boot/kernel/cryptodev.ko Reading symbols from /boot/kernel/ipmi.ko...Reading symbols from /usr/lib/d= ebug//boot/kernel/ipmi.ko.debug...done. done. Loaded symbols for /boot/kernel/ipmi.ko Reading symbols from /boot/kernel/hwpmc.ko...Reading symbols from /usr/lib/= debug//boot/kernel/hwpmc.ko.debug...done. done. Loaded symbols for /boot/kernel/hwpmc.ko Reading symbols from /boot/kernel/tcp_rack.ko...Reading symbols from /usr/l= ib/debug//boot/kernel/tcp_rack.ko.debug...done. done. Loaded symbols for /boot/kernel/tcp_rack.ko Reading symbols from /boot/kernel/mfip.ko...Reading symbols from /usr/lib/d= ebug//boot/kernel/mfip.ko.debug...done. done. Loaded symbols for /boot/kernel/mfip.ko Reading symbols from /boot/kernel/cam.ko...Reading symbols from /usr/lib/de= bug//boot/kernel/cam.ko.debug...done. done. Loaded symbols for /boot/kernel/cam.ko Reading symbols from /boot/kernel/usb.ko...Reading symbols from /usr/lib/de= bug//boot/kernel/usb.ko.debug...done. done. Loaded symbols for /boot/kernel/usb.ko Reading symbols from /boot/kernel/uhci.ko...Reading symbols from /usr/lib/d= ebug//boot/kernel/uhci.ko.debug...done. done. Loaded symbols for /boot/kernel/uhci.ko Reading symbols from /boot/kernel/xhci.ko...Reading symbols from /usr/lib/d= ebug//boot/kernel/xhci.ko.debug...done. done. Loaded symbols for /boot/kernel/xhci.ko Reading symbols from /boot/kernel/ehci.ko...Reading symbols from /usr/lib/d= ebug//boot/kernel/ehci.ko.debug...done. done. Loaded symbols for /boot/kernel/ehci.ko Reading symbols from /boot/kernel/ohci.ko...Reading symbols from /usr/lib/d= ebug//boot/kernel/ohci.ko.debug...done. done. Loaded symbols for /boot/kernel/ohci.ko Reading symbols from /boot/kernel/ukbd.ko...Reading symbols from /usr/lib/d= ebug//boot/kernel/ukbd.ko.debug...done. done. Loaded symbols for /boot/kernel/ukbd.ko Reading symbols from /boot/kernel/ums.ko...Reading symbols from /usr/lib/de= bug//boot/kernel/ums.ko.debug...done. done. Loaded symbols for /boot/kernel/ums.ko Reading symbols from /boot/kernel/usb_quirk.ko...Reading symbols from /usr/= lib/debug//boot/kernel/usb_quirk.ko.debug...done. done. Loaded symbols for /boot/kernel/usb_quirk.ko Reading symbols from /boot/kernel/usb_template.ko...Reading symbols from /u= sr/lib/debug//boot/kernel/usb_template.ko.debug...done. done. Loaded symbols for /boot/kernel/usb_template.ko Reading symbols from /boot/kernel/efirt.ko...Reading symbols from /usr/lib/= debug//boot/kernel/efirt.ko.debug...done. done. Loaded symbols for /boot/kernel/efirt.ko Reading symbols from /boot/kernel/nfscl.ko...Reading symbols from /usr/lib/= debug//boot/kernel/nfscl.ko.debug...done. done. Loaded symbols for /boot/kernel/nfscl.ko Reading symbols from /boot/kernel/nfscommon.ko...Reading symbols from /usr/= lib/debug//boot/kernel/nfscommon.ko.debug...done. done. Loaded symbols for /boot/kernel/nfscommon.ko Reading symbols from /boot/kernel/nfssvc.ko...Reading symbols from /usr/lib= /debug//boot/kernel/nfssvc.ko.debug...done. done. Loaded symbols for /boot/kernel/nfssvc.ko Reading symbols from /boot/kernel/nfslock.ko...Reading symbols from /usr/li= b/debug//boot/kernel/nfslock.ko.debug...done. done. Loaded symbols for /boot/kernel/nfslock.ko Reading symbols from /boot/kernel/nfsd.ko...Reading symbols from /usr/lib/d= ebug//boot/kernel/nfsd.ko.debug...done. done. Loaded symbols for /boot/kernel/nfsd.ko Reading symbols from /boot/kernel/nfslockd.ko...Reading symbols from /usr/l= ib/debug//boot/kernel/nfslockd.ko.debug...done. done. Loaded symbols for /boot/kernel/nfslockd.ko Reading symbols from /boot/kernel/ataintel.ko...Reading symbols from /usr/l= ib/debug//boot/kernel/ataintel.ko.debug...done. done. Loaded symbols for /boot/kernel/ataintel.ko Reading symbols from /boot/kernel/ata.ko...Reading symbols from /usr/lib/de= bug//boot/kernel/ata.ko.debug...done. done. Loaded symbols for /boot/kernel/ata.ko Reading symbols from /boot/kernel/atapci.ko...Reading symbols from /usr/lib= /debug//boot/kernel/atapci.ko.debug...done. done. Loaded symbols for /boot/kernel/atapci.ko Reading symbols from /boot/kernel/geom_label.ko...Reading symbols from /usr= /lib/debug//boot/kernel/geom_label.ko.debug...done. done. Loaded symbols for /boot/kernel/geom_label.ko Reading symbols from /boot/kernel/linux64.ko...Reading symbols from /usr/li= b/debug//boot/kernel/linux64.ko.debug...done. done. Loaded symbols for /boot/kernel/linux64.ko Reading symbols from /boot/kernel/smbios.ko...Reading symbols from /usr/lib= /debug//boot/kernel/smbios.ko.debug...done. done. Loaded symbols for /boot/kernel/smbios.ko Reading symbols from /boot/kernel/mac_ntpd.ko...Reading symbols from /usr/l= ib/debug//boot/kernel/mac_ntpd.ko.debug...done. done. Loaded symbols for /boot/kernel/mac_ntpd.ko Reading symbols from /boot/kernel/netgraph.ko...Reading symbols from /usr/l= ib/debug//boot/kernel/netgraph.ko.debug...done. done. Loaded symbols for /boot/kernel/netgraph.ko Reading symbols from /boot/kernel/ng_netflow.ko...Reading symbols from /usr= /lib/debug//boot/kernel/ng_netflow.ko.debug...done. done. Loaded symbols for /boot/kernel/ng_netflow.ko Reading symbols from /boot/kernel/ng_ksocket.ko...Reading symbols from /usr= /lib/debug//boot/kernel/ng_ksocket.ko.debug...done. done. Loaded symbols for /boot/kernel/ng_ksocket.ko Reading symbols from /boot/kernel/ng_ether.ko...Reading symbols from /usr/l= ib/debug//boot/kernel/ng_ether.ko.debug...done. done. Loaded symbols for /boot/kernel/ng_ether.ko Reading symbols from /boot/kernel/ng_socket.ko...Reading symbols from /usr/= lib/debug//boot/kernel/ng_socket.ko.debug...done. done. Loaded symbols for /boot/kernel/ng_socket.ko Reading symbols from /boot/kernel/dtraceall.ko...Reading symbols from /usr/= lib/debug//boot/kernel/dtraceall.ko.debug...done. done. Loaded symbols for /boot/kernel/dtraceall.ko Reading symbols from /boot/kernel/dtrace.ko...Reading symbols from /usr/lib= /debug//boot/kernel/dtrace.ko.debug...done. done. Loaded symbols for /boot/kernel/dtrace.ko Reading symbols from /boot/kernel/dtmalloc.ko...Reading symbols from /usr/l= ib/debug//boot/kernel/dtmalloc.ko.debug...done. done. Loaded symbols for /boot/kernel/dtmalloc.ko Reading symbols from /boot/kernel/fbt.ko...Reading symbols from /usr/lib/de= bug//boot/kernel/fbt.ko.debug...done. done. Loaded symbols for /boot/kernel/fbt.ko #0 doadump (textdump=3D1) at src/sys/amd64/include/pcpu.h:241 241 __asm("movq %%gs:%P1,%0" : "=3Dr" (td) : "n" (OFFSETOF_CURT= HREAD)); (kgdb) #0 doadump (textdump=3D1) at src/sys/amd64/include/pcpu.h:241 #1 0xffffffff80496320 in kern_reboot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:470 #2 0xffffffff80496799 in vpanic (fmt=3D, ap=3D) at /usr/src/sys/kern/kern_shutdown.c:896 #3 0xffffffff804964d3 in panic (fmt=3D) at /usr/src/sys/kern/kern_shutdown.c:823 #4 0xffffffff80767314 in trap_fatal (frame=3D0xfffffe00d74d7cd0, eva=3D0) at /usr/src/sys/amd64/amd64/trap.c:946 #5 0xffffffff80767379 in trap_pfault (frame=3D0xfffffe00d74d7cd0, usermode= =3D0) at src/sys/amd64/include/pcpu.h:241 #6 0xffffffff80766964 in trap (frame=3D0xfffffe00d74d7cd0) at /usr/src/sys/amd64/amd64/trap.c:441 #7 0xffffffff80740805 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:232 #8 0xffffffff825cb5ea in fbt_provide_module_function (lf=3D0xfffff800020ff= 000, symindx=3D30763, symval=3D0xfffffe00d74d7e00, opaque=3D0xfffffe00d74d7e= 50) at /usr/src/sys/cddl/dev/fbt/x86/fbt_isa.c:190 #9 0xffffffff804bf8f7 in link_elf_each_function_nameval ( file=3D0xfffff800020ff000, callback=3D0xffffffff825cb570 , opaque=3D0xfffffe00d74d7e50) at /usr/src/sys/kern/link_elf.c:1513 #10 0xffffffff825ca33e in fbt_provide_module (arg=3D, lf=3D0xfffff800020ff000) at /usr/src/sys/cddl/dev/fbt/fbt.c:204 #11 0xffffffff825ca242 in fbt_linker_file_cb (lf=3D, arg=3D) at /usr/src/sys/cddl/dev/fbt/fbt.c:1103 #12 0xffffffff8046d772 in linker_file_foreach ( predicate=3D0xffffffff825ca230 , context=3D0x0) at /usr/src/sys/kern/kern_linker.c:594 #13 0xffffffff8046cb58 in linker_load_module (kldname=3D, modname=3D0xffffffff81d792ae "fbt", parent=3D, verinfo=3D, lfpp=3D0x0) at /usr/src/sys/kern/kern_linker.c:236 #14 0xffffffff8046f1bd in linker_load_dependencies (lf=3D0xfffff8002389a400) at /usr/src/sys/kern/kern_linker.c:2200 #15 0xffffffff80797f3e in link_elf_load_file (cls=3D, filename=3D0xfffff80003d592c0 "/boot/kernel/dtraceall.ko", result=3D0xfffffe00d74d8898) at /usr/src/sys/kern/link_elf_obj.c:1010 #16 0xffffffff8046c96f in linker_load_module (kldname=3D, modname=3D0xfffff800231a7800 "dtraceall", parent=3D, verinfo=3D, lfpp=3D0xfffffe00d74d8a38) at linker_i= f.h:180 #17 0xffffffff8046e297 in kern_kldload (td=3D0xfffff80114df9000, file=3D, fileid=3D0xfffffe00d74d8a74) at /usr/src/sys/kern/kern_linker.c:1089 #18 0xffffffff8046e35b in sys_kldload (td=3D0xfffff80114df9000, uap=3D) at /usr/src/sys/kern/kern_linker.c:1115 #19 0xffffffff80767ddc in amd64_syscall (td=3D0xfffff80114df9000, traced=3D= 0) at src/sys/amd64/amd64/../../kern/subr_syscall.c:135 #20 0xffffffff807410ed in fast_syscall_common () at /usr/src/sys/amd64/amd64/exception.S:504 #21 0x00000008002de43a in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb) --=20 Larry Rosenman https://people.FreeBSD.org/~ler/ Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106 --swragf7kp7z3ilk7 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQHBBAABCgCrFiEEHjgknedhWzvJgwVzaXyZsatIp30FAlzTQZ0tFIAAAAAAFQAP cGthLWFkZHJlc3NAZ251cGcub3JnbGVyQEZyZWVCU0Qub3JnXxSAAAAAAC4AKGlz c3Vlci1mcHJAbm90YXRpb25zLm9wZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQxRTM4 MjQ5REU3NjE1QjNCQzk4MzA1NzM2OTdDOTlCMUFCNDhBNzdEAAoJEGl8mbGrSKd9 Vr4H/iGoHa8aC1NQbOWZHs6rJivb2MxjyTFWipvX36NwWM/sfm73X8DvgO1NfA7u GBsNZJh2IxH5ax1YuRY1G8CK5Qk+xylvMtUq3L5aEX1SggvEdQ9nGDm56Djqo1bu XPWPQAEc6TKudEsnM40dcWUBRxNUCbnFghMI9z83g+9HzlXOJ5WgoV3Sir2unQWx k08WM58EIpwYRF61cxe5OLknoixm0WXRMfyruuRSX7dSCuz6yJtdUYAtdoogjC1b zT/94Er9KkpbMIY2iqeM2oRaQ+/1qRTlWszdfPDskWDbdmKtOK7tkjmpF1Gzw2L+ 0iKj9YVZQP6qJrAk2iCZPVM65eE= =GJfH -----END PGP SIGNATURE----- --swragf7kp7z3ilk7--