From nobody Sun Oct 2 04:12:23 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Mg9Xn4yqhz4drJb for ; Sun, 2 Oct 2022 04:12:25 +0000 (UTC) (envelope-from ler@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Mg9Xn4hYQz41kr; Sun, 2 Oct 2022 04:12:25 +0000 (UTC) (envelope-from ler@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1664683945; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=q5I9s8//+6d91H4tR7S1jJoHG2Kma9LlOd1yOFaD0wU=; b=w2ivBOH/Hzz1zq9Zp9pDrAMle7a1+DPzH83ihtVzhZ/lwdkIKHftBGDSGjryByLAw5LNFf 2fJKlcwPna1Fys12wkrXMVTKg44igOex7AaHJJHBjqdG+QauV34idI13i7fZZ2jjTka+vZ GpQaNuTTVyY7Zw/qJyEGAjr+hsNxgSCCr26xld+7oizatPHIeOxN7cKxIu8XQwUSYcjqay md8gfgFsDdkHt2C3udklpQIiKcSQPBgcczCzuQz/lhU6D4k8o7nfnJgO1zN4yP8sBTmlbs A3/32lwNLLXWJbTMi/VCaad71rT5BmdBRiIAzZmN/RGlZ4RsZHYVz4b/OWvYzw== Received: from thebighonker.lerctr.org (unknown [IPv6:2602:fcdb:0:10:7ae3:b5ff:fe1b:23b4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.lerctr.org", Issuer "R3" (verified OK)) (Authenticated sender: ler/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Mg9Xn2T3Mz19ls; Sun, 2 Oct 2022 04:12:25 +0000 (UTC) (envelope-from ler@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=ler2019; h=Content-Type:Message-ID:References:In-Reply-To:Subject:To:From: Date:MIME-Version:Sender:Reply-To:Cc:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=q5I9s8//+6d91H4tR7S1jJoHG2Kma9LlOd1yOFaD0wU=; b=QNLaji5aBTCupN9YKjXeKRci+/ 5fHYu6OexUofJPU3JfWhTjc3wsYbHIcqCwFliMVGlKSIRJ6EqrCmWfU/EmDKBn6WReuz2NvplrAQK uXLUx3oZTiJ6gU6Up6ByBlL2FXSYihiD8ACryf6gq2wcDrQRIVzgQkGuM+cPDsPeNV9/PPFdNHiKI IBdTfmrWv0zFfJuZhb2N8pHkkBolJsNXSl4UeVnzAeHVVCk2742+qPBxAVHFzOMCPKzCPvoOaQPRf 8ToMJWKn3Y3aCgILiahVr72fxm5mvsUGqV6qYbLZG5djUcjrbUfUC0vWeFMdzONk8sOTWYfYzvG3V cbP5eGZw==; Authentication-Results: thebighonker.lerctr.org; iprev=pass (thebighonker.lerctr.org) smtp.remote-ip=2602:fcdb:0:10:7ae3:b5ff:fe1b:23b4; auth=pass (LOGIN) smtp.auth=ler@lerctr.org; spf=softfail smtp.mailfrom=FreeBSD.org; dmarc=skipped header.from=FreeBSD.org; arc=none Received-SPF: softfail (thebighonker.lerctr.org: transitioning domain of FreeBSD.org does not designate 2602:fcdb:0:10:7ae3:b5ff:fe1b:23b4 as permitted sender) client-ip=2602:fcdb:0:10:7ae3:b5ff:fe1b:23b4; envelope-from=ler@FreeBSD.org; helo=webmail.lerctr.org; Received: from thebighonker.lerctr.org ([2602:fcdb:0:10:7ae3:b5ff:fe1b:23b4]:18050 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.95 (FreeBSD)) (envelope-from ) id 1oeqL2-0003OH-3E; Sat, 01 Oct 2022 23:12:24 -0500 Received: from 2600:1700:210:b18f:8cb4:f417:c3e6:a42 by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Sat, 01 Oct 2022 23:12:23 -0500 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Date: Sat, 01 Oct 2022 23:12:23 -0500 From: Larry Rosenman To: Warner Losh , Freebsd current , Mark Johnston Subject: Re: BOOT CRASH -- Current -CURRENT In-Reply-To: <6d03cfe3d1c0244ffb9703b05af3bcb5@FreeBSD.org> References: <9de62c1d7c3eae83b97a1004780a8d0d@FreeBSD.org> <6d03cfe3d1c0244ffb9703b05af3bcb5@FreeBSD.org> Message-ID: <555871472eda375673b6b7d8eabc6264@FreeBSD.org> X-Sender: ler@FreeBSD.org Content-Type: multipart/alternative; boundary="=_c6fac64eb3c0d4c1aa3bdb883c568a3b" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1664683945; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references:dkim-signature; bh=q5I9s8//+6d91H4tR7S1jJoHG2Kma9LlOd1yOFaD0wU=; b=ZWO35zQjO+PJuqnciC8k/Ps9Xc7Pev1gm2SrFreNJ4PF7pz7W7PcM8SoIn75VLRZK5GE1e dUji9VBRpul06JKc+MNPts/FvChJsiGZsAugBS4jf9sG3+CY288Ey+1aqtirSjrlV3kCVk SL8Zv/q+uySndxdhzLV2JeqjnlHbqdHmJKzV+spkQ+79mlaKMx+RGHrswOndDgS7BM7IVM Y+TGTW78/mj1vDA1roygUEeBkiCpKbxWWyv/+buu8vxfF6IJwbjvypUK6KduG5vyiwheMU ui0DXBTM3GSiQ2XhneqivPxZ5xBLOEXlu7uniYutro3zYX52gHjgeudK7WpEJA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1664683945; a=rsa-sha256; cv=none; b=OyksYIzM7vbiLQyEZlW5cqrna0DqUPBLhCjGSav+0uX1QfzjPrtK/imBaVg+hs+kZRUpGR YgZKeo0LiaacAQC1DwWAbwazNzO1G8YO+ecekpUspshfitiS66RPoMR0Uao4IYbIDXjvFx PVkkwQlNdOfAsEGQkTbF8htS2wv1s19F7cEEHzV1sYsHQsYd/WvUyl+oPzjqp4Jdyh6bIj 5CpLGx4c8HQ7Ai3SCdFutQxdko6OeRMwb0w7A8Vzi+m8DAFTYTvnH6WvEJR/OO6GT9B+5S RjM1BPBQBJWOAUg/vPwn5wLnrc9BD599yPEHM1Top2xhpU744GyEp5mcWZDoUQ== ARC-Authentication-Results: i=1; thebighonker.lerctr.org; iprev=pass (thebighonker.lerctr.org) smtp.remote-ip=2602:fcdb:0:10:7ae3:b5ff:fe1b:23b4; auth=pass (LOGIN) smtp.auth=ler@lerctr.org; spf=softfail smtp.mailfrom=FreeBSD.org; dmarc=skipped header.from=FreeBSD.org; arc=none X-ThisMailContainsUnwantedMimeParts: N --=_c6fac64eb3c0d4c1aa3bdb883c568a3b Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8; format=flowed On 10/01/2022 10:43 pm, Larry Rosenman wrote: > On 10/01/2022 10:08 pm, Warner Losh wrote: > > On Sat, Oct 1, 2022 at 9:06 PM Larry Rosenman wrote: > > On 10/01/2022 10:04 pm, Warner Losh wrote: > > Do you have a /boot tarball that can be loaded in a VM that recreates > the problem (along with a clean hash)? > > But before you try that, have you tried a completely clean rebuild of > the kernel to preclude the possibility that something is somehow cross > threaded? > > Warner > > On Sat, Oct 1, 2022 at 8:39 PM Larry Rosenman wrote: > ❯ more info.11 > Dump header from device: /dev/mfid0p3 > Architecture: amd64 > Architecture Version: 2 > Dump Length: 126748815 > Blocksize: 512 > Compression: zstd > Dumptime: 2022-10-01 21:26:40 -0500 > Hostname: > Magic: FreeBSD Kernel Dump > Version String: FreeBSD 14.0-CURRENT #168 > ler/freebsd-main-changes-n258354-6cdd871ebc4: Sat Oct 1 21:13:01 CDT > 2022 > root@borg.lerctr.org:/usr/obj/usr/src/amd64.amd64/sys/LER-MINIMAL > Panic String: page fault > Dump Parity: 501115454 > Bounds: 11 > Dump Status: good > > I do have source and debug stuff, BUT kgdb croaks on me. > > I *CAN* give access to the machine. > > the console backtrace showed something about the kld load of > dependencies. > > -- > Larry Rosenman http://people.freebsd.org/~ler > Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org > US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 let me wipe /usr/obj, and rebuild everything (I *DO* use meta-mode). I've had fewer problems with it than non-meta mode, but this looks like a 'corruption' or 'cross threaded' crash I've chased in the past that went away with a rebuild. So it's better to be sure... Warner Still breaks -- did someone(tm) forget to make netlink a module? -- Larry Rosenman http://people.freebsd.org/~ler Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 ❯ sudo kgdb -c vmcore.12 /mnt/usr/lib/debug/boot/kernel/kernel.debug GNU gdb (GDB) 12.1 [GDB v12.1 for FreeBSD] Copyright (C) 2022 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-portbld-freebsd14.0". Type "show configuration" for configuration details. For bug reporting instructions, please see: . Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /mnt/usr/lib/debug/boot/kernel/kernel.debug... Unread portion of the kernel message buffer: ---<>--- Copyright (c) 1992-2022 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 14.0-CURRENT #0 ler/freebsd-main-changes-n258354-6cdd871ebc4: Sat Oct 1 22:30:48 CDT 2022 root@borg.lerctr.org:/usr/obj/usr/src/amd64.amd64/sys/LER-MINIMAL amd64 FreeBSD clang version 14.0.5 (https://github.com/llvm/llvm-project.git llvmorg-14.0.5-0-gc12386ae247c) VT(efifb): resolution 640x480 CPU: Intel(R) Xeon(R) CPU X5660 @ 2.80GHz (2793.16-MHz K8-class CPU) Origin="GenuineIntel" Id=0x206c2 Family=0x6 Model=0x2c Stepping=2 Features=0xbfebfbff Features2=0x29ee3ff AMD Features=0x2c100800 AMD Features2=0x1 Structured Extended Features3=0x9c000000 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID TSC: P-state invariant, performance statistics real memory = 137438953472 (131072 MB) avail memory = 133789515776 (127591 MB) CPU microcode: no matching update found Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 24 CPUs FreeBSD/SMP: 2 package(s) x 6 core(s) x 2 hardware threads random: unblocking device. ioapic1: MADT APIC ID 1 != hw id 0 ioapic0 irqs 0-23 ioapic1 irqs 32-55 Launching APs: 1 14 12 21 2 6 17 10 18 15 4 19 7 3 8 20 13 5 23 11 9 16 22 TCP_ratelimit: Is now initialized TCP Hpts created 24 swi interrupt threads and bound 24 to NUMA domains random: entropy device external interface kbd1 at kbdmux0 acpi0: acpi0: Power Button (fixed) apei0: on acpi0 cpu0: on acpi0 atrtc0: port 0x70-0x7f irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x5f irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 1.0 on pci0 pci1: on pcib1 pci1: at device 0.0 (no driver attached) pci1: at device 0.1 (no driver attached) pcib2: at device 3.0 on pci0 pci2: on pcib2 pci2: at device 0.0 (no driver attached) pci2: at device 0.1 (no driver attached) pcib3: at device 4.0 on pci0 pci3: on pcib3 mfi0: port 0xfc00-0xfcff mem 0xdf1bc000-0xdf1bffff,0xdf1c0000-0xdf1fffff irq 33 at device 0.0 on pci3 mfi0: Using MSI mfi0: Megaraid SAS driver Ver 4.23 mfi0: FW MaxCmds = 1008, limiting to 128 mfi0: 55346 (717997792s/0x0020/info) - Shutdown command received from host pcib4: mfi0: 55347 (boot + 33s/0x0020/info) - Firmware initialization started (PCI ID 0079/1000/1f17/1028) mfi0: 55348 (boot + 33s/0x0020/info) - Firmware version 2.100.03-4651 at device 5.0 on pci0 pci4: mfi0: 55349 (boot + 35s/0x0008/info) - Battery Present mfi0: 55350 (boot + 35s/0x0020/info) - Package version 12.10.7-0001 mfi0: 55351 (boot + 35s/0x0020/info) - Board Revision A00 on pcib4 pcib5: mfi0: 55352 (boot + 63s/0x0004/info) - Enclosure PD 20(c None/p0) communication restored at device 6.0 on pci0 pci5: mfi0: 55353 (boot + 63s/0x0002/info) - Inserted: Encl PD 20 mfi0: 55354 (boot + 63s/0x0002/info) - Inserted: PD 20(c None/p0) Info: enclPd=20, scsiType=d, portMap=00, sasAddr=5b4ba0b019ef on pcib5 pcib6: e700,0000000000000000 mfi0: 55355 (boot + 63s/0x0002/info) - Inserted: PD 00(e0x20/s0) mfi0: 55356 (boot + 63s/0x0002/info) - Inserted: PD 00(e0x20/s0) Info: enclPd=20, scsiType=0, portMap=04, sasAddr=4433221107000 at device 7.0 on pci0 pci6: 000,0000000000000000 mfi0: 55357 (boot + 63s/0x0002/WARN) - PD 00(e0x20/s0) is not a certified drive mfi0: 55358 (boot + 63s/0x0002/info) - Inserted: PD 01(e0x20/s1) on pcib6 pcib7: mfi0: 55359 (boot + 63s/0x0002/info) - Inserted: PD 01(e0x20/s1) Info: enclPd=20, scsiType=0, portMap=05, sasAddr=4433221106000 at device 9.0 on pci0 pci7: 000,0000000000000000 mfi0: 55360 (boot + 63s/0x0002/WARN) - PD 01(e0x20/s1) is not a certified drive mfi0: 55361 (boot + 63s/0x0002/info) - Inserted: PD 02(e0x20/s2) on pcib7 pci0: mfi0: 55362 (boot + 63s/0x0002/info) - Inserted: PD 02(e0x20/s2) Info: enclPd=20, scsiType=0, portMap=01, sasAddr=4433221105000 at device 20.0 (no driver attached) pci0: 000,0000000000000000 mfi0: 55363 (boot + 63s/0x0002/WARN) - PD 02(e0x20/s2) is not a certified drive mfi0: 55364 (boot + 63s/0x0002/info) - Inserted: PD 03(e0x20/s3) at device 20.1 (no driver attached) pci0: mfi0: 55365 (boot + 63s/0x0002/info) - Inserted: PD 03(e0x20/s3) Info: enclPd=20, scsiType=0, portMap=02, sasAddr=4433221104000 at device 20.2 (no driver attached) uhci0: 000,0000000000000000 port 0xec40-0xec5f irq 17 at device 26.0 on pci0 mfi0: 55366 (boot + 63s/0x0002/WARN) - PD 03(e0x20/s3) is not a certified drive mfi0: 55367 (boot + 63s/0x0002/info) - Inserted: PD 04(e0x20/s4) mfi0: 55368 (boot + 63s/0x0002/info) - Inserted: PD 04(e0x20/s4) Info: enclPd=20, scsiType=0, portMap=03, sasAddr=4433221103000000,0000000000000000 mfi0: 55369 (boot + 63s/0x0002/WARN) - PD 04(e0x20/s4) is not a certified drive mfi0: 55370 (boot + 63s/0x0002/info) - Inserted: PD 05(e0x20/s5) mfi0: 55371 (boot + 63s/0x0002/info) - Inserted: PD 05(e0x20/s5) Info: enclPd=20, scsiType=0, portMap=00, sasAddr=4433221102000usbus0000,0000000000000000 mfi0: 55372 (boot + 63s/0x0002/WARN) - PD 05(e0x20/s5) is not a certified drive mfi0: 55373 (boot + 100s/0x0008/info) - Battery temperature is normal on uhci0 mfi0: 55374 (boot + 100s/0x0008/info) - Current capacity of the battery is above threshold mfi0: 55375 (boot + 100s/0x0008/info) - Battery charge complete mfi0: 55376 (717997971s/0x0020/info) - Time established as 10/02/22 3:52:51; (148 seconds since power on) usbus0: 12Mbps Full Speed USB v1.0 uhci1: port 0xec60-0xec7f irq 18 at device 26.1 on pci0 usbus1 on uhci1 usbus1: 12Mbps Full Speed USB v1.0 ehci0: mem 0xdf0de000-0xdf0de3ff irq 19 at device 26.7 on pci0 usbus2: EHCI version 1.0 usbus2 on ehci0 usbus2: 480Mbps High Speed USB v2.0 uhci2: port 0xec80-0xec9f irq 21 at device 29.0 on pci0 usbus3 on uhci2 usbus3: 12Mbps Full Speed USB v1.0 uhci3: port 0xeca0-0xecbf irq 20 at device 29.1 on pci0 usbus4 on uhci3 usbus4: 12Mbps Full Speed USB v1.0 ehci1: mem 0xdf0df000-0xdf0df3ff irq 21 at device 29.7 on pci0 usbus5: EHCI version 1.0 usbus5 on ehci1 usbus5: 480Mbps High Speed USB v2.0 pcib8: at device 30.0 on pci0 pci8: on pcib8 vgapci0: mem 0xd5800000-0xd5ffffff,0xde7fc000-0xde7fffff,0xde800000-0xdeffffff irq 19 at device 3.0 on pci8 vgapci0: Boot video device isab0: at device 31.0 on pci0 isa0: on isab0 pci0: at device 31.2 (no driver attached) uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: console (115200,n,8,1) uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xcdfff,0xec000-0xeffff pnpid ORM0000 on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] est0: on cpu0 Timecounter "TSC-low" frequency 1396499773 Hz quality 1000 Timecounters tick every 1.000 msec ZFS filesystem version: 5 ZFS storage pool version: features support (5000) ugen0.1: at usbus0 ugen2.1: at usbus2 mfid0 on mfi0 mfid0: 1907200MB (3905945600 sectors) RAID volume 'Disk2' is optimal ugen1.1: at usbus1 uhub0 on usbus0 uhub0: on usbus0 ugen4.1: at usbus4 uhub1 on usbus1 uhub1: on usbus1 mfid1 on mfi0 mfid1: 1907200MB (3905945600 sectors) RAID volume 'Disk1' is optimal uhub2 on usbus4 uhub2: on usbus4 uhub3 on usbus2 uhub3: on usbus2 ugen5.1: at usbus5 ugen3.1: at usbus3 mfid2 on mfi0 mfid2: 2861056MB (5859442688 sectors) RAID volume 'Disk0' is optimal uhub4 on usbus5 uhub4: on usbus5 mfid3 on mfi0 mfid3: 2861056MB (5859442688 sectors) RAID volume 'Disk4' is optimal uhub5 on usbus3 uhub5: on usbus3 mfid4 on mfi0 mfid4: 1907200MB (3905945600 sectors) RAID volume 'Disk3' is optimal mfid5 on mfi0 mfid5: 1907200MB (3905945600 sectors) RAID volume 'DIsk5' is optimal Trying to mount root from zfs:zroot/ROOT/14-2022_10_01-2233 []... uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub5: 2 ports with 2 removable, self powered Root mount waiting for: usbus2 usbus5 uhub3: 4 ports with 4 removable, self powered uhub4: 4 ports with 4 removable, self powered Root mount waiting for: usbus2 usbus5 ugen2.2: at usbus2 uhub6 on uhub3 uhub6: on usbus2 uhub6: MTT enabled ugen0.2: at usbus0 ukbd0 on uhub0 ukbd0: on usbus0 kbd2 at ukbd0 uhub6: 3 ports with 3 removable, self powered ugen3.2: at usbus3 ugen3.3: at usbus3 ukbd1 on uhub5 ukbd1: on usbus3 kbd3 at ukbd1 Dual Console: Video Primary, Serial Secondary <118>sysctl: net.inet.tcp.functions_default=rack at line 55: No such file or directory <118>Setting hostuuid: 4c4c4544-0035-4310-8048-c7c04f524c31. <118>Setting hostid: 0xd7b851f2. <118>no pools available to import <118>Starting file system checks: <118>Mounting local filesystems:. <118>Loading kernel modules: aesni0: coretemp0: on cpu0 link_elf_obj: symbol nhgrp_get_group undefined linker_load_file: /boot/kernel/netlink.ko - unsupported file type KLD linux_common.ko: depends on netlink - not available or version mismatch linker_load_file: /boot/kernel/linux_common.ko - unsupported file type KLD linux.ko: depends on linux_common - not available or version mismatch linker_load_file: /boot/kernel/linux.ko - unsupported file type <118>kldload: an error occurred while loading module linux. Please check dmesg(8) for more details. <118>/etc/rc: WARNING: Unable to load kernel module linux ichwd0: on isa0 efirtc0: efirtc0: registered as a time-of-day clock, resolution 1.000000s ipmi0: on isa0 ipmi0: KCS mode found at io 0xca8 alignment 0x4 on isa ipmi0: IPMI device rev. 0, firmware rev. 2.92, version 2.0, device support mask 0xdf ipmi0: Number of channels 5 ipmi0: Attached watchdog ipmi0: Establishing power cycle handler hwpmc: SOFT/16/64/0x67 TSC/1/64/0x20 IAP/4/48/0x3ff IAF/3/48/0x67 UCP/8/48/0x3f8 UCF/1/48/0x60 mfip0: on mfi0 ioat0: mem 0xdf0e0000-0xdf0e3fff irq 51 at device 22.0 on pci0 ioat0: Capabilities: 77 ioat1: mem 0xdf0e4000-0xdf0e7fff irq 52 at device 22.1 on pci0 ioat1: Capabilities: 77 ioat2: mem 0xdf0e8000-0xdf0ebfff irq 53 at device 22.2 on pci0 ioat2: Capabilities: 77 ioat3: mem 0xdf0ec000-0xdf0effff irq 54 at device 22.3 on pci0 ioat3: Capabilities: 77 ioat4: mem 0xdf0f0000-0xdf0f3fff irq 51 at device 22.4 on pci0 ioat4: Capabilities: 77 ioat5: mem 0xdf0f4000-0xdf0f7fff irq 52 at device 22.5 on pci0 ioat5: Capabilities: 77 ioat6: mem 0xdf0f8000-0xdf0fbfff irq 53 at device 22.6 on pci0 ioat6: Capabilities: 77 ioat7: mem 0xdf0fc000-0xdf0fffff irq 54 at device 22.7 on pci0 ioat7: Capabilities: 77 bce0: mem 0xd6000000-0xd7ffffff irq 36 at device 0.0 on pci1 <6>bce0: Using defaults for TSO: 65518/35/2048 <6>bce0: Ethernet address: a4:ba:db:29:66:95 bce0: ASIC (0x57092003); Rev (C0); Bus (PCIe x4, 2.5Gbps); B/C (7.10.0); Bufs (RX:8;TX:8;PG:32); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.13) Coal (RX:6,6,18,18; TX:20,20,80,80) bce1: mem 0xd8000000-0xd9ffffff irq 48 at device 0.1 on pci1 <6>bce1: Using defaults for TSO: 65518/35/2048 <6>bce1: Ethernet address: a4:ba:db:29:66:97 bce1: ASIC (0x57092003); Rev (C0); Bus (PCIe x4, 2.5Gbps); B/C (7.10.0); Bufs (RX:8;TX:8;PG:32); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.13) Coal (RX:6,6,18,18; TX:20,20,80,80) bce2: mem 0xda000000-0xdbffffff irq 32 at device 0.0 on pci2 <6>bce2: Using defaults for TSO: 65518/35/2048 <6>bce2: Ethernet address: a4:ba:db:29:66:99 bce2: ASIC (0x57092003); Rev (C0); Bus (PCIe x4, 2.5Gbps); B/C (7.10.0); Bufs (RX:8;TX:8;PG:32); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.13) Coal (RX:6,6,18,18; TX:20,20,80,80) bce3: mem 0xdc000000-0xddffffff irq 42 at device 0.1 on pci2 <6>bce3: Using defaults for TSO: 65518/35/2048 <6>bce3: Ethernet address: a4:ba:db:29:66:9b bce3: ASIC (0x57092003); Rev (C0); Bus (PCIe x4, 2.5Gbps); B/C (7.10.0); Bufs (RX:8;TX:8;PG:32); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.13) Coal (RX:6,6,18,18; TX:20,20,80,80) miibus0: on bce0 brgphy0: PHY 1 on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow miibus1: on bce1 brgphy1: PHY 1 on miibus1 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow miibus2: on bce2 brgphy2: PHY 1 on miibus2 brgphy2: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow miibus3: on bce3 brgphy3: PHY 1 on miibus3 brgphy3: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow ums0 on uhub5 ums0: on usbus3 ums0: 3 buttons and [Z] coordinates ID=0 ses0 at mfi0 bus 0 scbus0 target 32 lun 0 ses0: Fixed Enclosure Services SPC-3 SCSI device ses0: 150.000MB/s transfers ses0: SES Device atapci0: port 0xec10-0xec17,0xec08-0xec0b,0xec18-0xec1f,0xec0c-0xec0f,0xec20-0xec2f,0xec30-0xec3f irq 23 at device 31.2 on pci0 atapci0: failed to add ata child device ata2: at channel 0 on atapci0 device_attach: ata2 attach returned 6 <118>Autoloading module: acpi_wmi acpi_wmi0: on acpi0 acpi_wmi0: cannot find EC device link_elf_obj: symbol nhgrp_get_group undefined linker_load_file: /boot/kernel/netlink.ko - unsupported file type KLD linux_common.ko: depends on netlink - not available or version mismatch Fatal trap 12: page fault while in kernel mode cpuid = 20; apic id = 12 fault virtual address = 0x0 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff8231c132 stack pointer = 0x28:0xfffffe02b296a530 frame pointer = 0x28:0xfffffe02b296a560 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 232 (kldload) rdi: fffff80128ca00b0 rsi: fffffe02b296a538 rdx: 0 rcx: ffffffff825dbde0 r8: 4 r9: f3 rax: 0 rbx: fffff80128b9ae80 rbp: fffffe02b296a560 r10: f3 r11: ff9094d191909292 r12: fffff80128b9aec0 r13: fffffe02b296a584 r14: fffffe02b296a584 r15: ffffffff825dbdd0 trap number = 12 panic: page fault cpuid = 20 time = 1664682860 KDB: stack backtrace: #0 0xffffffff80508e9d at kdb_backtrace+0x5d #1 0xffffffff804bc741 at vpanic+0x151 #2 0xffffffff804bc5e3 at panic+0x43 #3 0xffffffff807afe59 at trap_fatal+0x409 #4 0xffffffff807afeaf at trap_pfault+0x4f #5 0xffffffff80787d18 at calltrap+0x8 #6 0xffffffff8048fddb at linker_file_unload+0xdb #7 0xffffffff807bc858 at link_elf_load_file+0x198 #8 0xffffffff8048f659 at linker_load_module+0x999 #9 0xffffffff804921b3 at linker_load_dependencies+0xe3 #10 0xffffffff807bd7b3 at link_elf_load_file+0x10f3 #11 0xffffffff8048f659 at linker_load_module+0x999 #12 0xffffffff80491351 at kern_kldload+0x141 #13 0xffffffff8049145b at sys_kldload+0x5b #14 0xffffffff807b074c at amd64_syscall+0x10c #15 0xffffffff8078862b at fast_syscall_common+0xf8 Uptime: 22s Dumping 5423 out of 131021 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:5959 __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (offsetof(struct pcpu, (kgdb) bt full #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:59 td = #1 dump_savectx () at /usr/src/sys/kern/kern_shutdown.c:405 No locals. #2 0xffffffff804bc2e8 in dumpsys (di=0x0) at /usr/src/sys/x86/include/dump.h:87 No locals. #3 doadump (textdump=) at /usr/src/sys/kern/kern_shutdown.c:434 di = 0x0 error = coredump = 1 di = #4 kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:541 once = 0 #5 0xffffffff804bc7ae in vpanic (fmt=, ap=ap@entry=0xfffffe02b296a380) at /usr/src/sys/kern/kern_shutdown.c:979 buf = "page fault", '\000' other_cpus = {__bits = {15728639, 0, 0, 0}} td = 0xfffffe00e153d1e0 bootopt = newpanic = #6 0xffffffff804bc5e3 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:903 ap = {{gp_offset = 16, fp_offset = 48, overflow_arg_area = 0xfffffe02b296a3b0, reg_save_area = 0xfffffe02b296a350}} #7 0xffffffff807afe59 in trap_fatal (frame=0xfffffe02b296a470, eva=0) at /usr/src/sys/amd64/amd64/trap.c:955 softseg = {ssd_base = 0, ssd_limit = 1048575, ssd_type = 27, ssd_dpl = 0, ssd_p = 1, ssd_long = 1, ssd_def32 = 0, ssd_gran = 1} code = gdt = ss = 40 type = handled = #8 0xffffffff807afeaf in trap_pfault (frame=0xfffffe02b296a470, usermode=false, signo=, ucode=) at /usr/src/sys/amd64/amd64/trap.c:763 td = 0xfffffe00e153d1e0 p = eva = 0 map = ftype = rv = --Type for more, q to quit, c to continue without paging-- #9 No locals. #10 0xffffffff8231c132 in sdt_kld_unload_try (arg=, lf=, error=0xfffffe02b296a584) at /usr/src/sys/cddl/dev/sdt/sdt.c:345 begin = end = curr = 0xffffffff825dbdd0 prov = 0xfffff80128b9aec0 tmp = 0xfffff80128b9ae80 #11 0xffffffff8048fddb in linker_file_unload (file=file@entry=0xfffff80101392a80, flags=flags@entry=1) at /usr/src/sys/kern/kern_linker.c:675 _ep = 0xfffff80128b9af00 _t = 0xfffff80128b9af00 _el = 0xfffff80117b87280 error = 0 mod = next = ml = nextml = i = cp = #12 0xffffffff807bc858 in link_elf_load_file (cls=, filename=, result=) at /usr/src/sys/kern/link_elf_obj.c:1243 td = 0xfffffe00e153d1e0 error = 8 shdr = lf = 0xfffff80101392a80 mapsize = hdr = 0xfffff80117db0100 nd = 0xfffff8011793f300 flags = 1 resid = ef = nbytes = i = symtabindex = symstrindex = nsym = shstrindex = --Type for more, q to quit, c to continue without paging-- alignmask = mapbase = pb = rl = ra = j = es = #13 0xffffffff8048f659 in LINKER_LOAD_FILE (cls=0xffffffff80adb370 , result=0xfffffe02b296a800, filename=) at ./linker_if.h:214 _m = rc = _desc = _cep = _ce = #14 linker_load_file (filename=, result=) at /usr/src/sys/kern/kern_linker.c:461 lf = 0x0 lc = 0xffffffff80adb370 foundfile = 0 error = modules = _el = _ep = _t = _tid = _v = _v = _v = #15 linker_load_module (kldname=kldname@entry=0x0, modname=modname@entry=0xffffffff825c292c "linux_common", parent=parent@entry=0xfffff8011775e300, verinfo=verinfo@entry=0xffffffff825ce198 <_linuxelf_depend_on_linux_common>, lfpp=lfpp@entry=0x0) at /usr/src/sys/kern/kern_linker.c:2205 pathname = filename = error = lfdep = #16 0xffffffff804921b3 in linker_load_dependencies (lf=0x0, lf@entry=0xfffff8011775e300) at /usr/src/sys/kern/kern_linker.c:2295 error = 0 start = --Type for more, q to quit, c to continue without paging-- stop = mdp = 0xffffffff825cfab8 <__set_modmetadata_set_sym__mod_metadata_md_linuxelf_on_linux_common> modname = 0xffffffff825c292c "linux_common" verinfo = 0xffffffff825ce198 <_linuxelf_depend_on_linux_common> mp = nmodname = nmdp = nmp = mod = lfdep = ver = #17 0xffffffff807bd7b3 in link_elf_load_file (cls=, filename=0xfffff801018e3880 "/boot/kernel/linux.ko", result=0xfffffe02b296ac10) at /usr/src/sys/kern/link_elf_obj.c:1212 td = 0xfffffe00e153d1e0 error = shdr = 0xfffff801367be800 lf = 0xfffff80128ca00b0 mapsize = hdr = 0xfffff80136929c00 nd = 0xfffff80117958e00 flags = 1 resid = ef = 0xfffff80128ca00b0 nbytes = i = symtabindex = symstrindex = nsym = shstrindex = alignmask = mapbase = pb = rl = ra = j = es = #18 0xffffffff8048f659 in LINKER_LOAD_FILE (cls=0xffffffff80adb370 , result=0xfffffe02b296ac10, filename=) at ./linker_if.h:214 --Type for more, q to quit, c to continue without paging-- _m = rc = _desc = _cep = _ce = #19 linker_load_file (filename=, result=) at /usr/src/sys/kern/kern_linker.c:461 lf = 0x0 lc = 0xffffffff80adb370 foundfile = 0 error = modules = _el = _ep = _t = _tid = _v = _v = _v = #20 linker_load_module (kldname=kldname@entry=0x0, modname=modname@entry=0xfffff8013632f800 "linux", parent=parent@entry=0x0, verinfo=verinfo@entry=0x0, lfpp=lfpp@entry=0xfffffe02b296ada0) at /usr/src/sys/kern/kern_linker.c:2205 pathname = filename = error = lfdep = #21 0xffffffff80491351 in kern_kldload (td=td@entry=0xfffffe00e153d1e0, file=file@entry=0xfffff8013632f800 "linux", fileid=fileid@entry=0xfffffe02b296ade4) at /usr/src/sys/kern/kern_linker.c:1164 error = modname = 0xfffff8013632f800 "linux" kldname = 0x0 lf = 0xfffff80101633100 #22 0xffffffff8049145b in sys_kldload (td=0xfffffe00e153d1e0, uap=) at /usr/src/sys/kern/kern_linker.c:1187 pathname = 0xfffff8013632f800 "linux" error = 0 fileid = -2047 #23 0xffffffff807b074c in syscallenter (td=0xfffffe00e153d1e0) at /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:189 --Type for more, q to quit, c to continue without paging-- p = 0xfffffe00dfa4b008 sa = error = se = 0xffffffff80a6ffc0 traced = sy_thr_static = _tid = _v = _v = _audit_entered = _tid = _v = _v = _tid = _v = _v = #24 amd64_syscall (td=0xfffffe00e153d1e0, traced=0) at /usr/src/sys/amd64/amd64/trap.c:1200 ksi = {ksi_link = {tqe_next = 0xfffffe02b296af30, tqe_prev = 0xffffffff807af891 }, ksi_info = {si_signo = -1298747696, si_errno = -510, si_code = -2142476180, si_pid = -1, si_uid = 0, si_status = 0, si_addr = 0x0, si_value = {sival_int = 0, sival_ptr = 0x0, sigval_int = 0, sigval_ptr = 0x0}, _reason = {_fault = {_trapno = 0}, _timer = {_timerid = 0, _overrun = 0}, _mesgq = {_mqd = 0}, _poll = {_band = 0}, _capsicum = {_syscall = 0}, __spare__ = {__spare1__ = 0, __spare2__ = {0, 0, 0, 0, 0, 0, 0}}}}, ksi_flags = 0, ksi_sigq = 0x0} #25 No locals. #26 0x000006977a82871a in ?? () No symbol table info available. Backtrace stopped: Cannot access memory at address 0x69779933668 Sure looks like netlink needs to be made available as a kld..... -- Larry Rosenman http://people.freebsd.org/~ler Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 --=_c6fac64eb3c0d4c1aa3bdb883c568a3b Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=UTF-8

On 10/01/2022 10:43 pm, Larry Rosenman wrote:

On 10/01/2022 10:08 pm, Warner Losh wrote:

 

On Sat, Oct 1, 2022 at 9:06 PM La= rry Rosenman <ler@= freebsd.org> wrote:

On 10/01/2022 10:04 pm, Warn= er Losh wrote:

Do  you have a /boot tarball that can be loaded in a = VM that recreates the problem (along with a clean hash)?
 
But before you try that, have you tried a completely clean rebuild of = the kernel to preclude the possibility that something is somehow cross thre= aded?
 
Warner

On Sat, Oct 1, 2022 at 8:39 PM Larry Rosenman <ler@freebsd.org> wrot= e:

=E2=9D=AF more info.11
Dump header from= device: /dev/mfid0p3
   Architecture: amd64
  &nb= sp;Architecture Version: 2
   Dump Length: 126748815
&n= bsp;  Blocksize: 512
   Compression: zstd
  &= nbsp;Dumptime: 2022-10-01 21:26:40 -0500
   Hostname:
&= nbsp;  Magic: FreeBSD Kernel Dump
   Version String: Fr= eeBSD 14.0-CURRENT #168
ler/freebsd-main-changes-n258354-6cdd871ebc4:= Sat Oct  1 21:13:01 CDT
2022
     root@borg= =2Elerctr.org:/usr/obj/usr/src/amd64.amd64/sys/LER-MINIMAL
  &nbs= p;Panic String: page fault
   Dump Parity: 501115454
&n= bsp;  Bounds: 11
   Dump Status: good

I do h= ave source and debug stuff, BUT kgdb croaks on me.

I *CAN* give = access to the machine.

the console backtrace showed something ab= out the kld load of
dependencies.



--
Lar= ry Rosenman                  &= nbsp;  http://people.freebsd.org/~ler
Phone: +1= 214-642-9640                 = E-Mail: ler@FreeBSD.org
US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-= 2106

let me wipe /usr/obj, and rebuild everything (I *DO* use meta-mode).

 
I've had fewer problems with it than non-meta mode, but this looks lik= e a 'corruption' or 'cross threaded' crash I've chased in the past that wen= t away with a rebuild. So it's better to be sure...
 
Warner 

Still breaks -- did someone(tm) forget to make netlink a module?


-- 
Larry Rosenman       =               http://peopl= e.freebsd.org/~ler
Phone: +1 214-642-9640       &nb= sp;         E-Mail: ler@FreeBSD.org
US Mail: 5708 Sabbia Dr, Round = Rock, TX 78665-2106
=E2=9D=AF sudo kgdb -c vmcore.12 /mnt/usr/lib/debug/boot/kernel/= kernel.debug
GNU gdb (GDB) 12.1 [GDB v12.1 for FreeBSD]
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/li= censes/gpl.html>
This is free software: you are free to change and redistribute i= t.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-portbld-freebsd14.0".<= /div>
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:=
    <http://www.gnu.org/software/gdb/documentation/= >.
 
For help, type "help".
Type "apropos word" to search for commands related to "word"...<= /span>
Reading symbols from /mnt/usr/lib/debug/boot/kernel/kernel.debug= =2E..
 
Unread portion of the kernel message buffer:
---<<BOOT>>---
Copyright (c) 1992-2022 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 19= 93, 1994
The Regents of the University of California. All rights reserve= d.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 14.0-CURRENT #0 ler/freebsd-main-changes-n258354-6cdd871= ebc4: Sat Oct  1 22:30:48 CDT 2022
    root@borg.lerctr.org:/usr/obj/usr/src/amd64.amd64/= sys/LER-MINIMAL amd64
FreeBSD clang version 14.0.5 (https://github.com/llvm/llvm-proje= ct.git llvmorg-14.0.5-0-gc12386ae247c)
VT(efifb): resolution 640x480
CPU: Intel(R) Xeon(R) CPU          &nbs= p;X5660  @ 2.80GHz (2793.16-MHz K8-class CPU)
  Origin=3D"GenuineIntel"  Id=3D0x206c2  Family= =3D0x6  Model=3D0x2c  Stepping=3D2
  Features=3D0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,C= X8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,= SS,HTT,TM,PBE>
  Features2=3D0x29ee3ff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL= ,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AESNI&g= t;
  AMD Features=3D0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM= >
  AMD Features2=3D0x1<LAHF>
  Structured Extended Features3=3D0x9c000000<IBPB,STIBP,= L1DFL,SSBD>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
  TSC: P-state invariant, performance statistics
real memory  =3D 137438953472 (131072 MB)
avail memory =3D 133789515776 (127591 MB)
CPU microcode: no matching update found
Event timer "LAPIC" quality 600
ACPI APIC Table: <DELL   PE_SC3  >
FreeBSD/SMP: Multiprocessor System Detected: 24 CPUs
FreeBSD/SMP: 2 package(s) x 6 core(s) x 2 hardware threads
random: unblocking device.
ioapic1: MADT APIC ID 1 !=3D hw id 0
ioapic0 <Version 2.0> irqs 0-23
ioapic1 <Version 2.0> irqs 32-55
Launching APs: 1 14 12 21 2 6 17 10 18 15 4 19 7 3 8 20 13 5 23 = 11 9 16 22
TCP_ratelimit: Is now initialized
TCP Hpts created 24 swi interrupt threads and bound 24 to NUMA d= omains
random: entropy device external interface
kbd1 at kbdmux0
acpi0: <DELL PE_SC3>
acpi0: Power Button (fixed)
apei0: <ACPI Platform Error Interface> on acpi0
cpu0: <ACPI CPU> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x7f irq 8 on acpi0<= /span>
atrtc0: registered as a time-of-day clock, resolution 1.000000s<= /span>
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x5f irq 0 on acpi0<= /div>
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed= 003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 350
Event timer "HPET1" frequency 14318180 Hz quality 340
Event timer "HPET2" frequency 14318180 Hz quality 340
Event timer "HPET3" frequency 14318180 Hz quality 340
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900<= /div>
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80= b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0<= /div>
pci1: <ACPI PCI bus> on pcib1
pci1: <network, ethernet> at device 0.0 (no driver attache= d)
pci1: <network, ethernet> at device 0.1 (no driver attache= d)
pcib2: <ACPI PCI-PCI bridge> at device 3.0 on pci0<= /div>
pci2: <ACPI PCI bus> on pcib2
pci2: <network, ethernet> at device 0.0 (no driver attache= d)
pci2: <network, ethernet> at device 0.1 (no driver attache= d)
pcib3: <ACPI PCI-PCI bridge> at device 4.0 on pci0<= /div>
pci3: <ACPI PCI bus> on pcib3
mfi0: <Dell PERC H700 Integrated> port 0xfc00-0xfcff mem 0= xdf1bc000-0xdf1bffff,0xdf1c0000-0xdf1fffff irq 33 at device 0.0 on pci3
mfi0: Using MSI
mfi0: Megaraid SAS driver Ver 4.23
mfi0: FW MaxCmds =3D 1008, limiting to 128
mfi0: 55346 (717997792s/0x0020/info) - Shutdown command received= from host
pcib4: <ACPI PCI-PCI bridge>mfi0: 55347 (boot + 33s/0x0020= /info) - Firmware initialization started (PCI ID 0079/1000/1f17/1028)
mfi0: 55348 (boot + 33s/0x0020/info) - Firmware version 2.100.03= -4651
 at device 5.0 on pci0
pci4: <ACPI PCI bus>mfi0: 55349 (boot + 35s/0x0008/info) -= Battery Present
mfi0: 55350 (boot + 35s/0x0020/info) - Package version 12.10.7-0= 001
mfi0: 55351 (boot + 35s/0x0020/info) - Board Revision A00=
 on pcib4
pcib5: <ACPI PCI-PCI bridge>mfi0: 55352 (boot + 63s/0x0004= /info) - Enclosure PD 20(c None/p0) communication restored
 at device 6.0 on pci0
pci5: <ACPI PCI bus>mfi0: 55353 (boot + 63s/0x0002/info) -= Inserted: Encl PD 20
mfi0: 55354 (boot + 63s/0x0002/info) - Inserted: PD 20(c None/p0= ) Info: enclPd=3D20, scsiType=3Dd, portMap=3D00, sasAddr=3D5b4ba0b019ef on = pcib5
pcib6: <ACPI PCI-PCI bridge>e700,0000000000000000
mfi0: 55355 (boot + 63s/0x0002/info) - Inserted: PD 00(e0x20/s0)=
mfi0: 55356 (boot + 63s/0x0002/info) - Inserted: PD 00(e0x20/s0)= Info: enclPd=3D20, scsiType=3D0, portMap=3D04, sasAddr=3D4433221107000 at = device 7.0 on pci0
pci6: <ACPI PCI bus>000,0000000000000000
mfi0: 55357 (boot + 63s/0x0002/WARN) - PD 00(e0x20/s0) is not a = certified drive
mfi0: 55358 (boot + 63s/0x0002/info) - Inserted: PD 01(e0x20/s1)=
 on pcib6
pcib7: <ACPI PCI-PCI bridge>mfi0: 55359 (boot + 63s/0x0002= /info) - Inserted: PD 01(e0x20/s1) Info: enclPd=3D20, scsiType=3D0, portMap= =3D05, sasAddr=3D4433221106000 at device 9.0 on pci0
pci7: <ACPI PCI bus>000,0000000000000000
mfi0: 55360 (boot + 63s/0x0002/WARN) - PD 01(e0x20/s1) is not a = certified drive
mfi0: 55361 (boot + 63s/0x0002/info) - Inserted: PD 02(e0x20/s2)=
 on pcib7
pci0: <base peripheral, interrupt controller>mfi0: 55362 (= boot + 63s/0x0002/info) - Inserted: PD 02(e0x20/s2) Info: enclPd=3D20, scsi= Type=3D0, portMap=3D01, sasAddr=3D4433221105000 at device 20.0 (no driver a= ttached)
pci0: <base peripheral, interrupt controller>000,000000000= 0000000
mfi0: 55363 (boot + 63s/0x0002/WARN) - PD 02(e0x20/s2) is not a = certified drive
mfi0: 55364 (boot + 63s/0x0002/info) - Inserted: PD 03(e0x20/s3)=
 at device 20.1 (no driver attached)
pci0: <base peripheral, interrupt controller>mfi0: 55365 (= boot + 63s/0x0002/info) - Inserted: PD 03(e0x20/s3) Info: enclPd=3D20, scsi= Type=3D0, portMap=3D02, sasAddr=3D4433221104000 at device 20.2 (no driver a= ttached)
uhci0: <Intel 82801I (ICH9) USB controller>000,00000000000= 00000
 port 0xec40-0xec5f irq 17 at device 26.0 on pci0
mfi0: 55366 (boot + 63s/0x0002/WARN) - PD 03(e0x20/s3) is not a = certified drive
mfi0: 55367 (boot + 63s/0x0002/info) - Inserted: PD 04(e0x20/s4)=
mfi0: 55368 (boot + 63s/0x0002/info) - Inserted: PD 04(e0x20/s4)= Info: enclPd=3D20, scsiType=3D0, portMap=3D03, sasAddr=3D4433221103000000,= 0000000000000000
mfi0: 55369 (boot + 63s/0x0002/WARN) - PD 04(e0x20/s4) is not a = certified drive
mfi0: 55370 (boot + 63s/0x0002/info) - Inserted: PD 05(e0x20/s5)=
mfi0: 55371 (boot + 63s/0x0002/info) - Inserted: PD 05(e0x20/s5)= Info: enclPd=3D20, scsiType=3D0, portMap=3D00, sasAddr=3D4433221102000usbu= s0000,0000000000000000
mfi0: 55372 (boot + 63s/0x0002/WARN) - PD 05(e0x20/s5) is not a = certified drive
mfi0: 55373 (boot + 100s/0x0008/info) - Battery temperature is n= ormal
 on uhci0
mfi0: 55374 (boot + 100s/0x0008/info) - Current capacity of the = battery is above threshold
mfi0: 55375 (boot + 100s/0x0008/info) - Battery charge complete<= /span>
mfi0: 55376 (717997971s/0x0020/info) - Time established as 10/02= /22  3:52:51; (148 seconds since power on)
usbus0: 12Mbps Full Speed USB v1.0
uhci1: <Intel 82801I (ICH9) USB controller> port 0xec60-0x= ec7f irq 18 at device 26.1 on pci0
usbus1 on uhci1
usbus1: 12Mbps Full Speed USB v1.0
ehci0: <Intel 82801I (ICH9) USB 2.0 controller> mem 0xdf0d= e000-0xdf0de3ff irq 19 at device 26.7 on pci0
usbus2: EHCI version 1.0
usbus2 on ehci0
usbus2: 480Mbps High Speed USB v2.0
uhci2: <Intel 82801I (ICH9) USB controller> port 0xec80-0x= ec9f irq 21 at device 29.0 on pci0
usbus3 on uhci2
usbus3: 12Mbps Full Speed USB v1.0
uhci3: <Intel 82801I (ICH9) USB controller> port 0xeca0-0x= ecbf irq 20 at device 29.1 on pci0
usbus4 on uhci3
usbus4: 12Mbps Full Speed USB v1.0
ehci1: <Intel 82801I (ICH9) USB 2.0 controller> mem 0xdf0d= f000-0xdf0df3ff irq 21 at device 29.7 on pci0
usbus5: EHCI version 1.0
usbus5 on ehci1
usbus5: 480Mbps High Speed USB v2.0
pcib8: <ACPI PCI-PCI bridge> at device 30.0 on pci0=
pci8: <ACPI PCI bus> on pcib8
vgapci0: <VGA-compatible display> mem 0xd5800000-0xd5fffff= f,0xde7fc000-0xde7fffff,0xde800000-0xdeffffff irq 19 at device 3.0 on pci8<= /span>
vgapci0: Boot video device
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
pci0: <mass storage, ATA> at device 31.2 (no driver attach= ed)
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags = 0x10 on acpi0
uart0: console (115200,n,8,1)
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acp= i0
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0= xcdfff,0xec000-0xeffff pnpid ORM0000 on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 o= n isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
est0: <Enhanced SpeedStep Frequency Control> on cpu0
Timecounter "TSC-low" frequency 1396499773 Hz quality 1000
Timecounters tick every 1.000 msec
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
ugen0.1: <Intel UHCI root HUB> at usbus0
ugen2.1: <Intel EHCI root HUB> at usbus2
mfid0 on mfi0
mfid0: 1907200MB (3905945600 sectors) RAID volume 'Disk2' is opt= imal
ugen1.1: <Intel UHCI root HUB> at usbus1
uhub0 on usbus0
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1= > on usbus0
ugen4.1: <Intel UHCI root HUB> at usbus4
uhub1 on usbus1
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1= > on usbus1
mfid1 on mfi0
mfid1: 1907200MB (3905945600 sectors) RAID volume 'Disk1' is opt= imal
uhub2 on usbus4
uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1= > on usbus4
uhub3 on usbus2
uhub3: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1= > on usbus2
ugen5.1: <Intel EHCI root HUB> at usbus5
ugen3.1: <Intel UHCI root HUB> at usbus3
mfid2 on mfi0
mfid2: 2861056MB (5859442688 sectors) RAID volume 'Disk0' is opt= imal
uhub4 on usbus5
uhub4: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1= > on usbus5
mfid3 on mfi0
mfid3: 2861056MB (5859442688 sectors) RAID volume 'Disk4' is opt= imal
uhub5 on usbus3
uhub5: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1= > on usbus3
mfid4 on mfi0
mfid4: 1907200MB (3905945600 sectors) RAID volume 'Disk3' is opt= imal
mfid5 on mfi0
mfid5: 1907200MB (3905945600 sectors) RAID volume 'DIsk5' is opt= imal
Trying to mount root from zfs:zroot/ROOT/14-2022_10_01-2233 []..= =2E
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
uhub5: 2 ports with 2 removable, self powered
Root mount waiting for: usbus2 usbus5
uhub3: 4 ports with 4 removable, self powered
uhub4: 4 ports with 4 removable, self powered
Root mount waiting for: usbus2 usbus5
ugen2.2: <vendor 0x0424 product 0x2514> at usbus2
uhub6 on uhub3
uhub6: <vendor 0x0424 product 0x2514, class 9/0, rev 2.00/0.0= 0, addr 2> on usbus2
uhub6: MTT enabled
ugen0.2: <Raritan D2CIM-VUSB> at usbus0
ukbd0 on uhub0
ukbd0: <Keybd+Abs+Rel Mouse> on usbus0
kbd2 at ukbd0
uhub6: 3 ports with 3 removable, self powered
ugen3.2: <American Power Conversion Back-UPS RS 1500MS FW:952= =2Ee3 .D USB FW:e3> at usbus3
ugen3.3: <Avocent USB Composite Device-0> at usbus3=
ukbd1 on uhub5
ukbd1: <Keyboard> on usbus3
kbd3 at ukbd1
Dual Console: Video Primary, Serial Secondary
<118>sysctl: net.inet.tcp.functions_default=3Drack at line= 55: No such file or directory
<118>Setting hostuuid: 4c4c4544-0035-4310-8048-c7c04f524c3= 1.
<118>Setting hostid: 0xd7b851f2.
<118>no pools available to import
<118>Starting file system checks:
<118>Mounting local filesystems:.
<118>Loading kernel modules:
aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS>
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
link_elf_obj: symbol nhgrp_get_group undefined
linker_load_file: /boot/kernel/netlink.ko - unsupported file typ= e
KLD linux_common.ko: depends on netlink - not available or versi= on mismatch
linker_load_file: /boot/kernel/linux_common.ko - unsupported fil= e type
KLD linux.ko: depends on linux_common - not available or version= mismatch
linker_load_file: /boot/kernel/linux.ko - unsupported file type<= /span>
<118>kldload: an error occurred while loading module linux= =2E Please check dmesg(8) for more details.
<118>/etc/rc: WARNING: Unable to load kernel module linux<= /span>
ichwd0: <Intel ICH9 watchdog timer> on isa0
efirtc0: <EFI Realtime Clock>
efirtc0: registered as a time-of-day clock, resolution 1.000000s=
ipmi0: <IPMI System Interface> on isa0
ipmi0: KCS mode found at io 0xca8 alignment 0x4 on isa
ipmi0: IPMI device rev. 0, firmware rev. 2.92, version 2.0, devi= ce support mask 0xdf
ipmi0: Number of channels 5
ipmi0: Attached watchdog
ipmi0: Establishing power cycle handler
hwpmc: SOFT/16/64/0x67<INT,USR,SYS,REA,WRI> TSC/1/64/0x20&= lt;REA> IAP/4/48/0x3ff<INT,USR,SYS,EDG,THR,REA,WRI,INV,QUA,PRC> IA= F/3/48/0x67<INT,USR,SYS,REA,WRI> UCP/8/48/0x3f8<EDG,THR,REA,WRI,IN= V,QUA,PRC> UCF/1/48/0x60<REA,WRI>
mfip0: <SCSI Passthrough Bus> on mfi0
ioat0: <TBG IOAT Ch0> mem 0xdf0e0000-0xdf0e3fff irq 51 at = device 22.0 on pci0
ioat0: Capabilities: 77<Block_Fill,Move_CRC,DCA,Marker_Skippi= ng,CRC,Page_Break>
ioat1: <TBG IOAT Ch1> mem 0xdf0e4000-0xdf0e7fff irq 52 at = device 22.1 on pci0
ioat1: Capabilities: 77<Block_Fill,Move_CRC,DCA,Marker_Skippi= ng,CRC,Page_Break>
ioat2: <TBG IOAT Ch2> mem 0xdf0e8000-0xdf0ebfff irq 53 at = device 22.2 on pci0
ioat2: Capabilities: 77<Block_Fill,Move_CRC,DCA,Marker_Skippi= ng,CRC,Page_Break>
ioat3: <TBG IOAT Ch3> mem 0xdf0ec000-0xdf0effff irq 54 at = device 22.3 on pci0
ioat3: Capabilities: 77<Block_Fill,Move_CRC,DCA,Marker_Skippi= ng,CRC,Page_Break>
ioat4: <TBG IOAT Ch4> mem 0xdf0f0000-0xdf0f3fff irq 51 at = device 22.4 on pci0
ioat4: Capabilities: 77<Block_Fill,Move_CRC,DCA,Marker_Skippi= ng,CRC,Page_Break>
ioat5: <TBG IOAT Ch5> mem 0xdf0f4000-0xdf0f7fff irq 52 at = device 22.5 on pci0
ioat5: Capabilities: 77<Block_Fill,Move_CRC,DCA,Marker_Skippi= ng,CRC,Page_Break>
ioat6: <TBG IOAT Ch6> mem 0xdf0f8000-0xdf0fbfff irq 53 at = device 22.6 on pci0
ioat6: Capabilities: 77<Block_Fill,Move_CRC,DCA,Marker_Skippi= ng,CRC,Page_Break>
ioat7: <TBG IOAT Ch7> mem 0xdf0fc000-0xdf0fffff irq 54 at = device 22.7 on pci0
ioat7: Capabilities: 77<Block_Fill,Move_CRC,DCA,Marker_Skippi= ng,CRC,Page_Break>
bce0: <QLogic NetXtreme II BCM5709 1000Base-T (C0)> mem 0x= d6000000-0xd7ffffff irq 36 at device 0.0 on pci1
<6>bce0: Using defaults for TSO: 65518/35/2048
<6>bce0: Ethernet address: a4:ba:db:29:66:95
bce0: ASIC (0x57092003); Rev (C0); Bus (PCIe x4, 2.5Gbps); B/C (= 7.10.0); Bufs (RX:8;TX:8;PG:32); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.13)
Coal (RX:6,6,18,18; TX:20,20,80,80)
bce1: <QLogic NetXtreme II BCM5709 1000Base-T (C0)> mem 0x= d8000000-0xd9ffffff irq 48 at device 0.1 on pci1
<6>bce1: Using defaults for TSO: 65518/35/2048
<6>bce1: Ethernet address: a4:ba:db:29:66:97
bce1: ASIC (0x57092003); Rev (C0); Bus (PCIe x4, 2.5Gbps); B/C (= 7.10.0); Bufs (RX:8;TX:8;PG:32); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.13)
Coal (RX:6,6,18,18; TX:20,20,80,80)
bce2: <QLogic NetXtreme II BCM5709 1000Base-T (C0)> mem 0x= da000000-0xdbffffff irq 32 at device 0.0 on pci2
<6>bce2: Using defaults for TSO: 65518/35/2048
<6>bce2: Ethernet address: a4:ba:db:29:66:99
bce2: ASIC (0x57092003); Rev (C0); Bus (PCIe x4, 2.5Gbps); B/C (= 7.10.0); Bufs (RX:8;TX:8;PG:32); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.13)
Coal (RX:6,6,18,18; TX:20,20,80,80)
bce3: <QLogic NetXtreme II BCM5709 1000Base-T (C0)> mem 0x= dc000000-0xddffffff irq 42 at device 0.1 on pci2
<6>bce3: Using defaults for TSO: 65518/35/2048
<6>bce3: Ethernet address: a4:ba:db:29:66:9b
bce3: ASIC (0x57092003); Rev (C0); Bus (PCIe x4, 2.5Gbps); B/C (= 7.10.0); Bufs (RX:8;TX:8;PG:32); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.13)
Coal (RX:6,6,18,18; TX:20,20,80,80)
miibus0: <MII bus> on bce0
brgphy0: <BCM5709 10/100/1000baseT PHY> PHY 1 on miibus0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1= 000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto= -flow
miibus1: <MII bus> on bce1
brgphy1: <BCM5709 10/100/1000baseT PHY> PHY 1 on miibus1
brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1= 000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto= -flow
miibus2: <MII bus> on bce2
brgphy2: <BCM5709 10/100/1000baseT PHY> PHY 1 on miibus2
brgphy2:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1= 000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto= -flow
miibus3: <MII bus> on bce3
brgphy3: <BCM5709 10/100/1000baseT PHY> PHY 1 on miibus3
brgphy3:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1= 000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto= -flow
ums0 on uhub5
ums0: <Mouse> on usbus3
ums0: 3 buttons and [Z] coordinates ID=3D0
ses0 at mfi0 bus 0 scbus0 target 32 lun 0
ses0: <DP BACKPLANE 1.07> Fixed Enclosure Services SPC-3 S= CSI device
ses0: 150.000MB/s transfers
ses0: SES Device
atapci0: <Intel ATA controller> port 0xec10-0xec17,0xec08-= 0xec0b,0xec18-0xec1f,0xec0c-0xec0f,0xec20-0xec2f,0xec30-0xec3f irq 23 at de= vice 31.2 on pci0
atapci0: failed to add ata child device
ata2: <ATA channel> at channel 0 on atapci0
device_attach: ata2 attach returned 6
<118>Autoloading module: acpi_wmi
acpi_wmi0: <ACPI-WMI mapping> on acpi0
acpi_wmi0: cannot find EC device
link_elf_obj: symbol nhgrp_get_group undefined
linker_load_file: /boot/kernel/netlink.ko - unsupported file typ= e
KLD linux_common.ko: depends on netlink - not available or versi= on mismatch
 
 
Fatal trap 12: page fault while in kernel mode
cpuid =3D 20; apic id =3D 12
fault virtual address =3D 0x0
fault code =3D supervisor read data, page not present
instruction pointer =3D 0x20:0xffffffff8231c132
stack pointer         =3D 0x28:0xfffffe02b29= 6a530
frame pointer         =3D 0x28:0xfffffe02b29= 6a560
code segment =3D base 0x0, limit 0xfffff, type 0x1b
=3D DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags =3D interrupt enabled, resume, IOPL =3D 0
current process =3D 232 (kldload)
rdi: fffff80128ca00b0 rsi: fffffe02b296a538 rdx:    &n= bsp;           0
rcx: ffffffff825dbde0  r8:         = ;       4  r9:           =    f3
rax:                0 rb= x: fffff80128b9ae80 rbp: fffffe02b296a560
r10:               f3 r1= 1: ff9094d191909292 r12: fffff80128b9aec0
r13: fffffe02b296a584 r14: fffffe02b296a584 r15: ffffffff825dbdd= 0
trap number =3D 12
panic: page fault
cpuid =3D 20
time =3D 1664682860
KDB: stack backtrace:
#0 0xffffffff80508e9d at kdb_backtrace+0x5d
#1 0xffffffff804bc741 at vpanic+0x151
#2 0xffffffff804bc5e3 at panic+0x43
#3 0xffffffff807afe59 at trap_fatal+0x409
#4 0xffffffff807afeaf at trap_pfault+0x4f
#5 0xffffffff80787d18 at calltrap+0x8
#6 0xffffffff8048fddb at linker_file_unload+0xdb
#7 0xffffffff807bc858 at link_elf_load_file+0x198
#8 0xffffffff8048f659 at linker_load_module+0x999
#9 0xffffffff804921b3 at linker_load_dependencies+0xe3
#10 0xffffffff807bd7b3 at link_elf_load_file+0x10f3
#11 0xffffffff8048f659 at linker_load_module+0x999
#12 0xffffffff80491351 at kern_kldload+0x141
#13 0xffffffff8049145b at sys_kldload+0x5b
#14 0xffffffff807b074c at amd64_syscall+0x10c
#15 0xffffffff8078862b at fast_syscall_common+0xf8
Uptime: 22s
Dumping 5423 out of 131021 MB:..1%..11%..21%..31%..41%..51%..61%= =2E.71%..81%..91%
 
__curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:5959 __a= sm("movq %%gs:%P1,%0" : "=3Dr" (td) : "n" (offsetof(struct pcpu,
(kgdb) bt full
#0  __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h= :59
        td =3D <optimized out><= /div>
#1  dump_savectx () at /usr/src/sys/kern/kern_shutdown.c:40= 5
No locals.
#2  0xffffffff804bc2e8 in dumpsys (di=3D0x0) at /usr/src/sy= s/x86/include/dump.h:87
No locals.
#3  doadump (textdump=3D<optimized out>) at /usr/src/= sys/kern/kern_shutdown.c:434
        di =3D 0x0
        error =3D <optimized out>
        coredump =3D 1
        di =3D <optimized out><= /div>
#4  kern_reboot (howto=3D260) at /usr/src/sys/kern/kern_shu= tdown.c:541
        once =3D 0
#5  0xffffffff804bc7ae in vpanic (fmt=3D<optimized out&g= t;, ap=3Dap@entry=3D0xfffffe02b296a380)
    at /usr/src/sys/kern/kern_shutdown.c:979
        buf =3D "page fault", '\000' <rep= eats 245 times>
        other_cpus =3D {__bits =3D {15728639= , 0, 0, 0}}
        td =3D 0xfffffe00e153d1e0
        bootopt =3D <unavailable>
        newpanic =3D <optimized out>
#6  0xffffffff804bc5e3 in panic (fmt=3D<unavailable>)= at /usr/src/sys/kern/kern_shutdown.c:903
        ap =3D {{gp_offset =3D 16, fp_offset= =3D 48, overflow_arg_area =3D 0xfffffe02b296a3b0,
            reg_save_area =3D 0xff= fffe02b296a350}}
#7  0xffffffff807afe59 in trap_fatal (frame=3D0xfffffe02b29= 6a470, eva=3D0) at /usr/src/sys/amd64/amd64/trap.c:955
        softseg =3D {ssd_base =3D 0, ssd_lim= it =3D 1048575, ssd_type =3D 27, ssd_dpl =3D 0, ssd_p =3D 1, ssd_long =3D 1= ,
          ssd_def32 =3D 0, ssd_gran =3D= 1}
        code =3D <optimized out>
        gdt =3D <optimized out>=
        ss =3D 40
        type =3D <optimized out>
        handled =3D <optimized out>
#8  0xffffffff807afeaf in trap_pfault (frame=3D0xfffffe02b2= 96a470, usermode=3Dfalse, signo=3D<optimized out>,
    ucode=3D<optimized out>) at /usr/src/sys/amd= 64/amd64/trap.c:763
        td =3D 0xfffffe00e153d1e0
        p =3D <optimized out>
        eva =3D 0
        map =3D <optimized out>=
        ftype =3D <optimized out>
        rv =3D <optimized out><= /div>
--Type <RET> for more, q to quit, c to continue without pa= ging--
#9  <signal handler called>
No locals.
#10 0xffffffff8231c132 in sdt_kld_unload_try (arg=3D<optimize= d out>, lf=3D<optimized out>,
    error=3D0xfffffe02b296a584) at /usr/src/sys/cddl/d= ev/sdt/sdt.c:345
        begin =3D <optimized out>
        end =3D <optimized out>=
        curr =3D 0xffffffff825dbdd0
        prov =3D 0xfffff80128b9aec0
        tmp =3D 0xfffff80128b9ae80
#11 0xffffffff8048fddb in linker_file_unload (file=3Dfile@entry= =3D0xfffff80101392a80, flags=3Dflags@entry=3D1)
    at /usr/src/sys/kern/kern_linker.c:675
        _ep =3D 0xfffff80128b9af00
        _t =3D 0xfffff80128b9af00
        _el =3D 0xfffff80117b87280
        error =3D 0
        mod =3D <optimized out>=
        next =3D <optimized out>
        ml =3D <optimized out><= /div>
        nextml =3D <optimized out>
        i =3D <optimized out>
        cp =3D <optimized out><= /div>
#12 0xffffffff807bc858 in link_elf_load_file (cls=3D<optimize= d out>, filename=3D<optimized out>,
    result=3D<optimized out>) at /usr/src/sys/ke= rn/link_elf_obj.c:1243
        td =3D 0xfffffe00e153d1e0
        error =3D 8
        shdr =3D <optimized out>
        lf =3D 0xfffff80101392a80
        mapsize =3D <optimized out>
        hdr =3D 0xfffff80117db0100
        nd =3D 0xfffff8011793f300
        flags =3D 1
        resid =3D <optimized out>
        ef =3D <optimized out><= /div>
        nbytes =3D <optimized out>
        i =3D <optimized out>
        symtabindex =3D <optimized out>= ;
        symstrindex =3D <optimized out>= ;
        nsym =3D <optimized out>
        shstrindex =3D <optimized out>=
--Type <RET> for more, q to quit, c to continue without pa= ging--
        alignmask =3D <optimized out><= /span>
        mapbase =3D <optimized out>
        pb =3D <optimized out><= /div>
        rl =3D <optimized out><= /div>
        ra =3D <optimized out><= /div>
        j =3D <optimized out>
        es =3D <optimized out><= /div>
#13 0xffffffff8048f659 in LINKER_LOAD_FILE (cls=3D0xffffffff80ad= b370 <link_elf_class>,
    result=3D0xfffffe02b296a800, filename=3D<optimi= zed out>) at ./linker_if.h:214
        _m =3D <optimized out><= /div>
        rc =3D <optimized out><= /div>
        _desc =3D <optimized out>
        _cep =3D <optimized out>
        _ce =3D <optimized out>=
#14 linker_load_file (filename=3D<optimized out>, result= =3D<optimized out>) at /usr/src/sys/kern/kern_linker.c:461
        lf =3D 0x0
        lc =3D 0xffffffff80adb370 <link_e= lf_class>
        foundfile =3D 0
        error =3D <optimized out>
        modules =3D <optimized out>
        _el =3D <optimized out>=
        _ep =3D <optimized out>=
        _t =3D <optimized out><= /div>
        _tid =3D <optimized out>
        _v =3D <optimized out><= /div>
        _v =3D <optimized out><= /div>
        _v =3D <optimized out><= /div>
#15 linker_load_module (kldname=3Dkldname@entry=3D0x0, modname= =3Dmodname@entry=3D0xffffffff825c292c "linux_common",
    parent=3Dparent@entry=3D0xfffff8011775e300,=
    verinfo=3Dverinfo@entry=3D0xffffffff825ce198 <_= linuxelf_depend_on_linux_common>, lfpp=3Dlfpp@entry=3D0x0)
    at /usr/src/sys/kern/kern_linker.c:2205
        pathname =3D <optimized out>
        filename =3D <optimized out>
        error =3D <optimized out>
        lfdep =3D <optimized out>
#16 0xffffffff804921b3 in linker_load_dependencies (lf=3D0x0, lf= @entry=3D0xfffff8011775e300)
    at /usr/src/sys/kern/kern_linker.c:2295
        error =3D 0
        start =3D <optimized out>
--Type <RET> for more, q to quit, c to continue without pa= ging--
        stop =3D <optimized out>
        mdp =3D 0xffffffff825cfab8 <__set= _modmetadata_set_sym__mod_metadata_md_linuxelf_on_linux_common>
        modname =3D 0xffffffff825c292c "linu= x_common"
        verinfo =3D 0xffffffff825ce198 <_= linuxelf_depend_on_linux_common>
        mp =3D <optimized out><= /div>
        nmodname =3D <optimized out>
        nmdp =3D <optimized out>
        nmp =3D <optimized out>=
        mod =3D <optimized out>=
        lfdep =3D <optimized out>
        ver =3D <optimized out>=
#17 0xffffffff807bd7b3 in link_elf_load_file (cls=3D<optimize= d out>,
    filename=3D0xfffff801018e3880 "/boot/kernel/linux.= ko", result=3D0xfffffe02b296ac10)
    at /usr/src/sys/kern/link_elf_obj.c:1212
        td =3D 0xfffffe00e153d1e0
        error =3D <optimized out>
        shdr =3D 0xfffff801367be800
        lf =3D 0xfffff80128ca00b0
        mapsize =3D <optimized out>
        hdr =3D 0xfffff80136929c00
        nd =3D 0xfffff80117958e00
        flags =3D 1
        resid =3D <optimized out>
        ef =3D 0xfffff80128ca00b0
        nbytes =3D <optimized out>
        i =3D <optimized out>
        symtabindex =3D <optimized out>= ;
        symstrindex =3D <optimized out>= ;
        nsym =3D <optimized out>
        shstrindex =3D <optimized out>=
        alignmask =3D <optimized out><= /span>
        mapbase =3D <optimized out>
        pb =3D <optimized out><= /div>
        rl =3D <optimized out><= /div>
        ra =3D <optimized out><= /div>
        j =3D <optimized out>
        es =3D <optimized out><= /div>
#18 0xffffffff8048f659 in LINKER_LOAD_FILE (cls=3D0xffffffff80ad= b370 <link_elf_class>,
    result=3D0xfffffe02b296ac10, filename=3D<optimi= zed out>) at ./linker_if.h:214
--Type <RET> for more, q to quit, c to continue without pa= ging--
        _m =3D <optimized out><= /div>
        rc =3D <optimized out><= /div>
        _desc =3D <optimized out>
        _cep =3D <optimized out>
        _ce =3D <optimized out>=
#19 linker_load_file (filename=3D<optimized out>, result= =3D<optimized out>) at /usr/src/sys/kern/kern_linker.c:461
        lf =3D 0x0
        lc =3D 0xffffffff80adb370 <link_e= lf_class>
        foundfile =3D 0
        error =3D <optimized out>
        modules =3D <optimized out>
        _el =3D <optimized out>=
        _ep =3D <optimized out>=
        _t =3D <optimized out><= /div>
        _tid =3D <optimized out>
        _v =3D <optimized out><= /div>
        _v =3D <optimized out><= /div>
        _v =3D <optimized out><= /div>
#20 linker_load_module (kldname=3Dkldname@entry=3D0x0, modname= =3Dmodname@entry=3D0xfffff8013632f800 "linux",
    parent=3Dparent@entry=3D0x0, verinfo=3Dverinfo@ent= ry=3D0x0, lfpp=3Dlfpp@entry=3D0xfffffe02b296ada0)
    at /usr/src/sys/kern/kern_linker.c:2205
        pathname =3D <optimized out>
        filename =3D <optimized out>
        error =3D <optimized out>
        lfdep =3D <optimized out>
#21 0xffffffff80491351 in kern_kldload (td=3Dtd@entry=3D0xfffffe= 00e153d1e0,
    file=3Dfile@entry=3D0xfffff8013632f800 "linux", fi= leid=3Dfileid@entry=3D0xfffffe02b296ade4)
    at /usr/src/sys/kern/kern_linker.c:1164
        error =3D <optimized out>
        modname =3D 0xfffff8013632f800 "linu= x"
        kldname =3D 0x0
        lf =3D 0xfffff80101633100
#22 0xffffffff8049145b in sys_kldload (td=3D0xfffffe00e153d1e0, = uap=3D<optimized out>)
    at /usr/src/sys/kern/kern_linker.c:1187
        pathname =3D 0xfffff8013632f800 "lin= ux"
        error =3D 0
        fileid =3D -2047
#23 0xffffffff807b074c in syscallenter (td=3D0xfffffe00e153d1e0)=
    at /usr/src/sys/amd64/amd64/../../kern/subr_syscal= l.c:189
--Type <RET> for more, q to quit, c to continue without pa= ging--
        p =3D 0xfffffe00dfa4b008
        sa =3D <optimized out><= /div>
        error =3D <optimized out>
        se =3D 0xffffffff80a6ffc0 <sysent= +9728>
        traced =3D <optimized out>
        sy_thr_static =3D <optimized out&= gt;
        _tid =3D <optimized out>
        _v =3D <optimized out><= /div>
        _v =3D <optimized out><= /div>
        _audit_entered =3D <optimized out= >
        _tid =3D <optimized out>
        _v =3D <optimized out><= /div>
        _v =3D <optimized out><= /div>
        _tid =3D <optimized out>
        _v =3D <optimized out><= /div>
        _v =3D <optimized out><= /div>
#24 amd64_syscall (td=3D0xfffffe00e153d1e0, traced=3D0) at /usr/= src/sys/amd64/amd64/trap.c:1200
        ksi =3D {ksi_link =3D {tqe_next =3D = 0xfffffe02b296af30, tqe_prev =3D 0xffffffff807af891 <trap+1809>},
          ksi_info =3D {si_signo =3D -1= 298747696, si_errno =3D -510, si_code =3D -2142476180, si_pid =3D -1,
            si_uid =3D 0, si_statu= s =3D 0, si_addr =3D 0x0, si_value =3D {sival_int =3D 0, sival_ptr =3D 0x0,=
              sigval_int =3D = 0, sigval_ptr =3D 0x0}, _reason =3D {_fault =3D {_trapno =3D 0}, _timer =3D= {_timerid =3D 0,
                _overrun= =3D 0}, _mesgq =3D {_mqd =3D 0}, _poll =3D {_band =3D 0}, _capsicum =3D {_= syscall =3D 0},
              __spare__ =3D {= __spare1__ =3D 0, __spare2__ =3D {0, 0, 0, 0, 0, 0, 0}}}}, ksi_flags =3D 0,=
          ksi_sigq =3D 0x0}
#25 <signal handler called>
No locals.
#26 0x000006977a82871a in ?? ()
No symbol table info available.
Backtrace stopped: Cannot access memory at address 0x69779933668=
 
Sure looks like netlink needs to be made available as a kld.....=
= -- 
Larry Rosenman        = ;             http://people.fre= ebsd.org/~ler
Phone: +1 214-642-9640         &= nbsp;       E-Mail: ler@F= reeBSD.org
US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
--=_c6fac64eb3c0d4c1aa3bdb883c568a3b--