From owner-netperf-users@freebsd.org Fri Sep 14 13:12:40 2018 Return-Path: Delivered-To: netperf-users@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6DC85108331A for ; Fri, 14 Sep 2018 13:12:40 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [IPv6:2607:f3e0:80:80::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smarthost2.sentex.ca", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CD04A81A8B; Fri, 14 Sep 2018 13:12:39 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (lava.sentex.ca [IPv6:2607:f3e0:0:5:0:0:0:11]) by smarthost2.sentex.ca (8.15.2/8.15.2) with ESMTPS id w8EDCcAD052865 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 14 Sep 2018 09:12:38 -0400 (EDT) (envelope-from mike@sentex.net) Received: from [192.168.43.26] (saphire3.sentex.net [192.168.43.26]) by lava.sentex.ca (8.15.2/8.15.2) with ESMTP id w8EDCbs6085784; Fri, 14 Sep 2018 09:12:37 -0400 (EDT) (envelope-from mike@sentex.net) Subject: Re: update of zoo to r338656 12.0 (was Re: zoo vs 12.0 (was: zoo vs 11.2-rc2) From: Mike Tancsa To: George Neville-Neil Cc: Paul Holes , Mateusz Guzik , netperf-users@FreeBSD.org, netperf-admin@FreeBSD.org References: <8ca07d41-b753-9741-49be-150d42197edc@sentex.net> Openpgp: preference=signencrypt Autocrypt: addr=mike@sentex.net; prefer-encrypt=mutual; keydata= xsBNBEzcA24BCACpwI/iqOrs0GfQSfhA1v6Z8AcXVeGsRyKEKUpxoOYxXWc2z3vndbYlIP6E YJeifzKhS/9E+VjhhICaepLHfw865TDTUPr5D0Ed+edSsKjlnDtb6hfNJC00P7eoiuvi85TW F/gAxRY269A5d856bYrzLbkWp2lKUR3Bg6NnORtflGzx9ZWAltZbjYjjRqegPv0EQNYcHqWo eRpXilEo1ahT6nmOU8V7yEvT2j4wlLcQ6qg7w+N/vcBvyd/weiwHU+vTQ9mT61x5/wUrQhdw 2gJHeQXeDGMJV49RT2EEz+QVxaf477eyWsdQzPVjAKRMT3BVdK8WvpYAEfBAbXmkboOxABEB AAHNHG1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5jYT7CwHgEEwECACIFAkzcA24CGwMGCwkI BwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEJXHwM2kc8rX+sMH/2V6pTBKsQ5mpWWLgs6wVP2k BC+6r/YKNXv9Rw/PrC6+9hTbgA+sSjJ+8gxsCbJsOQXZrxF0x3l9oYdYfuKcwdwXFX1/FS8p HfBeDkmlH+dI709xT9wgrR4dS5aMmKp0scPrXPIAKiYVOHjOlNItcLYTEEWEFBepheEVsgmk GrNbcrHwOx/u4igUQ8vcpyXPyUki+BsftPw8ZQvBU887igh0OxaCR8AurJppQ5UQd63r81cX E1ZjoFoWCaGK/SjPb/OhpYpu5swoZIhOxQbn7OtakYPsDd5t2A5KhvjI8BMTnd5Go+2xsCmr jlIEq8Bi29gCcfQUvNiClevi13ifmnnOwE0ETNwDbgEIALWGNJHRAhpd0A4vtd3G0oRqMBcM FGThQr3qORmEBTPPEomTdBaHcn+Xl+3YUvTBD/67/mutWBwgp2R5gQOSqcM7axvgMSHbKqBL 9sd1LsLw0UT2O5AYxv3EwzhG84pwRg3XcUqvWA4lA8tIj/1q4Jzi5qOkg1zxq4W9qr9oiYK5 bBR638JUvr3eHMaz/Nz+sDVFgwHmXZj3M6aE5Ce9reCGbvrae7H5D5PPvtT3r22X8SqfVAiO TFKedCf/6jbSOedPN931FJQYopj9P6b3m0nI3ZiCDVSqeyOAIBLzm+RBUIU3brzoxDhYR8pz CJc2sK8l6YjqivPakrD86bFDff8AEQEAAcLAXwQYAQIACQUCTNwDbgIbDAAKCRCVx8DNpHPK 1+iQB/99aqNtez9ZTBWELj269La8ntuRx6gCpzfPXfn6SDIfTItDxTh1hrdRVP5QNGGF5wus N4EMwXouskva1hbFX3Pv72csYSxxEJXjW16oV8WK4KjKXoskLg2RyRP4uXqL7Mp2ezNtVY5F 9nu3fj4ydpHCSaqKy5xd70A8D50PfZsFgkrsa5gdQhPiGGEdxhq/XSeAAnZ4uVLJKarH+mj5 MEhgZPEBWkGrbDZpezl9qbFcUem/uT9x8FYT/JIztMVh9qDcdP5tzANW5J7nvgXjska+VFGY ryZK4SPDczh74mn6GI/+RBi7OUzXXPgpPBrhS5FByjwCqjjsSpTjTds+NGIY Organization: Sentex Communications Message-ID: <7dc50e6a-191b-002d-9adf-df16e591c9da@sentex.net> Date: Fri, 14 Sep 2018 09:12:36 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <8ca07d41-b753-9741-49be-150d42197edc@sentex.net> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.83 X-BeenThere: netperf-users@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Announcements and discussions related to the netperf cluster. " List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Sep 2018 13:12:40 -0000 On 9/13/2018 2:27 PM, Mike Tancsa wrote: > FYI, > I will be upgrading zoo's kernel (only the kernel) to -r338656 tomorrow > morning 9am zoo time, 13:00 UTC. The boot to a 12 kernel is not looking good. It seems to have issue with the mfi controller. Everything seems to post fine up until the timeouts. Going to try a hard power cycle of the box and then revert to the RELENG_11 kernel /boot/kernel/kernel text=0x16b3020 data=0x1cc848+0x76c970 syms=[0x8+0x17d6d8+0x8+0x199fe6] /boot/entropy size=0x1000 /boot/kernel/zfs.ko size 0x3a8008 at 0x2652000 loading required module 'opensolaris' /boot/kernel/opensolaris.ko size 0xa4f0 at 0x29fb000 /boot/kernel/if_em.ko size 0x72b58 at 0x2a06000 Booting... GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb ---<>--- Copyright (c) 1992-2018 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 12.0-ALPHA5 r338656 amd64 FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on LLVM 6.0.1) VT(vga): resolution 640x480 module_register: cannot register pci/em from kernel; already loaded from if_em.ko Module pci/em failed to register: 17 module_register: cannot register pci/igb from kernel; already loaded from if_em.ko Module pci/igb failed to register: 17 CPU: Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz (2300.05-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7dfefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 274877906944 (262144 MB) avail memory = 267743858688 (255340 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 40 CPUs FreeBSD/SMP: 2 package(s) x 10 core(s) x 2 hardware threads random: unblocking device. ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard ioapic2 irqs 48-71 on motherboard Launching APs: 1 12 16 6 7 21 22 25 23 14 20 38 11 4 28 2 15 5 39 9 3 37 32 27 36 30 35 8 13 33 19 18 17 31 24 10 29 26 34 Timecounter "TSC-low" frequency 1150024636 Hz quality 1000 random: entropy device external interface [ath_hal] loaded module_register_init: MOD_LOAD (vesa, 0xffffffff81128820, 0) error 19 random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" kbd1 at kbdmux0 netmap: loaded module nexus0 vtvga0: on motherboard cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) cpu0: on acpi0 atrtc0: port 0x70-0x71,0x74-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: on acpi0 pci0: on pcib0 pci0: at device 8.2 (no driver attached) pci0: at device 9.2 (no driver attached) pci0: at device 11.1 (no driver attached) pci0: at device 11.2 (no driver attached) pci0: at device 16.1 (no driver attached) pci0: at device 16.6 (no driver attached) pci0: at device 18.1 (no driver attached) pci0: at device 18.5 (no driver attached) pcib1: on acpi0 pci1: on pcib1 pci1: at device 8.2 (no driver attached) pci1: at device 9.2 (no driver attached) pci1: at device 11.1 (no driver attached) pci1: at device 11.2 (no driver attached) pci1: at device 16.1 (no driver attached) pci1: at device 16.6 (no driver attached) pci1: at device 18.1 (no driver attached) pci1: at device 18.5 (no driver attached) acpi_syscontainer0: on acpi0 acpi_syscontainer1: on acpi0 acpi_syscontainer2: on acpi0 acpi_syscontainer3: on acpi0 pcib2: port 0xcf8-0xcff numa-domain 0 on acpi0 pci2: numa-domain 0 on pcib2 pcib3: irq 26 at device 1.0 numa-domain 0 on pci2 pci3: numa-domain 0 on pcib3 AVAGO MegaRAID SAS FreeBSD mrsas driver version: 06.712.04.00-fbsd mfi0: port 0x6000-0x60ff mem 0xc7300000-0xc730ffff,0xc7200000-0xc72fffff irq 26 at device 0.0 numa-domain 0 on pci3 mfi0: Using MSI mfi0: Megaraid SAS driver Ver 4.23 mfi0: FW MaxCmds = 928, limiting to 128 mfi0: MaxCmd = 928, Drv MaxCmd = 128, MaxSgl = 70, state = 0xb73c03a0 mfi0: 1309 (589112779s/0x0020/info) - Shutdown command received from host mfi0: 1310 (boot + 8s/0x0020/info) - Firmware initialization started (PCI ID 005d/1000/0809/15d9) mfi0: 1311 (boot + 8s/0x0020/info) - Firmware version 4.210.10-2910 mfi0: 1312 (boot + 10s/0x0020/info) - Package version 24.1.1-0001 mfi0: 1313 (boot + 10s/0x0020/info) - Board Revision mfi0: 1314 (boot + 33s/0x0002/info) - Inserted: PD 10(e0xfc/s6) mfi0: 1315 (boot + 33s/0x0002/info) - Inserted: PD 10(e0xfc/s6) Info: enclPd=fc, scsiType=0, portMap=00, sasAddr=4433221106000000,0000000000000000 mfi0: 1316 (boot + 33s/0x0002/info) - Inserted: PD 11(e0xfc/s7) pcib4: irq 32 at device 2.0 numa-domain 0 on pci2 mfi0: 1317 (boot + 33s/0x0002/info) - Inserted: PD 11(e0xfc/s7) Info: enclPd=fc, scsiType=0, portMap=01, sasAddr=4433221107000000,0000000000000000 mfi0: 1318 (boot + 34s/0x0002/info) - Unexpected sense: PD 10(e0xfc/s6) Path 4433221106000000, CDB: 28 00 3a 38 60 2f 00 00 01 00, Sense: 6/29/00 mfi0: 1319 (boot + 34s/0x0002/info) - Unexpected sense: PD 11(e0xfc/s7) Path 4433221107000000, CDB: 28 00 3a 38 60 2f 00 00 01 00, Spci4: numa-domain 0 on pcib4 ense: 6/29/00 mfi0: 1320 (590245504s/0x0020/info) - Time established as 09/14/18 13:05:04; (37 seconds since power on) mfi0: 1321 (590245504s/0x0002/info) - Unexpected sense: PD 10(e0xfc/s6) Path 4433221106000000, CDB: 2e 00 3a 38 3d fb 00 00 01 00, Sense: 6/29/00 mfi0: 1322 (590245505s/0x0020/WARN) - Patrol Read can't be started, as PDs are either not ONLINE, or are in a VD with an active process, or are in an excluded VD nvme0: mem 0xc7410000-0xc7413fff irq 32 at device 0.0 numa-domain 0 on pci4 pcib5: irq 40 at device 3.0 numa-domain 0 on pci2 pci5: numa-domain 0 on pcib5 ix0: port 0x5020-0x503f mem 0xc5c00000-0xc5dfffff,0xc5e04000-0xc5e07fff irq 40 at device 0.0 numa-domain 0 on pci5 ix0: using 2048 tx descriptors and 2048 rx descriptors ix0: msix_init qsets capped at 16 ix0: pxm cpus: 10 queue msgs: 63 admincnt: 1 ix0: using 10 rx queues 10 tx queues ix0: Using MSIX interrupts with 11 vectors ix0: allocated for 10 queues ix0: allocated for 10 rx queues ix0: Ethernet address: 0c:c4:7a:1f:2f:a0 ix0: PCI Express Bus: Speed 5.0GT/s Width x8 ix0: netmap queues/slots: TX 10/2048, RX 10/2048 ix1: port 0x5000-0x501f mem 0xc5a00000-0xc5bfffff,0xc5e00000-0xc5e03fff irq 44 at device 0.1 numa-domain 0 on pci5 ix1: using 2048 tx descriptors and 2048 rx descriptors ix1: msix_init qsets capped at 16 ix1: pxm cpus: 10 queue msgs: 63 admincnt: 1 ix1: using 10 rx queues 10 tx queues ix1: Using MSIX interrupts with 11 vectors ix1: allocated for 10 queues ix1: allocated for 10 rx queues ix1: Ethernet address: 0c:c4:7a:1f:2f:a1 ix1: PCI Express Bus: Speed 5.0GT/s Width x8 ix1: netmap queues/slots: TX 10/2048, RX 10/2048 pcib6: irq 40 at device 3.2 numa-domain 0 on pci2 pci6: numa-domain 0 on pcib6 ix2: port 0x4020-0x403f mem 0xc5600000-0xc57fffff,0xc5804000-0xc5807fff irq 42 at device 0.0 numa-domain 0 on pci6 ix2: using 2048 tx descriptors and 2048 rx descriptors ix2: msix_init qsets capped at 16 ix2: pxm cpus: 10 queue msgs: 63 admincnt: 1 ix2: using 10 rx queues 10 tx queues ix2: Using MSIX interrupts with 11 vectors ix2: allocated for 10 queues ix2: allocated for 10 rx queues ix2: Ethernet address: 0c:c4:7a:1f:2f:a2 ix2: PCI Express Bus: Speed 5.0GT/s Width x8 ix2: netmap queues/slots: TX 10/2048, RX 10/2048 ix3: port 0x4000-0x401f mem 0xc5400000-0xc55fffff,0xc5800000-0xc5803fff irq 45 at device 0.1 numa-domain 0 on pci6 ix3: using 2048 tx descriptors and 2048 rx descriptors ix3: msix_init qsets capped at 16 ix3: pxm cpus: 10 queue msgs: 63 admincnt: 1 ix3: using 10 rx queues 10 tx queues ix3: Using MSIX interrupts with 11 vectors ix3: allocated for 10 queues ix3: allocated for 10 rx queues ix3: Ethernet address: 0c:c4:7a:1f:2f:a3 ix3: PCI Express Bus: Speed 5.0GT/s Width x8 ix3: netmap queues/slots: TX 10/2048, RX 10/2048 pci2: at device 17.0 (no driver attached) ahci0: port 0x7110-0x7117,0x7100-0x7103,0x70f0-0x70f7,0x70e0-0x70e3,0x7020-0x703f mem 0xc7538000-0xc75387ff irq 16 at device 17.4 numa-domain 0 on pci2 ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahciem0: on ahci0 xhci0: mem 0xc7500000-0xc750ffff irq 19 at device 20.0 numa-domain 0 on pci2 xhci0: 32 bytes context size, 64-bit DMA usbus0 numa-domain 0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pci2: at device 22.0 (no driver attached) pci2: at device 22.1 (no driver attached) ehci0: mem 0xc7534000-0xc75343ff irq 18 at device 26.0 numa-domain 0 on pci2 usbus1: EHCI version 1.0 usbus1 numa-domain 0 on ehci0 usbus1: 480Mbps High Speed USB v2.0 pcib7: irq 16 at device 28.0 numa-domain 0 on pci2 pci7: numa-domain 0 on pcib7 pcib8: irq 16 at device 28.4 numa-domain 0 on pci2 pci8: numa-domain 0 on pcib8 pcib9: at device 0.0 numa-domain 0 on pci8 pci9: numa-domain 0 on pcib9 vgapci0: port 0x3000-0x307f mem 0xc6000000-0xc6ffffff,0xc7000000-0xc701ffff irq 16 at device 0.0 numa-domain 0 on pci9 vgapci0: Boot video device ehci1: mem 0xc7533000-0xc75333ff irq 18 at device 29.0 numa-domain 0 on pci2 usbus2: EHCI version 1.0 usbus2 numa-domain 0 on ehci1 usbus2: 480Mbps High Speed USB v2.0 isab0: at device 31.0 numa-domain 0 on pci2 isa0: numa-domain 0 on isab0 ahci1: port 0x7070-0x7077,0x7060-0x7063,0x7050-0x7057,0x7040-0x7043,0x7000-0x701f mem 0xc7532000-0xc75327ff irq 16 at device 31.2 numa-domain 0 on pci2 ahci1: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported ahcich4: at channel 0 on ahci1 ahcich5: at channel 1 on ahci1 ahcich6: at channel 2 on ahci1 ahcich7: at channel 3 on ahci1 ahcich8: at channel 4 on ahci1 ahcich9: at channel 5 on ahci1 ahciem1: on ahci1 pcib10: numa-domain 1 on acpi0 pci10: numa-domain 1 on pcib10 pcib11: irq 56 at device 2.0 numa-domain 1 on pci10 pci11: numa-domain 1 on pcib11 ix4: port 0xf020-0xf03f mem 0xfbc80000-0xfbcfffff,0xfbd04000-0xfbd07fff irq 56 at device 0.0 numa-domain 1 on pci11 ix4: using 2048 tx descriptors and 2048 rx descriptors ix4: msix_init qsets capped at 16 ix4: pxm cpus: 10 queue msgs: 63 admincnt: 1 ix4: using 10 rx queues 10 tx queues ix4: Using MSIX interrupts with 11 vectors ix4: allocated for 10 queues ix4: allocated for 10 rx queues ix4: Ethernet address: 00:1b:21:47:3b:24 ix4: PCI Express Bus: Speed 5.0GT/s Width x8 ix4: netmap queues/slots: TX 10/2048, RX 10/2048 ix5: port 0xf000-0xf01f mem 0xfbc00000-0xfbc7ffff,0xfbd00000-0xfbd03fff irq 60 at device 0.1 numa-domain 1 on pci11 ix5: using 2048 tx descriptors and 2048 rx descriptors ix5: msix_init qsets capped at 16 ix5: pxm cpus: 10 queue msgs: 63 admincnt: 1 ix5: using 10 rx queues 10 tx queues ix5: Using MSIX interrupts with 11 vectors ix5: allocated for 10 queues ix5: allocated for 10 rx queues ix5: Ethernet address: 00:1b:21:47:3b:25 ix5: PCI Express Bus: Speed 5.0GT/s Width x8 ix5: netmap queues/slots: TX 10/2048, RX 10/2048 pcib12: irq 64 at device 3.0 numa-domain 1 on pci10 pci12: numa-domain 1 on pcib12 em0: port 0xe000-0xe01f mem 0xfbec0000-0xfbedffff,0xfbe00000-0xfbe7ffff,0xfbee0000-0xfbee3fff irq 64 at device 0.0 numa-domain 1 on pci12 em0: attach_pre capping queues at 2 Current cap: 0x460b em0: using 1024 tx descriptors and 1024 rx descriptors em0: msix_init qsets capped at 2 em0: pxm cpus: 10 queue msgs: 4 admincnt: 1 em0: using 2 rx queues 2 tx queues em0: Using MSIX interrupts with 3 vectors em0: allocated for 2 tx_queues em0: allocated for 2 rx_queues em0: Ethernet address: 68:05:ca:06:ee:3e em0: netmap queues/slots: TX 2/1024, RX 2/1024 acpi_button0: on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: console (115200,n,8,1) uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xcf7ff,0xcf800-0xd07ff,0xd0800-0xd17ff pnpid ORM0000 on isa0 est0: on cpu0 p4tcc0: on cpu0 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1.000 msec ugen0.1: <0x8086 XHCI root HUB> at usbus0 ugen2.1: at usbus2 ugen1.1: at usbus1 uhub0: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 uhub1: on usbus2 uhub2: on usbus1 mfid0 numa-domain 0 on mfi0 ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: ACS-2 ATA SATA 3.x device ada0: Serial Number WD-WCC137TALF5K ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 2861588MB (5860533168 512 byte sectors) ada0: quirks=0x1<4K> ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 ada1: ACS-2 ATA SATA 3.x device ada1: Serial Number WD-WCC134FU4DRY ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 2861588MB (5860533168 512 byte sectors) ada1: quirks=0x1<4K> ada2 at ahcich2 bus 0 scbus2 target 0 lun 0 ada2: ACS-2 ATA SATA 3.x device ada2: Serial Number WD-WCC137TALKU5 ada2: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada2: Command Queueing enabled ada2: 2861588MB (5860533168 512 byte sectors) ada2: quirks=0x1<4K> mfid0: 476416MB (975699968 sectors) RAID volume 'SSD1' is optimal ada3 at ahcich4 bus 0 scbus5 target 0 lun 0 ada3: ACS-2 ATA SATA 3.x device ada3: Serial Number WD-WCC5D2UU1U3Y ada3: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada3: Command Queueing enabled ada3: 3815447MB (7814037168 512 byte sectors) ada3: quirks=0x1<4K> mfid1 numa-domain 0 on mfi0 mfid1: 476416MB (975699968 sectors) RAID volume 'SSD2' is optimal ada4 at ahcich5 bus 0 scbus6 target 0 lun 0 ada4: ACS-2 ATA SATA 3.x device ada4: Serial Number WD-WCC5D2UU1SF9 ada4: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada4: Command Queueing enabled ada4: 3815447MB (7814037168 512 byte sectors) ada4: quirks=0x1<4K> ada5 at ahcich6 bus 0 scbus7 target 0 lun 0 ada5: ACS-2 ATA SATA 3.x device ada5: Serial Number WD-WCC5D2UU18NK ada5: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada5: Command Queueing enabled ada5: 3815447MB (7814037168 512 byte sectors) ada5: quirks=0x1<4K> ada6 at ahcich7 bus 0 scbus8 target 0 lun 0 ada6: ACS-2 ATA SATA 3.x device ada6: Serial Number WD-WCC5D2UU1N29 ada6: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada6: Command Queueing enabled ada6: 3815447MB (7814037168 512 byte sectors) ada6: quirks=0x1<4K> ses0 at ahciem0 bus 0 scbus4 target 0 lun 0 ses0: SEMB S-E-S 2.00 device ses0: SEMB SES Device ses1 at ahciem1 bus 0 scbus11 target 0 lun 0 ses1: SEMB S-E-S 2.00 device ses1: SEMB SES Device uhub1: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered nvd0: NVMe namespace nvd0: 763097MB (1562824368 512 byte sectors) Trying to mount root from zfs:zroot/ROOT/default []... uhub0: 21 ports with 21 removable, self powered ugen1.2: at usbus1 uhub3 numa-domain 0 on uhub2 uhub3: on usbus1 ugen2.2: at usbus2 uhub4 numa-domain 0 on uhub1 uhub4: on usbus2 uhub3: 6 ports with 6 removable, self powered uhub4: 8 ports with 8 removable, self powered ugen0.2: at usbus0 ukbd0 numa-domain 0 on uhub0 ukbd0: on usbus0 kbd2 at ukbd0 ugen0.3: at usbus0 ugen0.4: at usbus0 uhub5 numa-domain 0 on uhub0 uhub5: on usbus0 uhub5: 4 ports with 3 removable, self powered ugen0.5: at usbus0 ukbd1 numa-domain 0 on uhub5 ukbd1: on usbus0 kbd3 at ukbd1 mfi0: COMMAND 0xfffffe017c17aec0 TIMEOUT AFTER 54 SECONDS mfi0: COMMAND 0xfffffe017c17aec0 TIMEOUT AFTER 84 SECONDS mfi0: COMMAND 0xfffffe017c17aec0 TIMEOUT AFTER 117 SECONDS mfi0: COMMAND 0xfffffe017c17aec0 TIMEOUT AFTER 147 SECONDS mfi0: COMMAND 0xfffffe017c17aec0 TIMEOUT AFTER 177 SECONDS mfi0: COMMAND 0xfffffe017c17aec0 TIMEOUT AFTER 207 SECONDS mfi0: COMMAND 0xfffffe017c17aec0 TIMEOUT AFTER 240 SECONDS mfi0: COMMAND 0xfffffe017c17aec0 TIMEOUT AFTER 271 SECONDS mfi0: COMMAND 0xfffffe017c17aec0 TIMEOUT AFTER 301 SECONDS mfi0: COMMAND 0xfffffe017c17aec0 TIMEOUT AFTER 332 SECONDS its getting stuck here. -- ------------------- Mike Tancsa, tel +1 519 651 3400 x203 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada