Date: Wed, 3 Apr 2024 12:17:50 +0100 From: Sarder Kamal <bsd.m0osk@gmail.com> To: freebsd-drivers@freebsd.org Subject: Error with mpi3mr driver Message-ID: <CAMnT8ii7QhGhiZe48FNmko8BBfLu72LLTmkYO6=jsFEDMyJKXA@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
--00000000000024a7f606152f6009 Content-Type: text/plain; charset="UTF-8" Good afternoon everyone We recently procured a Dell PowerEdge R760xd2 server with PERC H965i Adapter, which has 24 SAS disks attached to it. I have set the disks to be non-raid (even though the controller is marked as RAID), and I can see the disks when I load mpi3mr driver. I am also able to create a raidz2 array with the disks (3 arrays of 7 disks and 3 hot spares). I have setup bhyve on the server and the relevant zpool is used for virtual machines. All seem to work, until I try to write back data from within the VMs. For example, if I try to get the ports using git clone -- it hangs after a while. There are no errors, it just hangs. I usually run the commands from a screen session, so I can open another screen session, but most of the commands that need to access the disks hangs -- even Ctrl+C does not work. In the meanwhile, in the host OS, I see these errors Apr 3 10:43:58 newDellServer kernel: mpi3mr0: bus_dmamap_load(): retcode = 36 Apr 3 10:43:58 newDellServer kernel: mpi3mr0: request load in progress Apr 3 10:43:58 newDellServer kernel: func: mpi3mr_action_scsiio line: 1143 Build SGLs failed Apr 3 10:43:58 newDellServer kernel: (da6:mpi3mr0:0:281:0): WRITE(10). CDB: 2a 00 15 6d f6 b8 00 07 e0 00 Apr 3 10:43:58 newDellServer kernel: (da6:mpi3mr0:0:281:0): CAM status: CCB request was invalid Apr 3 10:43:58 newDellServer kernel: (da6:mpi3mr0:0:281:0): Error 22, Unretryable error at different times it picks on different disks, but the essence of the error is the same My google search brought me to this earlier bug report https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=272469 but I'm not sure if this is relevant -- I do not have that many cores (the server has only 96 cores in two physical CPUs). here are some of the details from the system # cat /etc/os-release NAME=FreeBSD VERSION="14.0-RELEASE-p6" VERSION_ID="14.0" ID=freebsd ANSI_COLOR="0;31" PRETTY_NAME="FreeBSD 14.0-RELEASE-p6" CPE_NAME="cpe:/o:freebsd:freebsd:14.0" HOME_URL="https://FreeBSD.org/" BUG_REPORT_URL="https://bugs.FreeBSD.org/" # dmesg | grep SMP FreeBSD/SMP: Multiprocessor System Detected: 96 CPUs FreeBSD/SMP: 2 package(s) x 24 core(s) x 2 hardware threads # sysctl kern.sched.topology_spec kern.sched.topology_spec: <groups> <group level="1" cache-level="0"> <cpu count="96" mask="ffffffffffffffff,ffffffff,0,0,0,0,0,0,0,0,0,0,0,0,0,0">0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95</cpu> <children> <group level="2" cache-level="3"> <cpu count="48" mask="ffffffffffff,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47</cpu> <flags><flag name="NODE">NUMA node</flag></flags> <children> <group level="3" cache-level="2"> <cpu count="2" mask="3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">0, 1</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="c,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">2, 3</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="30,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">4, 5</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="c0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">6, 7</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="300,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">8, 9</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="c00,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">10, 11</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="3000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">12, 13</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="c000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">14, 15</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="30000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">16, 17</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="c0000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">18, 19</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="300000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">20, 21</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="c00000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">22, 23</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="3000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">24, 25</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="c000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">26, 27</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="30000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">28, 29</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="c0000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">30, 31</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="300000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">32, 33</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="c00000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">34, 35</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="3000000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">36, 37</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="c000000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">38, 39</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="30000000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">40, 41</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="c0000000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">42, 43</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="300000000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">44, 45</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="c00000000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">46, 47</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> </children> </group> <group level="2" cache-level="3"> <cpu count="48" mask="ffff000000000000,ffffffff,0,0,0,0,0,0,0,0,0,0,0,0,0,0">48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95</cpu> <flags><flag name="NODE">NUMA node</flag></flags> <children> <group level="3" cache-level="2"> <cpu count="2" mask="3000000000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">48, 49</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="c000000000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">50, 51</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="30000000000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">52, 53</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="c0000000000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">54, 55</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="300000000000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">56, 57</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="c00000000000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">58, 59</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="3000000000000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">60, 61</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="c000000000000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">62, 63</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="0,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0">64, 65</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="0,c,0,0,0,0,0,0,0,0,0,0,0,0,0,0">66, 67</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="0,30,0,0,0,0,0,0,0,0,0,0,0,0,0,0">68, 69</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="0,c0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">70, 71</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="0,300,0,0,0,0,0,0,0,0,0,0,0,0,0,0">72, 73</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="0,c00,0,0,0,0,0,0,0,0,0,0,0,0,0,0">74, 75</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="0,3000,0,0,0,0,0,0,0,0,0,0,0,0,0,0">76, 77</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="0,c000,0,0,0,0,0,0,0,0,0,0,0,0,0,0">78, 79</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="0,30000,0,0,0,0,0,0,0,0,0,0,0,0,0,0">80, 81</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="0,c0000,0,0,0,0,0,0,0,0,0,0,0,0,0,0">82, 83</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="0,300000,0,0,0,0,0,0,0,0,0,0,0,0,0,0">84, 85</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="0,c00000,0,0,0,0,0,0,0,0,0,0,0,0,0,0">86, 87</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="0,3000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0">88, 89</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="0,c000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0">90, 91</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="0,30000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0">92, 93</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> <group level="3" cache-level="2"> <cpu count="2" mask="0,c0000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0">94, 95</cpu> <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags> </group> </children> </group> </children> </group> </groups> # zpool status -v pool: Data02 state: ONLINE config: NAME STATE READ WRITE CKSUM Data02 ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 da0 ONLINE 0 0 0 da1 ONLINE 0 0 0 da2 ONLINE 0 0 0 da3 ONLINE 0 0 0 da4 ONLINE 0 0 0 da5 ONLINE 0 0 0 da6 ONLINE 0 0 0 raidz2-1 ONLINE 0 0 0 da7 ONLINE 0 0 0 da8 ONLINE 0 0 0 da9 ONLINE 0 0 0 da10 ONLINE 0 0 0 da11 ONLINE 0 0 0 da12 ONLINE 0 0 0 da13 ONLINE 0 0 0 raidz2-2 ONLINE 0 0 0 da14 ONLINE 0 0 0 da15 ONLINE 0 0 0 da16 ONLINE 0 0 0 da17 ONLINE 0 0 0 da18 ONLINE 0 0 0 da19 ONLINE 0 0 0 da20 ONLINE 0 0 0 spares da21 AVAIL da22 AVAIL da23 AVAIL errors: No known data errors # pcicon -lv (truncated) mpi3mr0@pci0:80:0:0: class=0x010400 rev=0x01 hdr=0x00 vendor=0x1000 device=0x00a5 subvendor=0x1028 subdevice=0x2114 vendor = 'Broadcom / LSI' device = 'Fusion-MPT 24GSAS/PCIe SAS40xx' class = mass storage subclass = RAID I am happy to provide more data/details if that helps, please let me also know how to obtain them. Any suggestion to remedy the situation is greatly appreciated. Kind regards SK --00000000000024a7f606152f6009 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Good afternoon everyone<div><br></div><div>We recently pro= cured a Dell=C2=A0PowerEdge R760xd2 server with=C2=A0PERC H965i Adapter, wh= ich has 24 SAS disks attached to it. I have=C2=A0set the disks to be non-ra= id (even though the controller is marked as RAID), and I can see the disks = when I load mpi3mr driver.</div><div><br></div><div>I am also able to creat= e a raidz2 array with the disks (3 arrays of 7 disks and 3 hot spares). I h= ave setup bhyve on the server and the relevant zpool is used for virtual ma= chines.</div><div><br></div><div>All seem to work, until I try to write bac= k data from within the VMs. For example, if I try to get the ports using gi= t clone -- it hangs after a while. There are no errors, it just hangs. I us= ually run the commands from a screen session, so I can open another screen = session, but most of the commands that need to access the disks hangs -- ev= en Ctrl+C does not work.</div><div><br></div><div>In the meanwhile, in the = host OS, I see these errors</div><div>Apr =C2=A03 10:43:58 newDellServer ke= rnel: mpi3mr0: bus_dmamap_load(): retcode =3D 36<br>Apr =C2=A03 10:43:58 ne= wDellServer kernel: mpi3mr0: request load in progress<br>Apr =C2=A03 10:43:= 58 newDellServer kernel: func: mpi3mr_action_scsiio line: 1143 Build SGLs f= ailed<br>Apr =C2=A03 10:43:58 newDellServer kernel: (da6:mpi3mr0:0:281:0): = WRITE(10). CDB: 2a 00 15 6d f6 b8 00 07 e0 00 <br>Apr =C2=A03 10:43:58 newD= ellServer kernel: (da6:mpi3mr0:0:281:0): CAM status: CCB request was invali= d<br>Apr =C2=A03 10:43:58 newDellServer kernel: (da6:mpi3mr0:0:281:0): Erro= r 22, Unretryable error<br><br></div><div>at different times it picks on di= fferent disks, but the essence of the error is the same</div><div><br></div= ><div>My google search brought me to this earlier bug report</div><div><a h= ref=3D"https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D272469">https://= bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D272469</a><br></div><div>but I&= #39;m not sure if this is relevant -- I do not have that many cores (the se= rver has only 96 cores in two physical CPUs).=C2=A0</div><div><br></div><di= v>here are some of the details from the system</div><div># cat /etc/os-rele= ase <br>NAME=3DFreeBSD<br>VERSION=3D"14.0-RELEASE-p6"<br>VERSION_= ID=3D"14.0"<br>ID=3Dfreebsd<br>ANSI_COLOR=3D"0;31"<br>P= RETTY_NAME=3D"FreeBSD 14.0-RELEASE-p6"<br>CPE_NAME=3D"cpe:/o= :freebsd:freebsd:14.0"<br>HOME_URL=3D"<a href=3D"https://FreeBSD.= org/">https://FreeBSD.org/</a>"<br>BUG_REPORT_URL=3D"<a href=3D"h= ttps://bugs.FreeBSD.org/">https://bugs.FreeBSD.org/</a>"<br><br># dmes= g | grep SMP<br>FreeBSD/SMP: Multiprocessor System Detected: 96 CPUs<br>Fre= eBSD/SMP: 2 package(s) x 24 core(s) x 2 hardware threads<br><br># sysctl ke= rn.sched.topology_spec<br>kern.sched.topology_spec: <groups><br>=C2= =A0<group level=3D"1" cache-level=3D"0"><br>=C2= =A0 <cpu count=3D"96" mask=3D"ffffffffffffffff,ffffffff,0= ,0,0,0,0,0,0,0,0,0,0,0,0,0">0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 1= 2, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, = 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49,= 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68= , 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 8= 7, 88, 89, 90, 91, 92, 93, 94, 95</cpu><br>=C2=A0 <children><br= >=C2=A0 =C2=A0<group level=3D"2" cache-level=3D"3"&g= t;<br>=C2=A0 =C2=A0 <cpu count=3D"48" mask=3D"fffffffffff= f,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, = 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29,= 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47<= /cpu><br>=C2=A0 =C2=A0 <flags><flag name=3D"NODE">= NUMA node</flag></flags><br>=C2=A0 =C2=A0 <children><br>= =C2=A0 =C2=A0 =C2=A0<group level=3D"3" cache-level=3D"2&q= uot;><br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D"2" mask=3D"= ;3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">0, 1</cpu><br>=C2=A0 =C2=A0= =C2=A0 <flags><flag name=3D"THREAD">THREAD group<= /flag><flag name=3D"SMT">SMT group</flag></flag= s><br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0<grou= p level=3D"3" cache-level=3D"2"><br>=C2=A0 =C2=A0 = =C2=A0 <cpu count=3D"2" mask=3D"c,0,0,0,0,0,0,0,0,0,0,0,0= ,0,0,0">2, 3</cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags><f= lag name=3D"THREAD">THREAD group</flag><flag name=3D&= quot;SMT">SMT group</flag></flags><br>=C2=A0 =C2=A0 =C2= =A0</group><br>=C2=A0 =C2=A0 =C2=A0<group level=3D"3" ca= che-level=3D"2"><br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D"= ;2" mask=3D"30,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">4, 5</cp= u><br>=C2=A0 =C2=A0 =C2=A0 <flags><flag name=3D"THREAD&quo= t;>THREAD group</flag><flag name=3D"SMT">SMT group= </flag></flags><br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0= =C2=A0 =C2=A0<group level=3D"3" cache-level=3D"2"&g= t;<br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D"2" mask=3D"c0,0,= 0,0,0,0,0,0,0,0,0,0,0,0,0,0">6, 7</cpu><br>=C2=A0 =C2=A0 =C2= =A0 <flags><flag name=3D"THREAD">THREAD group</fla= g><flag name=3D"SMT">SMT group</flag></flags>= ;<br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0<group le= vel=3D"3" cache-level=3D"2"><br>=C2=A0 =C2=A0 =C2=A0= <cpu count=3D"2" mask=3D"300,0,0,0,0,0,0,0,0,0,0,0,0,0,0= ,0">8, 9</cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags><flag = name=3D"THREAD">THREAD group</flag><flag name=3D"= ;SMT">SMT group</flag></flags><br>=C2=A0 =C2=A0 =C2=A0&= lt;/group><br>=C2=A0 =C2=A0 =C2=A0<group level=3D"3" cache-= level=3D"2"><br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D"2&q= uot; mask=3D"c00,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">10, 11</cpu= ><br>=C2=A0 =C2=A0 =C2=A0 <flags><flag name=3D"THREAD"= ;>THREAD group</flag><flag name=3D"SMT">SMT group&= lt;/flag></flags><br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 = =C2=A0 =C2=A0<group level=3D"3" cache-level=3D"2">= ;<br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D"2" mask=3D"3000,0= ,0,0,0,0,0,0,0,0,0,0,0,0,0,0">12, 13</cpu><br>=C2=A0 =C2=A0 = =C2=A0 <flags><flag name=3D"THREAD">THREAD group</= flag><flag name=3D"SMT">SMT group</flag></flags= ><br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0<group= level=3D"3" cache-level=3D"2"><br>=C2=A0 =C2=A0 =C2= =A0 <cpu count=3D"2" mask=3D"c000,0,0,0,0,0,0,0,0,0,0,0,0= ,0,0,0">14, 15</cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags><= ;flag name=3D"THREAD">THREAD group</flag><flag name= =3D"SMT">SMT group</flag></flags><br>=C2=A0 =C2=A0= =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0<group level=3D"3"= ; cache-level=3D"2"><br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D&= quot;2" mask=3D"30000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">16, = 17</cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags><flag name=3D"T= HREAD">THREAD group</flag><flag name=3D"SMT">= SMT group</flag></flags><br>=C2=A0 =C2=A0 =C2=A0</group><= br>=C2=A0 =C2=A0 =C2=A0<group level=3D"3" cache-level=3D"= 2"><br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D"2" mask=3D&q= uot;c0000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">18, 19</cpu><br>=C2= =A0 =C2=A0 =C2=A0 <flags><flag name=3D"THREAD">THREAD= group</flag><flag name=3D"SMT">SMT group</flag>= ;</flags><br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0 =C2= =A0<group level=3D"3" cache-level=3D"2"><br>=C2= =A0 =C2=A0 =C2=A0 <cpu count=3D"2" mask=3D"300000,0,0,0,0= ,0,0,0,0,0,0,0,0,0,0,0">20, 21</cpu><br>=C2=A0 =C2=A0 =C2=A0 = <flags><flag name=3D"THREAD">THREAD group</flag>= ;<flag name=3D"SMT">SMT group</flag></flags><br= >=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0<group level= =3D"3" cache-level=3D"2"><br>=C2=A0 =C2=A0 =C2=A0 &l= t;cpu count=3D"2" mask=3D"c00000,0,0,0,0,0,0,0,0,0,0,0,0,0,0= ,0">22, 23</cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags><fla= g name=3D"THREAD">THREAD group</flag><flag name=3D&qu= ot;SMT">SMT group</flag></flags><br>=C2=A0 =C2=A0 =C2= =A0</group><br>=C2=A0 =C2=A0 =C2=A0<group level=3D"3" ca= che-level=3D"2"><br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D"= ;2" mask=3D"3000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">24, 25= </cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags><flag name=3D"THR= EAD">THREAD group</flag><flag name=3D"SMT">SM= T group</flag></flags><br>=C2=A0 =C2=A0 =C2=A0</group><br= >=C2=A0 =C2=A0 =C2=A0<group level=3D"3" cache-level=3D"2&= quot;><br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D"2" mask=3D&quo= t;c000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">26, 27</cpu><br>=C2= =A0 =C2=A0 =C2=A0 <flags><flag name=3D"THREAD">THREAD= group</flag><flag name=3D"SMT">SMT group</flag>= ;</flags><br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0 =C2= =A0<group level=3D"3" cache-level=3D"2"><br>=C2= =A0 =C2=A0 =C2=A0 <cpu count=3D"2" mask=3D"30000000,0,0,0= ,0,0,0,0,0,0,0,0,0,0,0,0">28, 29</cpu><br>=C2=A0 =C2=A0 =C2= =A0 <flags><flag name=3D"THREAD">THREAD group</fla= g><flag name=3D"SMT">SMT group</flag></flags>= ;<br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0<group le= vel=3D"3" cache-level=3D"2"><br>=C2=A0 =C2=A0 =C2=A0= <cpu count=3D"2" mask=3D"c0000000,0,0,0,0,0,0,0,0,0,0,0,= 0,0,0,0">30, 31</cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags>&l= t;flag name=3D"THREAD">THREAD group</flag><flag name= =3D"SMT">SMT group</flag></flags><br>=C2=A0 =C2=A0= =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0<group level=3D"3"= ; cache-level=3D"2"><br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D&= quot;2" mask=3D"300000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">= 32, 33</cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags><flag name=3D&qu= ot;THREAD">THREAD group</flag><flag name=3D"SMT"= >SMT group</flag></flags><br>=C2=A0 =C2=A0 =C2=A0</group&= gt;<br>=C2=A0 =C2=A0 =C2=A0<group level=3D"3" cache-level=3D&q= uot;2"><br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D"2" mask= =3D"c00000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">34, 35</cpu>= ;<br>=C2=A0 =C2=A0 =C2=A0 <flags><flag name=3D"THREAD"&g= t;THREAD group</flag><flag name=3D"SMT">SMT group<= /flag></flags><br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2= =A0 =C2=A0<group level=3D"3" cache-level=3D"2"><b= r>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D"2" mask=3D"300000000= 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">36, 37</cpu><br>=C2=A0 =C2= =A0 =C2=A0 <flags><flag name=3D"THREAD">THREAD group&= lt;/flag><flag name=3D"SMT">SMT group</flag></f= lags><br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0<g= roup level=3D"3" cache-level=3D"2"><br>=C2=A0 =C2=A0= =C2=A0 <cpu count=3D"2" mask=3D"c000000000,0,0,0,0,0,0,0= ,0,0,0,0,0,0,0,0">38, 39</cpu><br>=C2=A0 =C2=A0 =C2=A0 <fl= ags><flag name=3D"THREAD">THREAD group</flag><f= lag name=3D"SMT">SMT group</flag></flags><br>=C2= =A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0<group level=3D&q= uot;3" cache-level=3D"2"><br>=C2=A0 =C2=A0 =C2=A0 <cpu= count=3D"2" mask=3D"30000000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0= ,0">40, 41</cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags><fla= g name=3D"THREAD">THREAD group</flag><flag name=3D&qu= ot;SMT">SMT group</flag></flags><br>=C2=A0 =C2=A0 =C2= =A0</group><br>=C2=A0 =C2=A0 =C2=A0<group level=3D"3" ca= che-level=3D"2"><br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D"= ;2" mask=3D"c0000000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">42= , 43</cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags><flag name=3D"= ;THREAD">THREAD group</flag><flag name=3D"SMT"&g= t;SMT group</flag></flags><br>=C2=A0 =C2=A0 =C2=A0</group>= ;<br>=C2=A0 =C2=A0 =C2=A0<group level=3D"3" cache-level=3D&quo= t;2"><br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D"2" mask=3D= "300000000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">44, 45</cpu>= ;<br>=C2=A0 =C2=A0 =C2=A0 <flags><flag name=3D"THREAD"&g= t;THREAD group</flag><flag name=3D"SMT">SMT group<= /flag></flags><br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2= =A0 =C2=A0<group level=3D"3" cache-level=3D"2"><b= r>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D"2" mask=3D"c00000000= 000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">46, 47</cpu><br>=C2=A0 =C2= =A0 =C2=A0 <flags><flag name=3D"THREAD">THREAD group&= lt;/flag><flag name=3D"SMT">SMT group</flag></f= lags><br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0 </childr= en><br>=C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0<group level=3D&qu= ot;2" cache-level=3D"3"><br>=C2=A0 =C2=A0 <cpu count= =3D"48" mask=3D"ffff000000000000,ffffffff,0,0,0,0,0,0,0,0,0,= 0,0,0,0,0">48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, = 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80,= 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95</cpu><br>= =C2=A0 =C2=A0 <flags><flag name=3D"NODE">NUMA node<= ;/flag></flags><br>=C2=A0 =C2=A0 <children><br>=C2=A0 =C2=A0= =C2=A0<group level=3D"3" cache-level=3D"2"><br>= =C2=A0 =C2=A0 =C2=A0 <cpu count=3D"2" mask=3D"30000000000= 00,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">48, 49</cpu><br>=C2=A0 =C2= =A0 =C2=A0 <flags><flag name=3D"THREAD">THREAD group&= lt;/flag><flag name=3D"SMT">SMT group</flag></f= lags><br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0<g= roup level=3D"3" cache-level=3D"2"><br>=C2=A0 =C2=A0= =C2=A0 <cpu count=3D"2" mask=3D"c000000000000,0,0,0,0,0,= 0,0,0,0,0,0,0,0,0,0">50, 51</cpu><br>=C2=A0 =C2=A0 =C2=A0 <= ;flags><flag name=3D"THREAD">THREAD group</flag>&l= t;flag name=3D"SMT">SMT group</flag></flags><br>= =C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0<group level= =3D"3" cache-level=3D"2"><br>=C2=A0 =C2=A0 =C2=A0 &l= t;cpu count=3D"2" mask=3D"30000000000000,0,0,0,0,0,0,0,0,0,0= ,0,0,0,0,0">52, 53</cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags>= ;<flag name=3D"THREAD">THREAD group</flag><flag na= me=3D"SMT">SMT group</flag></flags><br>=C2=A0 =C2= =A0 =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0<group level=3D"3&q= uot; cache-level=3D"2"><br>=C2=A0 =C2=A0 =C2=A0 <cpu count= =3D"2" mask=3D"c0000000000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0&= quot;>54, 55</cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags><flag n= ame=3D"THREAD">THREAD group</flag><flag name=3D"= SMT">SMT group</flag></flags><br>=C2=A0 =C2=A0 =C2=A0&l= t;/group><br>=C2=A0 =C2=A0 =C2=A0<group level=3D"3" cache-l= evel=3D"2"><br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D"2&qu= ot; mask=3D"300000000000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">56,= 57</cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags><flag name=3D"= THREAD">THREAD group</flag><flag name=3D"SMT">= ;SMT group</flag></flags><br>=C2=A0 =C2=A0 =C2=A0</group>= <br>=C2=A0 =C2=A0 =C2=A0<group level=3D"3" cache-level=3D"= ;2"><br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D"2" mask=3D&= quot;c00000000000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">58, 59</cpu&= gt;<br>=C2=A0 =C2=A0 =C2=A0 <flags><flag name=3D"THREAD"= >THREAD group</flag><flag name=3D"SMT">SMT group&l= t;/flag></flags><br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 = =C2=A0 =C2=A0<group level=3D"3" cache-level=3D"2">= ;<br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D"2" mask=3D"300000= 0000000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">60, 61</cpu><br>=C2= =A0 =C2=A0 =C2=A0 <flags><flag name=3D"THREAD">THREAD= group</flag><flag name=3D"SMT">SMT group</flag>= ;</flags><br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0 =C2= =A0<group level=3D"3" cache-level=3D"2"><br>=C2= =A0 =C2=A0 =C2=A0 <cpu count=3D"2" mask=3D"c0000000000000= 00,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0">62, 63</cpu><br>=C2=A0 =C2= =A0 =C2=A0 <flags><flag name=3D"THREAD">THREAD group&= lt;/flag><flag name=3D"SMT">SMT group</flag></f= lags><br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0<g= roup level=3D"3" cache-level=3D"2"><br>=C2=A0 =C2=A0= =C2=A0 <cpu count=3D"2" mask=3D"0,3,0,0,0,0,0,0,0,0,0,0,= 0,0,0,0">64, 65</cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags>&l= t;flag name=3D"THREAD">THREAD group</flag><flag name= =3D"SMT">SMT group</flag></flags><br>=C2=A0 =C2=A0= =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0<group level=3D"3"= ; cache-level=3D"2"><br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D&= quot;2" mask=3D"0,c,0,0,0,0,0,0,0,0,0,0,0,0,0,0">66, 67&l= t;/cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags><flag name=3D"THREA= D">THREAD group</flag><flag name=3D"SMT">SMT = group</flag></flags><br>=C2=A0 =C2=A0 =C2=A0</group><br>= =C2=A0 =C2=A0 =C2=A0<group level=3D"3" cache-level=3D"2&q= uot;><br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D"2" mask=3D"= ;0,30,0,0,0,0,0,0,0,0,0,0,0,0,0,0">68, 69</cpu><br>=C2=A0 =C2= =A0 =C2=A0 <flags><flag name=3D"THREAD">THREAD group&= lt;/flag><flag name=3D"SMT">SMT group</flag></f= lags><br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0<g= roup level=3D"3" cache-level=3D"2"><br>=C2=A0 =C2=A0= =C2=A0 <cpu count=3D"2" mask=3D"0,c0,0,0,0,0,0,0,0,0,0,0= ,0,0,0,0">70, 71</cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags>&= lt;flag name=3D"THREAD">THREAD group</flag><flag name= =3D"SMT">SMT group</flag></flags><br>=C2=A0 =C2=A0= =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0<group level=3D"3"= ; cache-level=3D"2"><br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D&= quot;2" mask=3D"0,300,0,0,0,0,0,0,0,0,0,0,0,0,0,0">72, 73= </cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags><flag name=3D"THR= EAD">THREAD group</flag><flag name=3D"SMT">SM= T group</flag></flags><br>=C2=A0 =C2=A0 =C2=A0</group><br= >=C2=A0 =C2=A0 =C2=A0<group level=3D"3" cache-level=3D"2&= quot;><br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D"2" mask=3D&quo= t;0,c00,0,0,0,0,0,0,0,0,0,0,0,0,0,0">74, 75</cpu><br>=C2=A0 = =C2=A0 =C2=A0 <flags><flag name=3D"THREAD">THREAD gro= up</flag><flag name=3D"SMT">SMT group</flag><= ;/flags><br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0&l= t;group level=3D"3" cache-level=3D"2"><br>=C2=A0 =C2= =A0 =C2=A0 <cpu count=3D"2" mask=3D"0,3000,0,0,0,0,0,0,0,= 0,0,0,0,0,0,0">76, 77</cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags= ><flag name=3D"THREAD">THREAD group</flag><flag= name=3D"SMT">SMT group</flag></flags><br>=C2=A0 = =C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0<group level=3D"= 3" cache-level=3D"2"><br>=C2=A0 =C2=A0 =C2=A0 <cpu cou= nt=3D"2" mask=3D"0,c000,0,0,0,0,0,0,0,0,0,0,0,0,0,0">= ;78, 79</cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags><flag name=3D&q= uot;THREAD">THREAD group</flag><flag name=3D"SMT"= ;>SMT group</flag></flags><br>=C2=A0 =C2=A0 =C2=A0</group= ><br>=C2=A0 =C2=A0 =C2=A0<group level=3D"3" cache-level=3D&= quot;2"><br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D"2" mask= =3D"0,30000,0,0,0,0,0,0,0,0,0,0,0,0,0,0">80, 81</cpu><br= >=C2=A0 =C2=A0 =C2=A0 <flags><flag name=3D"THREAD">TH= READ group</flag><flag name=3D"SMT">SMT group</fla= g></flags><br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0 = =C2=A0<group level=3D"3" cache-level=3D"2"><br>= =C2=A0 =C2=A0 =C2=A0 <cpu count=3D"2" mask=3D"0,c0000,0,0= ,0,0,0,0,0,0,0,0,0,0,0,0">82, 83</cpu><br>=C2=A0 =C2=A0 =C2= =A0 <flags><flag name=3D"THREAD">THREAD group</fla= g><flag name=3D"SMT">SMT group</flag></flags>= ;<br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0<group le= vel=3D"3" cache-level=3D"2"><br>=C2=A0 =C2=A0 =C2=A0= <cpu count=3D"2" mask=3D"0,300000,0,0,0,0,0,0,0,0,0,0,0,= 0,0,0">84, 85</cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags><= flag name=3D"THREAD">THREAD group</flag><flag name=3D= "SMT">SMT group</flag></flags><br>=C2=A0 =C2=A0 = =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0<group level=3D"3"= cache-level=3D"2"><br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D&q= uot;2" mask=3D"0,c00000,0,0,0,0,0,0,0,0,0,0,0,0,0,0">86, = 87</cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags><flag name=3D"T= HREAD">THREAD group</flag><flag name=3D"SMT">= SMT group</flag></flags><br>=C2=A0 =C2=A0 =C2=A0</group><= br>=C2=A0 =C2=A0 =C2=A0<group level=3D"3" cache-level=3D"= 2"><br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D"2" mask=3D&q= uot;0,3000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0">88, 89</cpu><br>= =C2=A0 =C2=A0 =C2=A0 <flags><flag name=3D"THREAD">THR= EAD group</flag><flag name=3D"SMT">SMT group</flag= ></flags><br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0 = =C2=A0<group level=3D"3" cache-level=3D"2"><br>= =C2=A0 =C2=A0 =C2=A0 <cpu count=3D"2" mask=3D"0,c000000,0= ,0,0,0,0,0,0,0,0,0,0,0,0,0">90, 91</cpu><br>=C2=A0 =C2=A0 =C2= =A0 <flags><flag name=3D"THREAD">THREAD group</fla= g><flag name=3D"SMT">SMT group</flag></flags>= ;<br>=C2=A0 =C2=A0 =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0<group le= vel=3D"3" cache-level=3D"2"><br>=C2=A0 =C2=A0 =C2=A0= <cpu count=3D"2" mask=3D"0,30000000,0,0,0,0,0,0,0,0,0,0,= 0,0,0,0">92, 93</cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags>&l= t;flag name=3D"THREAD">THREAD group</flag><flag name= =3D"SMT">SMT group</flag></flags><br>=C2=A0 =C2=A0= =C2=A0</group><br>=C2=A0 =C2=A0 =C2=A0<group level=3D"3"= ; cache-level=3D"2"><br>=C2=A0 =C2=A0 =C2=A0 <cpu count=3D&= quot;2" mask=3D"0,c0000000,0,0,0,0,0,0,0,0,0,0,0,0,0,0">9= 4, 95</cpu><br>=C2=A0 =C2=A0 =C2=A0 <flags><flag name=3D&quo= t;THREAD">THREAD group</flag><flag name=3D"SMT"&= gt;SMT group</flag></flags><br>=C2=A0 =C2=A0 =C2=A0</group&g= t;<br>=C2=A0 =C2=A0 </children><br>=C2=A0 =C2=A0</group><br>=C2= =A0 </children><br>=C2=A0</group><br></groups><br><br># z= pool status -v<br>=C2=A0 pool: Data02<br>=C2=A0state: ONLINE<br>config:<br>= <br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 NAME =C2=A0 =C2=A0 =C2=A0 =C2=A0STATE =C2= =A0 =C2=A0 READ WRITE CKSUM<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 Data02 =C2=A0 = =C2=A0 =C2=A0ONLINE =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0<= br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 raidz2-0 =C2=A0ONLINE =C2=A0 =C2=A0 = =C2=A0 0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 da0 =C2=A0 =C2=A0 ONLINE =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0 = =C2=A0 =C2=A0 0<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 da1 =C2=A0 =C2= =A0 ONLINE =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0<br>=C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 da2 =C2=A0 =C2=A0 ONLINE =C2=A0 =C2=A0 = =C2=A0 0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 da3 =C2=A0 =C2=A0 ONLINE =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0 = =C2=A0 =C2=A0 0<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 da4 =C2=A0 =C2= =A0 ONLINE =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0<br>=C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 da5 =C2=A0 =C2=A0 ONLINE =C2=A0 =C2=A0 = =C2=A0 0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 da6 =C2=A0 =C2=A0 ONLINE =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0 = =C2=A0 =C2=A0 0<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 raidz2-1 =C2=A0ONLINE= =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0<br>=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 da7 =C2=A0 =C2=A0 ONLINE =C2=A0 =C2=A0 =C2=A0 0= =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 da8 =C2=A0 =C2=A0 ONLINE =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0 =C2=A0 = =C2=A0 0<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 da9 =C2=A0 =C2=A0 ONL= INE =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0<br>=C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 da10 =C2=A0 =C2=A0ONLINE =C2=A0 =C2=A0 =C2=A0 = 0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 da11 =C2=A0 =C2=A0ONLINE =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0 =C2=A0 = =C2=A0 0<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 da12 =C2=A0 =C2=A0ONL= INE =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0<br>=C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 da13 =C2=A0 =C2=A0ONLINE =C2=A0 =C2=A0 =C2=A0 = 0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 rai= dz2-2 =C2=A0ONLINE =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0<b= r>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 da14 =C2=A0 =C2=A0ONLINE =C2=A0= =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0<br>=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 da15 =C2=A0 =C2=A0ONLINE =C2=A0 =C2=A0 =C2=A0 0 =C2=A0= =C2=A0 0 =C2=A0 =C2=A0 0<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 da16= =C2=A0 =C2=A0ONLINE =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0= <br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 da17 =C2=A0 =C2=A0ONLINE =C2= =A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0<br>=C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 da18 =C2=A0 =C2=A0ONLINE =C2=A0 =C2=A0 =C2=A0 0 =C2= =A0 =C2=A0 0 =C2=A0 =C2=A0 0<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 d= a19 =C2=A0 =C2=A0ONLINE =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0 =C2=A0 =C2= =A0 0<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 da20 =C2=A0 =C2=A0ONLINE= =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 0<br>=C2=A0 =C2=A0 = =C2=A0 =C2=A0 spares<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 da21 =C2=A0 =C2= =A0 =C2=A0AVAIL =C2=A0 <br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 da22 =C2=A0 = =C2=A0 =C2=A0AVAIL =C2=A0 <br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 da23 =C2= =A0 =C2=A0 =C2=A0AVAIL =C2=A0 <br><br>errors: No known data errors<br><br><= /div><div><br></div><div># pcicon -lv (truncated)<br>mpi3mr0@pci0:80:0:0: = =C2=A0 =C2=A0class=3D0x010400 rev=3D0x01 hdr=3D0x00 vendor=3D0x1000 device= =3D0x00a5 subvendor=3D0x1028 subdevice=3D0x2114<br>=C2=A0 =C2=A0 vendor =C2= =A0 =C2=A0 =3D 'Broadcom / LSI'<br>=C2=A0 =C2=A0 device =C2=A0 =C2= =A0 =3D 'Fusion-MPT 24GSAS/PCIe SAS40xx'<br>=C2=A0 =C2=A0 class =C2= =A0 =C2=A0 =C2=A0=3D mass storage<br>=C2=A0 =C2=A0 subclass =C2=A0 =3D RAID= <br><br></div><div><br></div><div>I am happy to provide more data/details i= f that helps, please let me also know how to obtain them.</div><div><br></d= iv><div>Any suggestion to remedy the situation is greatly appreciated.</div= ><div><br></div><div>Kind regards</div><div>SK</div><div><br></div></div> --00000000000024a7f606152f6009--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAMnT8ii7QhGhiZe48FNmko8BBfLu72LLTmkYO6=jsFEDMyJKXA>