Date: Mon, 13 Feb 2023 14:15:23 -0300 From: Nicolas Goldman <nicolasgoldman07@gmail.com> To: freebsd-questions@freebsd.org Subject: Kernel panics when given a high workload Message-ID: <CAPrTifWgxNqE%2B_OVgHm2fYANe2w-cKLt0Y_nULyxeTHOiCT=bA@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
--000000000000c44f8405f497fefb Content-Type: text/plain; charset="UTF-8" Hello! Good Monday to all the FreeBSD community. I am working on the FreeBSD kernel for my university thesis. The idea is to make changes to the FreeBSD short-time scheduler so that all its operations are based on the concept of Petri Nets. We already have the modeling of said scheduler and the first tests running. I am currently running into a problem that has left me out of ideas. Very randomly, the kernel throws page faults and reboots the OS. With my thesis partner, we tried to see when this problem happened but didn't find any pattern to reproduce it. We could see it mostly when the processor is heavily loaded, but as I said previously, only in some simulations. I leave some information about the logs found; any help is appreciated. Code: uname -a FreeBSD pielihueso 13.1-RELEASE FreeBSD 13.1-RELEASE DrudiGoldmanPI/update_petriNetScheduler-13.1.0-n250157-cb2e622cf22d PI_KERNELCONF amd64 Differences between PI_KERNELCONF and GENERIC are: 1. We are working on the 4BSD scheduler instead of the ULE: Code: # options SCHED_ULE # ULE scheduler options SCHED_4BSD # 4BSD scheduler 2. We added some debugger options: Code: options DDB options GDB options KDB_UNATTENDED ------- Code: *cd /usr/obj/usr/src/amd64.amd64/sys/PI_KERNELCONF/ kgdb kernel.debug /var/crash/vmcore.last* GNU gdb (GDB) 12.1 [GDB v12.1 for FreeBSD] Copyright (C) 2022 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>; This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-portbld-freebsd13.1". Type "show configuration" for configuration details. For bug reporting instructions, please see: <https://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /boot/kernel/kernel... Reading symbols from /usr/lib/debug//boot/kernel/kernel.debug... Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0xffffffffffffffa8 fault code = supervisor write data, page not present instruction pointer = 0x20:0xffffffff80ca0822 stack pointer = 0x28:0xfffffe00cd879b60 frame pointer = 0x28:0x0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 922 (sshd) trap number = 12 panic: page fault cpuid = 1 time = 1676286448 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00cd879920 vpanic() at vpanic+0x17f/frame 0xfffffe00cd879970 panic() at panic+0x43/frame 0xfffffe00cd8799d0 trap_fatal() at trap_fatal+0x385/frame 0xfffffe00cd879a30 trap_pfault() at trap_pfault+0x4f/frame 0xfffffe00cd879a90 calltrap() at calltrap+0x8/frame 0xfffffe00cd879a90 --- trap 0xc, rip = 0xffffffff80ca0822, rsp = 0xfffffe00cd879b60, rbp = 0 --- kern_select() at kern_select+0x942 Uptime: 34s Dumping 371 out of 8085 MB:..5%..13%..22%..31%..44%..52%..61%..74%..82%..91% __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 55 __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (offsetof(struct pcpu, *(kgdb) where* #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 #1 doadump (textdump=textdump@entry=1) at /usr/src/sys/kern/kern_shutdown.c:399 #2 0xffffffff80c2f521 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:487 #3 0xffffffff80c2f99e in vpanic (fmt=0xffffffff811dfeea "%s", ap=<optimized out>) at /usr/src/sys/kern/kern_shutdown.c:920 #4 0xffffffff80c2f7a3 in panic (fmt=<unavailable>) at /usr/src/sys/kern/kern_shutdown.c:844 #5 0xffffffff810d7855 in trap_fatal (frame=0xfffffe00cd879aa0, eva=18446744073709551528) at /usr/src/sys/amd64/amd64/trap.c:944 #6 0xffffffff810d78af in trap_pfault (frame=0xfffffe00cd879aa0, usermode=false, signo=<optimized out>, ucode=<optimized out>) at /usr/src/sys/amd64/amd64/trap.c:763 #7 <signal handler called> #8 0xffffffff80ca0822 in selrescan (td=<error reading variable: Cannot access memory at address 0xffffffffffffffd0>, ibits=<optimized out>, obits=<optimized out>) at /usr/src/sys/kern/sys_generic.c:1325 #9 kern_select (td=<optimized out>, nd=<error reading variable: Cannot access memory at address 0xffffffffffffff90>, fd_in=<optimized out>, fd_ou=<optimized out>, fd_ex=<optimized out>, tvp=<optimized out>, abi_nfdbits=<error reading variable: Cannot access memory at address 0x10>) at /usr/src/sys/kern/sys_generic.c:1206 Backtrace stopped: Cannot access memory at address 0x8 Code: # nm -n /boot/kernel/kernel | grep 0xffffffff80ca0822 # nm -n /boot/kernel/kernel | grep 0xffffffff80ca0822 # nm -n /boot/kernel/kernel | grep 0xffffffff80c # nm -n /boot/kernel/kernel | grep 0xffffffff # nm -n /boot/kernel/kernel | grep 0xfffff # nm -n /boot/kernel/kernel | grep 0xff # nm -n /boot/kernel/kernel | grep 0x ffffffff80388c30 t cam_compat_handle_0x17 ffffffff803891e0 t cam_compat_handle_0x18 ffffffff803895f0 t cam_compat_handle_0x19 ffffffff80389730 t cam_compat_translate_dev_match_0x18 ffffffff80aba6a0 t xl_check_maddr_90xB ffffffff80aba6f0 t xl_check_maddr_90x ffffffff80abad90 t xl_txeof_90xB ffffffff80abb090 t xl_start_90xB_locked ffffffff80ebac80 t mlx5e_fec_mask_10x_25x_handler ffffffff80ebb050 t mlx5e_fec_avail_10x_25x_handler ffffffff80ebb0f0 t mlx5e_fec_mask_50x_handler ffffffff80ebb4e0 t mlx5e_fec_avail_50x_handler ffffffff810af6c0 T Xint0x80_syscall_pti ffffffff810af740 T Xint0x80_syscall ffffffff810af743 t int0x80_syscall_common ffffffff817fe180 r db_inst_0f0x ffffffff8180cc50 r mouse10x14_120 ffffffff8180cd40 r mouse10x16_50 ffffffff8180cd90 r mouse10x16_75 ffffffff8180cde0 r mouse10x16_90 ffffffff8180ce30 r mouse10x16_100 ffffffff8180ce80 r mouse10x16_120 ffffffff8180ced0 r mouse10x16_133 We also tried with dtrace but with no luck. Do you have other recommendations on how we can keep debugging this issue? We know it's something we broke on the scheduler because the generic kernel is working decently. P.S.: If someone is interested in how we implemented the Petri Net for the scheduler, contact me through the mail, and I can give you the paper we are working on. --000000000000c44f8405f497fefb Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><span style=3D"color:rgb(20,20,20);font-family:"Segoe= UI","Helvetica Neue",Helvetica,Roboto,Oxygen,Ubuntu,Cantare= ll,"Fira Sans","Droid Sans",sans-serif;font-size:16px">= Hello! Good Monday to all the FreeBSD community.</span><br style=3D"box-siz= ing:border-box;margin-top:0px;color:rgb(20,20,20);font-family:"Segoe U= I","Helvetica Neue",Helvetica,Roboto,Oxygen,Ubuntu,Cantarell= ,"Fira Sans","Droid Sans",sans-serif;font-size:16px"><s= pan style=3D"color:rgb(20,20,20);font-family:"Segoe UI","Hel= vetica Neue",Helvetica,Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans&= quot;,"Droid Sans",sans-serif;font-size:16px">I am working on the= FreeBSD kernel for my university thesis. The idea is to make changes to th= e FreeBSD short-time scheduler so that all its operations are based on the = concept of Petri Nets. We already have the modeling of said scheduler and t= he first tests running.</span><br style=3D"box-sizing:border-box;color:rgb(= 20,20,20);font-family:"Segoe UI","Helvetica Neue",Helve= tica,Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans","Droid Sans&= quot;,sans-serif;font-size:16px"><br style=3D"box-sizing:border-box;color:r= gb(20,20,20);font-family:"Segoe UI","Helvetica Neue",He= lvetica,Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans","Droid Sa= ns",sans-serif;font-size:16px"><span style=3D"color:rgb(20,20,20);font= -family:"Segoe UI","Helvetica Neue",Helvetica,Roboto,Ox= ygen,Ubuntu,Cantarell,"Fira Sans","Droid Sans",sans-ser= if;font-size:16px">I am currently running into a problem that has left me o= ut of ideas. Very randomly, the kernel throws=C2=A0</span><span style=3D"bo= x-sizing:border-box;font-weight:700;color:rgb(20,20,20);font-family:"S= egoe UI","Helvetica Neue",Helvetica,Roboto,Oxygen,Ubuntu,Can= tarell,"Fira Sans","Droid Sans",sans-serif;font-size:16= px">page faults</span><span style=3D"color:rgb(20,20,20);font-family:"= Segoe UI","Helvetica Neue",Helvetica,Roboto,Oxygen,Ubuntu,Ca= ntarell,"Fira Sans","Droid Sans",sans-serif;font-size:1= 6px">=C2=A0and reboots the OS. With my thesis partner, we tried to see when= this problem happened but didn't find any pattern to reproduce it. We = could see it mostly when the processor is heavily loaded, but as I said pre= viously, only in some simulations.</span><br style=3D"box-sizing:border-box= ;color:rgb(20,20,20);font-family:"Segoe UI","Helvetica Neue&= quot;,Helvetica,Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans","= Droid Sans",sans-serif;font-size:16px"><br style=3D"box-sizing:border-= box;color:rgb(20,20,20);font-family:"Segoe UI","Helvetica Ne= ue",Helvetica,Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans",&qu= ot;Droid Sans",sans-serif;font-size:16px"><span style=3D"color:rgb(20,= 20,20);font-family:"Segoe UI","Helvetica Neue",Helvetic= a,Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans","Droid Sans&quo= t;,sans-serif;font-size:16px">I leave some information about the logs found= ; any help is appreciated.</span><br style=3D"box-sizing:border-box;color:r= gb(20,20,20);font-family:"Segoe UI","Helvetica Neue",He= lvetica,Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans","Droid Sa= ns",sans-serif;font-size:16px"><br style=3D"box-sizing:border-box;colo= r:rgb(20,20,20);font-family:"Segoe UI","Helvetica Neue"= ,Helvetica,Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans","Droid= Sans",sans-serif;font-size:16px"><div style=3D"box-sizing:border-box;= display:flow-root;margin:0.5em 0px;background:rgb(246,246,246);border-width= :1px 1px 1px 3px;border-style:solid;border-color:rgb(224,224,224) rgb(224,2= 24,224) rgb(224,224,224) rgb(217,217,217);color:rgb(20,20,20);font-family:&= quot;Segoe UI","Helvetica Neue",Helvetica,Roboto,Oxygen,Ubun= tu,Cantarell,"Fira Sans","Droid Sans",sans-serif;font-s= ize:16px"><div style=3D"box-sizing:border-box;padding:6px 10px;font-size:13= px;color:rgb(102,0,0);background:rgb(251,251,251)">Code:</div><div dir=3D"l= tr" style=3D"box-sizing:border-box;padding:6px 10px;max-height:70vh;overflo= w:auto"><pre dir=3D"ltr" style=3D"white-space:pre-wrap;box-sizing:border-bo= x;font-family:Monaco,Menlo,Consolas,"Roboto Mono","Andale Mo= no","Ubuntu Mono",monospace;font-size:13px;margin-top:0px;ma= rgin-bottom:0px;padding:10px 10px 0px;line-height:1.4;overflow:auto;directi= on:ltr"><code style=3D"box-sizing:border-box;font-family:inherit;font-size:= 1em">uname -a FreeBSD pielihueso 13.1-RELEASE FreeBSD 13.1-RELEASE DrudiGoldmanPI/update_= petriNetScheduler-13.1.0-n250157-cb2e622cf22d PI_KERNELCONF amd64</code></p= re></div></div><div><br style=3D"box-sizing:border-box"><span style=3D"colo= r:rgb(20,20,20);font-family:"Segoe UI","Helvetica Neue"= ,Helvetica,Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans","Droid= Sans",sans-serif;font-size:16px">Differences between PI_KERNELCONF an= d GENERIC are:</span><br style=3D"box-sizing:border-box;color:rgb(20,20,20)= ;font-family:"Segoe UI","Helvetica Neue",Helvetica,Robo= to,Oxygen,Ubuntu,Cantarell,"Fira Sans","Droid Sans",san= s-serif;font-size:16px"><br style=3D"box-sizing:border-box;color:rgb(20,20,= 20);font-family:"Segoe UI","Helvetica Neue",Helvetica,R= oboto,Oxygen,Ubuntu,Cantarell,"Fira Sans","Droid Sans",= sans-serif;font-size:16px"><span style=3D"color:rgb(20,20,20);font-family:&= quot;Segoe UI","Helvetica Neue",Helvetica,Roboto,Oxygen,Ubun= tu,Cantarell,"Fira Sans","Droid Sans",sans-serif;font-s= ize:16px">1. We are working on the 4BSD scheduler instead of the ULE:</span= ><br style=3D"box-sizing:border-box;color:rgb(20,20,20);font-family:"S= egoe UI","Helvetica Neue",Helvetica,Roboto,Oxygen,Ubuntu,Can= tarell,"Fira Sans","Droid Sans",sans-serif;font-size:16= px"><div style=3D"box-sizing:border-box;display:flow-root;margin:0.5em 0px;= background:rgb(246,246,246);border-width:1px 1px 1px 3px;border-style:solid= ;border-color:rgb(224,224,224) rgb(224,224,224) rgb(224,224,224) rgb(217,21= 7,217);color:rgb(20,20,20);font-family:"Segoe UI","Helvetica= Neue",Helvetica,Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans",= "Droid Sans",sans-serif;font-size:16px"><div style=3D"box-sizing:= border-box;padding:6px 10px;font-size:13px;color:rgb(102,0,0);background:rg= b(251,251,251)">Code:</div><div dir=3D"ltr" style=3D"box-sizing:border-box;= padding:6px 10px;max-height:70vh;overflow:auto"><pre dir=3D"ltr" style=3D"w= hite-space:pre-wrap;box-sizing:border-box;font-family:Monaco,Menlo,Consolas= ,"Roboto Mono","Andale Mono","Ubuntu Mono",mo= nospace;font-size:13px;margin-top:0px;margin-bottom:0px;padding:10px 10px 0= px;line-height:1.4;overflow:auto;direction:ltr"><code style=3D"box-sizing:b= order-box;font-family:inherit;font-size:1em"># options SCHED_ULE = # ULE scheduler options SCHED_4BSD # 4BSD scheduler</code></pre></div></div><br = style=3D"box-sizing:border-box;color:rgb(20,20,20);font-family:"Segoe = UI","Helvetica Neue",Helvetica,Roboto,Oxygen,Ubuntu,Cantarel= l,"Fira Sans","Droid Sans",sans-serif;font-size:16px"><= span style=3D"color:rgb(20,20,20);font-family:"Segoe UI","He= lvetica Neue",Helvetica,Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans= ","Droid Sans",sans-serif;font-size:16px">2. We added some d= ebugger options:</span><br style=3D"box-sizing:border-box;color:rgb(20,20,2= 0);font-family:"Segoe UI","Helvetica Neue",Helvetica,Ro= boto,Oxygen,Ubuntu,Cantarell,"Fira Sans","Droid Sans",s= ans-serif;font-size:16px"><div style=3D"box-sizing:border-box;display:flow-= root;margin:0.5em 0px;background:rgb(246,246,246);border-width:1px 1px 1px = 3px;border-style:solid;border-color:rgb(224,224,224) rgb(224,224,224) rgb(2= 24,224,224) rgb(217,217,217);color:rgb(20,20,20);font-family:"Segoe UI= ","Helvetica Neue",Helvetica,Roboto,Oxygen,Ubuntu,Cantarell,= "Fira Sans","Droid Sans",sans-serif;font-size:16px"><di= v style=3D"box-sizing:border-box;padding:6px 10px;font-size:13px;color:rgb(= 102,0,0);background:rgb(251,251,251)">Code:</div><div dir=3D"ltr" style=3D"= box-sizing:border-box;padding:6px 10px;max-height:70vh;overflow:auto"><pre = dir=3D"ltr" style=3D"white-space:pre-wrap;box-sizing:border-box;font-family= :Monaco,Menlo,Consolas,"Roboto Mono","Andale Mono",&quo= t;Ubuntu Mono",monospace;font-size:13px;margin-top:0px;margin-bottom:0= px;padding:10px 10px 0px;line-height:1.4;overflow:auto;direction:ltr"><code= style=3D"box-sizing:border-box;font-family:inherit;font-size:1em">options = DDB options GDB options KDB_UNATTENDED</code></pre></div></div>-------=C2=A0<br styl= e=3D"box-sizing:border-box;color:rgb(20,20,20);font-family:"Segoe UI&q= uot;,"Helvetica Neue",Helvetica,Roboto,Oxygen,Ubuntu,Cantarell,&q= uot;Fira Sans","Droid Sans",sans-serif;font-size:16px"><div = style=3D"box-sizing:border-box;display:flow-root;margin:0.5em 0px;backgroun= d:rgb(246,246,246);border-width:1px 1px 1px 3px;border-style:solid;border-c= olor:rgb(224,224,224) rgb(224,224,224) rgb(224,224,224) rgb(217,217,217);co= lor:rgb(20,20,20);font-family:"Segoe UI","Helvetica Neue&quo= t;,Helvetica,Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans","Dro= id Sans",sans-serif;font-size:16px"><div style=3D"box-sizing:border-bo= x;padding:6px 10px;font-size:13px;color:rgb(102,0,0);background:rgb(251,251= ,251)">Code:</div><div dir=3D"ltr" style=3D"box-sizing:border-box;padding:6= px 10px;max-height:70vh;overflow:auto"><pre dir=3D"ltr" style=3D"white-spac= e:pre-wrap;box-sizing:border-box;font-family:Monaco,Menlo,Consolas,"Ro= boto Mono","Andale Mono","Ubuntu Mono",monospace;f= ont-size:13px;margin-top:0px;margin-bottom:0px;padding:10px 10px 0px;line-h= eight:1.4;overflow:auto;direction:ltr"><code style=3D"box-sizing:border-box= ;font-family:inherit;font-size:1em"><b><u>cd /usr/obj/usr/src/amd64.amd64/s= ys/PI_KERNELCONF/ kgdb kernel.debug /var/crash/vmcore.last</u></b> GNU gdb (GDB) 12.1 [GDB v12.1 for FreeBSD] Copyright (C) 2022 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <<a href=3D"http://gnu.org/li= censes/gpl.html" target=3D"_blank">http://gnu.org/licenses/gpl.html</a>> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-portbld-freebsd13.1". Type "show configuration" for configuration details. For bug reporting instructions, please see: <<a href=3D"https://www.gnu.org/software/gdb/bugs/" target=3D"_blank">ht= tps://www.gnu.org/software/gdb/bugs/</a>>. Find the GDB manual and other documentation resources online at: <<a href=3D"http://www.gnu.org/software/gdb/documentation/" target= =3D"_blank">http://www.gnu.org/software/gdb/documentation/</a>>. For help, type "help". Type "apropos word" to search for commands related to "word&= quot;... Reading symbols from /boot/kernel/kernel... Reading symbols from /usr/lib/debug//boot/kernel/kernel.debug... Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid =3D 0; apic id =3D 00 fault virtual address =3D 0xffffffffffffffa8 fault code =3D supervisor write data, page not present instruction pointer =3D 0x20:0xffffffff80ca0822 stack pointer =3D 0x28:0xfffffe00cd879b60 frame pointer =3D 0x28:0x0 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 922 (sshd) trap number =3D 12 panic: page fault cpuid =3D 1 time =3D 1676286448 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00cd879= 920 vpanic() at vpanic+0x17f/frame 0xfffffe00cd879970 panic() at panic+0x43/frame 0xfffffe00cd8799d0 trap_fatal() at trap_fatal+0x385/frame 0xfffffe00cd879a30 trap_pfault() at trap_pfault+0x4f/frame 0xfffffe00cd879a90 calltrap() at calltrap+0x8/frame 0xfffffe00cd879a90 --- trap 0xc, rip =3D 0xffffffff80ca0822, rsp =3D 0xfffffe00cd879b60, rbp = =3D 0 --- kern_select() at kern_select+0x942 Uptime: 34s Dumping 371 out of 8085 MB:..5%..13%..22%..31%..44%..52%..61%..74%..82%..91= % __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 55 __asm("movq %%gs:%P1,%0" : "=3Dr" (td) : &quo= t;n" (offsetof(struct pcpu, <br></code></pre><pre dir=3D"ltr" style=3D"white-space:pre-wrap;box-sizing:= border-box;font-family:Monaco,Menlo,Consolas,"Roboto Mono","= Andale Mono","Ubuntu Mono",monospace;font-size:13px;margin-t= op:0px;margin-bottom:0px;padding:10px 10px 0px;line-height:1.4;overflow:aut= o;direction:ltr"><code style=3D"box-sizing:border-box;font-family:inherit;f= ont-size:1em"><b><u>(kgdb) where</u></b></code></pre><pre dir=3D"ltr" style= =3D"white-space:pre-wrap;box-sizing:border-box;font-family:Monaco,Menlo,Con= solas,"Roboto Mono","Andale Mono","Ubuntu Mono&quo= t;,monospace;font-size:13px;margin-top:0px;margin-bottom:0px;padding:10px 1= 0px 0px;line-height:1.4;overflow:auto;direction:ltr"><code style=3D"box-siz= ing:border-box;font-family:inherit;font-size:1em"> #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 #1 doadump (textdump=3Dtextdump@entry=3D1) at /usr/src/sys/kern/kern_shutd= own.c:399 #2 0xffffffff80c2f521 in kern_reboot (howto=3D260) at /usr/src/sys/kern/ke= rn_shutdown.c:487 #3 0xffffffff80c2f99e in vpanic (fmt=3D0xffffffff811dfeea "%s", = ap=3D<optimized out>) at /usr/src/sys/kern/kern_shutdown.c:920 #4 0xffffffff80c2f7a3 in panic (fmt=3D<unavailable>) at /usr/src/sys= /kern/kern_shutdown.c:844 #5 0xffffffff810d7855 in trap_fatal (frame=3D0xfffffe00cd879aa0, eva=3D184= 46744073709551528) at /usr/src/sys/amd64/amd64/trap.c:944 #6 0xffffffff810d78af in trap_pfault (frame=3D0xfffffe00cd879aa0, usermode= =3Dfalse, signo=3D<optimized out>, ucode=3D<optimized out>) at = /usr/src/sys/amd64/amd64/trap.c:763 #7 <signal handler called> #8 0xffffffff80ca0822 in selrescan (td=3D<error reading variable: Canno= t access memory at address 0xffffffffffffffd0>, ibits=3D<optimized ou= t>, obits=3D<optimized out>) at /usr/src/sys/kern/sys_generic.c:13= 25 #9 kern_select (td=3D<optimized out>, nd=3D<error reading variabl= e: Cannot access memory at address 0xffffffffffffff90>, fd_in=3D<opti= mized out>, fd_ou=3D<optimized out>, fd_ex=3D<optimized out>= , tvp=3D<optimized out>, abi_nfdbits=3D<error reading variable: Cannot access memory at addre= ss 0x10>) at /usr/src/sys/kern/sys_generic.c:1206 Backtrace stopped: Cannot access memory at address 0x8</code></pre></div></= div><br style=3D"box-sizing:border-box;color:rgb(20,20,20);font-family:&quo= t;Segoe UI","Helvetica Neue",Helvetica,Roboto,Oxygen,Ubuntu,= Cantarell,"Fira Sans","Droid Sans",sans-serif;font-size= :16px"><div style=3D"box-sizing:border-box;display:flow-root;margin:0.5em 0= px;background:rgb(246,246,246);border-width:1px 1px 1px 3px;border-style:so= lid;border-color:rgb(224,224,224) rgb(224,224,224) rgb(224,224,224) rgb(217= ,217,217);color:rgb(20,20,20);font-family:"Segoe UI","Helvet= ica Neue",Helvetica,Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans&quo= t;,"Droid Sans",sans-serif;font-size:16px"><div style=3D"box-sizi= ng:border-box;padding:6px 10px;font-size:13px;color:rgb(102,0,0);background= :rgb(251,251,251)">Code:</div><div dir=3D"ltr" style=3D"box-sizing:border-b= ox;padding:6px 10px;max-height:70vh;overflow:auto"><pre dir=3D"ltr" style= =3D"white-space:pre-wrap;box-sizing:border-box;font-family:Monaco,Menlo,Con= solas,"Roboto Mono","Andale Mono","Ubuntu Mono&quo= t;,monospace;font-size:13px;margin-top:0px;margin-bottom:0px;padding:10px 1= 0px 0px;line-height:1.4;overflow:auto;direction:ltr"><code style=3D"box-siz= ing:border-box;font-family:inherit;font-size:1em"># nm -n /boot/kernel/kern= el | grep 0xffffffff80ca0822 # nm -n /boot/kernel/kernel | grep 0xffffffff80ca0822 # nm -n /boot/kernel/kernel | grep 0xffffffff80c # nm -n /boot/kernel/kernel | grep 0xffffffff # nm -n /boot/kernel/kernel | grep 0xfffff # nm -n /boot/kernel/kernel | grep 0xff # nm -n /boot/kernel/kernel | grep 0x ffffffff80388c30 t cam_compat_handle_0x17 ffffffff803891e0 t cam_compat_handle_0x18 ffffffff803895f0 t cam_compat_handle_0x19 ffffffff80389730 t cam_compat_translate_dev_match_0x18 ffffffff80aba6a0 t xl_check_maddr_90xB ffffffff80aba6f0 t xl_check_maddr_90x ffffffff80abad90 t xl_txeof_90xB ffffffff80abb090 t xl_start_90xB_locked ffffffff80ebac80 t mlx5e_fec_mask_10x_25x_handler ffffffff80ebb050 t mlx5e_fec_avail_10x_25x_handler ffffffff80ebb0f0 t mlx5e_fec_mask_50x_handler ffffffff80ebb4e0 t mlx5e_fec_avail_50x_handler ffffffff810af6c0 T Xint0x80_syscall_pti ffffffff810af740 T Xint0x80_syscall ffffffff810af743 t int0x80_syscall_common ffffffff817fe180 r db_inst_0f0x ffffffff8180cc50 r mouse10x14_120 ffffffff8180cd40 r mouse10x16_50 ffffffff8180cd90 r mouse10x16_75 ffffffff8180cde0 r mouse10x16_90 ffffffff8180ce30 r mouse10x16_100 ffffffff8180ce80 r mouse10x16_120 ffffffff8180ced0 r mouse10x16_133</code></pre></div></div><br style=3D"box-= sizing:border-box;color:rgb(20,20,20);font-family:"Segoe UI",&quo= t;Helvetica Neue",Helvetica,Roboto,Oxygen,Ubuntu,Cantarell,"Fira = Sans","Droid Sans",sans-serif;font-size:16px"><span style=3D= "color:rgb(20,20,20);font-family:"Segoe UI","Helvetica Neue&= quot;,Helvetica,Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans","= Droid Sans",sans-serif;font-size:16px">We also tried with dtrace but w= ith no luck. Do you have other recommendations on how we can keep debugging= this issue? We know it's something we broke on the scheduler because t= he generic kernel is working decently.</span><br style=3D"box-sizing:border= -box;color:rgb(20,20,20);font-family:"Segoe UI","Helvetica N= eue",Helvetica,Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans",&q= uot;Droid Sans",sans-serif;font-size:16px"><br style=3D"box-sizing:bor= der-box;color:rgb(20,20,20);font-family:"Segoe UI","Helvetic= a Neue",Helvetica,Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans"= ,"Droid Sans",sans-serif;font-size:16px"><span style=3D"color:rgb= (20,20,20);font-family:"Segoe UI","Helvetica Neue",Helv= etica,Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans","Droid Sans= ",sans-serif;font-size:16px">P.S.: If someone is interested in how we = implemented the Petri Net for the scheduler, contact me through the mail, a= nd I can give you the paper we are working on.</span><br></div></div> --000000000000c44f8405f497fefb--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAPrTifWgxNqE%2B_OVgHm2fYANe2w-cKLt0Y_nULyxeTHOiCT=bA>