Date: Fri, 16 Apr 2021 00:43:02 +0000 From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 255104] FreeBSD 13.0-RELEASE panic/crash with ipfw/dummynet/divert & wlan Message-ID: <bug-255104-227@https.bugs.freebsd.org/bugzilla/>
next in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D255104 Bug ID: 255104 Summary: FreeBSD 13.0-RELEASE panic/crash with ipfw/dummynet/divert & wlan Product: Base System Version: 13.0-STABLE Hardware: Any OS: Any Status: New Severity: Affects Only Me Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: kumba@gentoo.org I have upgraded my router appliance to FreeBSD 13.0-RELEASE and when using = IPFW + dummynet(4) + divert(4), I can trigger the kernel to panic in a very rand= om fashion. Background on my setup: - Hardware is a Protectli FW6C (https://protectli.com/product/fw6c/) * 16GB RAM * KINGSTON SUV500MS120G on /dev/ada0 * 6x Intel 82583V GbE network ports supported by em(4) [em0 to em5] * Custom-added Qualcom AR9462 on ath0/wlan0 - Custom kernel config installed in /boot/kernel.custom * Also a /boot/CUSTOM symlink pointing to /boot/kernel.custom - em0 is WAN, DHCP via dhclient(8) to my cable modem - em1 is LAN, connected to a Netgear switch - wlan0 is wireless LAN on a separate RFC1918 subnet from em1 - Firewall setup is IPFW-based * Uses in-kernel NAT for em1 and wlan0 subnets * Uses dummynet(4) for fq_codel shaping * Uses divert(4) socket to route packets to Snort for inline inspection Synopsis of what causes the crash: - Having Snort up and running in a tmux session - wlan0 is active and has a client station connected - ipfw divert(4) socket is active, feeding packets to Snort - Sending/receiving WLAN traffic will eventually cause a random panic/reb= oot - Traffic on the LAN on em1 does NOT appear to trigger a crash (note, see crash #4) Here are samples of the crashes. I do not have the original kernel for som= e of these, so I cannot generate full backtraces, but I do have several of the c= ore dumps under /var/crash. Let me know what is needed to help debug this. No= te, I feel that the issue highlighted in PR#255069 may be related somehow. I a= lso tried patch D29772 posted in PR#255041, and that had no effect. Crash #6 is using this patched kernel, so I can run kgdb against it if needed. Crash #1 (Only kgdb backtrace is available): #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 #1 doadump (textdump=3D<optimized out>) at ../../../kern/kern_shutdown= .c:399 #2 0xffffffff8074e645 in kern_reboot (howto=3D260) at ../../../kern/kern_shutdown.c:486 #3 0xffffffff8074eac0 in vpanic (fmt=3D<optimized out>, ap=3D<optimize= d out>) at ../../../kern/kern_shutdown.c:919 #4 0xffffffff8074e8c3 in panic (fmt=3D<unavailable>) at ../../../kern/kern_shutdown.c:843 #5 0xffffffff80ad2037 in trap_fatal (frame=3D0xfffffe00dc46d8e0, eva= =3D8) at ../../../amd64/amd64/trap.c:915 #6 0xffffffff80ad2089 in trap_pfault (frame=3Dframe@entry=3D0xfffffe00dc46d8e0, usermode=3Dfalse, signo=3D<optim= ized out>, signo@entry=3D0x0, ucode=3D<optimized out>, ucode@entry=3D0x0) at ../../../amd64/amd64/trap.c:732 #7 0xffffffff80ad1709 in trap (frame=3D0xfffffe00dc46d8e0) at ../../../amd64/amd64/trap.c:398 #8 <signal handler called> #9 0xffffffff814f00a5 in dummynet_task () from /boot/CUSTOM/dummynet.ko #10 0xffffffff807aeda1 in taskqueue_run_locked (queue=3D0x8962c, queue@entry=3D0xfffff8000b02d300) at ../../../kern/subr_taskqueue.c:476 #11 0xffffffff807b00bc in taskqueue_thread_loop (arg=3D<optimized out>, arg@entry=3D0xffffffff814fa048 <dn_tq>) at ../../../kern/subr_taskqueue.c:7= 93 #12 0xffffffff8070e05d in fork_exit (callout=3D0xffffffff807b0010 <taskqueue_thread_loop>, arg=3D0xffffffff814fa048 <dn_tq>, frame=3D0xfffffe00dc46db00) at ../../../kern/kern_fork.c:1069 #13 <signal handler called> Crash #2 (kgdb backtrace data unavailable): Fatal trap 12: page fault while in kernel mode cpuid =3D 0; apic id =3D 00 fault virtual address =3D 0x8 fault code =3D supervisor read data, page not present instruction pointer =3D 0x20:0xffffffff814f00a5 stack pointer =3D 0x28:0xfffffe00dc46d9a0 frame pointer =3D 0x28:0xfffffe00dc46da00 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 0 (dummynet) trap number =3D 12 panic: page fault cpuid =3D 0 time =3D 1618402444 KDB: stack backtrace: #0 0xffffffff8079b0b5 at kdb_backtrace+0x65 #1 0xffffffff8074ea51 at vpanic+0x181 #2 0xffffffff8074e8c3 at panic+0x43 #3 0xffffffff80ad2037 at trap_fatal+0x387 #4 0xffffffff80ad2089 at trap_pfault+0x49 #5 0xffffffff80ad1709 at trap+0x259 #6 0xffffffff80aaa4e8 at calltrap+0x8 #7 0xffffffff807aeda1 at taskqueue_run_locked+0x181 #8 0xffffffff807b00bc at taskqueue_thread_loop+0xac #9 0xffffffff8070e05d at fork_exit+0x7d #10 0xffffffff80aab4ee at fork_trampoline+0xe Uptime: 9m23s Dumping 787 out of 16144 MB: (CTRL-C to abort) ..3%..11%..21%..31%..41%..51%..61%..72%..82%..92% Crash #3 (this happened when sending Ctrl+C to the Snort process): Fatal trap 12: page fault while in kernel mode cpuid =3D 0; apic id =3D 00 fault virtual address =3D 0x8 fault code =3D supervisor read data, page not present instruction pointer =3D 0x20:0xffffffff807ec20c stack pointer =3D 0x28:0xfffffe011d7d07d0 frame pointer =3D 0x28:0xfffffe011d7d0810 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 86334 (snort) trap number =3D 12 panic: page fault cpuid =3D 0 time =3D 1618439898 KDB: stack backtrace: #0 0xffffffff8079e8f5 at kdb_backtrace+0x65 #1 0xffffffff80752291 at vpanic+0x181 #2 0xffffffff80752103 at panic+0x43 #3 0xffffffff80b05a37 at trap_fatal+0x387 #4 0xffffffff80b05a89 at trap_pfault+0x49 #5 0xffffffff80b05109 at trap+0x259 #6 0xffffffff80addee8 at calltrap+0x8 #7 0xffffffff807eaf68 at sbdestroy+0x18 #8 0xffffffff807edd39 at sofree+0x309 #9 0xffffffff807ee824 at soclose+0x2e4 #10 0xffffffff806f8a91 at _fdrop+0x11 #11 0xffffffff806fbdcb at closef+0x24b #12 0xffffffff806f8d92 at closefp+0x82 #13 0xffffffff80b0621c at amd64_syscall+0x10c #14 0xffffffff80ade80e at fast_syscall_common+0xf8 Uptime: 21m57s Dumping 786 out of 16146 MB:..3%..11%..21%..31%..41%..51%..62%..72%..82%..92% __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 55 /usr/src/sys/amd64/include/pcpu_aux.h: No such file or directory. (kgdb) #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 #1 doadump (textdump=3D<optimized out>) at ../../../kern/kern_shutdown= .c:399 #2 0xffffffff80751e85 in kern_reboot (howto=3D260) at ../../../kern/kern_shutdown.c:486 #3 0xffffffff80752300 in vpanic (fmt=3D<optimized out>, ap=3D<optimize= d out>) at ../../../kern/kern_shutdown.c:919 #4 0xffffffff80752103 in panic (fmt=3D<unavailable>) at ../../../kern/kern_shutdown.c:843 #5 0xffffffff80b05a37 in trap_fatal (frame=3D0xfffffe011d7d0710, eva= =3D8) at ../../../amd64/amd64/trap.c:915 #6 0xffffffff80b05a89 in trap_pfault (frame=3Dframe@entry=3D0xfffffe011d7d0710,=20 usermode=3Dfalse, signo=3D<optimized out>, signo@entry=3D0x0,=20 ucode=3D<optimized out>, ucode@entry=3D0x0) at ../../../amd64/amd64/trap.c:732 #7 0xffffffff80b05109 in trap (frame=3D0xfffffe011d7d0710) at ../../../amd64/amd64/trap.c:398 #8 <signal handler called> #9 sbcut_internal (sb=3Dsb@entry=3D0xfffff802fa2d68a8, len=3D3404) at ../../../kern/uipc_sockbuf.c:1491 #10 0xffffffff807eaf68 in sbflush_internal (sb=3D0xfffff802fa2d68a8,=20 sb@entry=3D0xfffff802fa2d6760) at ../../../kern/uipc_sockbuf.c:1431 #11 sbrelease_internal (sb=3D0xfffff802fa2d68a8, sb@entry=3D0xfffff802f= a2d6760,=20 so=3D0xfffff802fa2d6760, so@entry=3D0xfffff802fa2d68a8) at ../../../kern/uipc_sockbuf.c:721 #12 sbdestroy (sb=3Dsb@entry=3D0xfffff802fa2d68a8, so=3Dso@entry=3D0xfffff802fa2d6760) at ../../../kern/uipc_sockbuf.c:749 #13 0xffffffff807edd39 in sofree (so=3Dso@entry=3D0xfffff802fa2d6760) at ../../../kern/uipc_socket.c:1158 #14 0xffffffff807ee824 in soclose (so=3D0xfffff802fa2d6760) at ../../../kern/uipc_socket.c:1235 #15 0xffffffff806f8a91 in fo_close (fp=3Dfp@entry=3D0xfffff80010895500, td=3D0xd4c,=20 td@entry=3D0xfffffe012053a000) at ../../../sys/file.h:377 #16 _fdrop (fp=3Dfp@entry=3D0xfffff80010895500, td=3D0xd4c,=20 td@entry=3D0xfffffe012053a000) at ../../../kern/kern_descrip.c:3510 #17 0xffffffff806fbdcb in closef (fp=3Dfp@entry=3D0xfffff80010895500,=20 td=3Dtd@entry=3D0xfffffe012053a000) at ../../../kern/kern_descrip.c= :2828 #18 0xffffffff806f8d92 in closefp_impl (fdp=3D<optimized out>, fd=3D4,= =20 fp=3D0xfffff80010895500, td=3D0xfffffe012053a000, audit=3Dtrue) at ../../../kern/kern_descrip.c:1271 #19 closefp (fdp=3D<optimized out>, fd=3D4, fp=3D0xfffff80010895500,=20 td=3D0xfffffe012053a000, holdleaders=3D<optimized out>, audit=3Dtru= e) at ../../../kern/kern_descrip.c:1328 #20 0xffffffff80b0621c in syscallenter (td=3D0xfffffe012053a000) at ../../../amd64/amd64/../../kern/subr_syscall.c:189 #21 amd64_syscall (td=3D0xfffffe012053a000, traced=3D0) at ../../../amd64/amd64/trap.c:1156 #22 <signal handler called> #23 0x000000080915b40a in ?? () Backtrace stopped: Cannot access memory at address 0x7fffff4b1458 Crash #4 (based on the stacktrace, this may have been caused by emX traffic= ): NOTE: I use an out-of-tree copy of em-7.7.8 from Intel upstream, modifed to compile under FreeBSD 13.0 (changes are trivial). Fatal trap 9: general protection fault while in kernel mode cpuid =3D 1; apic id =3D 02 instruction pointer =3D 0x20:0xffffffff8086e9dc stack pointer =3D 0x28:0xfffffe00c5b9f840 frame pointer =3D 0x28:0xfffffe00c5b9f890 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 0 (em0 que) trap number =3D 9 panic: general protection fault cpuid =3D 1 time =3D 1618440500 KDB: stack backtrace: #0 0xffffffff8079e8f5 at kdb_backtrace+0x65 #1 0xffffffff80752291 at vpanic+0x181 #2 0xffffffff80752103 at panic+0x43 #3 0xffffffff80b05a37 at trap_fatal+0x387 #4 0xffffffff80b055cf at trap+0x71f #5 0xffffffff80addee8 at calltrap+0x8 #6 0xffffffff8088c488 at netisr_dispatch_src+0xc8 #7 0xffffffff8086ddd9 at ether_input+0x69 #8 0xffffffff8086a69a at if_input+0xa #9 0xffffffff81b1f000 at em_rxeof+0x260 #10 0xffffffff81b20380 at em_handle_que+0x40 #11 0xffffffff807b25e1 at taskqueue_run_locked+0x181 #12 0xffffffff807b38fc at taskqueue_thread_loop+0xac #13 0xffffffff8071189d at fork_exit+0x7d #14 0xffffffff80adeeee at fork_trampoline+0xe Uptime: 9m14s Dumping 819 out of 16146 MB:..2%..12%..22%..32%..42%..51%..61%..71%..81%..92% __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 55 /usr/src/sys/amd64/include/pcpu_aux.h: No such file or directory. (kgdb) #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 #1 doadump (textdump=3D<optimized out>) at ../../../kern/kern_shutdown= .c:399 #2 0xffffffff80751e85 in kern_reboot (howto=3D260) at ../../../kern/kern_shutdown.c:486 #3 0xffffffff80752300 in vpanic (fmt=3D<optimized out>, ap=3D<optimize= d out>) at ../../../kern/kern_shutdown.c:919 #4 0xffffffff80752103 in panic (fmt=3D<unavailable>) at ../../../kern/kern_shutdown.c:843 #5 0xffffffff80b05a37 in trap_fatal (frame=3D0xfffffe00c5b9f780, eva= =3D0) at ../../../amd64/amd64/trap.c:915 #6 0xffffffff80b055cf in trap (frame=3D0xfffffe00c5b9f780) at ../../../amd64/amd64/trap.c:576 #7 <signal handler called> #8 ether_input_internal (ifp=3D0x5f48844900310210, m=3D0xfffff8039a9e9= d00) at ../../../net/if_ethersubr.c:524 #9 ether_nh_input (m=3D0xfffff8039a9e9d00) at ../../../net/if_ethersubr.c:739 #10 0xffffffff8088c488 in netisr_dispatch_src (proto=3Dproto@entry=3D5,= =20 source=3D<optimized out>, source@entry=3D0, m=3Dm@entry=3D0xfffff80= 39a9e9d00) at ../../../net/netisr.c:1143 #11 0xffffffff8088c76f in netisr_dispatch (proto=3D2594086144, proto@en= try=3D5,=20 m=3D0x2d, m@entry=3D0xfffff8039a9e9d00) at ../../../net/netisr.c:12= 34 #12 0xffffffff8086ddd9 in ether_input (ifp=3D<optimized out>,=20 m=3D0xfffff8039a9e9d00) at ../../../net/if_ethersubr.c:830 #13 0xffffffff8086a69a in if_input (ifp=3D0xfffff8039a9e9d00, sendmp=3D= 0x0) at ../../../net/if.c:4391 #14 0xffffffff81b1f000 in em_rxeof () from /boot/modules/if_em_updated.= ko #15 0xffffffff81b20380 in em_handle_que () from /boot/modules/if_em_updated.ko #16 0xffffffff807b25e1 in taskqueue_run_locked (queue=3D0xfffff80017500= 200,=20 queue@entry=3D0xfffff80002bdfa00) at ../../../kern/subr_taskqueue.c= :476 #17 0xffffffff807b38fc in taskqueue_thread_loop (arg=3D<optimized out>,= =20 arg@entry=3D0xfffffe002014e6a0) at ../../../kern/subr_taskqueue.c:7= 93 #18 0xffffffff8071189d in fork_exit ( callout=3D0xffffffff807b3850 <taskqueue_thread_loop>,=20 arg=3D0xfffffe002014e6a0, frame=3D0xfffffe00c5b9fb00) at ../../../kern/kern_fork.c:1069 #19 <signal handler called> Crash #5: Fatal trap 12: page fault while in kernel mode cpuid =3D 1; apic id =3D 02 fault virtual address =3D 0x0 fault code =3D supervisor read data, page not present instruction pointer =3D 0x20:0xffffffff8047ae0d stack pointer =3D 0x28:0xfffffe001d3fc550 frame pointer =3D 0x28:0xfffffe001d3fc590 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 12 (swi1: netisr 1) trap number =3D 12 panic: page fault cpuid =3D 1 time =3D 1618441084 KDB: stack backtrace: #0 0xffffffff8079e8f5 at kdb_backtrace+0x65 #1 0xffffffff80752291 at vpanic+0x181 #2 0xffffffff80752103 at panic+0x43 #3 0xffffffff80b05a37 at trap_fatal+0x387 #4 0xffffffff80b05a89 at trap_pfault+0x49 #5 0xffffffff80b05109 at trap+0x259 #6 0xffffffff80addee8 at calltrap+0x8 #7 0xffffffff808a73a3 at ieee80211_parent_xmitpkt+0x13 #8 0xffffffff808b988e at ieee80211_vap_pkt_send_dest+0x25e #9 0xffffffff808ba606 at ieee80211_vap_transmit+0x1d6 #10 0xffffffff8086d82b at ether_output_frame+0xab #11 0xffffffff8086d727 at ether_output+0x6b7 #12 0xffffffff808eb2e9 at ip_output_send+0x109 #13 0xffffffff808eb062 at ip_output+0x12a2 #14 0xffffffff808e8164 at ip_forward+0x394 #15 0xffffffff808e7d89 at ip_input+0x6c9 #16 0xffffffff8088cc1b at swi_net+0x12b #17 0xffffffff80714abd at ithread_loop+0x24d Uptime: 3m18s Dumping 849 out of 16146 MB:..2%..12%..21%..31%..42%..51%..61%..72%..81%..91% __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 55 /usr/src/sys/amd64/include/pcpu_aux.h: No such file or directory. (kgdb) #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 #1 doadump (textdump=3D<optimized out>) at ../../../kern/kern_shutdown= .c:399 #2 0xffffffff80751e85 in kern_reboot (howto=3D260) at ../../../kern/kern_shutdown.c:486 #3 0xffffffff80752300 in vpanic (fmt=3D<optimized out>, ap=3D<optimize= d out>) at ../../../kern/kern_shutdown.c:919 #4 0xffffffff80752103 in panic (fmt=3D<unavailable>) at ../../../kern/kern_shutdown.c:843 #5 0xffffffff80b05a37 in trap_fatal (frame=3D0xfffffe001d3fc490, eva= =3D0) at ../../../amd64/amd64/trap.c:915 #6 0xffffffff80b05a89 in trap_pfault (frame=3Dframe@entry=3D0xfffffe001d3fc490,=20 usermode=3Dfalse, signo=3D<optimized out>, signo@entry=3D0x0,=20 ucode=3D<optimized out>, ucode@entry=3D0x0) at ../../../amd64/amd64/trap.c:732 #7 0xffffffff80b05109 in trap (frame=3D0xfffffe001d3fc490) at ../../../amd64/amd64/trap.c:398 #8 <signal handler called> #9 ath_transmit (ic=3D<optimized out>, m=3D0xfffff801ed556200) at ../../../dev/ath/if_ath.c:3516 #10 0xffffffff808a73a3 in ieee80211_parent_xmitpkt (ic=3D0x0,=20 ic@entry=3D0xfffffe00d844f000, m=3Dm@entry=3D0xfffff8001e808300) at ../../../net80211/ieee80211_freebsd.c:717 #11 0xffffffff808b988e in ieee80211_vap_pkt_send_dest ( vap=3Dvap@entry=3D0xfffff8001e266000, m=3Dm@entry=3D0xfffff8001e808= 300,=20 ni=3Dni@entry=3D0xfffffe012c7b1000) at ../../../net80211/ieee80211_output.c:317 #12 0xffffffff808ba606 in ieee80211_start_pkt (vap=3D0xfffff8001e266000= ,=20 m=3D0xfffff8001e808300) at ../../../net80211/ieee80211_output.c:474 #13 ieee80211_vap_transmit (ifp=3D<optimized out>, m=3D<optimized out>) at ../../../net80211/ieee80211_output.c:534 #14 0xffffffff8086d82b in ether_output_frame ( ifp=3Difp@entry=3D0xfffff8001e188000, m=3D0xfffffe012c7b1000) at ../../../net/if_ethersubr.c:511 #15 0xffffffff8086d727 in ether_output (ifp=3D<optimized out>,=20 m=3D0xfffffe012c7b1000, dst=3D0xfffffe001d3fc8e0, ro=3D<optimized o= ut>) at ../../../net/if_ethersubr.c:438 #16 0xffffffff808eb2e9 in ip_output_send (inp=3Dinp@entry=3D0x0,=20 ifp=3D0xfffff8001e188000, m=3Dm@entry=3D0xfffff8001e808300, gw=3D<o= ptimized out>,=20 gw@entry=3D0xfffffe001d3fc8e0, ro=3D<optimized out>,=20 ro@entry=3D0xfffffe001d3fc8c0, stamp_tag=3D<optimized out>) at ../../../netinet/ip_output.c:275 #17 0xffffffff808eb062 in ip_output (m=3Dm@entry=3D0xfffff8001e808300,= =20 opt=3D<optimized out>, opt@entry=3D0x0, ro=3D<optimized out>,=20 ro@entry=3D0xfffffe001d3fc8c0, flags=3Dflags@entry=3D1, imo=3Dimo@e= ntry=3D0x0,=20 inp=3D<optimized out>, inp@entry=3D0x0) at ../../../netinet/ip_outp= ut.c:812 #18 0xffffffff808e8164 in ip_forward (m=3D0xfffff8001e808300,=20 srcrt=3D<optimized out>) at ../../../netinet/ip_input.c:1067 #19 0xffffffff808e7d89 in ip_input (m=3D0x0) at ../../../netinet/ip_input.c:789 #20 0xffffffff8088cc1b in netisr_process_workstream_proto ( nwsp=3D<optimized out>, proto=3D1) at ../../../net/netisr.c:919 #21 swi_net (arg=3D<optimized out>) at ../../../net/netisr.c:966 #22 0xffffffff80714abd in intr_event_execute_handlers (p=3D<optimized o= ut>,=20 ie=3D0xfffff80002826b00) at ../../../kern/kern_intr.c:1168 #23 ithread_execute_handlers (p=3D<optimized out>, ie=3D0xfffff80002826= b00) at ../../../kern/kern_intr.c:1181 #24 ithread_loop (arg=3Darg@entry=3D0xfffff80002833ac0) at ../../../kern/kern_intr.c:1269 #25 0xffffffff8071189d in fork_exit ( callout=3D0xffffffff80714870 <ithread_loop>, arg=3D0xfffff80002833a= c0,=20 frame=3D0xfffffe001d3fcb00) at ../../../kern/kern_fork.c:1069 #26 <signal handler called> Crash #6: Fatal trap 12: page fault while in kernel mode cpuid =3D 1; apic id =3D 02 fault virtual address =3D 0x388 fault code =3D supervisor read data, page not present instruction pointer =3D 0x20:0xffffffff8088cc07 stack pointer =3D 0x28:0xfffffe001d3fc9c0 frame pointer =3D 0x28:0xfffffe001d3fca20 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 12 (swi1: netisr 1) trap number =3D 12 panic: page fault cpuid =3D 1 time =3D 1618528473 KDB: stack backtrace: #0 0xffffffff8079e8f5 at kdb_backtrace+0x65 #1 0xffffffff80752291 at vpanic+0x181 #2 0xffffffff80752103 at panic+0x43 #3 0xffffffff80b05d07 at trap_fatal+0x387 #4 0xffffffff80b05d59 at trap_pfault+0x49 #5 0xffffffff80b053d9 at trap+0x259 #6 0xffffffff80ade1b8 at calltrap+0x8 #7 0xffffffff80714abd at ithread_loop+0x24d #8 0xffffffff8071189d at fork_exit+0x7d #9 0xffffffff80adf1be at fork_trampoline+0xe Uptime: 2m28s Dumping 781 out of 16146 MB:..3%..11%..21%..31%..41%..52%..62%..72%..82%..91% __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 55 /usr/src/sys/amd64/include/pcpu_aux.h: No such file or directory. (kgdb) #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 #1 doadump (textdump=3D<optimized out>) at ../../../kern/kern_shutdown= .c:399 #2 0xffffffff80751e85 in kern_reboot (howto=3D260) at ../../../kern/kern_shutdown.c:486 #3 0xffffffff80752300 in vpanic (fmt=3D<optimized out>, ap=3D<optimize= d out>) at ../../../kern/kern_shutdown.c:919 #4 0xffffffff80752103 in panic (fmt=3D<unavailable>) at ../../../kern/kern_shutdown.c:843 #5 0xffffffff80b05d07 in trap_fatal (frame=3D0xfffffe001d3fc900, eva= =3D904) at ../../../amd64/amd64/trap.c:915 #6 0xffffffff80b05d59 in trap_pfault (frame=3Dframe@entry=3D0xfffffe001d3fc900,=20 usermode=3Dfalse, signo=3D<optimized out>, signo@entry=3D0x0,=20 ucode=3D<optimized out>, ucode@entry=3D0x0) at ../../../amd64/amd64/trap.c:732 #7 0xffffffff80b053d9 in trap (frame=3D0xfffffe001d3fc900) at ../../../amd64/amd64/trap.c:398 #8 <signal handler called> #9 0xffffffff8088cc07 in netisr_process_workstream_proto ( nwsp=3D<optimized out>, proto=3D1) at ../../../net/netisr.c:918 #10 swi_net (arg=3D<optimized out>) at ../../../net/netisr.c:966 #11 0xffffffff80714abd in intr_event_execute_handlers (p=3D<optimized o= ut>,=20 ie=3D0xfffff80002826b00) at ../../../kern/kern_intr.c:1168 #12 ithread_execute_handlers (p=3D<optimized out>, ie=3D0xfffff80002826= b00) at ../../../kern/kern_intr.c:1181 #13 ithread_loop (arg=3Darg@entry=3D0xfffff80002833ac0) at ../../../kern/kern_intr.c:1269 #14 0xffffffff8071189d in fork_exit ( callout=3D0xffffffff80714870 <ithread_loop>, arg=3D0xfffff80002833a= c0,=20 frame=3D0xfffffe001d3fcb00) at ../../../kern/kern_fork.c:1069 #15 <signal handler called> ----------------------------------------------------------------------- I suspect the underlying flaw is somehow tied to an interaction with divert= (8) and dummynet(8) and the wlan0 adapter. Standard LAN traffic does not seem = to trigger the panic, or at least trigger it as easily. But WLAN traffic does trigger it very easily, usually within a minute or two of turning on the divert(8) rule, connecting a wireless station, and generating some wireless traffic. I also suspect Snort is applying memory pressure somehow. I am u= sing the standard Talos ruleset (30-day delayed release, several months old). This is how I start Snort-2.9.17: snort -c /usr/local/etc/snort/snort.conf -i em0 -k none -A console -Q --daq ipfw --daq-mode inline --daq-var port=3D8000 And this is the divert(8) rule: ipfw add 00049 divert 8000 all from any to any via em0 This is my NAT/dummynet configuration from the firewall: /sbin/ipfw nat 1 config if em0 deny_in same_ports unreg_only reset /sbin/ipfw pipe 1 config bw 294MBit/s burst 1048576 # Download pipe /sbin/ipfw pipe 2 config bw 12MBit/s # Upload pipe /sbin/ipfw sched 1 config pipe 1 type fq_codel target 5ms quantum 6000 flows 2048 interval 300 limit 15360 ecn /sbin/ipfw sched 2 config pipe 2 type fq_codel ecn /sbin/ipfw queue 01 config sched 2 weight 100 # Outbound TCP A= CK /sbin/ipfw queue 02 config sched 1 weight 100 # Inbound TCP ACK /sbin/ipfw queue 03 config sched 2 weight 90 # Outbound HTTP/HTTPS/RSYNC /sbin/ipfw queue 04 config sched 1 weight 90 # Inbound HTTP/HTTPS/RSYNC /sbin/ipfw queue 05 config sched 2 weight 85 # Outbound DNS /sbin/ipfw queue 06 config sched 1 weight 85 # Inbound DNS /sbin/ipfw queue 07 config sched 2 weight 65 # Outbound Steam Client /sbin/ipfw queue 08 config sched 1 weight 65 # Inbound Steam Client /sbin/ipfw queue 09 config sched 2 weight 55 # Outbound IMAP/POP3/SMTP /sbin/ipfw queue 10 config sched 1 weight 55 # Inbound IMAP/POP3/SMTP That's about all I can think that is relevant. Please let me know if any additional information is needed. The system is rolled back to FreeBSD 12.= 2, but I am keeping the FreeBSD 13.0 boot environment, so I can easily reboot = into 13.0 and try any patches out. --=20 You are receiving this mail because: You are the assignee for the bug.=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-255104-227>