Date: Fri, 19 Dec 2025 12:10:09 +0100 From: Kristof Provost <kp@FreeBSD.org> To: Gleb Smirnoff <glebius@FreeBSD.org> Cc: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Subject: Re: git: 0d469d23715d - main - net: attach IPv4 and IPv6 stacks to an interface with EVENTHANDLER(9) Message-ID: <4A394DAA-1FCE-440A-8E92-88BD9B4EE087@FreeBSD.org> In-Reply-To: <694452f3.32deb.4d0ab2a7@gitrepo.freebsd.org> References: <694452f3.32deb.4d0ab2a7@gitrepo.freebsd.org>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] On 18 Dec 2025, at 20:16, Gleb Smirnoff wrote: > The branch main has been updated by glebius: > > URL: > https://cgit.FreeBSD.org/src/commit/?id=0d469d23715d690b863787ebfa51529e1f6a9092 > > commit 0d469d23715d690b863787ebfa51529e1f6a9092 > Author: Gleb Smirnoff <glebius@FreeBSD.org> > AuthorDate: 2025-12-18 16:47:39 +0000 > Commit: Gleb Smirnoff <glebius@FreeBSD.org> > CommitDate: 2025-12-18 19:15:53 +0000 > > net: attach IPv4 and IPv6 stacks to an interface with > EVENTHANDLER(9) > > This change retires two historic relics: the if_afdata[] array and > the > dom_ifattach/dom_ifdetach methods. > > The if_afdata[] array is a relic of the era, when there was > expectation > that many transport protocols will coexist with IP, e.g. IPX or > NetAtalk. > The array hasn't had any members except AF_INET and AF_INET6 for > over a > decade already. This change removes the array and just leaves two > pointer > fields: if_inet and if_inet6. > > The dom_ifattach/dom_ifdetach predates the EVENTHANDLER(9) > framework and > was a good enough method to initialize protocol contexts back > then. Today > there is no good reason to treat IPv4 and IPv6 stacks differently > to other > protocols/features that attach and detach from an interface. > > The locking of if_afdata[] is a relic of SMPng times, when the > system > startup and the interface attach was even more convoluted than > before this > change, and we also had unloadable protocols that used a field in > if_afdata[]. Note that IPv4 and IPv6 are not unloadable. > > Note that this change removes NET_EPOCH_WAIT() from the interface > detach > sequence. This may surface several new races associated with > interface > removal. I failed to hit any with consecutive test suite runs, > though. > The expected general race scenario is that while struct ifnet is > freed > with proper epoch_call(9) itself, some structures hanging off > ifnet are > freed with direct free(9). The proper fix is either make if_foo > point at > some static "dead" structure providing SMP visibility of this > store, or > free those structure with epoch_call(9). All of these cases are > planned > to be found and resolved during 16.0-CURRENT lifetime. > > Reviewed by: zlei, gallatin, melifaro > Differential Revision: https://reviews.freebsd.org/D54089 I’m seeing panics on pfsync interface destruction now: panic: mld_change_state: bad ifp cpuid = 19 time = 1766142554 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe01843fd990 vpanic() at vpanic+0x136/frame 0xfffffe01843fdac0 panic() at panic+0x43/frame 0xfffffe01843fdb20 mld_change_state() at mld_change_state+0x6d0/frame 0xfffffe01843fdb90 in6_leavegroup_locked() at in6_leavegroup_locked+0xa9/frame 0xfffffe01843fdbf0 in6_leavegroup() at in6_leavegroup+0x32/frame 0xfffffe01843fdc10 pfsync_multicast_cleanup() at pfsync_multicast_cleanup+0x83/frame 0xfffffe01843fdc40 pfsync_clone_destroy() at pfsync_clone_destroy+0x260/frame 0xfffffe01843fdc90 ifc_simple_destroy_wrapper() at ifc_simple_destroy_wrapper+0x26/frame 0xfffffe01843fdca0 if_clone_destroyif_flags() at if_clone_destroyif_flags+0x69/frame 0xfffffe01843fdce0 if_clone_detach() at if_clone_detach+0xe6/frame 0xfffffe01843fdd10 vnet_pfsync_uninit() at vnet_pfsync_uninit+0xf0/frame 0xfffffe01843fdd30 vnet_destroy() at vnet_destroy+0x154/frame 0xfffffe01843fdd60 prison_deref() at prison_deref+0xaf5/frame 0xfffffe01843fddd0 sys_jail_remove() at sys_jail_remove+0x15c/frame 0xfffffe01843fde00 amd64_syscall() at amd64_syscall+0x169/frame 0xfffffe01843fdf30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01843fdf30 --- syscall (508, FreeBSD ELF64, jail_remove), rip = 0x2d8234c9e31a, rsp = 0x2d823179b928, rbp = 0x2d823179b9b0 --- KDB: enter: panic The pfsync:basic_ipv6 seems to trigger this reliably. Best regards, Kristof [-- Attachment #2 --] <!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" content="text/xhtml; charset=utf-8"> </head> <body><div style="font-family: sans-serif;"><div class="markdown" style="white-space: normal;"> <p dir="auto">On 18 Dec 2025, at 20:16, Gleb Smirnoff wrote:</p> <blockquote style="margin: 0 0 5px; padding-left: 5px; border-left: 2px solid #136BCE; color: #136BCE;"> <p dir="auto">The branch main has been updated by glebius:</p> <p dir="auto">URL: <a href="https://cgit.FreeBSD.org/src/commit/?id=0d469d23715d690b863787ebfa51529e1f6a9092">https://cgit.FreeBSD.org/src/commit/?id=0d469d23715d690b863787ebfa51529e1f6a9092</a></p>; <p dir="auto">commit 0d469d23715d690b863787ebfa51529e1f6a9092<br> Author: Gleb Smirnoff <a href="mailto:glebius@FreeBSD.org">glebius@FreeBSD.org</a><br> AuthorDate: 2025-12-18 16:47:39 +0000<br> Commit: Gleb Smirnoff <a href="mailto:glebius@FreeBSD.org">glebius@FreeBSD.org</a><br> CommitDate: 2025-12-18 19:15:53 +0000</p> <pre style="margin-left: 15px; margin-right: 15px; padding: 5px; border: thin solid gray; overflow-x: auto; max-width: 90vw; background-color: #E4E4E4;"><code style="padding: 0 0.25em; background-color: #E4E4E4;">net: attach IPv4 and IPv6 stacks to an interface with EVENTHANDLER(9) This change retires two historic relics: the if_afdata[] array and the dom_ifattach/dom_ifdetach methods. The if_afdata[] array is a relic of the era, when there was expectation that many transport protocols will coexist with IP, e.g. IPX or NetAtalk. The array hasn't had any members except AF_INET and AF_INET6 for over a decade already. This change removes the array and just leaves two pointer fields: if_inet and if_inet6. The dom_ifattach/dom_ifdetach predates the EVENTHANDLER(9) framework and was a good enough method to initialize protocol contexts back then. Today there is no good reason to treat IPv4 and IPv6 stacks differently to other protocols/features that attach and detach from an interface. The locking of if_afdata[] is a relic of SMPng times, when the system startup and the interface attach was even more convoluted than before this change, and we also had unloadable protocols that used a field in if_afdata[]. Note that IPv4 and IPv6 are not unloadable. Note that this change removes NET_EPOCH_WAIT() from the interface detach sequence. This may surface several new races associated with interface removal. I failed to hit any with consecutive test suite runs, though. The expected general race scenario is that while struct ifnet is freed with proper epoch_call(9) itself, some structures hanging off ifnet are freed with direct free(9). The proper fix is either make if_foo point at some static "dead" structure providing SMP visibility of this store, or free those structure with epoch_call(9). All of these cases are planned to be found and resolved during 16.0-CURRENT lifetime. Reviewed by: zlei, gallatin, melifaro Differential Revision: https://reviews.freebsd.org/D54089 </code></pre> </blockquote> <p dir="auto">I’m seeing panics on pfsync interface destruction now:</p> <pre style="margin-left: 15px; margin-right: 15px; padding: 5px; border: thin solid gray; overflow-x: auto; max-width: 90vw; background-color: #E4E4E4;"><code style="padding: 0 0.25em; background-color: #E4E4E4;">panic: mld_change_state: bad ifp cpuid = 19 time = 1766142554 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe01843fd990 vpanic() at vpanic+0x136/frame 0xfffffe01843fdac0 panic() at panic+0x43/frame 0xfffffe01843fdb20 mld_change_state() at mld_change_state+0x6d0/frame 0xfffffe01843fdb90 in6_leavegroup_locked() at in6_leavegroup_locked+0xa9/frame 0xfffffe01843fdbf0 in6_leavegroup() at in6_leavegroup+0x32/frame 0xfffffe01843fdc10 pfsync_multicast_cleanup() at pfsync_multicast_cleanup+0x83/frame 0xfffffe01843fdc40 pfsync_clone_destroy() at pfsync_clone_destroy+0x260/frame 0xfffffe01843fdc90 ifc_simple_destroy_wrapper() at ifc_simple_destroy_wrapper+0x26/frame 0xfffffe01843fdca0 if_clone_destroyif_flags() at if_clone_destroyif_flags+0x69/frame 0xfffffe01843fdce0 if_clone_detach() at if_clone_detach+0xe6/frame 0xfffffe01843fdd10 vnet_pfsync_uninit() at vnet_pfsync_uninit+0xf0/frame 0xfffffe01843fdd30 vnet_destroy() at vnet_destroy+0x154/frame 0xfffffe01843fdd60 prison_deref() at prison_deref+0xaf5/frame 0xfffffe01843fddd0 sys_jail_remove() at sys_jail_remove+0x15c/frame 0xfffffe01843fde00 amd64_syscall() at amd64_syscall+0x169/frame 0xfffffe01843fdf30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01843fdf30 --- syscall (508, FreeBSD ELF64, jail_remove), rip = 0x2d8234c9e31a, rsp = 0x2d823179b928, rbp = 0x2d823179b9b0 --- KDB: enter: panic </code></pre> <p dir="auto">The pfsync:basic_ipv6 seems to trigger this reliably.</p> <p dir="auto">Best regards,<br> Kristof</p> </div> </div> </body> </html>help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4A394DAA-1FCE-440A-8E92-88BD9B4EE087>