Date: Sat, 20 Oct 2018 20:18:50 +0000 From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 227654] [panic] repeatable crash with lagg+vlan+em Message-ID: <bug-227654-1689-tLH0HBkJMB@https.bugs.freebsd.org/bugzilla/> In-Reply-To: <bug-227654-1689@https.bugs.freebsd.org/bugzilla/> References: <bug-227654-1689@https.bugs.freebsd.org/bugzilla/>
next in thread | previous in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D227654 --- Comment #4 from Eugene Grosbein <eugen@freebsd.org> --- This is 100% repeatable using same command sequence under 12.0-BETA1/i386 installed with all defaults inside VirtualBox VM. This time it says: panic: vm_fault_hold: fault on nofault entry, addr: 0 It generates nice crashdump and reboots. I've uploaded kernel.debug (stock one from 12.0-BETA1/i386 installation ima= ge, 18M compressed) and vmcore.0.xz (9.2MB compressed) here: http://www.grosbein.net/freebsd/crash/20181021/ Here comes kgdb script: Script started on Sun Oct 21 02:31:58 2018 Command: kgdb kernel.debug /var/crash/vmcore.0 GNU gdb (GDB) 8.2 [GDB v8.2 for FreeBSD] Copyright (C) 2018 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.htm= l> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i386-portbld-freebsd12.0". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from kernel.debug...done. Unread portion of the kernel message buffer: <6>em0: link state changed to DOWN <6>vlan61: link state changed to DOWN panic: vm_fault_hold: fault on nofault entry, addr: 0 cpuid =3D 0 time =3D 1540062884 KDB: stack backtrace: #0 0x112d01f at kdb_backtrace+0x4f #1 0x10e03f7 at vpanic+0x147 #2 0x10e02ab at panic+0x1b #3 0x14289c5 at vm_fault_hold+0x2a45 #4 0x1425f2e at vm_fault+0x5e #5 0x16b6ef7 at trap_pfault+0xc7 #6 0x16b64af at trap+0x3cf #7 0xffc0315d at PTDpde+0x4165 #8 0x11e0122 at ether_output+0x6a2 #9 0x124887d at arprequest+0x44d #10 0x12493f9 at arp_ifinit+0x59 #11 0x124c5bb at arp_handle_ifllchange+0x3b #12 0x11db275 at if_setlladdr+0x275 #13 0x11eb900 at vlan_lladdr_fn+0x30 #14 0x113eba9 at taskqueue_run_locked+0x189 #15 0x113fd57 at taskqueue_thread_loop+0x97 #16 0x10a1af1 at fork_exit+0x71 #17 0xffc033ba at PTDpde+0x43c2 Uptime: 7m21s Physical memory: 2019 MB Dumping 84 MB: 69 53 37 21 5 __curthread () at ./machine/pcpu.h:226 226 ./machine/pcpu.h: No such file or directory. (kgdb) add-kld if_tap.ko add symbol table from file "if_tap.ko.debug" at .rodata_addr =3D 0x18c0c134 set_sysctl_set_addr =3D 0x18c0c4e0 set_modmetadata_set_addr =3D 0x18c0c4f8 .note.gnu.build-id_addr =3D 0x18c0c500 .dynsym_addr =3D 0x18c0c548 .gnu.hash_addr =3D 0x18c0cb18 .hash_addr =3D 0x18c0cb58 .dynstr_addr =3D 0x18c0ce48 .rel.dyn_addr =3D 0x18c0d348 .text_addr =3D 0x18c0f000 .data_addr =3D 0x18c12000 set_sysinit_set_addr =3D 0x18c12270 set_sysuninit_set_addr =3D 0x18c12278 .dynamic_addr =3D 0x18c13000 .bss_addr =3D 0x18c14000 (y or n) y Reading symbols from if_tap.ko.debug...done. (kgdb) add-kld if_lagg.ko add symbol table from file "if_lagg.ko.debug" at .rodata_addr =3D 0x18c15138 set_sysctl_set_addr =3D 0x18c16038 set_modmetadata_set_addr =3D 0x18c16054 .note.gnu.build-id_addr =3D 0x18c16060 .dynsym_addr =3D 0x18c160a8 .gnu.hash_addr =3D 0x18c16798 .hash_addr =3D 0x18c167e8 .dynstr_addr =3D 0x18c16b68 .rel.dyn_addr =3D 0x18c171e8 .text_addr =3D 0x18c1a000 .data_addr =3D 0x18c24000 set_vnet_addr =3D 0x18c24270 set_sysinit_set_addr =3D 0x18c242a0 set_sysuninit_set_addr =3D 0x18c242ac .dynamic_addr =3D 0x18c25000 .bss_addr =3D 0x18c26000 (y or n) y Reading symbols from if_lagg.ko.debug...done. (kgdb) bt full #0 __curthread () at ./machine/pcpu.h:226 td =3D <optimized out> #1 doadump (textdump=3D<optimized out>) at /usr/src/sys/kern/kern_shutdown= .c:366 error =3D <error reading variable error (Cannot access memory at ad= dress 0x0)> coredump =3D <optimized out> #2 0x010e0073 in kern_reboot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:446 once =3D <error reading variable once (Cannot access memory at addr= ess 0x0)> #3 0x010e0444 in vpanic ( fmt=3D0x179ab2c "%s: fault on nofault entry, addr: %#lx",=20 ap=3D0x16a1462c "U\327p\001") at /usr/src/sys/kern/kern_shutdown.c:872 buf =3D "vm_fault_hold: fault on nofault entry, addr: 0", '\000' <r= epeats 209 times> td =3D 0x7369700 newpanic =3D <error reading variable newpanic (Cannot access memory= at address 0x1)> bootopt =3D 260 other_cpus =3D <optimized out> #4 0x010e02ab in panic ( fmt=3D0x179ab2c "%s: fault on nofault entry, addr: %#lx") at /usr/src/sys/kern/kern_shutdown.c:799 ap =3D <unavailable> --Type <RET> for more, q to quit, c to continue without paging--c #5 0x014289c5 in vm_fault_hold (map=3D0x2bd5000, vaddr=3D0, fault_type=3D<= optimized out>, fault_flags=3D0, m_hold=3D0x0) at /usr/src/sys/vm/vm_fault.c:586 hardfault =3D <optimized out> nera =3D <optimized out> faultcount =3D <optimized out> wired =3D 0 prot =3D 7 '\a' result =3D 0 rv =3D <optimized out> behind =3D <optimized out> ahead =3D <optimized out> error =3D <optimized out> locked =3D <optimized out> vp =3D <optimized out> fs =3D <optimized out> dset =3D <optimized out> next_object =3D <optimized out> alloc_req =3D <optimized out> era =3D <optimized out> behavior =3D <optimized out> cluster_offset =3D <optimized out> e_start =3D <optimized out> e_end =3D <optimized out> is_first_object_locked =3D <error reading variable is_first_object_= locked (Cannot access memory at address 0x0)> retry_prot =3D <optimized out> retry_pindex =3D <optimized out> retry_object =3D <optimized out> dead =3D <optimized out> #6 0x01425f2e in vm_fault (map=3D0x2bd5000, vaddr=3D0, fault_type=3D1 '\00= 1', fault_flags=3D0) at /usr/src/sys/vm/vm_fault.c:536 td =3D 0x7369700 result =3D <optimized out> #7 0x016b6ef7 in trap_pfault (frame=3D0x16a14894, usermode=3D0, eva=3D28) = at /usr/src/sys/i386/i386/trap.c:882 td =3D 0x7369700 p =3D <optimized out> va =3D <optimized out> ftype =3D <unavailable> rv =3D <optimized out> map =3D <optimized out> #8 0x016b64af in trap (frame=3D0x16a14894) at /usr/src/sys/i386/i386/trap.= c:519 td =3D <optimized out> dr6 =3D <optimized out> addr =3D <optimized out> ucode =3D <optimized out> signo =3D <optimized out> p =3D 0x206c1e4 <proc0> type =3D 12 eva =3D 28 ksi =3D <optimized out> #9 0xffc0315d in ?? () No symbol table info available. #10 0x16a14894 in ?? () No symbol table info available. #11 0x011e0122 in ether_output (ifp=3D0x7240400, m=3D<unavailable>, dst=3D0= x16a149e8, ro=3D0x16a149a8) at /usr/src/sys/net/if_ethersubr.c:435 linkhdr =3D "\001\000\000\000\000\227\066\a\\I\241\026\372\251" error =3D <optimized out> pflags =3D 0 addref =3D 0 phdr =3D <optimized out> hlen =3D <optimized out> lle =3D <optimized out> eh =3D <optimized out> t =3D <optimized out> #12 0x0124887d in arprequest (ifp=3D0x7240400, sip=3D0x16a14a40, tip=3D0x16= a14a40, enaddr=3D0x1911d94a "") at /usr/src/sys/netinet/if_ether.c:428 linkhdr =3D "\377\377\377\377\377\377\000\275\024\247\377\000\b\006\000\000\000\000\000= \000\000\000\000" carpaddr =3D <optimized out> m =3D <optimized out> ah =3D <optimized out> linkhdrsize =3D <error reading variable linkhdrsize (Cannot access = memory at address 0x18)> error =3D 0 ro =3D {ro_rt =3D 0x0, ro_lle =3D 0x0, ro_prepend =3D 0x16a149d0 "\377\377\377\377\377\377", ro_plen =3D 14, ro_flags =3D 0, ro_mtu =3D 0, s= pare =3D 0, ro_dst =3D {sa_len =3D 0 '\000', sa_family =3D 0 '\000', sa_data =3D '\000'= <repeats 13 times>}} sa =3D {sa_len =3D 2 '\002', sa_family =3D 35 '#', sa_data =3D "\000\000J\331\021\031\000\000\000\000\000\227\066\a"} #13 0x012493f9 in arp_announce_ifaddr (ifp=3D0x7240400, addr=3D..., enaddr=3D<unavailable>) at /usr/src/sys/netinet/if_ether.c:1436 No locals. #14 arp_ifinit (ifp=3D0x7240400, ifa=3D0x1911d200) at /usr/src/sys/netinet/if_ether.c:1423 dst_in =3D 0x1911d250 dst =3D 0x1911d250 #15 0x0124c5bb in arp_handle_ifllchange (ifp=3D0x7240400) at /usr/src/sys/netinet/if_ether.c:1450 ifa =3D 0x1911d200 #16 0x011db275 in if_setlladdr (ifp=3D0x7240400, lladdr=3D0x1911d94a "", le= n=3D6) at /usr/src/sys/net/if.c:3867 _ep =3D <optimized out> _t =3D <optimized out> _el =3D <optimized out> ifa =3D <optimized out> sdl =3D <optimized out> nep_et =3D {datap =3D {0x0, 0x16a26c80, 0xdeadbeef}, datai =3D {1}} ifr =3D <optimized out> #17 0x011eb900 in vlan_lladdr_fn (arg=3D0x1911ab00, pending=3D1) at /usr/src/sys/net/if_vlan.c:1306 ifv =3D 0x1911ab00 ifp =3D <unavailable> #18 0x0113eba9 in taskqueue_run_locked (queue=3D0x736d600) at /usr/src/sys/kern/subr_taskqueue.c:465 tb_first =3D <optimized out> pending =3D 1 task =3D 0x1911ab28 tb =3D <optimized out> #19 0x0113fd57 in taskqueue_thread_loop (arg=3D0x206f918 <taskqueue_thread>= ) at /usr/src/sys/kern/subr_taskqueue.c:757 tqp =3D <optimized out> tq =3D 0x736d600 #20 0x010a1af1 in fork_exit (callout=3D0x113fcc0 <taskqueue_thread_loop>, arg=3D0x206f918 <taskqueue_thread>, frame=3D0x16a14ba8) at /usr/src/sys/kern/kern_fork.c:1057 td =3D 0x7369700 p =3D 0x206c1e4 <proc0> dtd =3D <optimized out> #21 0xffc033ba in ?? () No symbol table info available. (kgdb) frame 11 #11 0x011e0122 in ether_output (ifp=3D0x7240400, m=3D<unavailable>,=20 dst=3D0x16a149e8, ro=3D0x16a149a8) at /usr/src/sys/net/if_ethersubr.c:4= 35 435 return ether_output_frame(ifp, m); (kgdb) p *ifp $1 =3D {if_link =3D {cstqe_next =3D 0x0}, if_clones =3D {le_next =3D 0x0,=20 le_prev =3D 0x757ff1c}, if_groups =3D {cstqh_first =3D 0x757ec70,=20 cstqh_last =3D 0x756d1a4}, if_alloctype =3D 6 '\006', if_softc =3D 0x19= 11ab00,=20 if_llsoftc =3D 0x0, if_l2com =3D 0x0, if_dname =3D 0x1b08e5c <vlanname> "= vlan",=20 if_dunit =3D 61, if_index =3D 6, if_index_reserved =3D 0,=20 if_xname =3D "vlan61\000\000\000\000\000\000\000\000\000",=20 if_description =3D 0x0, if_flags =3D 34819, if_drv_flags =3D 64,=20 if_capabilities =3D 3, if_capenable =3D 3, if_linkmib =3D 0x1911ab14,=20 if_linkmiblen =3D 20, if_refcount =3D 1, if_type =3D 135 '\207',=20 if_addrlen =3D 6 '\006', if_hdrlen =3D 4 '\004', if_link_state =3D 1 '\00= 1',=20 if_mtu =3D 1500, if_metric =3D 0, if_baudrate =3D 0, if_hwassist =3D 6,=20 if_epoch =3D 421, if_lastchange =3D {tv_sec =3D 1540062864, tv_usec =3D 9= 21826},=20 if_snd =3D {ifq_head =3D 0x0, ifq_tail =3D 0x0, ifq_len =3D 0, ifq_maxlen= =3D 50,=20 ifq_mtx =3D {lock_object =3D {lo_name =3D 0x7240430 "vlan61",=20 lo_flags =3D 16973824, lo_data =3D 0, lo_witness =3D 0x0}, mtx_lock= =3D 0},=20 ifq_drv_head =3D 0x0, ifq_drv_tail =3D 0x0, ifq_drv_len =3D 0,=20 ifq_drv_maxlen =3D 0, altq_type =3D 0, altq_flags =3D 0, altq_disc =3D = 0x0,=20 altq_ifp =3D 0x7240400, altq_enqueue =3D 0x0, altq_dequeue =3D 0x0,=20 altq_request =3D 0x0, altq_clfier =3D 0x0, altq_classify =3D 0x0,=20 altq_tbr =3D 0x0, altq_cdnr =3D 0x0}, if_linktask =3D {ta_link =3D { stqe_next =3D 0x0}, ta_pending =3D 0, ta_priority =3D 0,=20 ta_func =3D 0x11d5460 <do_link_state_change>, ta_context =3D 0x7240400}= ,=20 if_addr_lock =3D {lock_object =3D {lo_name =3D 0x1704959 "if_addr_lock",= =20 lo_flags =3D 16973824, lo_data =3D 0, lo_witness =3D 0x0}, mtx_lock = =3D 0},=20 --Type <RET> for more, q to quit, c to continue without paging-- c if_addrhead =3D {cstqh_first =3D 0x1911d900, cstqh_last =3D 0x1911d214}, if_multiaddrs =3D {cstqh_first =3D 0x1911ac80, cstqh_last =3D 0x1911a0c0}, = if_amcount =3D 0, if_addr =3D 0x1911d900, if_hw_addr =3D 0x757ebd0, if_broadcastaddr = =3D 0x1b08424 <etherbroadcastaddr> "\377\377\377\377\377\377", if_afdata_lock =3D {lock_o= bject =3D {lo_name =3D 0x1771f6b "if_afdata", lo_flags =3D 16973824, lo_data =3D = 0, lo_witness =3D 0x0}, mtx_lock =3D 0}, if_afdata =3D {0x0, 0x0, 0x757ece0, 0= x0 <repeats 25 times>, 0x74cff80, 0x0 <repeats 13 times>}, if_afdata_initializ= ed =3D 2, if_fib =3D 0, if_vnet =3D 0x7017060, if_home_vnet =3D 0x7017060, if_vlan= trunk =3D 0x0, if_bpf =3D 0x1911aa80, if_pcount =3D 0, if_bridge =3D 0x0, if_lagg =3D= 0x0, if_pf_kif =3D 0x0, if_carp =3D 0x0, if_label =3D 0x0, if_netmap =3D 0x0, if= _output =3D 0x11dfa80 <ether_output>, if_input =3D 0x11e0720 <ether_input>, if_bridge_i= nput =3D 0x0, if_bridge_output =3D 0x0, if_bridge_linkstate =3D 0x0, if_start =3D 0x= 0, if_ioctl =3D 0x11ea700 <vlan_ioctl>, if_init =3D 0x11ea470 <vlan_init>, if_resolvemulti =3D 0x11e0780 <ether_resolvemulti>, if_qflush =3D 0x11ea6f0 <vlan_qflush>, if_transmit =3D 0x11ea480 <vlan_transmit>, if_reassign =3D 0= x11e0980 <ether_reassign>, if_get_counter =3D 0x11d5720 <if_get_counter_default>, if_requestencap =3D 0x11e08b0 <ether_requestencap>, if_counters =3D {0x16b6= 1000, 0x16b61010, 0x16b61020, 0x16b61030, 0x16b61040, 0x16b61050, 0x16b61060, 0x16b61070, 0x16b61080, 0x16b61090, 0x16b610a0, 0x16b610b0}, if_hw_tsomax = =3D 65518, if_hw_tsomaxsegcount =3D 35, if_hw_tsomaxsegsize =3D 2048, if_snd_ta= g_alloc =3D 0x0, if_snd_tag_modify =3D 0x0, if_snd_tag_query =3D 0x0, if_snd_tag_fr= ee =3D 0x0, if_pcp =3D 0 '\000', if_netdump_methods =3D 0x0, if_epoch_ctx =3D {data =3D= {0x0, 0x0}}, if_addr_et =3D {datap =3D {0x0, 0x0, 0x0}, datai =3D {0}}, if_maddr_= et =3D {datap =3D {0x0, 0x0, 0x0}, datai =3D {0}}, if_ispare =3D {0, 0, 0, 0}} (kgdb) p m $2 =3D <unavailable> (kgdb) l 430 if (m =3D=3D NULL) 431 return (0); 432 } 433=20=20=20=20=20 434 /* Continue with link-layer output */ 435 return ether_output_frame(ifp, m); 436 } 437=20=20=20=20=20 438 static bool 439 ether_set_pcp(struct mbuf **mp, struct ifnet *ifp, uint8_t pcp) (kgdb) p ifp->if_type $4 =3D 135 '\207' (kgdb) p ifp->if_transmit $5 =3D (if_transmit_fn_t) 0x11ea480 <vlan_transmit> (kgdb) p *((struct ifvlan *)ifp->if_softc)->ifv_trunk->parent $8 =3D {if_link =3D {cstqe_next =3D 0x757d000}, if_clones =3D {le_next =3D = 0x0,=20 le_prev =3D 0x0}, if_groups =3D {cstqh_first =3D 0x74fe980,=20 cstqh_last =3D 0x74fe984}, if_alloctype =3D 6 '\006', if_softc =3D 0x75= 2a000,=20 if_llsoftc =3D 0x0, if_l2com =3D 0x0, if_dname =3D 0x70444b0 "em", if_dun= it =3D 0,=20 if_index =3D 1, if_index_reserved =3D 0,=20 if_xname =3D "em0", '\000' <repeats 12 times>, if_description =3D 0x0,=20 if_flags =3D 34819, if_drv_flags =3D 64, if_capabilities =3D 8468635,=20 if_capenable =3D 8454299, if_linkmib =3D 0x0, if_linkmiblen =3D 0,=20 if_refcount =3D 4, if_type =3D 161 '\241', if_addrlen =3D 6 '\006',=20 if_hdrlen =3D 18 '\022', if_link_state =3D 1 '\001', if_mtu =3D 1500,=20 if_metric =3D 0, if_baudrate =3D 0, if_hwassist =3D 6, if_epoch =3D 1,=20 if_lastchange =3D {tv_sec =3D 1540062449, tv_usec =3D 317821}, if_snd =3D= { ifq_head =3D 0x0, ifq_tail =3D 0x0, ifq_len =3D 0, ifq_maxlen =3D 1023,= ifq_mtx =3D { lock_object =3D {lo_name =3D 0x752b430 "em0", lo_flags =3D 16973824,= =20 lo_data =3D 0, lo_witness =3D 0x0}, mtx_lock =3D 0}, ifq_drv_head = =3D 0x0,=20 ifq_drv_tail =3D 0x0, ifq_drv_len =3D 0, ifq_drv_maxlen =3D 1023, altq_= type =3D 0,=20 altq_flags =3D 1, altq_disc =3D 0x0, altq_ifp =3D 0x752b400, altq_enque= ue =3D 0x0,=20 altq_dequeue =3D 0x0, altq_request =3D 0x0, altq_clfier =3D 0x0,=20 altq_classify =3D 0x0, altq_tbr =3D 0x0, altq_cdnr =3D 0x0}, if_linktas= k =3D { ta_link =3D {stqe_next =3D 0x0}, ta_pending =3D 0, ta_priority =3D 0,=20 ta_func =3D 0x11d5460 <do_link_state_change>, ta_context =3D 0x752b400}= ,=20 if_addr_lock =3D {lock_object =3D {lo_name =3D 0x1704959 "if_addr_lock",= =20 lo_flags =3D 16973824, lo_data =3D 0, lo_witness =3D 0x0},=20 mtx_lock =3D 129987456}, if_addrhead =3D {cstqh_first =3D 0x74e0c00,=20 --Type <RET> for more, q to quit, c to continue without paging--q cstqh_last =3D Quit Command exit status: 0 Script done on Sun Oct 21 02:55:39 2018 --=20 You are receiving this mail because: You are on the CC list for the bug.=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-227654-1689-tLH0HBkJMB>