From owner-freebsd-net@freebsd.org Tue Jan 7 11:40:44 2020 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EED9D2255A8 for ; Tue, 7 Jan 2020 11:40:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 47sVn865K2z423x for ; Tue, 7 Jan 2020 11:40:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id CF3E02255A7; Tue, 7 Jan 2020 11:40:44 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CF02A2255A6 for ; Tue, 7 Jan 2020 11:40:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47sVn85CmMz423v for ; Tue, 7 Jan 2020 11:40:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id AE8CC2647F for ; Tue, 7 Jan 2020 11:40:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 007Beikp081933 for ; Tue, 7 Jan 2020 11:40:44 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 007BeiH0081932 for net@FreeBSD.org; Tue, 7 Jan 2020 11:40:44 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 243126] Assertion fl->ifl_cidx == cidx failed at /usr/src/sys/net/iflib.c:2531 Date: Tue, 07 Jan 2020 11:40:44 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jan 2020 11:40:45 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D243126 Andriy Gapon changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |pkelsey@freebsd.org --- Comment #1 from Andriy Gapon --- Correction, the panic happened with vmxnet3 network driver. The VM was later switched to em as a workaround and that got me confused. Some data from the crash: (kgdb) fr 14 #14 0xffffffff808b721e in rxd_frag_to_sd (rxq=3D0xfffffe00003fe000, irf=3D, unload=3D, sd=3D0xfffffe0011a54900, pf_rv=3D0xfffffe0011a549b0, ri=3D0xfffffe0011a54960) at /usr/src/sys/net/iflib.c:2531 2531 /usr/src/sys/net/iflib.c: No such file or directory. (kgdb) p cidx $1 =3D 142 (kgdb) p *rxq $3 =3D {ifr_ctx =3D 0xfffff80002d22400, ifr_fl =3D 0xfffff80002d1a000, ifr_= rx_irq =3D 0, pfil =3D 0xfffff8000436fb80, ifr_cq_cidx =3D 477, ifr_id =3D 0, ifr_nfl = =3D 2 '\002', ifr_ntxqirq =3D 1 '\001', ifr_txqid =3D "\000\000\000", ifr_fl_offset =3D 1 '\001', ifr_lc =3D {ifp =3D 0xfffff80002d1a800, lro_m= buf_data =3D 0xfffffe00da9c4000, lro_queued =3D 1753531, lro_flushed =3D 339087, lro_bad= _csum =3D 0, lro_cnt =3D 8, lro_mbuf_count =3D 0, lro_mbuf_max =3D 256, lro_ackcnt_lim =3D 65535, lro_length_lim =3D 65535, lro_hashsz =3D 251,= lro_hash =3D 0xfffff80002fa5000, lro_active =3D {lh_first =3D 0x0}, lro_free =3D {lh= _first =3D 0xfffffe00da9c5360}}, ifr_task =3D {gt_task =3D {ta_link =3D {stqe_next =3D= 0x0}, ta_flags =3D 2, ta_priority =3D 0, ta_func =3D 0xffffffff808b0bd0 <_task_fn_rx>, ta_context =3D 0xfffffe00003fe000}, gt_taskqueue =3D 0xfffff80002922600, gt_list =3D {le_next =3D 0x0, le_prev =3D 0xfffffe00117= e98a8}, gt_uniq =3D 0xfffffe00003fe000, gt_name =3D "rxq0", '\000' , gt_dev =3D 0xfffff80002d3b000, gt_irq =3D 0xfffff80002d17900, gt_cpu =3D 0}, ifr_filter_info =3D {ifi_filter =3D 0xffffffff80af8510 , ifi_filter_arg =3D 0xfffff80002fab800, ifi_task =3D 0xfffffe00003fe090,= ifi_ctx =3D 0xfffffe00003fe000}, ifr_ifdi =3D 0xfffff80002d17d80, ifr_frags =3D {{i= rf_flid =3D 0 '\000', irf_idx =3D 142, irf_len =3D 1514}, {irf_flid =3D 1 '\001', irf_idx =3D 76, irf_len =3D 2048}, {irf_flid =3D 1 '\001', irf_idx = =3D 77, irf_len =3D 1762}, {irf_flid =3D 1 '\001', irf_idx =3D 53, irf_len =3D 1038= }, {irf_flid =3D 1 '\001', irf_idx =3D 114, irf_len =3D 2048}, {irf_flid =3D 1 '\001', irf_idx =3D 115, irf_len =3D 2048}, {irf_flid =3D 1 '\001', irf_idx = =3D 116, irf_len =3D 2048}, {irf_flid =3D 1 '\001', irf_idx =3D 117, irf_len =3D 204= 8}, {irf_flid =3D 1 '\001', irf_idx =3D 118, irf_len =3D 2048}, {irf_flid =3D 1= '\001', irf_idx =3D 119, irf_len =3D 1906}, {irf_flid =3D 1 '\001', irf_idx = =3D 184, irf_len =3D 1306}, {irf_flid =3D 1 '\001', irf_idx =3D 17, irf_len =3D 706}= , {irf_flid =3D 1 '\001', irf_idx =3D 3, irf_len =3D 2048}, {irf_flid =3D 1 '\001', irf_idx =3D 4, irf_len =3D 2048}, {irf_flid =3D 1 '\001', irf_idx =3D= 5, irf_len =3D 2048}, {irf_flid =3D 1 '\001', irf_idx =3D 6, irf_len =3D 1202}, {irf_f= lid =3D 0 '\000', irf_idx =3D 0, irf_len =3D 0} }} (kgdb) p *ri $4 =3D {iri_qsidx =3D 0, iri_vtag =3D 0, iri_len =3D 1514, iri_cidx =3D 477= , iri_ifp =3D 0xfffff80002d1a800, iri_frags =3D 0xfffffe00003fe140, iri_flowid =3D 600473= 664, iri_csum_flags =3D 251658240, iri_csum_data =3D 65535, iri_flags =3D 0 '\00= 0', iri_nfrags =3D 1 '\001', iri_rsstype =3D 130 '\202', iri_pad =3D 0 '\000'} (kgdb) fr 17 #17 iflib_rxeof (rxq=3D, budget=3D16) at /usr/src/sys/net/iflib.c:2803 2803 in /usr/src/sys/net/iflib.c (kgdb) i loc ctx =3D scctx =3D lro_possible =3D v4_forwarding =3D v6_forwarding =3D sctx =3D 0xffffffff810e7780 rx_pkts =3D 1 rx_bytes =3D 1514 mh =3D 0x0 mt =3D 0x0 ifp =3D 0xfffff80002d1a800 cidxp =3D 0xfffffe00003fe020 avail =3D 17 i =3D fl =3D m =3D 0x0 budget_left =3D 16 ri =3D err =3D mf =3D lro_enabled =3D (kgdb) p *cidxp $1 =3D 477 (kgdb) p *$5.ifc_sctx $7 =3D {isc_magic =3D 3405705229, isc_driver =3D 0xffffffff810e7900 , isc_q_align =3D 512, isc_tx_maxsize =3D 65536, isc_tx_maxsegsize =3D 16383, isc_tso_maxsize =3D 65550, isc_tso_maxsegsize = =3D 16383, isc_rx_maxsize =3D 16383, isc_rx_maxsegsize =3D 16383, isc_rx_nsegments = =3D 1, isc_admin_intrcnt =3D 1, isc_vendor_info =3D 0xffffffff810e7930 , isc_driver_version =3D 0xffffffff80ba63e4 "2", isc_parse_devinfo =3D 0x0, isc_nrxd_min =3D {32, 32, 32, 0, 0, 0, 0, 0}, isc_nrxd_default =3D {256, 256, 256, 0, 0, 0, 0, 0}, isc_nrxd_max =3D {2048= , 2048, 2048, 0, 0, 0, 0, 0}, isc_ntxd_min =3D {32, 32, 0, 0, 0, 0, 0, 0}, isc_ntxd_default =3D {512, 512, 0, 0, 0, 0, 0, 0}, isc_ntxd_max =3D {4096= , 4096, 0, 0, 0, 0, 0, 0}, isc_nfl =3D 2, isc_ntxqs =3D 2, isc_nrxqs =3D 3, __spare= 0__ =3D 0, isc_tx_reclaim_thresh =3D 0, isc_flags =3D 9, isc_name =3D 0x0} (kgdb) p *rxq->ifr_ctx $5 =3D {ops =3D 0xfffff80002d18000, ifc_softc =3D 0xfffff80002d22000, ifc_d= ev =3D 0xfffff80002d3b000, ifc_ifp =3D 0xfffff80002d1a800, ifc_cpus =3D {__bits = =3D {255, 0, 0, 0}}, ifc_sctx =3D 0xffffffff810e7780 , ifc_softc_ctx =3D {isc_vectors =3D 9, isc_nrxqsets =3D 8, isc_ntxqsets = =3D 8, __spare0__ =3D 0, __spare1__ =3D 0, isc_msix_bar =3D 24, isc_tx_nsegments = =3D 32, isc_ntxd =3D {512, 512, 0, 0, 0, 0, 0, 0}, isc_nrxd =3D {512, 256, 256, 0, = 0, 0, 0, 0}, isc_txqsizes =3D {8192, 8192, 0, 0, 0, 0, 0, 0}, isc_rxqsizes =3D= {8192, 4096, 4096, 0, 0, 0, 0, 0}, isc_txd_size =3D "\000\000\000\000\000\000\000", isc_rxd_size =3D "\000\000\000\000\000\000\000", isc_tx_tso_segments_max =3D 32, isc_tx_tso_size_max =3D 65532, isc_tx_tso_segsize_max =3D 16383, isc_tx_csum_flags =3D 5654, isc_capabilit= ies =3D 6621115, isc_capenable =3D 6554555, isc_rss_table_size =3D 128, isc_rss_tab= le_mask =3D 127, isc_nrxqsets_max =3D 8, isc_ntxqsets_max =3D 8, __spare2__ =3D 0, isc_i= ntr =3D IFLIB_INTR_MSIX, isc_max_frame_size =3D 1522, isc_min_frame_size =3D 0, isc_pause_frames =3D 0, __spare3__ =3D 0, __spare4__ =3D 0, __spare5__ =3D = 0, __spare6__ =3D 0, __spare7__ =3D 0, __spare8__ =3D 0, __spare9__ =3D 0x0, isc_disable_msi= x =3D 0, isc_txrx =3D 0xffffffff810e7680 , isc_media =3D 0x0}, ifc_ctx= _sx =3D {lock_object =3D {lo_name =3D 0xffffffff80be034d "iflib ctx lock", lo_flags =3D 36896768, lo_data =3D 0, lo_witness =3D 0x0}, sx_lock = =3D 1}, ifc_state_mtx =3D {lock_object =3D {lo_name =3D 0xfffff80002dac7d0 "vmx0", = lo_flags =3D 16973824, lo_data =3D 0, lo_witness =3D 0x0}, mtx_lock =3D 0}, ifc_txqs =3D 0xfffffe00117e5000, ifc_rxqs =3D 0xfffffe00003fe000, ifc_if_= flags =3D 34819, ifc_flags =3D 112, ifc_max_fl_buf_size =3D 2048, ifc_rx_mbuf_sz =3D = 2048, ifc_link_state =3D 2, ifc_watchdog_events =3D 0, ifc_led_dev =3D 0x0, ifc_msix_mem =3D 0xfffff80002d15480, ifc_legacy_irq =3D {ii_res =3D 0x0, = __spare0__ =3D 0, ii_tag =3D 0x0}, ifc_admin_task =3D {gt_task =3D {ta_link =3D {stqe_= next =3D 0x0}, ta_flags =3D 2, ta_priority =3D 0, ta_func =3D 0xffffffff808acd60 <_task_fn_admin>, ta_context =3D 0xfffff80002d22400}, gt_taskqueue =3D 0xfffff80002923500, gt_list =3D {le_n= ext =3D 0x0, le_prev =3D 0xfffffe00007fb000}, gt_uniq =3D 0xfffff80002d22400, gt_name =3D "admin", '\000' , gt_dev =3D 0x0, gt_irq = =3D 0x0, gt_cpu =3D -1}, ifc_vflr_task =3D {gt_task =3D {ta_link =3D {stqe_next =3D = 0x0}, ta_flags =3D 0, ta_priority =3D 0, ta_func =3D 0x0, ta_context =3D 0x0}, gt_taskqueue =3D 0x0, gt_list =3D {le_next =3D 0x0, le_prev =3D 0x0}, g= t_uniq =3D 0x0, gt_name =3D '\000' , gt_dev =3D 0x0, gt_irq =3D 0x0,= gt_cpu =3D 0}, ifc_filter_info =3D { ifi_filter =3D 0xffffffff80af8550 , ifi_filter_arg = =3D 0xfffff80002d22000, ifi_task =3D 0xfffff80002d225f8, ifi_ctx =3D 0xfffff80002d22400}, ifc_media =3D {ifm_mask =3D -268435456, ifm_media =3D = 0, ifm_cur =3D 0xfffff8000437fcc0, ifm_list =3D {lh_first =3D 0xfffff80004= 37fcc0}, ifm_change =3D 0xffffffff808b6d40 , ifm_status =3D 0xffffffff808b6de0 }, ifc_mediap =3D 0xfffff80002d22708, ifc_sysctl_node =3D 0xfffff80002d16d80, ifc_sysctl_ntxqs =3D 0, ifc_sysct= l_nrxqs =3D 0, ifc_sysctl_qs_eq_override =3D 0, ifc_sysctl_rx_budget =3D 0, ifc_sysctl_tx_abdicate =3D 0, ifc_sysctl_core_offset =3D 0, ifc_sysctl_separate_txrx =3D 0 '\000', ifc_sysctl_ntxds =3D {0, 0, 0, 0, = 0, 0, 0, 0}, ifc_sysctl_nrxds =3D {0, 0, 0, 0, 0, 0, 0, 0}, ifc_txrx =3D {ift_txd_en= cap =3D 0xffffffff80af5190 , ift_txd_flush =3D 0xffffffff80af5430 , ift_txd_credits_update =3D 0xffffffff80af5480 , ift_rxd_available =3D 0xffffffff80af5560 , ift_rxd_pkt_get =3D 0xffffffff80af5690 , ift_rxd_refill =3D 0xffffffff80af5970 , ift_rxd_flu= sh =3D 0xffffffff80af5a50 , ift_legacy_intr =3D 0xffffffff80af5ab0 }, ifc_vlan_attach_event =3D 0xfffff80002da2300, ifc_vlan_detach_event =3D 0xfffff80002da22c0, ifc_mac =3D {octet =3D "\000PV\243\265\275"}} (kgdb) p $5.ifc_softc_ctx $8 =3D {isc_vectors =3D 9, isc_nrxqsets =3D 8, isc_ntxqsets =3D 8, __spare0= __ =3D 0, __spare1__ =3D 0, isc_msix_bar =3D 24, isc_tx_nsegments =3D 32, isc_ntxd = =3D {512, 512, 0, 0, 0, 0, 0, 0}, isc_nrxd =3D {512, 256, 256, 0, 0, 0, 0, 0}, isc_txqsizes =3D {8192, 8192, 0, 0, 0, 0, 0, 0}, isc_rxqsizes =3D {8192, = 4096, 4096, 0, 0, 0, 0, 0}, isc_txd_size =3D "\000\000\000\000\000\000\000", isc_rxd_size =3D "\000\000\000\000\000\000\000", isc_tx_tso_segments_max = =3D 32, isc_tx_tso_size_max =3D 65532, isc_tx_tso_segsize_max =3D 16383, isc_tx_csum_flags =3D 5654, isc_capabilities =3D 6621115, isc_capenable =3D= 6554555, isc_rss_table_size =3D 128, isc_rss_table_mask =3D 127, isc_nrxqsets_max = =3D 8, isc_ntxqsets_max =3D 8, __spare2__ =3D 0, isc_intr =3D IFLIB_INTR_MSIX, isc_max_frame_size =3D 1522, isc_min_frame_size =3D 0, isc_pause_frames =3D= 0, __spare3__ =3D 0, __spare4__ =3D 0, __spare5__ =3D 0, __spare6__ =3D 0, __s= pare7__ =3D 0, __spare8__ =3D 0, __spare9__ =3D 0x0, isc_disable_msix =3D 0, isc_txrx =3D 0xffffffff810e7680 , isc_media =3D 0x0} (kgdb) p/x $7.isc_flags $9 =3D 0x9 So, it seems that because isc_flags has IFLIB_HAS_RXCQ bit, cidx was taken = from rxq->ifr_cq_cidx (477) in iflib_rxeof(). But rxd_frag_to_sd() asserts that fl->ifl_cidx =3D=3D irf->irf_idx, but if = we look at rxq->ifr_frags, none of the fragments has that ID. In fact: (kgdb) p rxq->ifr_frags[0] $11 =3D {irf_flid =3D 0 '\000', irf_idx =3D 142, irf_len =3D 1514} And that's where the assertion trips, 142 !=3D 477. --=20 You are receiving this mail because: You are the assignee for the bug.=