From nobody Sun Dec 17 10:00:30 2023 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4StJNt3yDzz54PMb for ; Sun, 17 Dec 2023 10:00:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4StJNt0XRzz3Q9t for ; Sun, 17 Dec 2023 10:00:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1702807230; a=rsa-sha256; cv=none; b=CJbe81cDTKXklp5E8NRIoJoL7awq4x+Q9Qw9yUrDQYkmFgi6fnQFnBDjwvcadRXSPMtTZu UuZgCya08Ibw4RPGETPkFhTwk01bSyMqU08arCRvvPp4chBNPuOcDY9QhjWSK9d6eQ8jlQ VPbqIQaNMEc20Q/KW4nXHqiT8Ys+fDm9QxoI+P2DCo1Ri96iuis/mNLzD2v7WvnOL4WtZq 6c5w3Zci321ay/Ai0dlRA/onJSm2hSkNjn/QafT1cWzynehuoUWmQ1bIOrF1NOQo5wjoAW Wliniby5j3O1jOpplLS2cRisb4s6A8qB3WdArXKhXBmiZ+QHNF1yuB0fphJXeg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1702807230; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Vg0zFmEJJtinu6xtrzf3pPlyT2fMVPhWDw/Cq9bUwVM=; b=xGzQqhHW+D8oal/ZYm6ZzQ4S9wtHKSu7+4x6HX/CBgzTIjFE0GyulqcOqYBcnD1B3cXUHJ AEBOtdNtMLyrSYYYOGQTEZdb5jEk4v5KLUgvXL8ClemIVxkm1LDKlL+Dq18pTdGHHYkcyo 4L4E36BoDmrxvr9Fn6eDmQ0eTUrfqtss3CqbqePHPew2eBhrFp2Zt+cxCXHtL8BJSCRbAR Wlji3IDBQyolRUfPV9+oGPpKqu+w5Q5VtacM2ocELpo9bFMl1KGI9I4AhsCLHFOfKVtGD3 B9UtO+ZBSBuyKOgjDln2jxyw49fUL0FALYNAA3Sj7QXrjFpYG7rRwpZZnXECrg== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4StJNs6kW0zYWT for ; Sun, 17 Dec 2023 10:00:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 3BHA0TL6016370 for ; Sun, 17 Dec 2023 10:00:29 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 3BHA0TZE016369 for bugs@FreeBSD.org; Sun, 17 Dec 2023 10:00:29 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 275798] panic: sackhint bytes rtx >= 0 Date: Sun, 17 Dec 2023 10:00:30 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ddaniel@nvidia.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D275798 Bug ID: 275798 Summary: panic: sackhint bytes rtx >=3D 0 Product: Base System Version: CURRENT Hardware: Any OS: Any Status: New Severity: Affects Some People Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: ddaniel@nvidia.com During IPSEC regression, got Kernel Panic=20 when ran traffic over tunnel mode and did ipsec restart in parallel Unread portion of the kernel message buffer: [8118.158276] panic: sackhint bytes rtx >=3D 0 [8118.169026] cpuid =3D 8 [8118.183306] time =3D 1702046494 [8118.197593] KDB: stack backtrace: [8118.212342] db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe01579543b0 [8118.227297] vpanic() at vpanic+0x132/frame 0xfffffe01579544e0 [8118.241552] panic() at panic+0x43/frame 0xfffffe0157954540 [8118.255964] tcp_sack_doack() at tcp_sack_doack+0x83c/frame 0xfffffe015795= 45d0 [8118.270235] tcp_do_segment() at tcp_do_segment+0x20f2/frame 0xfffffe01579546b0 [8118.284532] tcp_input_with_port() at tcp_input_with_port+0x1157/frame 0xfffffe0157954800 [8118.298806] tcp_input() at tcp_input+0xb/frame 0xfffffe0157954810 [8118.313084] ip_input() at ip_input+0x2ab/frame 0xfffffe0157954870 [8118.327340] netisr_dispatch_src() at netisr_dispatch_src+0xad/frame 0xfffffe01579548d0 [8118.341622] gif_input() at gif_input+0x241/frame 0xfffffe0157954920 [8118.355939] in_gif_input() at in_gif_input+0x74/frame 0xfffffe0157954960 [8118.370225] encap_input() at encap_input+0xd7/frame 0xfffffe01579549d0 [8118.384523] encap4_input() at encap4_input+0x26/frame 0xfffffe0157954a00 [8118.398803] ip_input() at ip_input+0x2ab/frame 0xfffffe0157954a60 [8118.413058] netisr_dispatch_src() at netisr_dispatch_src+0xad/frame 0xfffffe0157954ac0 [8118.427347] ether_demux() at ether_demux+0x17a/frame 0xfffffe0157954af0 [8118.441647] ether_nh_input() at ether_nh_input+0x39f/frame 0xfffffe015795= 4b40 [8118.455928] netisr_dispatch_src() at netisr_dispatch_src+0xad/frame 0xfffffe0157954ba0 [8118.470195] ether_input() at ether_input+0xd9/frame 0xfffffe0157954c00 [8118.484537] tcp_lro_flush_all() at tcp_lro_flush_all+0xdc/frame 0xfffffe0157954c50 [8118.498580] mlx5e_rx_cq_comp() at mlx5e_rx_cq_comp+0x1241/frame 0xfffffe0157954d70 [8118.512841] mlx5_cq_completion() at mlx5_cq_completion+0x74/frame 0xfffffe0157954dd0 [8118.527130] mlx5_eq_int() at mlx5_eq_int+0x28a/frame 0xfffffe0157954e30 [8118.541408] mlx5_msix_handler() at mlx5_msix_handler+0x15/frame 0xfffffe0157954e40 [8118.555698] lkpi_irq_handler() at lkpi_irq_handler+0x29/frame 0xfffffe0157954e60 [8118.570117] ithread_loop() at ithread_loop+0x266/frame 0xfffffe0157954ef0 [8118.584389] fork_exit() at fork_exit+0x82/frame 0xfffffe0157954f30 [8118.598289] fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe015795= 4f30 [8118.611898] --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- [8118.626197] Uptime: 2h15m18s [8118.663701] Dumping 1731 out of 32701 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% __curthread () at /usr/kernel_git/sys/amd64/include/pcpu_aux.h:57 57 __asm("movq %%gs:%P1,%0" : "=3Dr" (td) : "n" (offsetof(stru= ct pcpu, (kgdb) #0 __curthread () at /usr/kernel_git/sys/amd64/include/pcpu_aux.h:57 #1 doadump (textdump=3Dtextdump@entry=3D1) at /usr/kernel_git/sys/kern/kern_shutdown.c:403 #2 0xffffffff80b3b390 in kern_reboot (howto=3D260) at /usr/kernel_git/sys/kern/kern_shutdown.c:521 #3 0xffffffff80b3b88f in vpanic (fmt=3D0xffffffff8114668e "sackhint bytes = rtx >=3D 0", ap=3Dap@entry=3D0xfffffe0157954520) at /usr/kernel_git/sys/kern/kern_shutdown.c:965 #4 0xffffffff80b3b633 in panic (fmt=3D) at /usr/kernel_git/sys/kern/kern_shutdown.c:889 #5 0xffffffff80d3ba6c in tcp_sack_doack (tp=3D, to=3D0xfffffe01579545e8, th_ack=3D) at /usr/kernel_git/sys/netinet/tcp_sack.c:779 #6 0xffffffff80d2cbe2 in tcp_do_segment (tp=3D0xfffff8022bcf0000, tp@entry=3D, m=3D0xfffff80368fe5000,=20 m@entry=3D, th=3D0xfffff80368fe5098, th@entry=3D,=20 drop_hdrlen=3D64, drop_hdrlen@entry=3D, tlen=3D,=20 tlen@entry=3D, iptos=3D, iptos@entry=3D) at /usr/kernel_git/sys/netinet/tcp_input.c:2501 #7 0xffffffff80d2a0a7 in tcp_input_with_port (mp=3D, offp=3D, proto=3D, port=3Dport@entry=3D0) at /usr/kernel_git/sys/netinet/tcp_input.c:1160 #8 0xffffffff80d2aadb in tcp_input (mp=3D, offp=3D, proto=3D) at /usr/kernel_git/sys/netinet/tcp_input.c:1492 #9 0xffffffff80d1888b in ip_input (m=3D0x0, m@entry=3D) at /usr/kernel_git/sys/netinet/ip_input.c:861 #10 0xffffffff80c9393d in netisr_dispatch_src (proto=3Dproto@entry=3D1, source=3Dsource@entry=3D0, m=3D0xfffff80368fe5000) at /usr/kernel_git/sys/net/netisr.c:1152 #11 0xffffffff80c93c7f in netisr_dispatch (proto=3D, proto@ent= ry=3D1, m=3D) at /usr/kernel_git/sys/net/netisr.c:1243 #12 0xffffffff80c77da1 in gif_input (m=3D, m@entry=3D0xfffff80368fe5000, ifp=3Difp@entry=3D0xfffff8000b26c800, proto=3Dproto@entry=3D4,=20 ecn=3D, ecn@entry=3D0 '\000') at /usr/kernel_git/sys/net/if_gif.c:587 #13 0xffffffff80d092c4 in in_gif_input (m=3D0xfffff80368fe5000, off=3D20, p= roto=3D4, arg=3D) at /usr/kernel_git/sys/netinet/in_gif.c:315 #14 0xffffffff80d15717 in encap_input (head=3D, m=3D0xfffff80368fe5000, off=3D20, proto=3Dproto@entry=3D4) at /usr/kernel_git/sys/netinet/ip_encap.c:328 #15 0xffffffff80d15606 in encap4_input (mp=3D0xfffffe0157954a30, offp=3D0xfffffe0157954a2c, proto=3D4) at /usr/kernel_git/sys/netinet/ip_enc= ap.c:370 #16 0xffffffff80d1888b in ip_input (m=3D0xfffff80368fe5000, m@entry=3D) at /usr/kernel_git/sys/netinet/ip_input.c:861 #17 0xffffffff80c9393d in netisr_dispatch_src (proto=3Dproto@entry=3D1, source=3Dsource@entry=3D0, m=3D0xfffff80368fe5000) at /usr/kernel_git/sys/net/netisr.c:1152 #18 0xffffffff80c93c7f in netisr_dispatch (proto=3D, proto@ent= ry=3D1, m=3D) at /usr/kernel_git/sys/net/netisr.c:1243 #19 0xffffffff80c7610a in ether_demux (ifp=3Difp@entry=3D0xfffff80112341800, m=3D) at /usr/kernel_git/sys/net/if_ethersubr.c:952 #20 0xffffffff80c7779f in ether_input_internal (ifp=3D0xfffff80112341800, m=3D) at /usr/kernel_git/sys/net/if_ethersubr.c:716 #21 ether_nh_input (m=3D, m@entry=3D) at /usr/kernel_git/sys/net/if_ethersubr.c:746 #22 0xffffffff80c9393d in netisr_dispatch_src (proto=3Dproto@entry=3D5, source=3Dsource@entry=3D0, m=3Dm@entry=3D0xfffff80368fe5000) at /usr/kernel_git/sys/net/netisr.c:1152 #23 0xffffffff80c93c7f in netisr_dispatch (proto=3D, proto@ent= ry=3D5, m=3D, m@entry=3D0xfffff80368fe5000) at /usr/kernel_git/sys/net/netisr.c:1243 #24 0xffffffff80c76569 in ether_input (ifp=3D0xfffff80112341800, ifp@entry= =3D, m=3D0xfffff80368fe5000,=20 m@entry=3D) at /usr/kernel_git/sys/net/if_ethersubr.c:857 #25 0xffffffff80d33f4c in tcp_lro_flush_all (lc=3D0xfffffe0158ae5dc0) at /usr/kernel_git/sys/netinet/tcp_lro.c:1255 #26 0xffffffff82778971 in mlx5_cq_arm (cq=3D0xfffffe0158ae5ce0, cmd=3D0, uar_page=3D, doorbell_lock=3D0x0, cons_index=3D) --Type for more, q to quit, c to continue without paging-- at ../../dev/mlx5/cq.h:149 #27 mlx5e_cq_arm (cq=3D0xfffffe0158ae5cc0, dblock=3D0x0) at ../../dev/mlx5/mlx5_en/en.h:1273 #28 mlx5e_rx_cq_comp (mcq=3D, eqe=3D) at /tmp/ofed_freebsd_3_0/sys/modules/mlx5en/../../dev/mlx5/mlx5_en/mlx5_en_rx.= c:672 #29 0xffffffff826a80a4 in mlx5_cq_completion (dev=3D0xfffffe01581e6000, eqe=3D0xfffffe0158316bc0) at /tmp/ofed_freebsd_3_0/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_cq.c:73 #30 0xffffffff826ab2ba in mlx5_eq_int (dev=3D0xfffffe01581e6000, eq=3D0xfffff800056a5700) at /tmp/ofed_freebsd_3_0/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_eq.c:2= 75 #31 0xffffffff826aab35 in kfree (ptr=3D0xfffff80003c2de00) at ../../compat/linuxkpi/common/include/linux/slab.h:201 #32 mlx5_create_map_eq (dev=3D0xfffff8004e10e740, eq=3D0xfffff80003c2de58, vecidx=3D, nent=3D, mask=3D8590000128) at /tmp/ofed_freebsd_3_0/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_eq.c:4= 98 #33 0xffffffff826f3a99 in ?? () at /tmp/ofed_freebsd_3_0/sys/modules/linuxkpi/../../compat/linuxkpi/common/src= /linux_interrupt.c:110 from /boot/kernel/linuxkpi.ko #34 0x0000000000000000 in ?? () (kgdb)=20 PANIC: r-freeb08.mtr.labs.mlnx dumped core - see /var/crash/vmcore.6 Fri Dec 8 16:44:17 IST 2023 FreeBSD r-freeb08.mtr.labs.mlnx 15.0-CURRENT FreeBSD 15.0-CURRENT #142 ipsec-n266961-b5163c1700a8: Fri Dec 8 14:14:09 IST 2023=20=20=20=20 root@r-freeb08.mtr.labs.mlnx:/usr/obj/usr/kernel_git/amd64.amd64/sys/LATEST= =20 amd64 --=20 You are receiving this mail because: You are the assignee for the bug.=