From nobody Fri Aug 4 21:11:48 2023 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RHdgn2CK3z4m3fT for ; Fri, 4 Aug 2023 21:11:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RHdgm6d64z4YZd for ; Fri, 4 Aug 2023 21:11:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691183508; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=xURcVAo39fihzL24Vrn1ZJ2i2h0EGhyxBySN2JvHHeo=; b=pUCiVLpLdzM/XE5jteHIr4F+uwnU5ulnsNLY0aZwFUcUnWSjSL5+PfyFLAttCMGrqc3Qya Rf94oq8S2P2LFRqeyqeXa55lglOBm3A64MLhAvp2lTlKNQnvEgtEgVw1F3jX2pgSwhbhrD m5X/zq73SNz/35G1jEjr1yl09vHSyoPFeQxg9wXhRAzRr4UlKELQ4DK0I6+QDfHyZcCvrY mb9Kk4foiQaxJx2InjZiGT0Gq6+aqdFt0VjxL9M14vGNSSG0/RAWsdAGxXThWSqR0hDAoS COPdfTz9NnAacgY4KWr7ToHliCgEnDrDP5fQuuRVbMRJ6zinQmoITGfEFfDBag== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1691183508; a=rsa-sha256; cv=none; b=WtXpSS2fuXqvzz5UA9ombY5ZnoptDAp98l+601W6WJ+EjsCB/G3MA64moxH0IuqreJ3UDP VorM7QWlozHjLsMhdwG7T+Mp/eoIvqs9LOoGcIZWCo4TJz1ZmmX1kPGVMStPvA/alBINfn pevGmW3ntmCjWMVsVzu18QaP+udG24z2HwC7+DDiLf8amW5m49HT/Ajk2zWOaqyvO7XQ0y 9SUMfJAPgccQ2FsC9G8WJwT7IArZuJyMVY0bAZBiIBVMUqibFV5Zl4udZos1UL9UaMq7lu 64jX8ycfV3L0DYTgb5oVE+MneyrZQymIncS8QknUbAOgO3OtPdLkRoPBTmijPg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4RHdgm5hBNz1HgT for ; Fri, 4 Aug 2023 21:11:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 374LBmYL030846 for ; Fri, 4 Aug 2023 21:11:48 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 374LBmfw030845 for bugs@FreeBSD.org; Fri, 4 Aug 2023 21:11:48 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 272947] cxgbei: kernel panic in soreceive when hw.cxgbe.nofldtxq="-24" Date: Fri, 04 Aug 2023 21:11:48 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: asomers@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D272947 Bug ID: 272947 Summary: cxgbei: kernel panic in soreceive when hw.cxgbe.nofldtxq=3D"-24" Product: Base System Version: CURRENT Hardware: Any OS: Any Status: New Severity: Affects Only Me Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: asomers@FreeBSD.org I just hit this kernel panic four times in a row when attempting to use cxg= bei to serve zvol-backed CTL targets. The key thing seems to be that hw.cxgbe.{nofldtxq,nofldrxq}=3D"-24". The panic did not happen when the va= lue was "-8" or the default. After starting ctld, the panic happens within a second or two of connecting the clients. < in /boot/loader.conf > hw.cxgbe.nofldtxq=3D"-24" hw.cxgbe.nofldrxq=3D"-24" t4_tom_load=3D"YES" cxgbei_load=3D"YES" < in /etc/ctl.conf > portal-group pg0 { discovery-auth-group ag0 discovery-filter portal-name-auth listen 172.30.10.58 offload cxgbei } target iqn.2018-10.net.MYDOMAIN.MYHOST:zd0 { auth-group no-authentication portal-group pg0 portal-group pg1 portal-group pg2 portal-group pg3 lun 0 { path /dev/zvol/MYPOOL/MYDATASET blocksize 4096 option pblocksize 32768 } } $ sudo ifconfig cc0 toe $ sudo ifconfig cc1 toe $ sudo ifconfig cc2 toe $ sudo ifconfig cc3 toe $ sudo zfs load-key -a $ sudo service ctld onestart < Connect clients, if they weren't already connected from before reboot > Fatal trap 12: page fault while in kernel mode cpuid =3D 13; apic id =3D 42=20 fault virtual address =3D 0x0 fault code =3D supervisor read instruction, page not present instruction pointer =3D 0x20:0x0 stack pointer =3D 0x28:0xfffffe06058c2d38 frame pointer =3D 0x28:0xfffffe06058c2d50 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 2255 (ctld) rdi: fffff8018e70e000 rsi: 0000000000000000 rdx: fffffe06058c2da0 rcx: 0000000000000000 r8: 0000000000000000 r9: 0000000000000000 rax: ffffffff81d5c148 rbx: fffffe04dedf2ac0 rbp: fffffe06058c2d50 r10: d55588aa548cb865 r11: fffffe04dedf2fe0 r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 r15: fffffe04dedf2ac0 trap number =3D 12 panic: page fault cpuid =3D 13 time =3D 1691182043 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe06058c2= af0 vpanic() at vpanic+0x148/frame 0xfffffe06058c2b40 panic() at panic+0x43/frame 0xfffffe06058c2ba0 trap_fatal() at trap_fatal+0x40b/frame 0xfffffe06058c2c00 trap_pfault() at trap_pfault+0x4f/frame 0xfffffe06058c2c60 calltrap() at calltrap+0x8/frame 0xfffffe06058c2c60 --- trap 0xc, rip =3D 0, rsp =3D 0xfffffe06058c2d38, rbp =3D 0xfffffe06058c= 2d50 --- ??() at 0/frame 0xfffffe06058c2d50 dofileread() at dofileread+0x83/frame 0xfffffe06058c2d90 sys_read() at sys_read+0xbc/frame 0xfffffe06058c2e00 amd64_syscall() at amd64_syscall+0x770/frame 0xfffffe06058c2f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe06058c2f30 --- syscall (3, FreeBSD ELF64, read), rip =3D 0x36bdda97469a, rsp =3D 0x36bdd3d2c558, rbp =3D 0x36bdd3d2c580 --- KDB: enter: panic __curthread () at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/amd64/include/pcpu_aux= .h:59 59 __asm("movq %%gs:%P1,%0" : "=3Dr" (td) : "n" (offsetof(stru= ct pcpu, (kgdb) #0 __curthread () at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/amd64/include/pcpu_aux= .h:59 #1 doadump (textdump=3Dtextdump@entry=3D0) at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/kern/kern_shutdown.c:4= 07 #2 0xffffffff804a2c8a in db_dump (dummy=3D, dummy2=3D, dummy3=3D, dummy4=3D) at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/ddb/db_command.c:593 #3 0xffffffff804a2a90 in db_command (last_cmdp=3D, cmd_table=3D, dopager=3Dfalse) at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/ddb/db_command.c:506 #4 0xffffffff804a2bd6 in db_command_script ( command=3Dcommand@entry=3D0xffffffff817986d0 "dump") at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/ddb/db_command.c:571 #5 0xffffffff804a8198 in db_script_exec ( scriptname=3Dscriptname@entry=3D0xfffffe06058c27d0 "kdb.enter.panic", warnifnotfound=3Dwarnifnotfound@entry=3D0) at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/ddb/db_script.c= :304 #6 0xffffffff804a7fb2 in db_script_kdbenter (eventname=3D) at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/ddb/db_script.c= :326 #7 0xffffffff804a6051 in db_trap (type=3D, code=3D) at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/ddb/db_main.c:2= 69 #8 0xffffffff80ba4956 in kdb_trap (type=3Dtype@entry=3D3, code=3D, code@entry=3D0, tf=3Dtf@entry=3D0xfffffe06058c2a30) at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/kern/subr_kdb.c= :792 #9 0xffffffff81028851 in trap (frame=3D0xfffffe06058c2a30) at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/amd64/amd64/trap.c:610 #10 #11 kdb_enter (why=3D, msg=3D) at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/kern/subr_kdb.c= :558 #12 0xffffffff80b54dc9 in vpanic (fmt=3D, ap=3Dap@entry=3D0xfffffe06058c2b80) at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/kern/kern_shutdown.c:9= 60 #13 0xffffffff80b54c43 in panic ( fmt=3D0xffffffff816b7740 "\020\2722\201\377\377\377\377\0= 01") at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/kern/kern_shutdown.c:8= 96 #14 0xffffffff81028d0b in trap_fatal (frame=3D0xfffffe06058c2c70, eva=3D0) at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/amd64/amd64/trap.c:954 #15 0xffffffff81028d5f in trap_pfault (frame=3D0xfffffe06058c2c70, usermode=3Dfalse, signo=3D, ucode=3D) at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/amd64/amd64/trap.c:762 #16 #17 0x0000000000000000 in ?? () #18 0xffffffff80c00cbf in soreceive (so=3D0xfffff8018e70e000, psa=3D0x0, uio=3D0xfffffe06058c2da0, mp0=3D0x0, controlp=3D0x0, flagsp=3D0x0) at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/kern/uipc_socket.c:2910 #19 0xffffffff80bc65f3 in fo_read (uio=3D0xfffffe06058c2da0, active_cred=3D0xfffffe06058c2da0, td=3D0xfffffe04dedf2ac0, fp=3D, flags=3D) at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/sys/file.h:343 #20 dofileread (td=3Dtd@entry=3D0xfffffe04dedf2ac0, fd=3Dfd@entry=3D11, fp=3D, auio=3Dauio@entry=3D0xfffffe06058c2da0, offset=3Doffset@entry=3D-1, flags=3Dflags@entry=3D0) at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/kern/sys_generic.c:369 #21 0xffffffff80bc614c in kern_readv (td=3D0xfffffe04dedf2ac0, fd=3D11, auio=3D0xfffffe06058c2da0) at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/kern/sys_generic.c:290 #22 sys_read (td=3D0xfffffe04dedf2ac0, uap=3D) at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/kern/sys_generic.c:206 #23 0xffffffff81029c50 in syscallenter (td=3D0xfffffe04dedf2ac0) at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/amd64/amd64/../../kern= /subr_syscall.c:162 #24 amd64_syscall (td=3D0xfffffe04dedf2ac0, traced=3D0) at /usr/home/asomers/src/github/Axcient/freebsd-src/sys/amd64/amd64/trap.c:1199 #25 #26 0x000036bdda97469a in ?? () --=20 You are receiving this mail because: You are the assignee for the bug.=