From nobody Thu Jul 20 11:19:33 2023 X-Original-To: net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4R69FL5Sgpz4pBM3 for ; Thu, 20 Jul 2023 11:19:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4R69FL4BN5z3h7v for ; Thu, 20 Jul 2023 11:19:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1689851974; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=6bClb+ts8JyrWgDJY+btx3CiNV5oTojwW2mhiseSMhc=; b=rhIJVik1xS5qeAoDQyfo2qkrifhQYGk4E7DRgUaE1eOql/KxtbDbxuvirJWlBuBhw4rQCe XpbBo39VQXmxd8uTzaSdRzyQ8ahXkheeBR5D5OsVfmV/d7CQzECDNTy+g9GuOk93ndgnV+ RuLmdSynGs+QgE20OjbeMIIUe9Kd9dFaNA5taUuePLQ97dFXQB9kezN1y1G31eSLzX6t39 8ltbcByhxeWAflMKMedhkLJbMnrlFRazCAvK7f+rNpg7Z/IJJs84FJMetAEN8dDOFnszpa pzqifrU3D6H1YQXhi5w1ntCqQm0jsnAraOrHCvC7FRBj1neErveWLtUmSH1hSQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1689851974; a=rsa-sha256; cv=none; b=gpfeZH7oumBRczWuBYMyxhy1gLtvJ2Hyvy9BsoD1ZRd899KXvxGQGineFa6k1K44AIvnOy Nw3B5Pf2f25F33FVSZR0xWuMpNK22Wxmd1FweFZDgYYZ4PfgrucMFXY4mbu2G9uQ/WQe4Z 0T+pej2PIp0e2p3yuKbkP8930AuOU6tVD1SgvXCLxq25yzbjVu0T7SkHIg6bPCHyCl97oy jG3WdwgVSSvzMV+4snSu4KK/2zhO0Bg8pKJ1wfd/A1F6PfCMPJOdg4LzygM1esixovlGli 6fdi6sx8glwPfRTCVB3uN8hZ62FNYH9FgRDnyxUhnkpprORb9/lX6HDIqMM2gQ== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4R69FL3DnPzTNb for ; Thu, 20 Jul 2023 11:19:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 36KBJY1M021187 for ; Thu, 20 Jul 2023 11:19:34 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 36KBJYHC021186 for net@FreeBSD.org; Thu, 20 Jul 2023 11:19:34 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 272616] [panic] Reproducible kernel panic related to sendfile and IPSec Date: Thu, 20 Jul 2023 11:19:33 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: eugen@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter cc Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D272616 Bug ID: 272616 Summary: [panic] Reproducible kernel panic related to sendfile and IPSec Product: Base System Version: 13.2-STABLE Hardware: Any OS: Any Status: New Severity: Affects Some People Priority: --- Component: kern Assignee: net@FreeBSD.org Reporter: eugen@freebsd.org CC: ae@FreeBSD.org, glebius@FreeBSD.org, kib@FreeBSD.org This PR is similar to https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D2= 54419 except of pf(4) not in use. I can reproduce the panic every attempt by fetching small plain text file (residing on ZFS) over HTTP/1.1 from my Apache httpd server using sendfile(= ). The traffic in question goes through gif(4) interface with mtu=3D1500 over = ixl0 10Gbps interface with mtu=3D1500, so some IP fragmentation should occur. First time it happened, the kernel generated crashdump just fine, rebooted = and the crashdump was saved. Next my attempt reproduced same panic but kernel h= ang after printing "Uptime: 22m27s". I can experiment with this machine freely = as it is my workstation not in service. And I have iKVM plus IPMI SOL working (serial console). Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid =3D 2; apic id =3D 04 fault virtual address =3D 0x0 fault code =3D supervisor read data, page not present instruction pointer =3D 0x20:0xffffffff810bad5a stack pointer =3D 0x28:0xfffffe011dd8f4b0 frame pointer =3D 0x28:0xfffffe011dd8f4b0 code segment =3D base 0x0, limit 0xfffff, type 0x1b Fatal trap 12: page fault while in kernel mode cpuid =3D 1; apic id =3D 02 fault virtual address =3D 0x0 fault code =3D supervisor read data, page not present instruction pointer =3D 0x20:0xffffffff810bad5a stack pointer =3D 0x28:0xfffffe01771db4e0 frame pointer =3D 0x28:0xfffffe01771db4e0 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, =3D DPL 0= , pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 81478 (httpd) trap number =3D 12 panic: page fault cpuid =3D 2 time =3D 1689822623 KDB: stack backtrace: #0 0xffffffff80c53f15 at kdb_backtrace+0x65 #1 0xffffffff80c07852 at vpanic+0x152 #2 0xffffffff80c076f3 at panic+0x43 #3 0xffffffff810bede7 at trap_fatal+0x387 #4 0xffffffff810bee3f at trap_pfault+0x4f #5 0xffffffff81096a78 at calltrap+0x8 #6 0xffffffff80c9c999 at m_unshare+0x3a9 #7 0xffffffff82d19534 at esp_output+0x184 #8 0xffffffff82d15fc6 at ipsec4_perform_request+0x3b6 #9 0xffffffff82d16113 at ipsec4_common_output+0x83 #10 0xffffffff80e3894c at ipsec_kmod_output+0x2c #11 0xffffffff80dbc6df at ip_output+0xb8f #12 0xffffffff80dd3a54 at tcp_output+0x1d74 #13 0xffffffff80de599f at tcp_usr_send+0x17f #14 0xffffffff80c04ff1 at vn_sendfile+0x1251 #15 0xffffffff80c05fa7 at sendfile+0x117 #16 0xffffffff810bf6dc at amd64_syscall+0x10c #17 0xffffffff8109738b at fast_syscall_common+0xf8 Uptime: 4d5h15m40s Dumping 2283 out of 16249 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..= 91% warning: Could not load shared library symbols for nvidia.ko. Do you need "set solib-search-path" or "set sysroot"? __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 55 __asm("movq %%gs:%P1,%0" : "=3Dr" (td) : "n" (offsetof(stru= ct pcpu, ESC[?2004h(kgdb) bt ESC[?2004l#0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 #1 doadump (textdump=3D) at /usr/src/sys/kern/kern_shutdown= .c:396 #2 0xffffffff80c07419 in kern_reboot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:484 #3 0xffffffff80c078bf in vpanic (fmt=3D, ap=3Dap@entry=3D0xfffffe011dd8f300) at /usr/src/sys/kern/kern_shutdown.c:923 #4 0xffffffff80c076f3 in panic (fmt=3D) at /usr/src/sys/kern/kern_shutdown.c:847 #5 0xffffffff810bede7 in trap_fatal (frame=3D0xfffffe011dd8f3f0, eva=3D0) at /usr/src/sys/amd64/amd64/trap.c:942 #6 0xffffffff810bee3f in trap_pfault (frame=3D0xfffffe011dd8f3f0, usermode=3Dfalse, signo=3D, ucode=3D) at /usr/src/sys/amd64/amd64/trap.c:761 #7 #8 memcpy_erms () at /usr/src/sys/amd64/amd64/support.S:553 #9 0xffffffff80c9c999 in m_unshare (m0=3D0xfffff80146cc8200, how=3D1) at /usr/src/sys/kern/uipc_mbuf.c:2047 #10 0xffffffff82d19534 in esp_output () from /boot/kernel/ipsec.ko #11 0xffffffff82d15fc6 in ipsec4_perform_request () from /boot/kernel/ipsec= .ko #12 0xffffffff82d16113 in ipsec4_common_output () from /boot/kernel/ipsec.ko #13 0xffffffff80e3894c in ipsec_kmod_output (sc=3D0xfffff8001828ea00, sc@entry=3D0x18, m=3D0xfffff8002a388925, inp=3D0x3f8, inp@entry=3D0xfffff80133df99b0) at /usr/src/sys/netipsec/subr_ipsec.c:369 #14 0xffffffff80dbc6df in ip_output (m=3D0x0, m@entry=3D0xfffff80146cc8200, opt=3D, ro=3D, flags=3D0, imo=3D0x10, imo@entry=3D0x0, inp=3D0xfffff80133df99b0) at /usr/src/sys/netinet/ip_output.c:680 #15 0xffffffff80dd3a54 in tcp_output (tp=3D0xfffffe011d38d518) at /usr/src/sys/netinet/tcp_output.c:1541 #16 0xffffffff80de599f in tcp_usr_send (so=3D0xfffff8002a50cb10, flags=3D0,= m=3D0x0, nam=3D0x0, control=3D, td=3D0xfffffe0176dcb720) at /usr/src/sys/netinet/tcp_usrreq.c:1178 #17 0xffffffff80c04ff1 in vn_sendfile (fp=3D, sockfd=3D22, hdr_uio=3D0x0, trl_uio=3D0x0, offset=3D, nbytes=3D1038, sent=3D0xfffffe011dd8fdc8, fla= gs=3D0, td=3D0xfffffe0176dcb720) at /usr/src/sys/kern/kern_sendfile.c:1188 #18 0xffffffff80c05fa7 in fo_sendfile (fp=3D0xfffff8002a388925, sockfd=3D0, hdr_uio=3D0x3f8, trl_uio=3D0x3f8, offset=3D-2194227530512, nbytes=3D9, sent=3D0xfffffe01= 1dd8fdc8, flags=3D708348197, td=3D0xfffffe0176dcb720) at /usr/src/sys/sys/file.h:416 #19 sendfile (td=3D0xfffffe0176dcb720, uap=3D0xfffffe0176dcbb08, compat=3D<= optimized out>) at /usr/src/sys/kern/kern_sendfile.c:1326 #20 0xffffffff810bf6dc in syscallenter (td=3D0xfffffe0176dcb720) at /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:190 #21 amd64_syscall (td=3D0xfffffe0176dcb720, traced=3D0) at /usr/src/sys/amd64/amd64/trap.c:1183 #22 #23 0x0000000828695a5a in ?? () Backtrace stopped: Cannot access memory at address 0x82077d418 --=20 You are receiving this mail because: You are the assignee for the bug.=