From nobody Fri Feb 9 21:18:09 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TWmsy5Gncz5BMB7 for ; Fri, 9 Feb 2024 21:18:14 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qv1-xf29.google.com (mail-qv1-xf29.google.com [IPv6:2607:f8b0:4864:20::f29]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TWmsy1qQrz46Y1 for ; Fri, 9 Feb 2024 21:18:14 +0000 (UTC) (envelope-from markjdb@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-qv1-xf29.google.com with SMTP id 6a1803df08f44-68cc9061c78so7392926d6.3 for ; Fri, 09 Feb 2024 13:18:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707513492; x=1708118292; darn=freebsd.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:from:to:cc:subject:date:message-id :reply-to; bh=oO//GNpG3BkJGDlsxBWQOKjg9nrKalFUp6sJ//1eTzo=; b=Tq8SWaZ64dkp9HHSqfBCHFDxZ3g2O4y0a04hefzmswMazvRyFsjvw9nOPk7gDu7Yag W4DiY/oZ6YfBjA+z66UpsDDOqmgJbiGxx6S1THoCu6ICTxCaf6wWPjQFS0LmpAxouelV YBLMOSWf6A+eLjR2Y+LWsyxpiNtYr/DaMjBSTRO55HfjtFb7oz09B3zsEik+mhhzWRke 74L5raJr/EoMoqiDUeGsa+WguaQ2xsMpZI4yQejYcJe+IABJdeebVeTabYHlsiznp4cn 3myAV7jbY9mTf+wDohYJDS22ptHguZIjgQmt3Tf1OvCF3Ov5OIzvcw5YfOAFo6YxwbPG HquA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707513492; x=1708118292; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oO//GNpG3BkJGDlsxBWQOKjg9nrKalFUp6sJ//1eTzo=; b=b3qvPwGC8JpQc630HKxu9EYpYUkB+uI8CYA4Ipz0PFmKkSY8fU1UkXrtca5XIeeX+V MsGM6mklDLtil2zCq8xhR/WeRmHSKrqLyHEidow20nKRV3kH0n8k6yLxpuFex/KAwkbx Xl/WQqtexHOjzd7SlPCKncQTxsC+sZlzdwmN9bGwWS+Q63dwkhZKWi6R+dIOJar9rfTH x5sq0ZZXV64tNpezZhv49Dma17oFghyTWN3c6xuD8GtIgF4nwhQKt6mOcclYi5w4C/vi aw827ZzqpxWykek9U5N3i06WhJfK9YmvBSNNQqbd26adIV0l/klnLWBiLAxNF13bHSi4 PjhA== X-Gm-Message-State: AOJu0YxYmA3FC+GDEGdiGPNAShhA5/nMU8nSX45NXn87k+/mWERFIoTn WC44OoNFx1xnoB4vTBEP+K/8UIm43/fcei6J9Qmqxiq7sj2v++ys X-Google-Smtp-Source: AGHT+IHTZFJ412UrE4/gIz4jk25MdM4NjqaGsjsBupQiTEwU6eY90UU3OztNTQ6IWXInUZBzaELAPQ== X-Received: by 2002:a05:6214:21c9:b0:68c:a9a7:22cb with SMTP id d9-20020a05621421c900b0068ca9a722cbmr385666qvh.22.1707513492151; Fri, 09 Feb 2024 13:18:12 -0800 (PST) Received: from nuc (192-0-220-237.cpe.teksavvy.com. [192.0.220.237]) by smtp.gmail.com with ESMTPSA id a7-20020a0cefc7000000b0068cc143e310sm1154314qvt.143.2024.02.09.13.18.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Feb 2024 13:18:11 -0800 (PST) Date: Fri, 9 Feb 2024 16:18:09 -0500 From: Mark Johnston To: "Matthew L. Dailey" Cc: "freebsd-current@freebsd.org" Subject: Re: FreeBSD panics possibly caused by nfs clients Message-ID: References: List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4TWmsy1qQrz46Y1 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] On Fri, Feb 09, 2024 at 06:23:08PM +0000, Matthew L. Dailey wrote: > I had my first kernel panic with a KASAN kernel after only 01:27. This > first panic was a "double fault," which isn't anything we've seen > previously - usually we've seen trap 9 or trap 12, but sometimes others. > Based on the backtrace, it definitely looks like KASAN caught something, > but I don't have the expertise to know if this points to anything > specific. From the backtrace, it looks like this might have originated > in ipfw code. A double fault is rather unexpected. I presume you're running releng/14.0? Is it at all possible to test with FreeBSD-CURRENT? Did you add INVARIANTS etc. to the kernel configuration used here, or just KASAN? > Please let me know what other info I can provide or what I can do to dig > deeper. If you could repeat the test several times, I'd be interested in seeing if you always get the same result. If you're willing to share the vmcore (or several), I'd be willing to take a look at it. > Thanks!! > > Panic message: > [5674] Fatal double fault > [5674] rip 0xffffffff812f6e32 rsp 0xfffffe014677afe0 rbp 0xfffffe014677b430 > [5674] rax 0x1fffffc028cef620 rdx 0xf2f2f2f8f2f2f2f2 rbx 0x1 > [5674] rcx 0xdffff7c000000000 rsi 0xfffffe004086a4a0 rdi 0xf8f8f8f8f2f2f2f8 > [5674] r8 0xf8f8f8f8f8f8f8f8 r9 0x162a r10 0x835003002d3a64e1 > [5674] r11 0 r12 0xfffff78028cef620 r13 0xfffffe004086a440 > [5674] r14 0xfffffe01488c0560 r15 0x26f40 rflags 0x10006 > [5674] cs 0x20 ss 0x28 ds 0x3b es 0x3b fs 0x13 gs 0x1b > [5674] fsbase 0x95d1d81a130 gsbase 0xffffffff84a14000 kgsbase 0 > [5674] cpuid = 4; apic id = 08 > [5674] panic: double fault > [5674] cpuid = 4 > [5674] time = 1707498420 > [5674] KDB: stack backtrace: > [5674] Uptime: 1h34m34s > > Backtrace: > #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 > #1 doadump (textdump=) at > /usr/src/sys/kern/kern_shutdown.c:405 > #2 0xffffffff8128b7dc in kern_reboot (howto=howto@entry=260) > at /usr/src/sys/kern/kern_shutdown.c:526 > #3 0xffffffff8128c000 in vpanic ( > fmt=fmt@entry=0xffffffff82589a00 "double fault", > ap=ap@entry=0xfffffe0040866de0) at > /usr/src/sys/kern/kern_shutdown.c:970 > #4 0xffffffff8128bd75 in panic (fmt=0xffffffff82589a00 "double > fault") > at /usr/src/sys/kern/kern_shutdown.c:894 > #5 0xffffffff81c4b335 in dblfault_handler (frame=) > at /usr/src/sys/amd64/amd64/trap.c:1012 > #6 > #7 0xffffffff812f6e32 in sched_clock (td=td@entry=0xfffffe01488c0560, > cnt=cnt@entry=1) at /usr/src/sys/kern/sched_ule.c:2601 > #8 0xffffffff8119e2a7 in statclock (cnt=cnt@entry=1, > usermode=usermode@entry=0) at /usr/src/sys/kern/kern_clock.c:760 > #9 0xffffffff8119fb67 in handleevents (now=now@entry=24371855699832, > fake=fake@entry=0) at /usr/src/sys/kern/kern_clocksource.c:195 > #10 0xffffffff811a10cc in timercb (et=, arg=) > at /usr/src/sys/kern/kern_clocksource.c:353 > #11 0xffffffff81dcd280 in lapic_handle_timer (frame=0xfffffe014677b750) > at /usr/src/sys/x86/x86/local_apic.c:1343 > #12 > #13 __asan_load8_noabort (addr=18446741880219689232) > at /usr/src/sys/kern/subr_asan.c:1113 > #14 0xffffffff851488b8 in ?? () from /boot/thayer/ipfw.ko > #15 0xfffffe0100000000 in ?? () > #16 0xffffffff8134dcd5 in pcpu_find (cpuid=1238425856) > at /usr/src/sys/kern/subr_pcpu.c:286 > #17 0xffffffff85151f6f in ?? () from /boot/thayer/ipfw.ko > #18 0x0000000000000000 in ?? ()