From nobody Fri Feb 9 22:20:28 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TWpG82psrz5BRlG for ; Fri, 9 Feb 2024 22:20:48 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Received: from mail-pg1-x52a.google.com (mail-pg1-x52a.google.com [IPv6:2607:f8b0:4864:20::52a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TWpG816K5z4HvN; Fri, 9 Feb 2024 22:20:48 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pg1-x52a.google.com with SMTP id 41be03b00d2f7-5cf2d73a183so1917287a12.1; Fri, 09 Feb 2024 14:20:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707517246; x=1708122046; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=BKl8fko2BlWet28JRrZayCGX3dd0QVWg8EA/m6waNwk=; b=i9l4NnktXIBbXSuzXVVgTLxJJAlOIKmYDaGTb5cIA91Fg8z5iCQG25wK9Jn1FIt+B7 fKlFKovziit3J8y5Igc6lQrBgtBCbXlSx6bSxmr5+IO8y0Y6cqNrroHCI+PEP319/Sxy BsAQJMFTPm3gW4ORslqDvAwDnBblG3wVwBrz19ncu/e7EKqzjaD06x11pegsFtr+9Rbk lzYZ/vGF8VR4llq9Y6FHWMQ/ufddU+d471crsC+CIFjEDgJavMBQl+7TN49yexmvF221 NyYar4GnJyCauTpu3ftFtgrFYmtU86MAwD4KZpjpkM6JaA2Qm+PQ0mKFCAfNvUweuC4w wH5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707517246; x=1708122046; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BKl8fko2BlWet28JRrZayCGX3dd0QVWg8EA/m6waNwk=; b=Q+KlBqCbgKtTHPRAAK5E502rIJd8ph6B+c1wHPhUiUwVZHwQ1VRKmisGUPFaB4LdQj eWmyIXxjR0tW6H4WL/Zhl1GzBW7cYki9nrfhLTUwElkz/KbJCGRK5li8/kCPhmyB4yHw 3dGO2K0cwnj2VA+3FxP6I3qMHqsoKFFeuxyAyTYWVFHO8Kpwe5wC5d6hWzPThfomGXTX PIO9wTckxbgQkO8ZgQSBx658kZWX362ohXPdrbsFOTxf3Ody6bxvP4sZIaKC5RsA4UYA FYKVaUNaTpCeg/fB7Z8AJIcjGArbt4ZJXk5GtM8mBAHlduDG3Qgz8eYkIxvtI6o1Czgp gHhA== X-Forwarded-Encrypted: i=1; AJvYcCUDyRB06viFrThuWWLTWfyvig+JupBwvJshKYBzpzhesvpFiIDMh9iB4dl1yTJ/smJX/L3CRcvURlJRlv1D2Yma+hC0qaLgzy8FO9c= X-Gm-Message-State: AOJu0YxN0/k/I0NMBrVvVQZM5W4v1KIxLmAeBd5W5vSCbBSt8mj04Ajk CaxS1RIRPOHseMlWc2e9YUnUWj37h4tRPi89ok7VGOPuD0GSoYIyOXrBTdkYKYuQl9jvWwIpMd1 JmX03V0rRKmTF+WFKhwRQwDDGaA== X-Google-Smtp-Source: AGHT+IEfJpkQg1O8torNwx4RJe5ldVHoIQmV/3yaAbJBWOIgZh1SHDx3AW/w6JJuBXxUhkj/YWu3m+mMS54MtiP7EM8= X-Received: by 2002:a17:90a:9c2:b0:295:cd6f:86e0 with SMTP id 60-20020a17090a09c200b00295cd6f86e0mr3286104pjo.17.1707517245567; Fri, 09 Feb 2024 14:20:45 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Rick Macklem Date: Fri, 9 Feb 2024 14:20:28 -0800 Message-ID: Subject: Re: FreeBSD panics possibly caused by nfs clients To: Zaphod Beeblebrox Cc: Mark Johnston , "Matthew L. Dailey" , "freebsd-current@freebsd.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4TWpG816K5z4HvN X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] On Fri, Feb 9, 2024 at 2:04=E2=80=AFPM Zaphod Beeblebrox wrote: > > Just in case it's relevant, I'm carrying around this patch on my fairly b= usy little RISC-V machine. > > diff --git a/sys/fs/nfsclient/nfs_clvnops.c b/sys/fs/nfsclient/nfs_clvnop= s.c > index 0b8c587a542c..85c0ebd7a10f 100644 > --- a/sys/fs/nfsclient/nfs_clvnops.c > +++ b/sys/fs/nfsclient/nfs_clvnops.c > @@ -2459,6 +2459,16 @@ nfs_readdir(struct vop_readdir_args *ap) > return (EINVAL); > uio->uio_resid -=3D left; > > + /* > + * For readdirplus, if starting to read the directory, > + * purge the name cache, since it will be reloaded by > + * this directory read. > + * This removes potentially stale name cache entries. > + */ > + if (uio->uio_offset =3D=3D 0 && > + (VFSTONFS(vp->v_mount)->nm_flag & NFSMNT_RDIRPLUS) !=3D 0) > + cache_purge(vp); > + > /* > * Call ncl_bioread() to do the real work. > */ > ... without it, I can panic. This is not of interest to Matthew, since he is using Linux clients against= a FreeBSD server. However, it is of interest to me. This is the first time = I've seen this (unless I just forgot;-) and since readdirplus is not a default, = I suspect few test/use it. I will take a look at this, since it sounds reasonable. Thanks for posting it, rick > > > On Fri, Feb 9, 2024 at 4:18=E2=80=AFPM Mark Johnston = wrote: >> >> On Fri, Feb 09, 2024 at 06:23:08PM +0000, Matthew L. Dailey wrote: >> > I had my first kernel panic with a KASAN kernel after only 01:27. This >> > first panic was a "double fault," which isn't anything we've seen >> > previously - usually we've seen trap 9 or trap 12, but sometimes other= s. >> > Based on the backtrace, it definitely looks like KASAN caught somethin= g, >> > but I don't have the expertise to know if this points to anything >> > specific. From the backtrace, it looks like this might have originated >> > in ipfw code. >> >> A double fault is rather unexpected. I presume you're running >> releng/14.0? Is it at all possible to test with FreeBSD-CURRENT? >> >> Did you add INVARIANTS etc. to the kernel configuration used here, or >> just KASAN? >> >> > Please let me know what other info I can provide or what I can do to d= ig >> > deeper. >> >> If you could repeat the test several times, I'd be interested in seeing >> if you always get the same result. If you're willing to share the >> vmcore (or several), I'd be willing to take a look at it. >> >> > Thanks!! >> > >> > Panic message: >> > [5674] Fatal double fault >> > [5674] rip 0xffffffff812f6e32 rsp 0xfffffe014677afe0 rbp 0xfffffe01467= 7b430 >> > [5674] rax 0x1fffffc028cef620 rdx 0xf2f2f2f8f2f2f2f2 rbx 0x1 >> > [5674] rcx 0xdffff7c000000000 rsi 0xfffffe004086a4a0 rdi 0xf8f8f8f8f2f= 2f2f8 >> > [5674] r8 0xf8f8f8f8f8f8f8f8 r9 0x162a r10 0x835003002d3a64e1 >> > [5674] r11 0 r12 0xfffff78028cef620 r13 0xfffffe004086a440 >> > [5674] r14 0xfffffe01488c0560 r15 0x26f40 rflags 0x10006 >> > [5674] cs 0x20 ss 0x28 ds 0x3b es 0x3b fs 0x13 gs 0x1b >> > [5674] fsbase 0x95d1d81a130 gsbase 0xffffffff84a14000 kgsbase 0 >> > [5674] cpuid =3D 4; apic id =3D 08 >> > [5674] panic: double fault >> > [5674] cpuid =3D 4 >> > [5674] time =3D 1707498420 >> > [5674] KDB: stack backtrace: >> > [5674] Uptime: 1h34m34s >> > >> > Backtrace: >> > #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 >> > #1 doadump (textdump=3D) at >> > /usr/src/sys/kern/kern_shutdown.c:405 >> > #2 0xffffffff8128b7dc in kern_reboot (howto=3Dhowto@entry=3D260) >> > at /usr/src/sys/kern/kern_shutdown.c:526 >> > #3 0xffffffff8128c000 in vpanic ( >> > fmt=3Dfmt@entry=3D0xffffffff82589a00 "double fault", >> > ap=3Dap@entry=3D0xfffffe0040866de0) at >> > /usr/src/sys/kern/kern_shutdown.c:970 >> > #4 0xffffffff8128bd75 in panic (fmt=3D0xffffffff82589a00 "doubl= e >> > fault") >> > at /usr/src/sys/kern/kern_shutdown.c:894 >> > #5 0xffffffff81c4b335 in dblfault_handler (frame=3D) >> > at /usr/src/sys/amd64/amd64/trap.c:1012 >> > #6 >> > #7 0xffffffff812f6e32 in sched_clock (td=3Dtd@entry=3D0xfffffe01488c0= 560, >> > cnt=3Dcnt@entry=3D1) at /usr/src/sys/kern/sched_ule.c:2601 >> > #8 0xffffffff8119e2a7 in statclock (cnt=3Dcnt@entry=3D1, >> > usermode=3Dusermode@entry=3D0) at /usr/src/sys/kern/kern_clock.c:= 760 >> > #9 0xffffffff8119fb67 in handleevents (now=3Dnow@entry=3D243718556998= 32, >> > fake=3Dfake@entry=3D0) at /usr/src/sys/kern/kern_clocksource.c:19= 5 >> > #10 0xffffffff811a10cc in timercb (et=3D, arg=3D) >> > at /usr/src/sys/kern/kern_clocksource.c:353 >> > #11 0xffffffff81dcd280 in lapic_handle_timer (frame=3D0xfffffe014677b7= 50) >> > at /usr/src/sys/x86/x86/local_apic.c:1343 >> > #12 >> > #13 __asan_load8_noabort (addr=3D18446741880219689232) >> > at /usr/src/sys/kern/subr_asan.c:1113 >> > #14 0xffffffff851488b8 in ?? () from /boot/thayer/ipfw.ko >> > #15 0xfffffe0100000000 in ?? () >> > #16 0xffffffff8134dcd5 in pcpu_find (cpuid=3D1238425856) >> > at /usr/src/sys/kern/subr_pcpu.c:286 >> > #17 0xffffffff85151f6f in ?? () from /boot/thayer/ipfw.ko >> > #18 0x0000000000000000 in ?? () >>