Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 29 Aug 2023 11:53:38 +0000
From:      bugzilla-noreply@freebsd.org
To:        bugs@FreeBSD.org
Subject:   [Bug 273418] [panic] Repeating kernel panic on open(/dev/console)
Message-ID:  <bug-273418-227@https.bugs.freebsd.org/bugzilla/>

next in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D273418

            Bug ID: 273418
           Summary: [panic] Repeating kernel panic on open(/dev/console)
           Product: Base System
           Version: 13.2-STABLE
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: eugen@freebsd.org

A 13.2-STABLE/amd64 server sometimes runs flawlessly for several weeks, but
sometimes panices with same backtrace at midnight after newsyslog rotates a=
nd
compresses logs then sends SIGHUP to the syslogd that closes and reopens all
channels including /dev/console. The kernel panices on open(/dev/console)
system call sometimes. The system was source-updated from 12.4-STABLE/amd64=
 to
the commit https://cgit.freebsd.org/src/commit/?h=3Dstable/13&id=3D8711fd210

This is regression since 12.4-STABLE.

# conscontrol
Configured: ttyv0
 Available: uart,ttyv0,gdb
    Muting: off
# sysctl kern.vty
kern.vty: vt
# last | grep boot | head -5
boot time                                  Mon Aug 28 00:08
boot time                                  Sun Aug 27 00:07
boot time                                  Sat Aug 26 00:09
boot time                                  Tue Aug 22 00:08
boot time                                  Mon Aug 21 00:09

And I have 5 crashdumps with same backtrace. Custom kernel has debugging
options that point to use-after-free (0xdeadc0dedeadc0de, see below).

options         KDB                     # Enable kernel debugger support.
options         KDB_UNATTENDED
options         KDB_TRACE
options         DDB                     # Support DDB.
options         GDB                     # Support remote GDB.
options        INVARIANTS              # Enable calls of extra sanity check=
in
options        INVARIANT_SUPPORT       # Extra sanity checks of internal
structures, required by IN
options        WITNESS                 # Enable checks to detect deadlocks =
and
cycles
options        WITNESS_SKIPSPIN        # Don't run witness on spinlocks for
speedoptions

The backtrace:

#0  __curthread () at /data/src/sys/amd64/include/pcpu_aux.h:55
#1  doadump (textdump=3Dtextdump@entry=3D1) at
/data/src/sys/kern/kern_shutdown.c:396
#2  0xffffffff80c0dd43 in kern_reboot (howto=3D260) at
/data/src/sys/kern/kern_shutdown.c:484
#3  0xffffffff80c0e1af in vpanic (fmt=3D<optimized out>,
ap=3Dap@entry=3D0xfffffe0152e8d600)
    at /data/src/sys/kern/kern_shutdown.c:923
#4  0xffffffff80c0df33 in panic (fmt=3D<unavailable>) at
/data/src/sys/kern/kern_shutdown.c:847
#5  0xffffffff811178b7 in trap_fatal (frame=3D0xfffffe0152e8d690, eva=3D0)
    at /data/src/sys/amd64/amd64/trap.c:942
#6  <signal handler called>
#7  devfs_populate_loop (dm=3Ddm@entry=3D0xfffff8044020a000,
cleanup=3Dcleanup@entry=3D0)
    at /data/src/sys/fs/devfs/devfs_devs.c:533
#8  0xffffffff80a9a0fa in devfs_populate (dm=3Ddm@entry=3D0xfffff8044020a00=
0)
    at /data/src/sys/fs/devfs/devfs_devs.c:677
#9  0xffffffff80a9f318 in devfs_populate_vp (vp=3D0xfffff804401d9988)
    at /data/src/sys/fs/devfs/devfs_vnops.c:359
#10 0xffffffff80a9d61b in devfs_lookup (ap=3D0xfffffe0152e8da30)
    at /data/src/sys/fs/devfs/devfs_vnops.c:1187
#11 0xffffffff80cecbb1 in VOP_LOOKUP (dvp=3D0xfffff804401d9988,
vpp=3D0xfffffe0152e8dd10,
    cnp=3D0xfffffe0152e8dd38) at ./vnode_if.h:69
#12 lookup (ndp=3Dndp@entry=3D0xfffffe0152e8dcb8) at
/data/src/sys/kern/vfs_lookup.c:1092
#13 0xffffffff80cebba2 in namei (ndp=3Dndp@entry=3D0xfffffe0152e8dcb8)
    at /data/src/sys/kern/vfs_lookup.c:617
#14 0xffffffff80d11f90 in vn_open_cred (ndp=3Dndp@entry=3D0xfffffe0152e8dcb=
8,
    flagp=3Dflagp@entry=3D0xfffffe0152e8ddd4, cmode=3Dcmode@entry=3D0,
    vn_open_flags=3Dvn_open_flags@entry=3D16, cred=3D0xfffff80440282500,
fp=3D0xfffff8005a296af0)
    at /data/src/sys/kern/vfs_vnops.c:328
#15 0xffffffff80d08c58 in kern_openat (td=3D0xfffffe003753b000, fd=3D-100,
    path=3D0x1fc80b443e60 <error: Cannot access memory at address
0x1fc80b443e60>,
    pathseg=3DUIO_USERSPACE, flags=3D6, mode=3D<optimized out>)
    at /data/src/sys/kern/vfs_syscalls.c:1158
#16 0xffffffff81118283 in syscallenter (td=3D<optimized out>)
    at /data/src/sys/amd64/amd64/../../kern/subr_syscall.c:190
#17 amd64_syscall (td=3D0xfffffe003753b000, traced=3D0) at
/data/src/sys/amd64/amd64/trap.c:1183
#18 <signal handler called>
#19 0x00001fc80cc4504a in ?? ()
Backtrace stopped: Cannot access memory at address 0x1fc80b443d58

(kgdb) frame 15
#15 0xffffffff80d08c58 in kern_openat (td=3D0xfffffe003753b000, fd=3D-100,
    path=3D0x1fc80b443e60 <error: Cannot access memory at address
0x1fc80b443e60>,
    pathseg=3DUIO_USERSPACE, flags=3D6, mode=3D<optimized out>)
    at /data/src/sys/kern/vfs_syscalls.c:1158
1158            error =3D vn_open_cred(&nd, &flags, cmode, VN_OPEN_WANTIOCT=
LCAPS,

(kgdb) p nd
$4 =3D {ni_dirp =3D 0x1fc80b443e60 <error: Cannot access memory at address
0x1fc80b443e60>,
  ni_segflg =3D UIO_USERSPACE, ni_rightsneeded =3D 0xfffffe0152e8ddb0, ni_s=
tartdir
=3D 0x0,
  ni_rootdir =3D 0xfffff80003f36000, ni_topdir =3D 0x0, ni_dirfd =3D -100, =
ni_lcf =3D
0, ni_filecaps =3D {
    fc_rights =3D {cr_rights =3D {0, 0}}, fc_ioctls =3D 0x0, fc_nioctls =3D=
 -1,
fc_fcntls =3D 0},
  ni_vp =3D 0x0, ni_dvp =3D 0xfffff804401d9988, ni_resflags =3D 1, ni_debug=
flags =3D 3,
ni_loopcnt =3D 0,
  ni_pathlen =3D 1, ni_next =3D 0xfffff8039aa8200c "", ni_cnd =3D {cn_origf=
lags =3D
8683588,
    cn_flags =3D 344227908, cn_thread =3D 0xfffffe003753b000, cn_cred =3D
0xfffff80440282500,
    cn_nameiop =3D LOOKUP, cn_lkflags =3D 532480, cn_pnbuf =3D 0xfffff8039a=
a82000
"/dev/console",
    cn_nameptr =3D 0xfffff8039aa82005 "console", cn_namelen =3D 7}, ni_cap_=
tracker
=3D {
    tqh_first =3D 0x0, tqh_last =3D 0xfffffe0152e8dd78}, ni_dvp_seqc =3D 92=
8231424,
  ni_vp_seqc =3D 4294966784}

(kgdb) frame 7
#7  devfs_populate_loop (dm=3Ddm@entry=3D0xfffff8044020a000,
cleanup=3Dcleanup@entry=3D0)
    at /data/src/sys/fs/devfs/devfs_devs.c:533
533                         cdp->cdp_dirents[dm->dm_idx] !=3D NULL) {
(kgdb) p cdp->cdp_dirents[dm->dm_idx]
Cannot access memory at address 0xdeadc0dedeadc0de

--=20
You are receiving this mail because:
You are the assignee for the bug.=



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-273418-227>