Date: Fri, 20 Aug 2004 15:31:26 -0500 From: Richard Todd <rmtodd@ichotolot.servalan.com> To: freebsd-current@freebsd.org Subject: Panic 'kernel trap doesn't have ucred' in last night's -current Message-ID: <20040820211618.62A1C7E4@mx2.synetsystems.com>
next in thread | raw e-mail | index | archive | help
Hi. Upgraded to -current last night and got the following panic. The panic
seems to be fairly repeatable and is triggered by a minute or so's worth of
database activity with mysqld. The version of mysql is 3.23.58_1 from ports,
linked against libpthread; using libmap.conf to force usage of libc_r avoids
the panic, so it's definitely a thread/KSE issue. (Also note the oddness
of the backtrace in the frames between doreti_ast and sched_switch -- is this
normal, is the stack mangled, or is gdb just hallucinating?) My kernel config
is fairly unexceptional and is attached to the end after the gdb script.
Oh yes, this is on a dual PII/400 system.
Script started on Fri Aug 20 15:15:26 2004
GNU gdb 20040720 [GDB v6.x for FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-portbld-freebsd5.2"...
panic: kernel trap doesn't have ucred
panic messages:
---
panic: kernel trap doesn't have ucred
cpuid = 0;
boot() called on cpu#0
Uptime: 14m27s
Dumping 638 MB
16 32 48 64 80 96 112 128 144 160 176 192 208 224 240 256 272 288 304 320 336 352 368 384 400 416 432 448 464 480 496 512 528 544 560 576 592 608 624
---
#0 doadump () at pcpu.h:159
159 __asm __volatile("movl %%fs:0,%0" : "=r" (td));
doadump () at pcpu.h:159
159 __asm __volatile("movl %%fs:0,%0" : "=r" (td));
(kgdb) bt
#0 doadump () at pcpu.h:159
#1 0xc0615c14 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:396
#2 0xc0615f33 in panic (fmt=0xc085ff0d "kernel trap doesn't have ucred")
at /usr/src/sys/kern/kern_shutdown.c:558
#3 0xc07c7570 in trap (frame=
{tf_fs = -1067450344, tf_es = -1064042480, tf_ds = 16, tf_edi = -1036714944, tf_esi = -1034233440, tf_ebp = -548918068, tf_isp = -548918104, tf_ebx = 0, tf_edx = 4, tf_ecx = 0, tf_eax = -1036714944, tf_trapno = 12, tf_err = 0, tf_eip = -1067372413, tf_cs = 8, tf_eflags = 66118, tf_esp = 0, tf_ss = -1065121135})
at /usr/src/sys/i386/i386/trap.c:413
#4 0xc07b68da in calltrap () at /usr/src/sys/i386/i386/exception.s:140
#5 0xc0600018 in exec_copyout_strings (imgp=0x0)
at /usr/src/sys/kern/kern_exec.c:996
#6 0xc06066fb in thread_alloc_spare (td=0x6cb)
at /usr/src/sys/kern/kern_kse.c:1021
#7 0xc0606a2f in thread_user_enter (p=0xc234fc40, td=0xc25ad9a0)
at /usr/src/sys/kern/kern_kse.c:1182
#8 0xc0633788 in ast (framep=0xdf482d48) at /usr/src/sys/kern/subr_trap.c:166
#9 0xc07b724d in doreti_ast () at /usr/src/sys/i386/i386/exception.s:294
#10 0xdf482d48 in ?? ()
#11 0x0000002f in ?? ()
#12 0x0000002f in ?? ()
#13 0x0000002f in ?? ()
#14 0x0b113600 in ?? ()
#15 0x0832e000 in ?? ()
#16 0xbfbfe578 in ?? ()
#17 0xdf482d74 in ?? ()
#18 0x2032b83c in ?? ()
#19 0x0b0f6600 in ?? ()
#20 0x08328900 in ?? ()
#21 0x00000000 in ?? ()
#22 0x00000016 in ?? ()
#23 0x00000002 in ?? ()
#24 0x20327d3f in ?? ()
#25 0x0000001f in ?? ()
#26 0x00000202 in ?? ()
#27 0xbfbfe54c in ?? ()
#28 0x0000002f in ?? ()
#29 0x00000000 in ?? ()
#30 0x00000000 in ?? ()
#31 0x00000000 in ?? ()
#32 0x00000000 in ?? ()
#33 0x07dda000 in ?? ()
#34 0xc1bfc420 in ?? ()
#35 0xc1ca0280 in ?? ()
#36 0xdf482c9c in ?? ()
#37 0xdf482c84 in ?? ()
#38 0xc25ad9a0 in ?? ()
#39 0xc062570b in sched_switch (td=0x2032b83c, newtd=0xb113600)
---Type <return> to continue, or q <return> to quit---
at /usr/src/sys/kern/sched_ule.c:1207
Previous frame inner to this frame (corrupt stack?)
(kgdb) fr 3
#3 0xc07c7570 in trap (frame=
{tf_fs = -1067450344, tf_es = -1064042480, tf_ds = 16, tf_edi = -1036714944, tf_esi = -1034233440, tf_ebp = -548918068, tf_isp = -548918104, tf_ebx = 0, tf_edx = 4, tf_ecx = 0, tf_eax = -1036714944, tf_trapno = 12, tf_err = 0, tf_eip = -1067372413, tf_cs = 8, tf_eflags = 66118, tf_esp = 0, tf_ss = -1065121135})
at /usr/src/sys/i386/i386/trap.c:413
413 KASSERT(cold || td->td_ucred != NULL,
(kgdb) p td
$1 = (struct thread *) 0xc25ad9a0
(kgdb) p *td
$2 = {td_proc = 0xc234fc40, td_ksegrp = 0xc1e42630, td_plist = {
tqe_next = 0xc201cb00, tqe_prev = 0xc2ab8428}, td_kglist = {
tqe_next = 0x0, tqe_prev = 0xc2ab9850}, td_slpq = {tqe_next = 0xc2ab9840,
tqe_prev = 0xdf64fc78}, td_lockq = {tqe_next = 0x0, tqe_prev = 0x0},
td_runq = {tqe_next = 0x0, tqe_prev = 0xc1e42654}, td_selq = {
tqh_first = 0x0, tqh_last = 0xc25ad9d0}, td_sleepqueue = 0xc2a19320,
td_turnstile = 0xc2428dc0, td_tid = 100183, td_flags = 65538,
td_inhibitors = 0, td_pflags = 128, td_last_kse = 0xc1ca0280,
td_kse = 0xc1ca0280, td_dupfd = 0, td_wchan = 0x0, td_wmesg = 0x0,
td_lastcpu = 0 '\0', td_oncpu = 0 '\0', td_locks = 0, td_blocked = 0x0,
td_ithd = 0x0, td_lockname = 0x0, td_contested = {lh_first = 0x0},
td_sleeplocks = 0x0, td_intr_nesting_level = 0, td_pinned = 0,
td_mailbox = 0x0, td_ucred = 0x0, td_standin = 0xc2ae09a0, td_prticks = 0,
td_upcall = 0xc26cb000, td_sticks = 1437, td_uuticks = 0, td_usticks = 1,
td_intrval = 0, td_oldsigmask = {__bits = {0, 0, 0, 0}}, td_sigmask = {
__bits = {4294901503, 4294967295, 4294967295, 4294967295}}, td_siglist = {
__bits = {0, 0, 0, 0}}, td_waitset = 0x0, td_umtx = {tqe_next = 0x0,
tqe_prev = 0x0}, td_generation = 3, td_sigstk = {ss_sp = 0x0, ss_size = 0,
ss_flags = 0}, td_kflags = 0, td_xsig = 0, td_profil_addr = 0,
td_profil_ticks = 0, td_base_pri = 104 'h', td_priority = 160 ' ',
td_pcb = 0xdf482da0, td_state = TDS_RUNNING, td_retval = {0, 185558528},
td_slpcallout = {c_links = {sle = {sle_next = 0xc2ab995c}, tqe = {
tqe_next = 0xc2ab995c, tqe_prev = 0xce9ef790}}, c_time = 92526,
c_arg = 0xc25ad9a0, c_func = 0, c_flags = 8}, td_frame = 0xdf482d48,
td_kstack_obj = 0xc2490e70, td_kstack = 3746041856, td_kstack_pages = 2,
td_altkstack_obj = 0x0, td_altkstack = 0, td_altkstack_pages = 0,
td_critnest = 1, td_md = {md_savecrit = 582}, td_sched = 0xc25adafc}
(kgdb) p *td->td_proc
$3 = {p_list = {le_next = 0xc215be00, le_prev = 0xc244a700}, p_ksegrps = {
tqh_first = 0xc29335a0, tqh_last = 0xc1e42634}, p_threads = {
tqh_first = 0xc2ae0420, tqh_last = 0xc25ab848}, p_suspended = {
tqh_first = 0x0, tqh_last = 0xc234fc58}, p_ucred = 0xc2188c80,
p_fd = 0xc2164900, p_fdtol = 0x0, p_stats = 0xdf356000,
p_limit = 0xc2163a00, p_upages_obj = 0xc23144a4, p_sigacts = 0xc240d000,
p_flag = 49410, p_sflag = 1, p_state = PRS_NORMAL, p_pid = 688, p_hash = {
le_next = 0x0, le_prev = 0xc1ba8ac0}, p_pglist = {le_next = 0x0,
le_prev = 0xc234fe54}, p_pptr = 0xc234fe00, p_sibling = {le_next = 0x0,
le_prev = 0xc234fe68}, p_children = {lh_first = 0x0}, p_mtx = {
mtx_object = {lo_class = 0xc08a00bc, lo_name = 0xc08385cb "process lock",
lo_type = 0xc08385cb "process lock", lo_flags = 4390912, lo_list = {
tqe_next = 0xc234fe6c, tqe_prev = 0xc234fafc},
lo_witness = 0xc09154a0}, mtx_lock = 4, mtx_recurse = 0}, p_oppid = 0,
p_vmspace = 0xc201e000, p_swtime = 0, p_realtimer = {it_interval = {
tv_sec = 0, tv_usec = 0}, it_value = {tv_sec = 29667,
tv_usec = 310390}}, p_runtime = {sec = 74, frac = 9024260264433929926},
p_uu = 17286797, p_su = 48742844, p_iu = 1, p_uticks = 2515,
p_sticks = 9030, p_iticks = 0, p_profthreads = 0, p_maxthrwaits = 0,
p_traceflag = 0, p_tracevp = 0x0, p_tracecred = 0x0, p_textvp = 0xc2420b58,
p_siglist = {__bits = {0, 0, 0, 0}}, p_lock = 0 '\0', p_sigiolst = {
slh_first = 0x0}, p_sigparent = 20, p_sig = 0, p_code = 0, p_stops = 0,
p_stype = 0, p_step = 0 '\0', p_pfsflags = 0 '\0', p_nlminfo = 0x0,
p_aioinfo = 0x0, p_singlethread = 0x0, p_suspcount = 0, p_xthread = 0x0,
p_magic = 3203398350, p_comm = "mysqld", '\0' <repeats 13 times>,
p_pgrp = 0xc1f8b6c0, p_sysent = 0xc08e2360, p_args = 0xc233cd00,
p_cpulimit = 9223372036854775807, p_nice = 0 '\0', p_xstat = 0, p_klist = {
kl_lock = 0xc234fcac, kl_list = {slh_first = 0x0}}, p_numthreads = 7,
p_numksegrps = 6, p_md = {md_ldt = 0xc23d98c0}, p_itcallout = {c_links = {
sle = {sle_next = 0x0}, tqe = {tqe_next = 0x0, tqe_prev = 0xce9dcea8}},
c_time = 2966609, c_arg = 0xc234fc40, c_func = 0xc0620c88 <realitexpire>,
c_flags = 14}, p_uarea = 0xdf356000, p_acflag = 0, p_ru = 0x0,
p_peers = 0x0, p_leader = 0xc234fc40, p_emuldata = 0x0, p_label = 0x0,
p_sched = 0xc234fe00}
(kgdb) q
Script done on Fri Aug 20 15:16:48 2004
kernel config file below:
#
# GENERIC -- Generic kernel configuration file for FreeBSD/i386
#
# For more information on this file, please read the handbook section on
# Kernel Configuration Files:
#
# http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html
#
# The handbook is also available locally in /usr/share/doc/handbook
# if you've installed the doc distribution, otherwise always see the
# FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the
# latest information.
#
# An exhaustive list of options and more detailed explanations of the
# device lines is also present in the ../../conf/NOTES and NOTES files.
# If you are in doubt as to the purpose or necessity of a line, check first
# in NOTES.
#
# $FreeBSD: src/sys/i386/conf/GENERIC,v 1.413 2004/08/11 01:34:18 rwatson Exp $
machine i386
cpu I686_CPU
ident GENERIC
# To statically compile in device wiring instead of /boot/device.hints
#hints "GENERIC.hints" # Default places to look for devices.
makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols
options SCHED_ULE # ULE scheduler
options INET # InterNETworking
options INET6 # IPv6 communications protocols
options FFS # Berkeley Fast Filesystem
options SOFTUPDATES # Enable FFS soft updates support
options UFS_ACL # Support for access control lists
options UFS_DIRHASH # Improve performance on big directories
options MD_ROOT # MD is a potential root device
options NFSCLIENT # Network Filesystem Client
options NFSSERVER # Network Filesystem Server
options NFS_ROOT # NFS usable as /, requires NFSCLIENT
options MSDOSFS # MSDOS Filesystem
options CD9660 # ISO 9660 Filesystem
options PROCFS # Process filesystem (requires PSEUDOFS)
options PSEUDOFS # Pseudo-filesystem framework
options GEOM_GPT # GUID Partition Tables.
options COMPAT_43 # Compatible with BSD 4.3 [KEEP THIS!]
options COMPAT_FREEBSD4 # Compatible with FreeBSD4
options SCSI_DELAY=15000 # Delay (in ms) before probing SCSI
options KTRACE # ktrace(1) support
options SYSVSHM # SYSV-style shared memory
options SYSVMSG # SYSV-style message queues
options SYSVSEM # SYSV-style semaphores
options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions
options KBD_INSTALL_CDEV # install a CDEV entry in /dev
options AHC_REG_PRETTY_PRINT # Print register bitfields in debug
# output. Adds ~128k to driver.
options AHD_REG_PRETTY_PRINT # Print register bitfields in debug
# output. Adds ~215k to driver.
options PFIL_HOOKS # pfil(9) framework
options ADAPTIVE_GIANT # Giant mutex is adaptive.
# Debugging for use in -current
options KDB # Enable kernel debugger support.
options DDB # Support DDB.
options GDB # Support remote GDB.
options INVARIANTS # Enable calls of extra sanity checking
options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS
options WITNESS # Enable checks to detect deadlocks and cycles
options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed
# To make an SMP kernel, the next two are needed
options SMP # Symmetric MultiProcessor Kernel
device apic # I/O APIC
# Bus support. Do not remove isa, even if you have no isa slots
device isa
device eisa
device pci
# Floppy drives
device fdc
# ATA and ATAPI devices
device ata
device atadisk # ATA disk drives
device ataraid # ATA RAID drives
device atapicd # ATAPI CDROM drives
device atapifd # ATAPI floppy drives
device atapist # ATAPI tape drives
options ATA_STATIC_ID # Static device numbering
# SCSI Controllers
device ahb # EISA AHA1742 family
device ahc # AHA2940 and onboard AIC7xxx devices
device ahd # AHA39320/29320 and onboard AIC79xx devices
device amd # AMD 53C974 (Tekram DC-390(T))
device isp # Qlogic family
device mpt # LSI-Logic MPT-Fusion
#device ncr # NCR/Symbios Logic
device sym # NCR/Symbios Logic (newer chipsets + those of `ncr')
device trm # Tekram DC395U/UW/F DC315U adapters
device adv # Advansys SCSI adapters
device adw # Advansys wide SCSI adapters
device aha # Adaptec 154x SCSI adapters
device aic # Adaptec 15[012]x SCSI adapters, AIC-6[23]60.
device bt # Buslogic/Mylex MultiMaster SCSI adapters
device ncv # NCR 53C500
device nsp # Workbit Ninja SCSI-3
device stg # TMC 18C30/18C50
# SCSI peripherals
device scbus # SCSI bus (required for SCSI)
device ch # SCSI media changers
device da # Direct Access (disks)
device sa # Sequential Access (tape etc)
device cd # CD
device pass # Passthrough device (direct SCSI access)
device ses # SCSI Environmental Services (and SAF-TE)
# RAID controllers interfaced to the SCSI subsystem
device amr # AMI MegaRAID
device asr # DPT SmartRAID V, VI and Adaptec SCSI RAID
device ciss # Compaq Smart RAID 5*
device dpt # DPT Smartcache III, IV - See NOTES for options
device iir # Intel Integrated RAID
device ips # IBM (Adaptec) ServeRAID
device mly # Mylex AcceleRAID/eXtremeRAID
device twa # 3ware 9000 series PATA/SATA RAID
# RAID controllers
device aac # Adaptec FSA RAID
device aacp # SCSI passthrough for aac (requires CAM)
device ida # Compaq Smart RAID
device mlx # Mylex DAC960 family
device pst # Promise Supertrak SX6000
device twe # 3ware ATA RAID
# atkbdc0 controls both the keyboard and the PS/2 mouse
device atkbdc # AT keyboard controller
device atkbd # AT keyboard
device psm # PS/2 mouse
device vga # VGA video card driver
device splash # Splash screen and screen saver support
# syscons is the default console driver, resembling an SCO console
device sc
# Enable this for the pcvt (VT220 compatible) console driver
#device vt
#options XSERVER # support for X server on a vt console
#options FAT_CURSOR # start with block cursor
device agp # support several AGP chipsets
# Floating point support - do not disable.
device npx
# Power management support (see NOTES for more options)
#device apm
# Add suspend/resume support for the i8254.
device pmtimer
# PCCARD (PCMCIA) support
# PCMCIA and cardbus bridge support
device cbb # cardbus (yenta) bridge
device pccard # PC Card (16-bit) bus
device cardbus # CardBus (32-bit) bus
# Serial (COM) ports
device sio # 8250, 16[45]50 based serial ports
# Parallel port
device ppc
device ppbus # Parallel port bus (required)
device lpt # Printer
device plip # TCP/IP over parallel
device ppi # Parallel port interface device
#device vpo # Requires scbus and da
# If you've got a "dumb" serial or parallel PCI card that is
# supported by the puc(4) glue driver, uncomment the following
# line to enable it (connects to the sio and/or ppc drivers):
#device puc
# PCI Ethernet NICs.
device de # DEC/Intel DC21x4x (``Tulip'')
device em # Intel PRO/1000 adapter Gigabit Ethernet Card
device ixgb # Intel PRO/10GbE Ethernet Card
device txp # 3Com 3cR990 (``Typhoon'')
device vx # 3Com 3c590, 3c595 (``Vortex'')
# PCI Ethernet NICs that use the common MII bus controller code.
# NOTE: Be sure to keep the 'device miibus' line in order to use these NICs!
device miibus # MII bus support
device bfe # Broadcom BCM440x 10/100 Ethernet
device bge # Broadcom BCM570xx Gigabit Ethernet
device dc # DEC/Intel 21143 and various workalikes
device fxp # Intel EtherExpress PRO/100B (82557, 82558)
device pcn # AMD Am79C97x PCI 10/100 (precedence over 'lnc')
device re # RealTek 8139C+/8169/8169S/8110S
device rl # RealTek 8129/8139
device sf # Adaptec AIC-6915 (``Starfire'')
device sis # Silicon Integrated Systems SiS 900/SiS 7016
device sk # SysKonnect SK-984x & SK-982x gigabit Ethernet
device ste # Sundance ST201 (D-Link DFE-550TX)
device ti # Alteon Networks Tigon I/II gigabit Ethernet
device tl # Texas Instruments ThunderLAN
device tx # SMC EtherPower II (83c170 ``EPIC'')
device vr # VIA Rhine, Rhine II
device wb # Winbond W89C840F
device xl # 3Com 3c90x (``Boomerang'', ``Cyclone'')
# ISA Ethernet NICs. pccard NICs included.
device cs # Crystal Semiconductor CS89x0 NIC
# 'device ed' requires 'device miibus'
device ed # NE[12]000, SMC Ultra, 3c503, DS8390 cards
device ex # Intel EtherExpress Pro/10 and Pro/10+
device ep # Etherlink III based cards
device fe # Fujitsu MB8696x based cards
device ie # EtherExpress 8/16, 3C507, StarLAN 10 etc.
device lnc # NE2100, NE32-VL Lance Ethernet cards
device sn # SMC's 9000 series of Ethernet chips
device xe # Xircom pccard Ethernet
# ISA devices that use the old ISA shims
#device le
# Wireless NIC cards
device wlan # 802.11 support
device an # Aironet 4500/4800 802.11 wireless NICs.
device awi # BayStack 660 and others
device wi # WaveLAN/Intersil/Symbol 802.11 wireless NICs.
#device wl # Older non 802.11 Wavelan wireless NIC.
# Pseudo devices.
device loop # Network loopback
device mem # Memory and kernel memory devices
device io # I/O device
device random # Entropy device
device ether # Ethernet support
device sl # Kernel SLIP
device ppp # Kernel PPP
device tun # Packet tunnel.
device pty # Pseudo-ttys (telnet etc)
device md # Memory "disks"
device gif # IPv6 and IPv4 tunneling
device faith # IPv6-to-IPv4 relaying (translation)
# The `bpf' device enables the Berkeley Packet Filter.
# Be aware of the administrative consequences of enabling this!
device bpf # Berkeley packet filter
# USB support
device uhci # UHCI PCI->USB interface
device ohci # OHCI PCI->USB interface
device usb # USB Bus (required)
#device udbp # USB Double Bulk Pipe devices
device ugen # Generic
device uhid # "Human Interface Devices"
device ukbd # Keyboard
device ulpt # Printer
device umass # Disks/Mass storage - Requires scbus and da
device ums # Mouse
device urio # Diamond Rio 500 MP3 player
device uscanner # Scanners
# USB Ethernet, requires mii
device aue # ADMtek USB Ethernet
device axe # ASIX Electronics USB Ethernet
device cue # CATC USB Ethernet
device kue # Kawasaki LSI USB Ethernet
device rue # RealTek RTL8150 USB Ethernet
# FireWire support
device firewire # FireWire bus code
device sbp # SCSI over FireWire (Requires scbus and da)
device fwe # Ethernet over FireWire (non-standard!)
device "snd_sb16"
device "snd_sb8"
device snd_sbc
device sound
options INVARIANTS
options INVARIANT_SUPPORT
options IPSEC
options IPSEC_ESP
options KDB
options KDB_UNATTENDED
options WITNESS
options DIAGNOSTIC
options DUMMYNET
options IPFIREWALL
options IPFIREWALL_DEFAULT_TO_ACCEPT
options SW_WATCHDOG
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20040820211618.62A1C7E4>