Date: Fri, 1 Jun 2012 18:19:56 +0530 From: "Desai, Kashyap" <Kashyap.Desai@lsi.com> To: Konstantin Belousov <kostikbel@gmail.com> Cc: "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org>, "freebsd-scsi@freebsd.org" <freebsd-scsi@freebsd.org>, "McConnell, Stephen" <Stephen.McConnell@lsi.com> Subject: RE: Kernel panic in FreeBSD-8.3 from UFS Message-ID: <B2FD678A64EAAD45B089B123FDFC3ED72B9F6C1F7F@inbmail01.lsi.com> In-Reply-To: <20120601124338.GU2358@deviant.kiev.zoral.com.ua> References: <B2FD678A64EAAD45B089B123FDFC3ED72B9F6C1F79@inbmail01.lsi.com> <20120601124338.GU2358@deviant.kiev.zoral.com.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
Thanks for the information. *YES* to me also looks like memory leaks only..=
but it is CAM XPT who is using "366195K " memory..
See below output of "vmstat -m"
vmstat -m
Type InUse MemUse HighUse Requests Size(s)
feeder 7 1K - 7 16
acpiintr 1 1K - 1 32
isadev 9 1K - 9 64
acpica 3179 172K - 73127 16,32,64,128,256,512,1024,2048
cdev 7 1K - 7 128
acpitask 1 1K - 1 1024
sigio 1 1K - 1 32
filedesc 50 14K - 2926 16,256,512,1024
kenv 121 9K - 130 16,32,64,128,4096
kqueue 0 0K - 266 128,1024
CAM dev queue 8 1K - 8 128
proc-args 26 2K - 4746 16,32,64,128
hhook 2 1K - 2 128
ithread 128 11K - 128 16,64,128
CAM queue 43 9K - 595257 16,32,64,128,256,512,1024,2048=
,4096
KTRACE 100 13K - 100 128
acpisem 21 3K - 21 64,128
linker 157 6K - 166 16,32,256
lockf 1751 61K - 2311 32,64,128,256,512,1024
loginclass 2 1K - 96 64
ip6ndp 12 1K - 13 64,128
ip6opt 0 0K - 3 32
temp 56 233K - 11165 16,32,64,128,256,512,1024,2048=
,4096
devbuf 5248 4507K - 5360 16,32,64,128,256,512,1024,2048=
,4096
module 493 31K - 493 64,128
mtx_pool 2 8K - 2 4096
CAM SIM 8 1K - 8 128
subproc 216 219K - 3091 256,4096
proc 2 8K - 2 4096
session 18 2K - 109 64
pgrp 25 2K - 129 64
cred 62 6K - 13960 64,128
uidinfo 3 2K - 88 64,1024
plimit 18 5K - 1389 256
scsi_cd 0 0K - 4 16
CAM periph 22 3K - 84532 16,32,64,128
CAM XPT 183208 366195K - 722021 16,32,64,256,1024,2048
sysctltmp 0 0K - 453 16,32,64,128,4096
sysctloid 5010 158K - 5286 16,32,64,128
sysctl 0 0K - 763 16,32,64
tidhash 1 8K - 1 =20
callout 7 1792K - 7 =20
umtx 750 71K - 750 64,128
p1003.1b 1 1K - 1 16
SWAP 2 4373K - 2 64
bus-sc 97 417K - 6298 16,32,64,128,256,512,1024,2048=
,4096
bus 1382 64K - 9711 16,32,64,128,256,1024
devstat 16 33K - 16 16,4096
eventhandler 73 4K - 73 32,64,128
UART 6 3K - 6 16,256,1024
kobj 358 716K - 634 2048
Per-cpu 1 1K - 1 16
ata_pci 2 1K - 2 32
rman 226 13K - 424 16,32,64
sbuf 0 0K - 1636 16,32,64,128,256,512,1024,2048=
,4096
scsi_da 0 0K - 186 16
stack 0 0K - 2 128
taskqueue 15 1K - 15 16,64
Unitno 14 1K - 5912 16,64
iov 0 0K - 1847877 16,64,128,256
select 9 1K - 9 64
ioctlops 0 0K - 5848 16,32,64,128,256,512,1024,2048
msg 4 25K - 4 1024,4096
sem 4 101K - 4 1024,4096
shm 1 12K - 1 =20
tty 21 11K - 23 512,2048
mbuf_tag 0 0K - 549 32,64
shmfd 1 4K - 1 4096
pcb 18 79K - 567 16,32,64,512,1024,2048,4096
soname 3 1K - 16885 16,32,128
vfscache 1 512K - 1 =20
cl_savebuf 0 0K - 48 32
vfs_hash 1 256K - 1 =20
acpidev 50 2K - 50 32
vnodes 2 1K - 2 128
vnodemarker 0 0K - 6497 512
mount 94 4K - 197 16,32,64,128,256
BPF 10 1K - 10 64
ether_multi 21 1K - 24 16,32,64
ifaddr 90 17K - 90 16,32,64,128,256,512,2048
ifnet 11 11K - 11 64,1024
USBdev 35 9K - 35 32,128,1024
clone 6 24K - 6 4096
arpcom 2 1K - 2 16
lltable 23 6K - 23 256
USB 66 40K - 69 16,32,64,128,256,1024,4096
routetbl 29 4K - 245 16,64,128,256
igmp 10 2K - 10 128
in_multi 1 1K - 1 128
sctp_iter 0 0K - 3 256
sctp_ifn 2 1K - 2 128
sctp_ifa 4 1K - 4 128
sctp_vrf 1 1K - 1 64
sctp_a_it 0 0K - 3 16
hostcache 1 16K - 1 =20
syncache 1 72K - 1 =20
entropy 1024 64K - 1024 64
in6_multi 15 2K - 15 16,256
pci_link 16 2K - 16 64,128
mld 10 2K - 10 128
rpc 2 1K - 2 128
audit_evclass 179 3K - 218 16
jblocks 2 1K - 2 128
savedino 0 0K - 121 256
sbdep 0 0K - 464 32
jsegdep 1 1K - 6778 32
jseg 1 1K - 4558 128
jfreefrag 0 0K - 179 64
jnewblk 0 0K - 5965 64
jremref 0 0K - 317 64
jaddref 0 0K - 317 64
freedep 0 0K - 9 32
freework 1 1K - 268 32,128
newdirblk 0 0K - 6 32
dirrem 0 0K - 305 64
mkdir 0 0K - 12 64
diradd 0 0K - 305 64
freefile 0 0K - 72 32
freeblks 0 0K - 157 128
freefrag 0 0K - 179 64
indirdep 1 1K - 4235 64
newblk 2 65K - 5966 128
bmsafemap 2 5K - 4389 128,4096
inodedep 2 257K - 4997 256
pagedep 1 64K - 51 128
ufs_dirhash 8 4K - 24 16,32,64,512
ufs_mount 21 390K - 21 256,4096
vm_pgdata 2 65K - 2 64
UMAHash 1 1K - 1 256
acpi_perf 2 1K - 2 256
DEVFS1 127 32K - 187 256
atkbddev 2 1K - 2 32
DEVFS3 141 18K - 223 128,256
DEVFS 24 1K - 25 16,64
memdesc 1 4K - 1 4096
apmdev 1 1K - 1 64
io_apic 2 2K - 2 1024
pfs_nodes 21 3K - 21 128
msi 3 1K - 3 64
nexusdev 5 1K - 5 16
GEOM 117 19K - 2291 16,32,64,128,256,512,1024,2048
SCSI SES 2 4K - 2 2048
kbdmux 7 18K - 7 16,256,1024,2048
mps 22 280K - 24141 16,32,64,128,256,512,2048,4096
mps_user 0 0K - 662 32,64
` Kashyap
> -----Original Message-----
> From: Konstantin Belousov [mailto:kostikbel@gmail.com]
> Sent: Friday, June 01, 2012 6:14 PM
> To: Desai, Kashyap
> Cc: freebsd-scsi@freebsd.org; freebsd-fs@freebsd.org; McConnell, Stephen
> Subject: Re: Kernel panic in FreeBSD-8.3 from UFS
>=20
> On Fri, Jun 01, 2012 at 05:30:39PM +0530, Desai, Kashyap wrote:
> > Hi,
> >
> > We have seen kernel panic while doing IO along with HBA reset.
> > This looks to be very rare but not sure if someone can help me to
> > understand what is a issue here. To me it does not look any issue with
> > underline Device Driver <mps>
> >
> > See below back trace.
> You did not specified the panic message. Was it 'kmem_map too small' ?
>=20
> Unless HBA driver causes memory leak, this is probably indeed unrelated.
> >
> >
> > #0 doadump (textdump=3D1) at pcpu.h:244
> > 244 pcpu.h: No such file or directory.
> > in pcpu.h
> > (kgdb) #0 doadump (textdump=3D1) at pcpu.h:244
> > #1 0xc0a1845a in kern_reboot (howto=3D260)
> > at /usr/src/sys/kern/kern_shutdown.c:442
> > #2 0xc0a186f1 in panic (fmt=3DVariable "fmt" is not available.
> > ) at /usr/src/sys/kern/kern_shutdown.c:607
> > #3 0xc0c7ceda in kmem_malloc (map=3D0xc15c808c, size=3D32768, flags=3D=
2)
> > at /usr/src/sys/vm/vm_kern.c:334
> > #4 0xc0c708e7 in page_alloc (zone=3D0x0, bytes=3D32768, pflag=3D0xf198=
39bf
> "\002",
> > wait=3D2) at /usr/src/sys/vm/uma_core.c:994
> > #5 0xc0c72fe0 in uma_large_malloc (size=3D32768, wait=3D2)
> > at /usr/src/sys/vm/uma_core.c:3067
> > #6 0xc0a04fac in malloc (size=3D32768, mtp=3D0xc102b808, flags=3D2)
> > at /usr/src/sys/kern/kern_malloc.c:492
> > #7 0xc0c42e89 in softdep_disk_io_initiation (bp=3D0xdef881fc)
> > at /usr/src/sys/ufs/ffs/ffs_softdep.c:10126
> > #8 0xc0c5208f in ffs_geom_strategy (bo=3D0xc5fc30ac, bp=3D0xdef881fc)
> > at buf.h:411
> > #9 0xc0c65a43 in ufs_strategy (ap=3D0xf1983b00)
> > at /usr/src/sys/ufs/ufs/ufs_vnops.c:2317
> > #10 0xc0d6a6dd in VOP_STRATEGY_APV (vop=3D0xc102e4a0, a=3D0xf1983b00)
> > at vnode_if.c:2171
> > #11 0xc0a8d19e in bufstrategy (bo=3D0xc6b901bc, bp=3D0xdef881fc) at
> > vnode_if.h:940
> > #12 0xc0a9352e in bufwrite (bp=3D0xdef881fc) at buf.h:404
> > #13 0xc0a8db5c in vfs_bio_awrite (bp=3D0xdef881fc) at buf.h:392
> > #14 0xc0c584c5 in ffs_syncvnode (vp=3D0xc6b90110, waitfor=3D1)
> > at /usr/src/sys/ufs/ffs/ffs_vnops.c:288
> > #15 0xc0c58739 in ffs_fsync (ap=3D0xf1983c4c)
> > at /usr/src/sys/ufs/ffs/ffs_vnops.c:187
> > #16 0xc0d69712 in VOP_FSYNC_APV (vop=3D0xc102dfc0, a=3D0xf1983c4c)
> > at vnode_if.c:1267
> > #17 0xc0ab5d49 in sys_fsync (td=3D0xc64ea8a0, uap=3D0xf1983cec) at
> > vnode_if.h:549
> > #18 0xc0d49315 in syscall (frame=3D0xf1983d28) at subr_syscall.c:131
> > #19 0xc0d32af1 in Xint0x80_syscall ()
> > at /usr/src/sys/i386/i386/exception.s:266
> > #20 0x00000033 in ?? (
> >
> >
> > To me it looks like UFS is doing something to crash the kernel.
>=20
> You might try to use vmstat -z and vmstat -m on core to see what has
> used KVA.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?B2FD678A64EAAD45B089B123FDFC3ED72B9F6C1F7F>