Date: Sat, 22 Aug 2009 11:59:58 +0200 From: Marc UBM <ubm.freebsd@googlemail.com> To: current@freebsd.org Subject: panic: apic_free_vector: Thread already bound. Message-ID: <20090822115958.f93fcf29.ubm.freebsd@gmail.com>
next in thread | raw e-mail | index | archive | help
[-- Attachment #1 --]
Hiho! :-)
I'm seeing above panic in the final stages of the shutdown process.
Coredump is available, bt as follows, textdump attached.
FreeBSD hamstor 8.0-BETA2 FreeBSD 8.0-BETA2 #14: Wed Aug 19 22:43:17
CEST 2009 oni0@hamstor:/usr/obj/usr/src/sys/HAMSTOR amd64
(kgdb) bt
#0 doadump () at pcpu.h:223
#1 0xffffffff801b561c in db_fncall (dummy1=Variable "dummy1" is not
#available.
) at /usr/src/sys/ddb/db_command.c:548
#2 0xffffffff801b5951 in db_command (last_cmdp=0xffffffff808594a0,
#cmd_table=Variable "cmd_table" is not available.
) at /usr/src/sys/ddb/db_command.c:445
#3 0xffffffff801b5ba0 in db_command_loop ()
#at /usr/src/sys/ddb/db_command.c:498 4 0xffffffff801b7b69 in db_trap
#(type=Variable "type" is not available.
) at /usr/src/sys/ddb/db_main.c:229
#5 0xffffffff8038d775 in kdb_trap (type=3, code=0,
#tf=0xffffff8000017760) at /usr/src/sys/kern/subr_kdb.c:535 6
#0xffffffff8060e170 in trap (frame=0xffffff8000017760)
#at /usr/src/sys/amd64/amd64/trap.c:611 7 0xffffffff805f3b23 in
#calltrap () at /usr/src/sys/amd64/amd64/exception.S:224 8
#0xffffffff8038d94d in kdb_enter (why=0xffffffff80681954 "panic",
#msg=0xa <Address 0xa out of bounds>) at cpufunc.h:63 9
#0xffffffff8035e04b in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:562
#10 0xffffffff805fafbb in apic_free_vector (apic_id=Variable "apic_id"
#is not available.
) at /usr/src/sys/amd64/amd64/local_apic.c:995
#11 0xffffffff805f7560 in intr_remove_handler (cookie=Variable "cookie"
#is not available.
) at /usr/src/sys/amd64/amd64/intr_machdep.c:203
#12 0xffffffff806250d4 in hpt_shutdown_vbus
#(vbus_ext=0xffffff8000254000, howto=Variable "howto" is not available.
) at /usr/src/sys/dev/hptrr/hptrr_osm_bsd.c:361
#13 0xffffffff8035da8b in boot (howto=16392)
#at /usr/src/sys/kern/kern_shutdown.c:419 14 0xffffffff8035e126 in
#reboot (td=Variable "td" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:173
#15 0xffffffff8060dbba in syscall (frame=0xffffff8000017c80)
#at /usr/src/sys/amd64/amd64/trap.c:982 16 0xffffffff805f3e01 in
#Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:373 17
#0x000000000040892c in ?? ()
Previous frame inner to this frame (corrupt stack?)
frame #12 & #13 make me suspect the hptrr driver (again :-)), but I'm
not sure.
Bye
Marc
--
Marc "UBM" Bocklet <ubm.freebsd@gmail.com>
[-- Attachment #2 --]
hamstor dumped core - see /var/crash/vmcore.4
Sat Aug 22 11:07:03 CEST 2009
FreeBSD hamstor 8.0-BETA2 FreeBSD 8.0-BETA2 #14: Wed Aug 19 22:43:17 CEST 2009 oni0@hamstor:/usr/obj/usr/src/sys/HAMSTOR amd64
panic: apic_free_vector: Thread already bound.
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...
Unread portion of the kernel message buffer:
<118>Stopping devd.
<118>Writing entropy file:
<118>.
<118>Terminated
<118>.
<118>Aug 22 10:47:26 hamstor syslogd: exiting on signal 15
Waiting (max 60 seconds) for system process `vnlru' to stop...done
Waiting (max 60 seconds) for system process `bufdaemon' to stop...done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining...1 1 1 0 0 done
All buffers synced.
zfs_umount:971[0]: Force unmount is experimental - report any problems.
Uptime: 2h18m27s
panic: apic_free_vector: Thread already bound.
cpuid = 0
KDB: enter: panic
Physical memory: 4022 MB
Dumping 1592 MB: 1577 1561 1545 1529 1513 1497 1481 1465 1449 1433 1417 1401 1385 1369 1353 1337 1321 1305 1289 1273 1257 1241 1225 1209 1193 1177 1161 1145 1129 1113 1097 1081 1065 1049 1033 1017 1001 985 969 953 937 921 905 889 873 857 841 825 809 793 777 761 745 729 713 697 681 665 649 633 617 601 585 569 553 537 521 505 489 473 457 441 425 409 393 377 361 345 329 313 297 281 265 249 233 217 201 185 169 153 137 121 105 89 73 57 41 25 9
Reading symbols from /boot/kernel/ahci.ko...Reading symbols from /boot/kernel/ahci.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ahci.ko
Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /boot/kernel/zfs.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/zfs.ko
Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /boot/kernel/opensolaris.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/opensolaris.ko
Reading symbols from /boot/kernel/blank_saver.ko...Reading symbols from /boot/kernel/blank_saver.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/blank_saver.ko
#0 doadump () at pcpu.h:223
223 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) #0 doadump () at pcpu.h:223
#1 0xffffffff801b561c in db_fncall (dummy1=Variable "dummy1" is not available.
)
at /usr/src/sys/ddb/db_command.c:548
#2 0xffffffff801b5951 in db_command (last_cmdp=0xffffffff808594a0, cmd_table=Variable "cmd_table" is not available.
) at /usr/src/sys/ddb/db_command.c:445
#3 0xffffffff801b5ba0 in db_command_loop ()
at /usr/src/sys/ddb/db_command.c:498
#4 0xffffffff801b7b69 in db_trap (type=Variable "type" is not available.
) at /usr/src/sys/ddb/db_main.c:229
#5 0xffffffff8038d775 in kdb_trap (type=3, code=0, tf=0xffffff8000017760)
at /usr/src/sys/kern/subr_kdb.c:535
#6 0xffffffff8060e170 in trap (frame=0xffffff8000017760)
at /usr/src/sys/amd64/amd64/trap.c:611
#7 0xffffffff805f3b23 in calltrap ()
at /usr/src/sys/amd64/amd64/exception.S:224
#8 0xffffffff8038d94d in kdb_enter (why=0xffffffff80681954 "panic",
msg=0xa <Address 0xa out of bounds>) at cpufunc.h:63
#9 0xffffffff8035e04b in panic (fmt=Variable "fmt" is not available.
)
at /usr/src/sys/kern/kern_shutdown.c:562
#10 0xffffffff805fafbb in apic_free_vector (apic_id=Variable "apic_id" is not available.
)
at /usr/src/sys/amd64/amd64/local_apic.c:995
#11 0xffffffff805f7560 in intr_remove_handler (cookie=Variable "cookie" is not available.
)
at /usr/src/sys/amd64/amd64/intr_machdep.c:203
#12 0xffffffff806250d4 in hpt_shutdown_vbus (vbus_ext=0xffffff8000254000,
howto=Variable "howto" is not available.
) at /usr/src/sys/dev/hptrr/hptrr_osm_bsd.c:361
#13 0xffffffff8035da8b in boot (howto=16392)
at /usr/src/sys/kern/kern_shutdown.c:419
#14 0xffffffff8035e126 in reboot (td=Variable "td" is not available.
)
at /usr/src/sys/kern/kern_shutdown.c:173
#15 0xffffffff8060dbba in syscall (frame=0xffffff8000017c80)
at /usr/src/sys/amd64/amd64/trap.c:982
#16 0xffffffff805f3e01 in Xfast_syscall ()
at /usr/src/sys/amd64/amd64/exception.S:373
#17 0x000000000040892c in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb)
------------------------------------------------------------------------
ps -axl
Segmentation fault
------------------------------------------------------------------------
vmstat -s
0 cpu context switches
0 device interrupts
0 software interrupts
0 traps
0 system calls
0 kernel threads created
0 fork() calls
0 vfork() calls
0 rfork() calls
0 swap pager pageins
0 swap pager pages paged in
0 swap pager pageouts
0 swap pager pages paged out
0 vnode pager pageins
0 vnode pager pages paged in
0 vnode pager pageouts
0 vnode pager pages paged out
0 page daemon wakeups
0 pages examined by the page daemon
236 pages reactivated
0 copy-on-write faults
0 copy-on-write optimized faults
0 zero fill pages zeroed
0 zero fill pages prezeroed
0 intransit blocking page faults
0 total VM faults taken
0 pages affected by kernel thread creation
0 pages affected by fork()
0 pages affected by vfork()
0 pages affected by rfork()
261 pages cached
0 pages freed
0 pages freed by daemon
120628 pages freed by exiting processes
24 pages active
5 pages inactive
0 pages in VM cache
108008 pages wired down
885012 pages free
4096 bytes per page
67272 total name lookups
cache hits (87% pos + 6% neg) system 0% per-directory
deletions 0%, falsehits 0%, toolong 0%
------------------------------------------------------------------------
vmstat -m
Type InUse MemUse HighUse Requests Size(s)
ad_driver 3 1K - 3 32
sigio 0 0K - 1 64
filedesc 57 29K - 1661 32,512,1024,4096
kenv 72 11K - 77 16,32,64,128
kqueue 0 0K - 86 256,2048
proc-args 1 1K - 2459 16,32,64,128,256
ithread 65 11K - 66 32,128,256
ar_driver 0 0K - 18 512,2048
KTRACE 100 13K - 100 128
linker 137 604K - 186 16,32,64,128,256,512,1024,2048,4096
lockf 2 1K - 6002 64,128
ip6ndp 4 1K - 4 64,128
temp 19 15K - 40765 16,32,64,128,256,512,1024,2048,4096
devbuf 16136 30191K - 16200 16,32,64,128,256,512,1024,2048,4096
acpisem 16 2K - 16 128
module 250 32K - 250 128
mtx_pool 2 16K - 2
CAM XPT 40 14K - 7380 32,64,128,256,2048
osd 3 1K - 4 16,64
CAM periph 10 3K - 274 16,32,64,128,256
subproc 158 282K - 1703 512,4096
proc 2 16K - 2
session 1 1K - 98 128
pgrp 1 1K - 153 128
cred 10 2K - 11818 64,256
uidinfo 2 3K - 789 128,2048
plimit 1 1K - 1120 256
sysctltmp 0 0K - 335 16,32,64,128
sysctloid 3753 184K - 3860 16,32,64,128
sysctl 0 0K - 595 16,32,64
umtx 144 18K - 144 128
p1003.1b 1 1K - 1 16
SWAP 0 0K - 2 64
scsi_da 0 0K - 24 16
bus-sc 88 104K - 2260 16,32,64,128,256,512,1024,2048,4096
bus 809 77K - 5341 16,32,64,128,256,512,1024
devstat 10 21K - 10 32,4096
eventhandler 66 6K - 67 64,128
acpidev 94 6K - 94 64
kobj 160 640K - 209 4096
Per-cpu 1 1K - 1 32
rman 173 21K - 571 16,32,128
CAM dev queue 2 1K - 2 128
sbuf 0 0K - 606 16,32,64,128,256,512,1024,2048,4096
CAM queue 14 2K - 1800 16,32,64,128,256,512,1024,2048
pci_link 76 6K - 76 16,32,128
stack 0 0K - 4 256
taskqueue 13 2K - 13 16,32,128
Unitno 10 1K - 54 32,64
iov 0 0K - 286 16,64,256,512
select 14 2K - 14 128
ioctlops 0 0K - 14321 16,32,64,128,256,512,1024,2048,4096
msg 4 30K - 4 2048,4096
sem 4 11K - 4 512,1024
shm 1 20K - 1
tty 20 20K - 25 1024,2048
pts 0 0K - 1 256
mbuf_tag 0 0K - 9 32,128
shmfd 1 8K - 1
kbdmux 7 10K - 7 16,512,1024,2048,4096
pcb 13 157K - 116 16,32,128,1024,2048,4096
soname 0 0K - 832 16,32,128
biobuf 0 0K - 6 2048
vfscache 1 1024K - 1
cl_savebuf 0 0K - 2 64
vfs_hash 1 512K - 1
vnodes 2 1K - 2 256
vnodemarker 0 0K - 2237 512
mount 1 1K - 121 16,32,64,128,256
BPF 3 1K - 3 128
ether_multi 15 1K - 16 16,32,64
ifaddr 35 12K - 35 32,64,128,256,512,4096
ifnet 4 7K - 4 128,2048
clone 5 20K - 5 4096
arpcom 1 1K - 1 16
lltable 13 5K - 18 256,512
acpi_perf 1 1K - 1 128
ddb_capture 1 48K - 1
acpica 2224 197K - 75028 16,32,64,128,256,512,1024,2048
ppbusdev 3 1K - 3 256
routetbl 18 4K - 66 32,64,128,256,512
igmp 3 1K - 3 256
entropy 1024 64K - 1024 64
UART 3 2K - 3 16,512,1024
acpitask 1 2K - 1 2048
in_multi 2 1K - 2 256
sctp_iter 0 0K - 3 256
sctp_ifn 2 1K - 2 128
sctp_ifa 4 1K - 4 128
sctp_vrf 1 1K - 1 64
sctp_a_it 0 0K - 3 16
hostcache 1 28K - 1
syncache 1 96K - 1
USBdev 12 3K - 12 64,128,1024
USB 23 19K - 24 16,32,64,128,2048,4096
in6_multi 12 2K - 12 32,256
DEVFS1 96 48K - 104 512
mld 3 1K - 3 128
NFS FHA 1 2K - 1 2048
rpc 2 9K - 2 256
savedino 0 0K - 45 256
dirrem 0 0K - 168 64
mkdir 0 0K - 2 64
diradd 0 0K - 168 64
freefile 0 0K - 56 64
freeblks 0 0K - 51 256
freefrag 0 0K - 13 64
allocdirect 0 0K - 82 256
bmsafemap 0 0K - 44 128
newblk 1 1K - 83 64,512
inodedep 1 512K - 226 256
pagedep 1 128K - 35 128
ufs_dirhash 0 0K - 48 16,32,64,128,512
ufs_mount 0 0K - 9 512,2048,4096
DEVFS3 113 29K - 122 256
vm_pgdata 1 128K - 2 128
DEVFS 14 1K - 15 16,128
pfs_nodes 20 5K - 20 256
io_apic 1 2K - 1 2048
CAM SIM 2 1K - 2 256
memdesc 1 4K - 1 4096
nexusdev 3 1K - 3 16
GEOM 117 32K - 963 16,32,64,128,256,512,1024
isadev 7 1K - 7 128
atkbddev 2 1K - 2 64
ata_generic 3 3K - 3 1024
cdev 7 2K - 7 256
solaris 17483 347616K - 126875 16,32,64,128,256,512,1024,2048,4096
kstat_data 2 1K - 2 64
------------------------------------------------------------------------
vmstat -z
ITEM SIZE LIMIT USED FREE REQUESTS FAILURES
UMA Kegs: 208, 0, 93, 9, 93, 0
UMA Zones: 224, 0, 93, 9, 93, 0
UMA Slabs: 568, 0, 5332, 639, 45675, 0
UMA RCntSlabs: 568, 0, 314, 1, 314, 0
UMA Hash: 256, 0, 3, 12, 3, 0
16 Bucket: 152, 0, 58, 17, 58, 0
32 Bucket: 280, 0, 48, 8, 48, 0
64 Bucket: 536, 0, 86, 5, 86, 409
128 Bucket: 1048, 0, 213, 0, 213, 0
VM OBJECT: 216, 0, 139, 1301, 24601, 0
MAP: 232, 0, 7, 25, 7, 0
KMAP ENTRY: 120, 148800, 570, 19, 94058, 0
MAP ENTRY: 120, 0, 7, 1016, 45896, 0
DP fakepg: 120, 0, 0, 0, 0, 0
SG fakepg: 120, 0, 0, 0, 0, 0
mt_zone: 2056, 0, 237, 10, 237, 0
16: 16, 0, 1813, 203, 42675, 0
32: 32, 0, 2516, 615, 13114, 0
64: 64, 0, 15701, 651, 108867, 0
128: 128, 0, 3973, 1479, 32168, 0
256: 256, 0, 504, 366, 33091, 0
512: 512, 0, 9289, 2121, 21661, 0
1024: 1024, 0, 134, 3210, 12076, 0
2048: 2048, 0, 455, 367, 8531, 0
4096: 4096, 0, 325, 210, 9047, 0
Files: 80, 0, 0, 180, 8543, 0
TURNSTILE: 136, 0, 145, 35, 145, 0
umtx pi: 96, 0, 0, 0, 0, 0
PROC: 1120, 0, 57, 42, 1602, 0
THREAD: 912, 0, 128, 16, 141, 0
SLEEPQUEUE: 64, 0, 145, 79, 145, 0
VMSPACE: 392, 0, 1, 39, 1514, 0
cpuset: 72, 0, 2, 98, 2, 0
mbuf_packet: 256, 0, 264, 124, 1154230, 0
mbuf: 256, 0, 124, 392, 2593412, 0
mbuf_cluster: 2048, 25600, 384, 6, 384, 0
mbuf_jumbo_page: 4096, 12800, 60, 59, 313097, 0
mbuf_jumbo_9k: 9216, 19200, 0, 0, 0, 0
mbuf_jumbo_16k: 16384, 12800, 0, 0, 0, 0
mbuf_ext_refcnt: 4, 0, 0, 0, 0, 0
g_bio: 232, 0, 0, 272, 35602, 0
ttyinq: 160, 0, 0, 144, 300, 0
ttyoutq: 256, 0, 0, 90, 160, 0
ata_request: 312, 0, 1, 155, 281018, 0
ata_composite: 336, 0, 0, 0, 0, 0
VNODE: 472, 0, 9, 2591, 2640, 0
VNODEPOLL: 112, 0, 0, 0, 0, 0
NAMEI: 1024, 0, 0, 12, 26016, 0
S VFS Cache: 108, 0, 0, 1650, 2859, 0
L VFS Cache: 328, 0, 0, 1104, 1095, 0
DIRHASH: 1024, 0, 0, 56, 55, 0
NFSMOUNT: 608, 0, 0, 0, 0, 0
NFSNODE: 648, 0, 0, 0, 0, 0
pipe: 728, 0, 0, 15, 846, 0
ksiginfo: 112, 0, 91, 965, 91, 0
itimer: 344, 0, 0, 0, 0, 0
KNOTE: 120, 0, 0, 62, 88, 0
socket: 680, 25602, 0, 36, 448, 0
ipq: 56, 819, 0, 0, 0, 0
udp_inpcb: 336, 25608, 0, 22, 314, 0
udpcb: 16, 25704, 0, 168, 314, 0
tcp_inpcb: 336, 25608, 1, 21, 27, 0
tcpcb: 880, 25600, 0, 12, 27, 0
tcptw: 72, 5150, 1, 99, 6, 0
syncache: 144, 15366, 0, 52, 3, 0
hostcache: 136, 15372, 1, 55, 2, 0
tcpreass: 40, 1680, 0, 168, 19771, 0
sackhole: 32, 0, 0, 202, 51, 0
sctp_ep: 1248, 25602, 0, 0, 0, 0
sctp_asoc: 2176, 40000, 0, 0, 0, 0
sctp_laddr: 48, 80064, 0, 144, 3, 0
sctp_raddr: 592, 80004, 0, 0, 0, 0
sctp_chunk: 144, 400010, 0, 0, 0, 0
sctp_readq: 104, 400032, 0, 0, 0, 0
sctp_stream_msg_out: 96, 400026, 0, 0, 0, 0
sctp_asconf: 40, 400008, 0, 0, 0, 0
sctp_asconf_ack: 48, 400032, 0, 0, 0, 0
ripcb: 336, 25608, 0, 0, 0, 0
unpcb: 240, 25600, 0, 32, 106, 0
rtentry: 200, 0, 8, 30, 8, 0
selfd: 56, 0, 15, 111, 17678, 0
SWAPMETA: 288, 116519, 0, 0, 0, 0
Mountpoints: 752, 0, 1, 9, 5, 0
FFS inode: 168, 0, 0, 638, 658, 0
FFS1 dinode: 128, 0, 0, 0, 0, 0
FFS2 dinode: 256, 0, 0, 615, 658, 0
taskq_ent_cache: 64, 0, 604, 12, 1206, 0
taskq_cache: 288, 0, 13, 13, 26, 0
zio_cache: 720, 0, 0, 3130, 39949, 0
dmu_buf_impl_t: 224, 0, 364, 2713, 3197, 0
dnode_t: 768, 0, 310, 1955, 2282, 0
arc_buf_hdr_t: 208, 0, 3454, 20, 3678, 0
arc_buf_t: 72, 0, 2733, 767, 3678, 0
zil_lwb_cache: 200, 0, 0, 38, 24, 0
zfs_znode_cache: 376, 0, 0, 1960, 1947, 0
------------------------------------------------------------------------
vmstat -i
interrupt total rate
irq14: ata0 2719 23
irq16: hptrr0 4836 40
irq17: em0 323725 2743
irq21: ohci0+ 8019 67
irq22: ehci0 2 0
cpu0: timer 16623574 140877
Total 16962875 143753
------------------------------------------------------------------------
pstat -T
0/12328 files
0M/0M swap space
------------------------------------------------------------------------
pstat -s
Device 512-blocks Used Avail Capacity
------------------------------------------------------------------------
iostat
iostat: kvm_read(_tk_nin): invalid address (0x0)
iostat: disabling TTY statistics
iostat: kvm_getcptime: invalid address (0x0)
iostat: disabling CPU time statistics
ad0 ad4 ad6
KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s
15.78 21 0.33 44.65 31 1.37 44.48 31 1.36
------------------------------------------------------------------------
ipcs -a
Message Queues:
T ID KEY MODE OWNER GROUP CREATOR CGROUP CBYTES QNUM QBYTES LSPID LRPID STIME RTIME CTIME
Shared Memory:
T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME
Semaphores:
T ID KEY MODE OWNER GROUP CREATOR CGROUP NSEMS OTIME CTIME
------------------------------------------------------------------------
ipcs -T
msginfo:
msgmax: 16384 (max characters in a message)
msgmni: 40 (# of message queues)
msgmnb: 2048 (max characters in a message queue)
msgtql: 40 (max # of messages in system)
msgssz: 8 (size of a message segment)
msgseg: 2048 (# of message segments in system)
shminfo:
shmmax: 33554432 (max shared memory segment size)
shmmin: 1 (min shared memory segment size)
shmmni: 192 (max number of shared memory identifiers)
shmseg: 128 (max shared memory segments per process)
shmall: 8192 (max amount of shared memory in pages)
seminfo:
semmap: 30 (# of entries in semaphore map)
semmni: 10 (# of semaphore identifiers)
semmns: 60 (# of semaphores in system)
semmnu: 30 (# of undo structures in system)
semmsl: 60 (max # of semaphores per id)
semopm: 100 (max # of operations per semop call)
semume: 10 (max # of undo entries per process)
semusz: 152 (size in bytes of undo structure)
semvmx: 32767 (semaphore maximum value)
semaem: 16384 (adjust on exit max value)
------------------------------------------------------------------------
nfsstat
Client Info:
Rpc Counts:
Getattr Setattr Lookup Readlink Read Write Create Remove
0 0 0 0 0 0 0 0
Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access
0 0 0 0 0 0 0 0
Mknod Fsstat Fsinfo PathConf Commit
0 0 0 0 0
Rpc Info:
TimedOut Invalid X Replies Retries Requests
0 0 0 0 0
Cache Info:
Attr Hits Misses Lkup Hits Misses BioR Hits Misses BioW Hits Misses
0 0 0 0 0 0 0 0
BioRLHits Misses BioD Hits Misses DirE Hits Misses
0 0 0 0 0 0
Server Info:
Getattr Setattr Lookup Readlink Read Write Create Remove
0 0 0 0 0 0 0 0
Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access
0 0 0 0 0 0 0 0
Mknod Fsstat Fsinfo PathConf Commit
0 0 0 0 0
Server Ret-Failed
0
Server Faults
0
Server Cache Stats:
Inprog Idem Non-idem Misses
0 0 0 0
Server Write Gathering:
WriteOps WriteRPC Opsaved
0 0 0
------------------------------------------------------------------------
netstat -s
tcp:
1096371 packets sent
868640 data packets (1254645848 bytes)
1541 data packets (2214040 bytes) retransmitted
0 data packets unnecessarily retransmitted
0 resends initiated by MTU discovery
169438 ack-only packets (32 delayed)
0 URG only packets
0 window probe packets
56735 window update packets
17 control packets
1032059 packets received
494026 acks (for 1254645850 bytes)
4675 duplicate acks
0 acks for unsent data
308280 packets (442142813 bytes) received in-sequence
1321 completely duplicate packets (1912808 bytes)
31 old duplicate packets
0 packets with some dup. data (0 bytes duped)
19771 out-of-order packets (28628408 bytes)
0 packets (0 bytes) of data after window
0 window probes
204743 window update packets
0 packets received after close
0 discarded for bad checksums
0 discarded for bad header offset fields
0 discarded because packet too short
0 discarded due to memory problems
13 connection requests
3 connection accepts
0 bad connection attempts
0 listen queue overflows
0 ignored RSTs in the windows
8 connections established (including accepts)
26 connections closed (including 1 drop)
6 connections updated cached RTT on close
7 connections updated cached RTT variance on close
5 connections updated cached ssthresh on close
8 embryonic connections dropped
479533 segments updated rtt (of 263306 attempts)
26 retransmit timeouts
1 connection dropped by rexmit timeout
0 persist timeouts
0 connections dropped by persist timeout
0 Connections (fin_wait_2) dropped because of timeout
0 keepalive timeouts
0 keepalive probes sent
0 connections dropped by keepalive
13229 correct ACK header predictions
307074 correct data packet header predictions
3 syncache entries added
0 retransmitted
0 dupsyn
0 dropped
3 completed
0 bucket overflow
0 cache overflow
0 reset
0 stale
0 aborted
0 badack
0 unreach
0 zone failures
3 cookies sent
0 cookies received
25 SACK recovery episodes
696 segment rexmits in SACK recovery episodes
1007808 byte rexmits in SACK recovery episodes
3709 SACK options (SACK blocks) received
18267 SACK options (SACK blocks) sent
0 SACK scoreboard overflow
0 packets with ECN CE bit set
0 packets with ECN ECT(0) bit set
0 packets with ECN ECT(1) bit set
0 successful ECN handshakes
0 times ECN reduced the congestion window
udp:
294 datagrams received
0 with incomplete header
0 with bad data length field
0 with bad checksum
0 with no checksum
0 dropped due to no socket
0 broadcast/multicast datagrams undelivered
0 dropped due to full socket buffers
0 not for hashed pcb
294 delivered
110 datagrams output
0 times multicast source filter matched
ip:
1032345 total packets received
0 bad header checksums
0 with size smaller than minimum
0 with data size < data length
0 with ip length > max ip packet size
0 with header length < data size
0 with data length < header length
0 with bad options
0 with incorrect version number
0 fragments received
0 fragments dropped (dup or out of space)
0 fragments dropped after timeout
0 packets reassembled ok
1032345 packets for this host
0 packets for unknown/unsupported protocol
0 packets forwarded (0 packets fast forwarded)
0 packets not forwardable
0 packets received for unknown multicast group
0 redirects sent
1096481 packets sent from this host
0 packets sent with fabricated ip header
0 output packets dropped due to no bufs, etc.
0 output packets discarded due to no route
0 output datagrams fragmented
0 fragments created
0 datagrams that can't be fragmented
0 tunneling packets that can't find gif
0 datagrams with bad address in header
icmp:
0 calls to icmp_error
0 errors not generated in response to an icmp message
0 messages with bad code fields
0 messages less than the minimum length
0 messages with bad checksum
0 messages with bad length
0 multicast echo requests ignored
0 multicast timestamp requests ignored
0 message responses generated
0 invalid return addresses
0 no return routes
igmp:
0 messages received
0 messages received with too few bytes
0 messages received with wrong TTL
0 messages received with bad checksum
0 V1/V2 membership queries received
0 V3 membership queries received
0 membership queries received with invalid field(s)
0 general queries received
0 group queries received
0 group-source queries received
0 group-source queries dropped
0 membership reports received
0 membership reports received with invalid field(s)
0 membership reports received for groups to which we belong
0 V3 reports received without Router Alert
0 membership reports sent
ip6:
8 total packets received
0 with size smaller than minimum
0 with data size < data length
0 with bad options
0 with incorrect version number
0 fragments received
0 fragments dropped (dup or out of space)
0 fragments dropped after timeout
0 fragments that exceeded limit
0 packets reassembled ok
8 packets for this host
0 packets forwarded
0 packets not forwardable
0 redirects sent
8 packets sent from this host
0 packets sent with fabricated ip header
0 output packets dropped due to no bufs, etc.
0 output packets discarded due to no route
0 output datagrams fragmented
0 fragments created
0 datagrams that can't be fragmented
0 packets that violated scope rules
0 multicast packets which we don't join
Input histogram:
TCP: 8
Mbuf statistics:
8 one mbuf
0 one ext mbuf
0 two or more ext mbuf
0 packets whose headers are not continuous
0 tunneling packets that can't find gif
0 packets discarded because of too many headers
0 failures of source address selection
Source addresses selection rule applied:
8 first candidate
8 same address
icmp6:
0 calls to icmp6_error
0 errors not generated in response to an icmp6 message
0 errors not generated because of rate limitation
0 messages with bad code fields
0 messages < minimum length
0 bad checksums
0 messages with bad length
Histogram of error messages to be generated:
0 no route
0 administratively prohibited
0 beyond scope
0 address unreachable
0 port unreachable
0 packet too big
0 time exceed transit
0 time exceed reassembly
0 erroneous header field
0 unrecognized next header
0 unrecognized option
0 redirect
0 unknown
0 message responses generated
0 messages with too many ND options
0 messages with bad ND options
0 bad neighbor solicitation messages
0 bad neighbor advertisement messages
0 bad router solicitation messages
0 bad router advertisement messages
0 bad redirect messages
0 path MTU changes
rip6:
0 messages received
0 checksum calculations on inbound
0 messages with bad checksum
0 messages dropped due to no socket
0 multicast messages dropped due to no socket
0 messages dropped due to full socket buffers
0 delivered
0 datagrams output
------------------------------------------------------------------------
netstat -m
388/516/904 mbufs in use (current/cache/total)
260/130/390/25600 mbuf clusters in use (current/cache/total/max)
264/124 mbuf+clusters out of packet secondary zone in use (current/cache)
60/59/119/12800 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/19200 9k jumbo clusters in use (current/cache/total/max)
0/0/0/12800 16k jumbo clusters in use (current/cache/total/max)
857K/625K/1482K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
0 calls to protocol drain routines
------------------------------------------------------------------------
netstat -id
Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll Drop
em0 9000 <Link#1> 00:1b:21:17:2e:5d 1032504 0 1096425 0 0 0
em0 9000 192.168.0.0 h4mst0r.marines 1032337 - 1096473 - - -
plip0 1500 <Link#2> 0 0 0 0 0 0
lo0 16384 <Link#3> 16 0 16 0 0 0
lo0 16384 fe80:3::1 fe80:3::1 0 - 0 - - -
lo0 16384 localhost.mar ::1 0 - 8 - - -
lo0 16384 your-net localhost.marines 8 - 8 - - -
------------------------------------------------------------------------
netstat -anr
Routing tables
Internet:
Destination Gateway Flags Refs Use Netif Expire
default 192.168.0.1 UGS 0 0 em0
127.0.0.1 link#3 UH 0 8 lo0
192.168.0.0/24 link#1 U 0 1096473 em0
192.168.0.222 link#3 UHS 0 0 lo0
Internet6:
Destination Gateway Flags Netif Expire
::1 ::1 UH lo0
fe80::%lo0/64 link#3 U lo0
ff01:3::/32 fe80::1%lo0 U lo0
ff02::%lo0/32 fe80::1%lo0 U lo0
------------------------------------------------------------------------
netstat -anA
Active Internet connections (including servers)
Tcpcb Proto Recv-Q Send-Q Local Address Foreign Address (state)
ffffff005e9b1ca8 tcp4 0 0 192.168.0.222.22 192.168.0.23.35754 TIME_WAIT
------------------------------------------------------------------------
netstat -aL
Current listen queue sizes (qlen/incqlen/maxqlen)
Proto Listen Local Address
------------------------------------------------------------------------
fstat
Segmentation fault
------------------------------------------------------------------------
dmesg
Copyright (c) 1992-2009 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.0-BETA2 #14: Wed Aug 19 22:43:17 CEST 2009
oni0@hamstor:/usr/obj/usr/src/sys/HAMSTOR
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: AMD Sempron(tm) Processor LE-1100 (1908.70-MHz K8-class CPU)
Origin = "AuthenticAMD" Id = 0x70ff1 Stepping = 1
Features=0x78bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2>
Features2=0x2001<SSE3,CX16>
AMD Features=0xea500800<SYSCALL,NX,MMX+,FFXSR,RDTSCP,LM,3DNow!+,3DNow!>
AMD Features2=0x119<LAHF,ExtAPIC,CR8,Prefetch>
real memory = 2147483648 (2048 MB)
avail memory = 4044324864 (3856 MB)
ACPI APIC Table: <GBT NVDAACPI>
ioapic0: Changing APIC ID to 2
ioapic0 <Version 1.1> irqs 0-23 on motherboard
kbd1 at kbdmux0
acpi0: <GBT NVDAACPI> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, cbef0000 (3) failed
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
acpi_hpet0: <High Precision Event Timer> iomem 0xfeff0000-0xfeff03ff on acpi0
Timecounter "HPET" frequency 25000000 Hz quality 900
acpi_button0: <Power Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pci0: <memory, RAM> at device 0.0 (no driver attached)
isab0: <PCI-ISA bridge> at device 1.0 on pci0
isa0: <ISA bus> on isab0
pci0: <serial bus, SMBus> at device 1.1 (no driver attached)
pci0: <memory, RAM> at device 1.2 (no driver attached)
ohci0: <nVidia nForce MCP61 USB Controller> mem 0xef007000-0xef007fff irq 21 at device 2.0 on pci0
ohci0: [ITHREAD]
usbus0: <nVidia nForce MCP61 USB Controller> on ohci0
ehci0: <NVIDIA nForce MCP61 USB 2.0 controller> mem 0xef004000-0xef0040ff irq 22 at device 2.1 on pci0
ehci0: [ITHREAD]
usbus1: EHCI version 1.0
usbus1: <NVIDIA nForce MCP61 USB 2.0 controller> on ehci0
pcib1: <ACPI PCI-PCI bridge> at device 4.0 on pci0
pci1: <ACPI PCI bus> on pcib1
em0: <Intel(R) PRO/1000 Network Connection 6.9.14> port 0xa000-0xa03f mem 0xec020000-0xec03ffff,0xec000000-0xec01ffff irq 17 at device 7.0 on pci1
em0: [FILTER]
em0: Ethernet address: 00:1b:21:17:2e:5d
pci0: <multimedia, HDA> at device 5.0 (no driver attached)
atapci0: <nVidia nForce MCP61 UDMA133 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xf000-0xf00f at device 6.0 on pci0
ata0: <ATA channel 0> on atapci0
ata0: [ITHREAD]
ata1: <ATA channel 1> on atapci0
ata1: [ITHREAD]
pci0: <bridge> at device 7.0 (no driver attached)
atapci1: <nVidia nForce MCP61 SATA300 controller> port 0x9f0-0x9f7,0xbf0-0xbf3,0x970-0x977,0xb70-0xb73,0xdc00-0xdc0f mem 0xef006000-0xef006fff irq 21 at device 8.0 on pci0
atapci1: [ITHREAD]
ata2: <ATA channel 0> on atapci1
ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci1
ata3: [ITHREAD]
pcib2: <ACPI PCI-PCI bridge> at device 9.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <PCI-PCI bridge> at device 0.0 on pci2
pci3: <PCI bus> on pcib3
pcib4: <PCI-PCI bridge> at device 0.2 on pci2
pci4: <PCI bus> on pcib4
hptrr0: <sx508x> port 0xb000-0xb0ff mem 0xe9000000-0xe90fffff irq 16 at device 4.0 on pci4
hptrr: adapter at PCI 4:4:0, IRQ 16
vgapci0: <VGA-compatible display> mem 0xea000000-0xeaffffff,0xd0000000-0xdfffffff,0xed000000-0xedffffff irq 22 at device 13.0 on pci0
atrtc1: <AT realtime clock> port 0x70-0x73 on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: [FILTER]
ppc0: <Parallel port> port 0x378-0x37f irq 7 on acpi0
ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode
ppc0: [ITHREAD]
ppbus0: <Parallel port bus> on ppc0
plip0: <PLIP network interface> on ppbus0
plip0: [ITHREAD]
lpt0: <Printer> on ppbus0
lpt0: [ITHREAD]
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
cpu0: <ACPI CPU> on acpi0
powernow0: <PowerNow! K8> on cpu0
orm0: <ISA Option ROMs> at iomem 0xd0000-0xd3fff,0xd4000-0xd4fff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
atrtc0: <AT Real Time Clock> at port 0x70 irq 8 on isa0
atrtc0: Warning: Couldn't map I/O.
atrtc0: Warning: Couldn't map Interrupt.
Timecounter "TSC" frequency 1908696477 Hz quality 800
Timecounters tick every 1.000 msec
hptrr: start channel [0,0]
hptrr: start channel [0,1]
hptrr: start channel [0,2]
hptrr: start channel [0,3]
hptrr: channel [0,0] started successfully
hptrr: channel [0,1] started successfully
hptrr: channel [0,2] started successfully
hptrr: channel [0,3] started successfully
hptrr0: [GIANT-LOCKED]
hptrr0: [ITHREAD]
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 480Mbps High Speed USB v2.0
ad0: 78166MB <Maxtor 4R080L0 RAMB1TU0> at ata0-master UDMA133
ad4: 715404MB <SAMSUNG HD753LJ 1AA01109> at ata2-master SATA300
ad6: 715404MB <SAMSUNG HD753LJ 1AA01109> at ata3-master SATA300
GEOM: ad0s1: geometry does not match label (255h,63s != 16h,63s).
ugen0.1: <nVidia> at usbus0
uhub0: <nVidia OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
uhub0: 10 ports with 10 removable, self powered
ugen1.1: <nVidia> at usbus1
uhub1: <nVidia EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
da0 at hptrr0 bus 0 target 0 lun 0
da0: <HPT DISK 0_0 4.00> Fixed Direct Access SCSI-0 device
da1 at hptrr0 bus 0 target 1 lun 0
da1: <HPT DISK 0_1 4.00> Fixed Direct Access SCSI-0 device
da2 at hptrr0 bus 0 target 2 lun 0
da2: <HPT DISK 0_2 4.00> Fixed Direct Access SCSI-0 device
da3 at hptrr0 bus 0 target 3 lun 0
da3: <HPT DISK 0_3 4.00> Fixed Direct Access SCSI-0 device
Root mount waiting for: usbus1
Root mount waiting for: usbus1
Root mount waiting for: usbus1
Root mount waiting for: usbus1
uhub1: 10 ports with 10 removable, self powered
Trying to mount root from ufs:/dev/ad0s1a
ugen0.2: <Darfon> at usbus0
ukbd0: <Darfon USB Combo Keyboard, class 0/0, rev 1.10/3.02, addr 2> on usbus0
kbd2 at ukbd0
Entropy harvesting:
interrupts
ethernet
point_to_point
kickstart
.
/dev/ad0s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/ad0s1a: clean, 51888 free (1392 frags, 6312 blocks, 0.3% fragmentation)
/dev/ad0s1f: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/ad0s1f: clean, 30688854 free (155622 frags, 3816654 blocks, 0.5% fragmentation)
/dev/ad0s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/ad0s1d: clean, 2367444 free (612 frags, 295854 blocks, 0.0% fragmentation)
WARNING: ZFS is considered to be an experimental feature in FreeBSD.
ZFS filesystem version 13
ZFS storage pool version 13
Starting Network: lo0 em0.
add net default: gateway 192.168.0.1
Configuring keyboard:
keymap
keyrate
.
Running: /usr/local/sbin/pure-ftpd -c50 -B -C8 -D -E -fftp -H -I15 -lunix -L2000:8 -m4 -S192.168.0.222,65222 -U022:022 -w -R
em0: link state changed to UP
Removing stale Samba tdb files:
.
.
done
Configuring syscons:
keymap
keyrate
font8x16
font8x14
font8x8
blanktime
screensaver
.
Sat Aug 22 08:29:28 CEST 2009
Aug 22 10:01:22 hamstor su: oni0 to root on /dev/pts/0
Aug 22 10:47:23 hamstor shutdown: power-down by oni0:
Stopping inetd.
Stopping cron.
Stopping sshd.
Stopping powerd.
Stopping smbd.
Stopping nmbd.
Stopping pureftpd.
Stopping devd.
Writing entropy file:
.
Terminated
.
Aug 22 10:47:26 hamstor syslogd: exiting on signal 15
Waiting (max 60 seconds) for system process `vnlru' to stop...done
Waiting (max 60 seconds) for system process `bufdaemon' to stop...done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining...1 1 1 0 0 done
All buffers synced.
zfs_umount:971[0]: Force unmount is experimental - report any problems.
Uptime: 2h18m27s
panic: apic_free_vector: Thread already bound.
cpuid = 0
KDB: enter: panic
Physical memory: 4022 MB
Dumping 1592 MB: 1577 1561 1545 1529 1513 1497 1481 1465 1449 1433 1417 1401 1385 1369 1353 1337 1321 1305 1289 1273 1257 1241 1225 1209 1193 1177 1161 1145 1129 1113 1097 1081 1065 1049 1033 1017 1001 985 969 953 937 921 905 889 873 857 841 825 809 793 777 761 745 729 713 697 681 665 649 633 617 601 585 569 553 537 521 505 489 473 457 441 425 409 393 377 361 345 329 313 297 281 265 249 233 217 201 185 169 153 137 121 105 89 73 57 41 25 9
------------------------------------------------------------------------
kernel config
config: File /boot/kernel/kernel doesn't contain configuration file. Either unsupported, or not compiled with INCLUDE_CONFIG_FILE
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20090822115958.f93fcf29.ubm.freebsd>