Date: Mon, 1 Apr 2013 13:05:49 -0500 (CDT) From: Larry Rosenman <ler@lerctr.org> To: freebsd-current@freebsd.org, freebsd-fs@freebsd.org Subject: [CRASH] ZFS recv (fwd)/CURRENT Message-ID: <alpine.BSF.2.00.1304011305010.33680@borg>
next in thread | raw e-mail | index | archive | help
Re-Sending. Any ideas, guys/gals? This really gets in my way. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 512-248-2683 E-Mail: ler@lerctr.org US Mail: 430 Valona Loop, Round Rock, TX 78681-3893 ---------- Forwarded message ---------- Date: Mon, 25 Mar 2013 09:01:30 -0500 (CDT) From: Larry Rosenman <ler@lerctr.org> To: freebsd-current@freebsd.org Subject: [CRASH] ZFS recv Greetings, I'm getting a zfs recv crash on 10.0-CURRENT.... I was ssh'd to my 8.4 box and did: zfs send -R -D vault@2013-03-24 | ssh home.lerctr.org zfs recv -F -u -v -d zroot/backups/TBH And on the 2nd (or so) filesystem got the following panic. I have the vmcore as well as sources, and can give ssh access. I can also reproduce this at will. ideas? borg.lerctr.org dumped core - see /var/crash/vmcore.4 Mon Mar 25 08:52:46 CDT 2013 FreeBSD borg.lerctr.org 10.0-CURRENT FreeBSD 10.0-CURRENT #129 r248695: Mon Mar 25 05:03:32 CDT 2013 root@borg.lerctr.org:/usr/obj/usr/src/sys/BORG-DTRACE amd64 panic: page fault GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x378 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80531426 stack pointer = 0x28:0xffffff91579193d0 frame pointer = 0x28:0xffffff9157919470 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 1044 (zfs) trap number = 12 panic: page fault cpuid = 0 Uptime: 2m10s Dumping 4913 out of 64747 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /boot/kernel/zfs.ko.symbols...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/acl_nfs4.ko...Reading symbols from /boot/kernel/acl_nfs4.ko.symbols...done. done. Loaded symbols for /boot/kernel/acl_nfs4.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /boot/kernel/opensolaris.ko.symbols...done. done. Loaded symbols for /boot/kernel/opensolaris.ko Reading symbols from /boot/kernel/linux.ko...Reading symbols from /boot/kernel/linux.ko.symbols...done. done. Loaded symbols for /boot/kernel/linux.ko Reading symbols from /boot/kernel/coretemp.ko...Reading symbols from /boot/kernel/coretemp.ko.symbols...done. done. Loaded symbols for /boot/kernel/coretemp.ko Reading symbols from /boot/kernel/ichsmb.ko...Reading symbols from /boot/kernel/ichsmb.ko.symbols...done. done. Loaded symbols for /boot/kernel/ichsmb.ko Reading symbols from /boot/kernel/smbus.ko...Reading symbols from /boot/kernel/smbus.ko.symbols...done. done. Loaded symbols for /boot/kernel/smbus.ko Reading symbols from /boot/kernel/ichwd.ko...Reading symbols from /boot/kernel/ichwd.ko.symbols...done. done. Loaded symbols for /boot/kernel/ichwd.ko Reading symbols from /boot/kernel/cpuctl.ko...Reading symbols from /boot/kernel/cpuctl.ko.symbols...done. done. Loaded symbols for /boot/kernel/cpuctl.ko Reading symbols from /boot/kernel/crypto.ko...Reading symbols from /boot/kernel/crypto.ko.symbols...done. done. Loaded symbols for /boot/kernel/crypto.ko Reading symbols from /boot/kernel/cryptodev.ko...Reading symbols from /boot/kernel/cryptodev.ko.symbols...done. done. Loaded symbols for /boot/kernel/cryptodev.ko Reading symbols from /boot/kernel/dtraceall.ko...Reading symbols from /boot/kernel/dtraceall.ko.symbols...done. done. Loaded symbols for /boot/kernel/dtraceall.ko Reading symbols from /boot/kernel/profile.ko...Reading symbols from /boot/kernel/profile.ko.symbols...done. done. Loaded symbols for /boot/kernel/profile.ko Reading symbols from /boot/kernel/cyclic.ko...Reading symbols from /boot/kernel/cyclic.ko.symbols...done. done. Loaded symbols for /boot/kernel/cyclic.ko Reading symbols from /boot/kernel/dtrace.ko...Reading symbols from /boot/kernel/dtrace.ko.symbols...done. done. Loaded symbols for /boot/kernel/dtrace.ko Reading symbols from /boot/kernel/systrace_freebsd32.ko...Reading symbols from /boot/kernel/systrace_freebsd32.ko.symbols...done. done. Loaded symbols for /boot/kernel/systrace_freebsd32.ko Reading symbols from /boot/kernel/systrace.ko...Reading symbols from /boot/kernel/systrace.ko.symbols...done. done. Loaded symbols for /boot/kernel/systrace.ko Reading symbols from /boot/kernel/sdt.ko...Reading symbols from /boot/kernel/sdt.ko.symbols...done. done. Loaded symbols for /boot/kernel/sdt.ko Reading symbols from /boot/kernel/lockstat.ko...Reading symbols from /boot/kernel/lockstat.ko.symbols...done. done. Loaded symbols for /boot/kernel/lockstat.ko Reading symbols from /boot/kernel/fasttrap.ko...Reading symbols from /boot/kernel/fasttrap.ko.symbols...done. done. Loaded symbols for /boot/kernel/fasttrap.ko Reading symbols from /boot/kernel/fbt.ko...Reading symbols from /boot/kernel/fbt.ko.symbols...done. done. Loaded symbols for /boot/kernel/fbt.ko Reading symbols from /boot/kernel/dtnfscl.ko...Reading symbols from /boot/kernel/dtnfscl.ko.symbols...done. done. Loaded symbols for /boot/kernel/dtnfscl.ko Reading symbols from /boot/kernel/dtmalloc.ko...Reading symbols from /boot/kernel/dtmalloc.ko.symbols...done. done. Loaded symbols for /boot/kernel/dtmalloc.ko Reading symbols from /boot/kernel/dtio.ko...Reading symbols from /boot/kernel/dtio.ko.symbols...done. done. Loaded symbols for /boot/kernel/dtio.ko Reading symbols from /boot/modules/vboxdrv.ko...done. Loaded symbols for /boot/modules/vboxdrv.ko Reading symbols from /boot/kernel/fdescfs.ko...Reading symbols from /boot/kernel/fdescfs.ko.symbols...done. done. Loaded symbols for /boot/kernel/fdescfs.ko Reading symbols from /boot/kernel/uhid.ko...Reading symbols from /boot/kernel/uhid.ko.symbols...done. done. Loaded symbols for /boot/kernel/uhid.ko Reading symbols from /boot/modules/vboxnetflt.ko...done. Loaded symbols for /boot/modules/vboxnetflt.ko Reading symbols from /boot/kernel/netgraph.ko...Reading symbols from /boot/kernel/netgraph.ko.symbols...done. done. Loaded symbols for /boot/kernel/netgraph.ko Reading symbols from /boot/kernel/ng_ether.ko...Reading symbols from /boot/kernel/ng_ether.ko.symbols...done. done. Loaded symbols for /boot/kernel/ng_ether.ko Reading symbols from /boot/modules/vboxnetadp.ko...done. Loaded symbols for /boot/modules/vboxnetadp.ko #0 doadump (textdump=<value optimized out>) at pcpu.h:229 229 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump (textdump=<value optimized out>) at pcpu.h:229 #1 0xffffffff80529060 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:447 #2 0xffffffff805293e7 in panic (fmt=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:754 #3 0xffffffff80732cd5 in trap_fatal (frame=<value optimized out>, eva=<value optimized out>) at /usr/src/sys/amd64/amd64/trap.c:872 #4 0xffffffff807330d9 in trap_pfault (frame=0x0, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:730 #5 0xffffffff807326bc in trap (frame=0xffffff9157919320) at /usr/src/sys/amd64/amd64/trap.c:463 #6 0xffffffff8071cc62 in calltrap () at exception.S:228 #7 0xffffffff80531426 in _sx_xlock_hard (sx=0xfffffe01d5b64db8, tid=18446741877920023696, opts=<value optimized out>, file=0x0, line=250541281) at /usr/src/sys/kern/kern_sx.c:556 #8 0xffffffff80530f8c in _sx_xlock (sx=0x0, opts=0, file=0x0, line=0) at sx.h:152 #9 0xffffffff80e67251 in dmu_objset_from_ds (ds=0xfffffe01d5b64c00, osp=0xffffff91579195b0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c:434 #10 0xffffffff80e649ef in dmu_recv_stream (drc=0xffffff9157919750, fp=<value optimized out>, voffp=0xffffff9157919740, cleanup_fd=8, action_handlep=<value optimized out>) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c:1318 #11 0xffffffff80ed9226 in zfs_ioc_recv (zc=0xffffff8012cc8000) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:4084 #12 0xffffffff80ed44a1 in zfsdev_ioctl (dev=<value optimized out>, zcmd=<value optimized out>, arg=0xffffff8012cc8000 "zroot/backups/TBH/home", flag=<value optimized out>, td=<value optimized out>) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:5902 #13 0xffffffff80427e0f in devfs_ioctl_f (fp=0xfffffe00c0b360a0, com=3517471259, data=0xffffff8012cc8000, cred=<value optimized out>, td=0xfffffe00c0bec490) at /usr/src/sys/fs/devfs/devfs_vnops.c:757 #14 0xffffffff8057359b in kern_ioctl (td=0xfffffe00c0bec490, fd=<value optimized out>, com=18446741877920023696) at file.h:306 #15 0xffffffff8057331f in sys_ioctl (td=0xfffffe00c0bec490, uap=0xffffff9157919b80) at /usr/src/sys/kern/sys_generic.c:693 #16 0xffffffff807335d7 in amd64_syscall (td=0xfffffe00c0bec490, traced=0) at subr_syscall.c:134 #17 0xffffffff8071cf4b in Xfast_syscall () at exception.S:387 #18 0x00000008019dacea in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb) ------------------------------------------------------------------------ ps -axl UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND 0 0 0 0 -8 0 0 0 - DLs - 0:02.51 [kernel] 0 1 0 0 20 0 9428 0 wait DLs - 0:00.10 [init] 0 2 0 0 -16 0 0 0 crypto_w DL - 0:00.00 [crypto] 0 3 0 0 -16 0 0 0 crypto_r DL - 0:00.00 [crypto returns 0 4 0 0 -16 0 0 0 - DL - 0:00.00 [fdc0] 0 5 0 0 -8 0 0 0 tx->tx_s DL - 0:00.08 [zfskern] 0 6 0 0 -16 0 0 0 waiting_ DL - 0:00.00 [sctp_iterator] 0 7 0 0 -16 0 0 0 ccb_scan DL - 0:00.00 [xpt_thrd] 0 8 0 0 -16 0 0 0 - RL - 0:00.00 [pagedaemon] 0 9 0 0 -16 0 0 0 psleep DL - 0:00.00 [vmdaemon] 0 10 0 0 -16 0 0 0 audit_wo DL - 0:00.00 [audit] 0 11 0 0 155 0 0 0 - RL - 6:41.28 [idle] 0 12 0 0 -84 0 0 0 - WL - 0:00.26 [intr] 0 13 0 0 -8 0 0 0 - DL - 0:00.33 [geom] 0 14 0 0 -16 0 0 0 - DL - 0:00.01 [yarrow] 0 15 0 0 -68 0 0 0 - DL - 0:00.01 [usb] 0 16 0 0 -20 0 0 0 VBoxIS DL - 0:00.00 [TIMER] 0 17 0 0 155 0 0 0 pgzero DL - 0:00.00 [pagezero] 0 18 0 0 -16 0 0 0 psleep DL - 0:00.00 [bufdaemon] 0 19 0 0 16 0 0 0 - RL - 0:00.00 [syncer] 0 20 0 0 -16 0 0 0 vlruwt DL - 0:00.00 [vnlru] 0 632 1 0 20 0 13196 0 select Ds - 0:00.00 [devd] 0 757 1 0 20 0 14376 0 select Ds - 0:00.04 [syslogd] 0 760 1 0 -52 0 6180 2088 - Rs - 0:00.00 [watchdogd] 0 774 1 0 20 0 16456 0 select Ds - 0:00.01 [rpcbind] 0 809 1 0 52 0 38948 0 select Ds - 0:00.02 [mountd] 0 815 1 0 52 0 36792 0 select Ds - 0:00.04 [nfsd] 0 817 815 0 52 0 12216 0 rpcsvc D - 0:00.00 [nfsd] 0 853 1 0 20 0 25196 0 select Ds - 0:00.01 [ntpd] 0 874 0 0 -16 0 0 0 sleep DL - 0:00.00 [ng_queue] 0 886 1 0 20 0 34708 0 nanslp Ds - 0:00.01 [perl] 0 890 1 0 52 0 30780 0 nanslp D - 0:00.43 [smartd] 70 899 1 0 20 0 84416 0 select Ds - 0:00.22 [postgres] 70 903 899 0 20 0 84416 0 select Ds - 0:00.00 [postgres] 70 904 899 0 20 0 84416 0 select Ds - 0:00.00 [postgres] 70 905 899 0 20 0 84416 0 select Ds - 0:00.00 [postgres] 70 906 899 0 20 0 44120 0 select Ds - 0:00.00 [postgres] 26 928 1 0 52 0 48644 0 select Ds - 0:00.00 [exim-4.80.1-2] 0 938 1 0 20 0 103216 0 - Rs - 0:00.02 [cupsd] 1028 944 1 0 155 0 40308 0 select Ds - 0:00.05 [boinc_client] 910 948 1 0 20 0 59412 0 uwait Ds - 0:00.00 [bacula-sd] 0 951 1 0 20 0 57068 0 uwait Ds - 0:00.00 [bacula-fd] 910 954 1 0 20 0 72404 0 - Rs - 0:00.00 [bacula-dir] 0 959 1 0 20 0 56264 0 select Ds - 0:00.00 [sshd] 0 969 1 0 20 0 16460 0 nanslp Ds - 0:00.00 [cron] 0 990 1 0 52 0 18576 0 select Ds - 0:00.00 [inetd] 0 1010 1 0 21 0 47584 0 wait Ds - 0:00.00 [login] 0 1011 1 0 52 0 14364 0 ttyin Ds+ - 0:00.00 [getty] 0 1012 1 0 52 0 14364 0 ttyin Ds+ - 0:00.00 [getty] 0 1013 1 0 52 0 14364 0 ttyin Ds+ - 0:00.00 [getty] 0 1014 1 0 52 0 14364 0 ttyin Ds+ - 0:00.00 [getty] 0 1015 1 0 52 0 14364 0 ttyin Ds+ - 0:00.00 [getty] 0 1016 1 0 52 0 14364 0 ttyin Ds+ - 0:00.00 [getty] 0 1017 1 0 52 0 14364 0 ttyin Ds+ - 0:00.00 [getty] 0 1020 1010 0 20 0 16928 0 wait D - 0:00.00 [sh] 0 1022 1020 0 20 0 47640 0 select D+ - 0:00.00 [ssh] 1028 1024 944 0 155 19 54272 0 - RN - 0:17.51 [wcg_hpf2_roset 1028 1025 944 0 155 19 54272 0 - RN - 0:18.11 [wcg_hpf2_roset 1028 1026 944 0 155 19 129292 0 - RN - 0:03.57 [wcgrid_faah_7. 1028 1027 944 0 155 19 1892 0 - RN - 0:00.03 [wcgrid_cep2_6. 1028 1028 944 0 155 19 74216 0 m DN - 0:17.23 [setiathome-6.1 1028 1029 944 0 155 19 74216 0 - RN - 0:19.34 [setiathome-6.1 1028 1030 944 0 155 19 74216 0 m DN - 0:16.21 [setiathome-6.1 1028 1031 944 0 155 19 129292 0 i DN - 0:00.00 [wcgrid_faah_7. 1028 1032 944 0 155 19 74216 0 m DN - 0:17.25 [setiathome-6.1 1028 1033 1027 0 155 19 1892 0 i DN - 0:00.00 [wcgrid_cep2_6. 1028 1034 1033 0 155 19 1892 0 i DN - 0:00.00 [wcgrid_cep2_6. 1028 1035 1024 0 155 19 54272 0 6 DN - 0:00.00 [wcg_hpf2_roset 1028 1036 1035 0 155 19 54272 0 - RN - 0:00.00 [wcg_hpf2_roset 1028 1037 1025 0 155 19 54272 0 6 DN - 0:00.00 [wcg_hpf2_roset 1028 1038 1037 0 155 19 54272 0 6 DN - 0:00.00 [wcg_hpf2_roset 0 1040 959 0 20 0 81532 0 select Ds - 0:00.00 [sshd] 0 1042 1040 0 27 0 20304 0 pause Ds - 0:00.00 [csh] 0 1044 1042 0 21 0 39968 0 - R - 0:00.00 [zfs] ------------------------------------------------------------------------ vmstat -s 339379 cpu context switches 36990 device interrupts 6821 software interrupts 362408 traps 34071033 system calls 21 kernel threads created 834 fork() calls 182 vfork() calls 7 rfork() calls 0 swap pager pageins 0 swap pager pages paged in 0 swap pager pageouts 0 swap pager pages paged out 1567 vnode pager pageins 11054 vnode pager pages paged in 8 vnode pager pageouts 16 vnode pager pages paged out 0 page daemon wakeups 0 pages examined by the page daemon 11 pages reactivated 33160 copy-on-write faults 209 copy-on-write optimized faults 284971 zero fill pages zeroed 0 zero fill pages prezeroed 107 intransit blocking page faults 405648 total VM faults taken 3427 page faults requiring I/O 0 pages affected by kernel thread creation 32298 pages affected by fork() 6408 pages affected by vfork() 22897 pages affected by rfork() 0 pages cached 407800 pages freed 0 pages freed by daemon 0 pages freed by exiting processes 67444 pages active 20205 pages inactive 8 pages in VM cache 860785 pages wired down 15199607 pages free 4096 bytes per page 247996 total name lookups cache hits (93% pos + 1% neg) system 0% per-directory deletions 0%, falsehits 0%, toolong 0% ------------------------------------------------------------------------ vmstat -m Type InUse MemUse HighUse Requests Size(s) DEVFS3 207 52K - 226 256 filedesc 78 251K - 1058 16,32,64,128,2048,4096 filedesc_to_leader 11 1K - 11 64 sigio 1 1K - 1 64 kdtrace 500 112K - 1633 64,256 kenv 84 11K - 112 16,32,64,128 kqueue 2 3K - 52 256,2048 proc-args 46 4K - 373 16,32,64,128,256 DEVFS1 181 91K - 199 512 hhook 2 1K - 2 128 ithread 119 21K - 119 32,128,256 KTRACE 100 13K - 100 128 DEVFS 40 1K - 41 16,128 linker 351 171K - 477 16,32,64,128,256,512,1024,2048,4096 DEVFSP 1 1K - 11 64 lockf 50 6K - 86 64,128 loginclass 2 1K - 6 64 cache 1 1K - 1 32 devbuf 17037 34759K - 17243 16,32,64,128,256,512,1024,2048,4096 temp 56 35K - 10350 16,32,64,128,256,512,1024,2048,4096 ip6ndp 5 1K - 6 64,128 module 293 37K - 293 128 mtx_pool 2 16K - 2 osd 8 1K - 20 16,32,64,128 pmchooks 1 1K - 1 128 pgrp 36 5K - 51 128 session 34 5K - 45 128 proc 2 256K - 2 subproc 231 350K - 1202 512,4096 cred 108 17K - 8456 64,256 plimit 19 5K - 202 256 uidinfo 6 33K - 9 128 NFS fh 1 1K - 7 16 sysctl 0 0K - 380 16,32,64 sysctloid 6406 317K - 6553 16,32,64,128 sysctltmp 0 0K - 338 16,32,64,128,256,4096 tidhash 1 256K - 1 callout 9 3208K - 9 umtx 1062 133K - 1062 128 p1003.1b 1 1K - 1 16 SWAP 12 19681K - 12 64 bus 900 86K - 4751 16,32,64,128,256,512,1024 bus-sc 135 290K - 1989 16,32,64,128,256,512,1024,2048,4096 devstat 24 49K - 24 32,4096 eventhandler 86 7K - 86 64,128 kobj 169 676K - 681 4096 Per-cpu 1 1K - 1 32 rman 292 33K - 686 16,32,128 sbuf 0 0K - 2785 16,32,64,128,256,512,1024,2048,4096 stack 0 0K - 2 256 taskqueue 105 16K - 135 16,32,64,256,1024 Unitno 19 2K - 227 32,64 ioctlops 1 8K - 13701 16,32,64,128,256,512,1024,2048 select 34 5K - 34 128 iov 0 0K - 1301 16,64,128,256,512 msg 4 30K - 4 2048,4096 sem 4 106K - 4 2048,4096 shm 11 40K - 22 2048 tty 19 19K - 21 1024,2048 mbuf_tag 0 0K - 58 32,128 shmfd 1 8K - 1 soname 7 1K - 697 16,32,128 pcb 38 8341K - 95 16,32,128,1024,2048 vfscache 1 16384K - 1 vfs_hash 1 8192K - 1 vnodes 1 1K - 1 256 mount 397 20K - 1897 16,32,64,128,256,512 vnodemarker 0 0K - 202 512 BPF 3 1K - 3 128 ifnet 4 7K - 4 128,2048 ifaddr 48 15K - 48 32,64,128,256,512,2048,4096 ether_multi 40 3K - 46 16,32,64 clone 6 1K - 6 128 arpcom 2 1K - 2 16 lltable 12 5K - 12 256,512 newnfsmnt 1 1K - 1 1024 pfs_nodes 21 6K - 21 256 pfs_vncache 76 5K - 81 64 GEOM 344 60K - 2226 16,32,64,128,256,512,1024,2048 routetbl 35 5K - 240 32,64,128,256,512 igmp 3 1K - 3 256 in_multi 2 1K - 2 256 ppbusdev 2 1K - 2 256 entropy 1024 64K - 1024 64 sctp_a_it 0 0K - 3 16 sctp_vrf 1 1K - 1 64 sctp_ifa 5 1K - 5 128 sctp_ifn 2 1K - 2 128 sctp_iter 0 0K - 3 256 hostcache 1 28K - 1 syncache 1 100K - 1 in6_mfilter 1 1K - 1 1024 in6_multi 22 3K - 22 32,256 ip6_moptions 2 1K - 2 32,256 CAM DEV 15 30K - 24 2048 UART 6 5K - 6 16,1024 mld 3 1K - 3 128 ata_pci 1 1K - 1 64 CAM CCB 13 26K - 101 2048 rpc 22 6K - 28 32,64,128,256,512,1024 audit_evclass 188 6K - 227 32 vm_pgdata 7 8193K - 7 128 UMAHash 1 1K - 1 512 raid_data 0 0K - 336 32,128,256 acpiintr 1 1K - 1 64 acpica 1830 187K - 57239 16,32,64,128,256,512,1024,2048 CAM path 22 1K - 68 32 CAM periph 16 4K - 41 16,32,64,128,256 acpitask 1 8K - 1 acpisem 21 3K - 21 128 CAM queue 39 2K - 133 16,32,64 memdesc 1 4K - 1 4096 acpidev 36 3K - 36 64 atkbddev 2 1K - 2 64 CAM dev queue 8 1K - 8 128 USB 38 41K - 40 16,32,64,128,256,512,1024,4096 md_nvidia_data 0 0K - 54 512 USBdev 30 6K - 40 64,128,512 md_sii_data 0 0K - 54 512 CAM SIM 8 2K - 8 256 CAM XPT 58 4K - 204 16,32,64,128,1024 kbdmux 7 18K - 7 16,512,1024,2048 apmdev 1 1K - 1 128 madt_table 0 0K - 1 4096 LED 4 1K - 4 16,128 isadev 5 1K - 5 128 io_apic 2 4K - 2 2048 MCA 8 1K - 8 128 pci_link 16 2K - 16 32,64,128 msi 2 1K - 2 128 nexusdev 4 1K - 4 16 acpi_perf 8 1K - 8 64 scsi_cd 0 0K - 11 16 cdev 8 2K - 8 256 solaris 62374 344297K - 749641 16,32,64,128,256,512,1024,2048,4096 linux 36 2K - 36 32,64 cpuctl 1 1K - 9 64,4096 crypto 1 1K - 1 512 xform 0 0K - 206 16,32 cyclic 32 3K - 32 16,64,128 kstat_data 5 1K - 5 64 fbt 1 256K - 1 iprtheap 23 56K - 23 32,64,128,256,2048 fdesc_mount 1 1K - 1 16 netgraph_node 4 1K - 4 128,256 netgraph 2 1K - 2 64 ------------------------------------------------------------------------ vmstat -z ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP UMA Kegs: 208, 0, 254, 1, 254, 0, 0 UMA Zones: 1408, 0, 254, 0, 254, 0, 0 UMA Slabs: 568, 0, 16533, 120, 31690, 0, 0 UMA RCntSlabs: 568, 0, 1217, 1, 1217, 0, 0 UMA Hash: 256, 0, 82, 8, 83, 0, 0 16 Bucket: 152, 0, 391, 9, 391, 0, 0 32 Bucket: 280, 0, 317, 5, 317, 15, 0 64 Bucket: 536, 0, 358, 6, 358, 84, 0 128 Bucket: 1048, 0, 1365, 0, 1365,4185, 0 VM OBJECT: 240, 0, 3359, 257, 14724, 0, 0 RADIX NODE: 144, 16148184, 20410,16127826, 76441, 0, 0 MAP: 232, 0, 8, 24, 8, 0, 0 KMAP ENTRY: 120, 4230725, 255, 1233, 48042, 0, 0 MAP ENTRY: 120, 0, 1670, 407, 34695, 0, 0 fakepg: 104, 0, 0, 0, 0, 0, 0 mt_zone: 4112, 0, 283, 22, 283, 0, 0 16: 16, 0, 6, 1170, 8417, 0, 0 16: 16, 0, 3274, 758, 56874, 0, 0 16: 16, 0, 2, 166, 2, 0, 0 16: 16, 0, 87, 1089, 23969, 0, 0 16: 16, 0, 2339, 1021, 2911, 0, 0 16: 16, 0, 550, 962, 3590, 0, 0 16: 16, 0, 8, 832, 41, 0, 0 16: 16, 0, 62, 946, 270, 0, 0 32: 32, 0, 31, 878, 4358, 0, 0 32: 32, 0, 2152, 1989, 31736, 0, 0 32: 32, 0, 29, 577, 29, 0, 0 32: 32, 0, 122, 787, 4245, 0, 0 32: 32, 0, 2388, 945, 2665, 0, 0 32: 32, 0, 241, 971, 2379, 0, 0 32: 32, 0, 71, 838, 280, 0, 0 32: 32, 0, 200, 406, 329, 0, 0 64: 64, 0, 84, 476, 810, 0, 0 64: 64, 0, 24198, 3130, 222936, 0, 0 64: 64, 0, 1107, 517, 2080, 0, 0 64: 64, 0, 729, 559, 25092, 0, 0 64: 64, 0, 324, 460, 358, 0, 0 64: 64, 0, 8983, 873, 16224, 0, 0 64: 64, 0, 33, 135, 33, 0, 0 64: 64, 0, 65, 495, 237, 0, 0 128: 128, 0, 34, 198, 80, 0, 0 128: 128, 0, 9778, 1300, 81357, 0, 0 128: 128, 0, 85, 234, 100, 0, 0 128: 128, 0, 1220, 404, 2193, 0, 0 128: 128, 0, 2953, 585, 3025, 0, 0 128: 128, 0, 567, 361, 1209, 0, 0 128: 128, 0, 16, 100, 16, 0, 0 128: 128, 0, 80, 239, 427, 0, 0 256: 256, 0, 10, 20, 12, 0, 0 256: 256, 0, 2880, 3525, 105923, 0, 0 256: 256, 0, 490, 80, 783, 0, 0 256: 256, 0, 240, 75, 1045, 0, 0 256: 256, 0, 5, 70, 7, 0, 0 256: 256, 0, 302, 208, 4700, 0, 0 256: 256, 0, 22, 23, 22, 0, 0 256: 256, 0, 129, 216, 1507, 0, 0 512: 512, 0, 169, 41, 210, 0, 0 512: 512, 0, 10146, 347, 128131, 0, 0 512: 512, 0, 4, 59, 76, 0, 0 512: 512, 0, 217, 77, 1180, 0, 0 512: 512, 0, 12, 44, 120, 0, 0 512: 512, 0, 137, 136, 357, 0, 0 512: 512, 0, 0, 0, 0, 0, 0 512: 512, 0, 0, 252, 877, 0, 0 1024: 1024, 0, 2, 66, 84, 0, 0 1024: 1024, 0, 325, 187, 2938, 0, 0 1024: 1024, 0, 4, 4, 4, 0, 0 1024: 1024, 0, 6, 14, 1917, 0, 0 1024: 1024, 0, 19, 9, 19, 0, 0 1024: 1024, 0, 15, 321, 2674, 0, 0 1024: 1024, 0, 1, 7, 1, 0, 0 1024: 1024, 0, 13, 15, 164, 0, 0 2048: 2048, 0, 38, 96, 326, 0, 0 2048: 2048, 0, 411, 237, 4477, 0, 0 2048: 2048, 0, 0, 0, 0, 0, 0 2048: 2048, 0, 81, 133, 1080, 0, 0 2048: 2048, 0, 1, 7, 3, 0, 0 2048: 2048, 0, 23, 95, 170, 0, 0 2048: 2048, 0, 0, 0, 0, 0, 0 2048: 2048, 0, 0, 8, 65, 0, 0 4096: 4096, 0, 67, 100, 1038, 0, 0 4096: 4096, 0, 6394, 4781, 99018, 0, 0 4096: 4096, 0, 169, 100, 681, 0, 0 4096: 4096, 0, 24, 41, 28, 0, 0 4096: 4096, 0, 10, 44, 12, 0, 0 4096: 4096, 0, 27, 63, 7787, 0, 0 4096: 4096, 0, 0, 0, 0, 0, 0 4096: 4096, 0, 0, 0, 0, 0, 0 Files: 80, 0, 192, 393, 17327, 0, 0 TURNSTILE: 136, 0, 532, 108, 532, 0, 0 rl_entry: 40, 0, 75, 681, 75, 0, 0 umtx pi: 96, 0, 0, 0, 0, 0, 0 MAC labels: 40, 0, 0, 0, 0, 0, 0 PROC: 1208, 0, 73, 89, 1044, 0, 0 THREAD: 1168, 0, 425, 106, 587, 0, 0 SLEEPQUEUE: 80, 0, 532, 164, 532, 0, 0 VMSPACE: 392, 0, 46, 134, 1022, 0, 0 cpuset: 72, 0, 274, 326, 425, 0, 0 cyclic_id_cache: 64, 0, 0, 0, 0, 0, 0 audit_record: 1240, 0, 0, 0, 0, 0, 0 mbuf_packet: 256, 26520435, 1023, 1163, 1760, 0, 0 mbuf: 256, 26520435, 7, 1027, 1874, 0, 0 mbuf_cluster: 2048, 4143818, 2176, 230, 2176, 0, 0 mbuf_jumbo_page: 4096, 2071909, 0, 14, 7, 0, 0 mbuf_jumbo_9k: 9216, 1841694, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 1381272, 0, 0, 0, 0, 0 mbuf_ext_refcnt: 4, 0, 0, 0, 0, 0, 0 dtrace_state_cache: 4096, 0, 0, 0, 0, 0, 0 g_bio: 248, 0, 2, 538, 109938, 0, 0 ttyinq: 160, 0, 120, 144, 255, 0, 0 ttyoutq: 256, 0, 64, 101, 136, 0, 0 ata_request: 336, 0, 0, 143, 77, 0, 0 ata_composite: 336, 0, 0, 0, 0, 0, 0 vtnet_tx_hdr: 24, 0, 0, 0, 0, 0, 0 cryptop: 88, 0, 0, 0, 0, 0, 0 cryptodesc: 72, 0, 0, 0, 0, 0, 0 FPU_save_area: 512, 0, 0, 0, 0, 0, 0 taskq_zone: 48, 0, 0, 504, 92, 0, 0 VNODE: 472, 0, 9393, 279, 9518, 0, 0 VNODEPOLL: 112, 0, 0, 0, 0, 0, 0 NAMEI: 1024, 0, 0, 128, 53101, 0, 0 S VFS Cache: 108, 0, 9441, 360, 11402, 0, 0 STS VFS Cache: 148, 0, 0, 0, 0, 0, 0 L VFS Cache: 328, 0, 84, 132, 112, 0, 0 LTS VFS Cache: 368, 0, 0, 0, 0, 0, 0 NCLNODE: 528, 0, 1, 13, 1, 0, 0 pipe: 744, 0, 12, 93, 553, 0, 0 space_seg_cache: 64, 0, 9593, 80007, 278530, 0, 0 zio_cache: 944, 0, 5, 7883, 318981, 0, 0 zio_link_cache: 48, 0, 3, 8709, 303395, 0, 0 zio_buf_512: 512, 0, 0, 0, 0, 0, 0 zio_data_buf_512: 512, 0, 0, 0, 0, 0, 0 zio_buf_1024: 1024, 0, 0, 0, 0, 0, 0 zio_data_buf_1024: 1024, 0, 0, 0, 0, 0, 0 zio_buf_1536: 1536, 0, 0, 0, 0, 0, 0 zio_data_buf_1536: 1536, 0, 0, 0, 0, 0, 0 zio_buf_2048: 2048, 0, 0, 0, 0, 0, 0 zio_data_buf_2048: 2048, 0, 0, 0, 0, 0, 0 zio_buf_2560: 2560, 0, 0, 0, 0, 0, 0 zio_data_buf_2560: 2560, 0, 0, 0, 0, 0, 0 zio_buf_3072: 3072, 0, 0, 0, 0, 0, 0 zio_data_buf_3072: 3072, 0, 0, 0, 0, 0, 0 zio_buf_3584: 3584, 0, 0, 0, 0, 0, 0 zio_data_buf_3584: 3584, 0, 0, 0, 0, 0, 0 zio_buf_4096: 4096, 0, 0, 0, 0, 0, 0 zio_data_buf_4096: 4096, 0, 0, 0, 0, 0, 0 zio_buf_5120: 5120, 0, 0, 0, 0, 0, 0 zio_data_buf_5120: 5120, 0, 0, 0, 0, 0, 0 zio_buf_6144: 6144, 0, 0, 0, 0, 0, 0 zio_data_buf_6144: 6144, 0, 0, 0, 0, 0, 0 zio_buf_7168: 7168, 0, 0, 0, 0, 0, 0 zio_data_buf_7168: 7168, 0, 0, 0, 0, 0, 0 zio_buf_8192: 8192, 0, 0, 0, 0, 0, 0 zio_data_buf_8192: 8192, 0, 0, 0, 0, 0, 0 zio_buf_10240: 10240, 0, 0, 0, 0, 0, 0 zio_data_buf_10240: 10240, 0, 0, 0, 0, 0, 0 zio_buf_12288: 12288, 0, 0, 0, 0, 0, 0 zio_data_buf_12288: 12288, 0, 0, 0, 0, 0, 0 zio_buf_14336: 14336, 0, 0, 0, 0, 0, 0 zio_data_buf_14336: 14336, 0, 0, 0, 0, 0, 0 zio_buf_16384: 16384, 0, 0, 0, 0, 0, 0 zio_data_buf_16384: 16384, 0, 0, 0, 0, 0, 0 zio_buf_20480: 20480, 0, 0, 0, 0, 0, 0 zio_data_buf_20480: 20480, 0, 0, 0, 0, 0, 0 zio_buf_24576: 24576, 0, 0, 0, 0, 0, 0 zio_data_buf_24576: 24576, 0, 0, 0, 0, 0, 0 zio_buf_28672: 28672, 0, 0, 0, 0, 0, 0 zio_data_buf_28672: 28672, 0, 0, 0, 0, 0, 0 zio_buf_32768: 32768, 0, 0, 0, 0, 0, 0 zio_data_buf_32768: 32768, 0, 0, 0, 0, 0, 0 zio_buf_36864: 36864, 0, 0, 0, 0, 0, 0 zio_data_buf_36864: 36864, 0, 0, 0, 0, 0, 0 zio_buf_40960: 40960, 0, 0, 0, 0, 0, 0 zio_data_buf_40960: 40960, 0, 0, 0, 0, 0, 0 zio_buf_45056: 45056, 0, 0, 0, 0, 0, 0 zio_data_buf_45056: 45056, 0, 0, 0, 0, 0, 0 zio_buf_49152: 49152, 0, 0, 0, 0, 0, 0 zio_data_buf_49152: 49152, 0, 0, 0, 0, 0, 0 zio_buf_53248: 53248, 0, 0, 0, 0, 0, 0 zio_data_buf_53248: 53248, 0, 0, 0, 0, 0, 0 zio_buf_57344: 57344, 0, 0, 0, 0, 0, 0 zio_data_buf_57344: 57344, 0, 0, 0, 0, 0, 0 zio_buf_61440: 61440, 0, 0, 0, 0, 0, 0 zio_data_buf_61440: 61440, 0, 0, 0, 0, 0, 0 zio_buf_65536: 65536, 0, 0, 0, 0, 0, 0 zio_data_buf_65536: 65536, 0, 0, 0, 0, 0, 0 zio_buf_69632: 69632, 0, 0, 0, 0, 0, 0 zio_data_buf_69632: 69632, 0, 0, 0, 0, 0, 0 zio_buf_73728: 73728, 0, 0, 0, 0, 0, 0 zio_data_buf_73728: 73728, 0, 0, 0, 0, 0, 0 zio_buf_77824: 77824, 0, 0, 0, 0, 0, 0 zio_data_buf_77824: 77824, 0, 0, 0, 0, 0, 0 zio_buf_81920: 81920, 0, 0, 0, 0, 0, 0 zio_data_buf_81920: 81920, 0, 0, 0, 0, 0, 0 zio_buf_86016: 86016, 0, 0, 0, 0, 0, 0 zio_data_buf_86016: 86016, 0, 0, 0, 0, 0, 0 zio_buf_90112: 90112, 0, 0, 0, 0, 0, 0 zio_data_buf_90112: 90112, 0, 0, 0, 0, 0, 0 zio_buf_94208: 94208, 0, 0, 0, 0, 0, 0 zio_data_buf_94208: 94208, 0, 0, 0, 0, 0, 0 zio_buf_98304: 98304, 0, 0, 0, 0, 0, 0 zio_data_buf_98304: 98304, 0, 0, 0, 0, 0, 0 zio_buf_102400: 102400, 0, 0, 0, 0, 0, 0 zio_data_buf_102400: 102400, 0, 0, 0, 0, 0, 0 zio_buf_106496: 106496, 0, 0, 0, 0, 0, 0 zio_data_buf_106496: 106496, 0, 0, 0, 0, 0, 0 zio_buf_110592: 110592, 0, 0, 0, 0, 0, 0 zio_data_buf_110592: 110592, 0, 0, 0, 0, 0, 0 zio_buf_114688: 114688, 0, 0, 0, 0, 0, 0 zio_data_buf_114688: 114688, 0, 0, 0, 0, 0, 0 zio_buf_118784: 118784, 0, 0, 0, 0, 0, 0 zio_data_buf_118784: 118784, 0, 0, 0, 0, 0, 0 zio_buf_122880: 122880, 0, 0, 0, 0, 0, 0 zio_data_buf_122880: 122880, 0, 0, 0, 0, 0, 0 zio_buf_126976: 126976, 0, 0, 0, 0, 0, 0 zio_data_buf_126976: 126976, 0, 0, 0, 0, 0, 0 zio_buf_131072: 131072, 0, 0, 0, 0, 0, 0 zio_data_buf_131072: 131072, 0, 0, 0, 0, 0, 0 sa_cache: 80, 0, 9194, 211, 9286, 0, 0 dnode_t: 856, 0, 9829, 335, 11053, 0, 0 dmu_buf_impl_t: 224, 0, 17993, 384, 21797, 0, 0 arc_buf_hdr_t: 216, 0, 11850, 264, 12999, 0, 0 arc_buf_t: 72, 0, 10407, 443, 14007, 0, 0 zil_lwb_cache: 192, 0, 4, 176, 56, 0, 0 zfs_znode_cache: 368, 0, 9194, 226, 9286, 0, 0 Mountpoints: 816, 0, 30, 25, 30, 0, 0 ksiginfo: 112, 0, 313, 842, 3205, 0, 0 itimer: 352, 0, 1, 21, 1, 0, 0 KNOTE: 128, 0, 6, 226, 67, 0, 0 socket: 680, 2071914, 57, 93, 480, 0, 0 unpcb: 240, 2071920, 11, 165, 114, 0, 0 ipq: 56, 129528, 0, 0, 0, 0, 0 udp_inpcb: 392, 2071920, 21, 99, 324, 0, 0 udpcb: 16, 2071944, 21, 1155, 324, 0, 0 tcp_inpcb: 392, 2071920, 24, 96, 36, 0, 0 tcpcb: 1016, 2071916, 24, 44, 36, 0, 0 tcptw: 72, 27800, 0, 100, 2, 0, 0 syncache: 152, 15375, 0, 50, 1, 0, 0 hostcache: 136, 15372, 0, 0, 0, 0, 0 tcpreass: 40, 259056, 0, 0, 0, 0, 0 sackhole: 32, 0, 0, 0, 0, 0, 0 sctp_ep: 1408, 2071914, 0, 0, 0, 0, 0 sctp_asoc: 2344, 40000, 0, 0, 0, 0, 0 sctp_laddr: 48, 80064, 0, 288, 4, 0, 0 sctp_raddr: 728, 80000, 0, 0, 0, 0, 0 sctp_chunk: 136, 400008, 0, 0, 0, 0, 0 sctp_readq: 104, 400032, 0, 0, 0, 0, 0 sctp_stream_msg_out: 104, 400032, 0, 0, 0, 0, 0 sctp_asconf: 40, 400008, 0, 0, 0, 0, 0 sctp_asconf_ack: 48, 400032, 0, 0, 0, 0, 0 ripcb: 392, 2071920, 0, 0, 0, 0, 0 rtentry: 200, 0, 17, 135, 17, 0, 0 selfd: 56, 0, 83, 484, 5931, 0, 0 SWAPMETA: 288, 8074092, 0, 0, 0, 0, 0 NetGraph items: 72, 4118, 0, 0, 0, 0, 0 NetGraph data items: 72, 522, 0, 0, 0, 0, 0 ------------------------------------------------------------------------ vmstat -i interrupt total rate irq1: atkbd0 2 0 irq6: fdc0 22 0 irq14: ata0 131 0 irq17: uhci0 ehci0 371 1 irq19: uhci1 ahci0+ 35224 136 cpu0:timer 21581 83 irq256: em0 1240 4 cpu7:timer 18860 72 cpu1:timer 17194 66 cpu2:timer 22448 86 cpu3:timer 17242 66 cpu4:timer 21918 84 cpu6:timer 19999 77 cpu5:timer 20715 79 Total 196947 760 ------------------------------------------------------------------------ pstat -T 192/2071909 files 0M/147455M swap space ------------------------------------------------------------------------ pstat -s Device 512-blocks Used Avail Capacity /dev/gpt/swap0 50331392 0 50331392 0% /dev/gpt/swap1 50331392 0 50331392 0% /dev/gpt/swap2 50331392 0 50331392 0% /dev/gpt/swap3 50331392 0 50331392 0% /dev/gpt/swap4 50331392 0 50331392 0% /dev/gpt/swap5 50331392 0 50331392 0% Total 301988352 0 301988352 0% ------------------------------------------------------------------------ iostat iostat: kvm_read(_tk_nin): invalid address (0x0) iostat: disabling TTY statistics ada0 ada1 ada2 cpu KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 14.49 24 0.33 14.56 23 0.32 14.28 24 0.33 0 10 2 0 88 ------------------------------------------------------------------------ ipcs -a Message Queues: T ID KEY MODE OWNER GROUP CREATOR CGROUP CBYTES QNUM QBYTES LSPID LRPID STIME RTIME CTIME Shared Memory: T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME m 65536 5432001 --rw------- pgsql pgsql pgsql pgsql 4 41263104 899 899 8:44:47 8:45:50 8:44:47 m 65537 28975112 --rw-rw-rw- boinc nobody boinc nobody 2 8192 944 1028 8:45:34 8:45:34 8:45:33 m 65538 28975113 --rw-rw-rw- boinc nobody boinc nobody 2 8192 944 1029 8:45:34 8:45:34 8:45:33 m 65539 28975114 --rw-rw-rw- boinc nobody boinc nobody 2 8192 944 1030 8:45:34 8:45:34 8:45:33 m 65540 28975115 --rw-rw-rw- boinc nobody boinc nobody 2 8192 944 1032 8:45:34 8:45:34 8:45:34 Semaphores: T ID KEY MODE OWNER GROUP CREATOR CGROUP NSEMS OTIME CTIME s 65536 5432001 --rw------- pgsql pgsql pgsql pgsql 17 8:44:47 8:44:47 s 65537 5432002 --rw------- pgsql pgsql pgsql pgsql 17 8:44:47 8:44:47 s 65538 5432003 --rw------- pgsql pgsql pgsql pgsql 17 8:44:47 8:44:47 s 65539 5432004 --rw------- pgsql pgsql pgsql pgsql 17 8:44:47 8:44:47 s 65540 5432005 --rw------- pgsql pgsql pgsql pgsql 17 8:44:47 8:44:47 s 65541 5432006 --rw------- pgsql pgsql pgsql pgsql 17 8:44:47 8:44:47 s 65542 5432007 --rw------- pgsql pgsql pgsql pgsql 17 8:44:47 8:44:47 ------------------------------------------------------------------------ ipcs -T msginfo: msgmax: 16384 (max characters in a message) msgmni: 40 (# of message queues) msgmnb: 2048 (max characters in a message queue) msgtql: 40 (max # of messages in system) msgssz: 8 (size of a message segment) msgseg: 2048 (# of message segments in system) shminfo: shmmax: 536870912 (max shared memory segment size) shmmin: 1 (min shared memory segment size) shmmni: 192 (max number of shared memory identifiers) shmseg: 128 (max shared memory segments per process) shmall: 131072 (max amount of shared memory in pages) seminfo: semmni: 50 (# of semaphore identifiers) semmns: 340 (# of semaphores in system) semmnu: 150 (# of undo structures in system) semmsl: 340 (max # of semaphores per id) semopm: 100 (max # of operations per semop call) semume: 50 (max # of undo entries per process) semusz: 632 (size in bytes of undo structure) semvmx: 32767 (semaphore maximum value) semaem: 16384 (adjust on exit max value) ------------------------------------------------------------------------ nfsstat Client Info: Rpc Counts: Getattr Setattr Lookup Readlink Read Write Create Remove 2 0 0 0 0 0 0 0 Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access 0 0 0 0 0 0 0 0 Mknod Fsstat Fsinfo PathConf Commit 0 1 1 0 0 Rpc Info: TimedOut Invalid X Replies Retries Requests 0 0 0 0 4 Cache Info: Attr Hits Misses Lkup Hits Misses BioR Hits Misses BioW Hits Misses 0 0 0 0 0 0 0 0 BioRLHits Misses BioD Hits Misses DirE Hits Misses Accs Hits Misses 0 0 0 0 0 0 0 0 Server Info: Getattr Setattr Lookup Readlink Read Write Create Remove 0 0 0 0 0 0 0 0 Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access 0 0 0 0 0 0 0 0 Mknod Fsstat Fsinfo PathConf Commit 0 0 0 0 0 Server Ret-Failed 0 Server Faults 0 Server Cache Stats: Inprog Idem Non-idem Misses 0 0 0 0 Server Write Gathering: WriteOps WriteRPC Opsaved 0 0 0 ------------------------------------------------------------------------ netstat -s tcp: 541 packets sent 224 data packets (18558 bytes) 0 data packets (0 bytes) retransmitted 0 data packets unnecessarily retransmitted 0 resends initiated by MTU discovery 308 ack-only packets (123 delayed) 0 URG only packets 0 window probe packets 0 window update packets 9 control packets 541 packets received 208 acks (for 17554 bytes) 3 duplicate acks 0 acks for unsent data 511 packets (432616 bytes) received in-sequence 0 completely duplicate packets (0 bytes) 0 old duplicate packets 0 packets with some dup. data (0 bytes duped) 0 out-of-order packets (0 bytes) 0 packets (0 bytes) of data after window 0 window probes 1 window update packet 0 packets received after close 0 discarded for bad checksums 0 discarded for bad header offset fields 0 discarded because packet too short 0 discarded due to memory problems 6 connection requests 1 connection accept 0 bad connection attempts 0 listen queue overflows 0 ignored RSTs in the windows 7 connections established (including accepts) 12 connections closed (including 0 drops) 0 connections updated cached RTT on close 0 connections updated cached RTT variance on close 0 connections updated cached ssthresh on close 0 embryonic connections dropped 208 segments updated rtt (of 178 attempts) 0 retransmit timeouts 0 connections dropped by rexmit timeout 0 persist timeouts 0 connections dropped by persist timeout 0 Connections (fin_wait_2) dropped because of timeout 0 keepalive timeouts 0 keepalive probes sent 0 connections dropped by keepalive 7 correct ACK header predictions 318 correct data packet header predictions 1 syncache entry added 0 retransmitted 0 dupsyn 0 dropped 1 completed 0 bucket overflow 0 cache overflow 0 reset 0 stale 0 aborted 0 badack 0 unreach 0 zone failures 1 cookie sent 0 cookies received 0 hostcache entries added 0 bucket overflow 0 SACK recovery episodes 0 segment rexmits in SACK recovery episodes 0 byte rexmits in SACK recovery episodes 0 SACK options (SACK blocks) received 0 SACK options (SACK blocks) sent 0 SACK scoreboard overflow 0 packets with ECN CE bit set 0 packets with ECN ECT(0) bit set 0 packets with ECN ECT(1) bit set 0 successful ECN handshakes 0 times ECN reduced the congestion window udp: 120 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 0 with no checksum 0 dropped due to no socket 26 broadcast/multicast datagrams undelivered 0 dropped due to full socket buffers 0 not for hashed pcb 94 delivered 95 datagrams output 0 times multicast source filter matched ip: 612 total packets received 0 bad header checksums 0 with size smaller than minimum 0 with data size < data length 0 with ip length > max ip packet size 0 with header length < data size 0 with data length < header length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 packets reassembled ok 610 packets for this host 2 packets for unknown/unsupported protocol 0 packets forwarded (0 packets fast forwarded) 0 packets not forwardable 0 packets received for unknown multicast group 0 redirects sent 585 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 0 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 tunneling packets that can't find gif 0 datagrams with bad address in header icmp: 0 calls to icmp_error 0 errors not generated in response to an icmp message 0 messages with bad code fields 0 messages less than the minimum length 0 messages with bad checksum 0 messages with bad length 0 multicast echo requests ignored 0 multicast timestamp requests ignored 0 message responses generated 0 invalid return addresses 0 no return routes igmp: 2 messages received 0 messages received with too few bytes 0 messages received with wrong TTL 0 messages received with bad checksum 1 V1/V2 membership query received 0 V3 membership queries received 0 membership queries received with invalid field(s) 1 general query received 0 group queries received 0 group-source queries received 0 group-source queries dropped 1 membership report received 0 membership reports received with invalid field(s) 0 membership reports received for groups to which we belong 0 V3 reports received without Router Alert 0 membership reports sent arp: 4 ARP requests sent 3 ARP replies sent 40 ARP requests received 2 ARP replies received 42 ARP packets received 1 total packet dropped due to no ARP entry 0 ARP entrys timed out 0 Duplicate IPs seen ip6: 51 total packets received 0 with size smaller than minimum 0 with data size < data length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 fragments that exceeded limit 0 packets reassembled ok 51 packets for this host 0 packets forwarded 0 packets not forwardable 0 redirects sent 58 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 9 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 packets that violated scope rules 0 multicast packets which we don't join Input histogram: UDP: 51 Mbuf statistics: 27 one mbuf 24 one ext mbuf 0 two or more ext mbuf 0 packets whose headers are not contiguous 0 tunneling packets that can't find gif 0 packets discarded because of too many headers 0 failures of source address selection Source addresses selection rule applied: icmp6: 0 calls to icmp6_error 0 errors not generated in response to an icmp6 message 0 errors not generated because of rate limitation Output histogram: neighbor solicitation: 1 0 messages with bad code fields 0 messages < minimum length 0 bad checksums 0 messages with bad length Histogram of error messages to be generated: 0 no route 0 administratively prohibited 0 beyond scope 0 address unreachable 0 port unreachable 0 packet too big 0 time exceed transit 0 time exceed reassembly 0 erroneous header field 0 unrecognized next header 0 unrecognized option 0 redirect 0 unknown 0 message responses generated 0 messages with too many ND options 0 messages with bad ND options 0 bad neighbor solicitation messages 0 bad neighbor advertisement messages 0 bad router solicitation messages 0 bad router advertisement messages 0 bad redirect messages 0 path MTU changes rip6: 0 messages received 0 checksum calculations on inbound 0 messages with bad checksum 0 messages dropped due to no socket 0 multicast messages dropped due to no socket 0 messages dropped due to full socket buffers 0 delivered 0 datagrams output ------------------------------------------------------------------------ netstat -m 1030/2190/3220 mbufs in use (current/cache/total) 1013/1393/2406/4143818 mbuf clusters in use (current/cache/total/max) 1023/1163 mbuf+clusters out of packet secondary zone in use (current/cache) 0/14/14/2071909 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/1841694 9k jumbo clusters in use (current/cache/total/max) 0/0/0/1381272 16k jumbo clusters in use (current/cache/total/max) 2283K/3389K/5673K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines ------------------------------------------------------------------------ netstat -id Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll Drop em0 1500 <Link#1> 00:30:48:f2:29:9c 654 0 0 591 0 0 0 em0 1500 192.168.200.0 borg 588 - - 585 - - - em0 1500 fe80::230:48f fe80::230:48ff:fe 0 - - 4 - - - em1* 1500 <Link#2> 00:30:48:f2:29:9d 0 0 0 0 0 0 0 lo0 16384 <Link#3> 51 0 0 51 0 0 0 lo0 16384 localhost ::1 51 - - 51 - - - lo0 16384 fe80::1%lo0 fe80::1 0 - - 0 - - - lo0 16384 your-net localhost 0 - - 0 - - - ------------------------------------------------------------------------ netstat -anr Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 192.168.200.11 UGS 0 560 em0 127.0.0.1 link#3 UH 0 0 lo0 192.168.200.0/24 link#1 U 0 25 em0 192.168.200.4 link#1 UHS 0 0 lo0 Internet6: Destination Gateway Flags Netif Expire ::/96 ::1 UGRS lo0 ::1 link#3 UH lo0 ::ffff:0.0.0.0/96 ::1 UGRS lo0 fe80::/10 ::1 UGRS lo0 fe80::%em0/64 link#1 U em0 fe80::230:48ff:fef2:299c%em0 link#1 UHS lo0 fe80::%lo0/64 link#3 U lo0 fe80::1%lo0 link#3 UHS lo0 ff01::%em0/32 fe80::230:48ff:fef2:299c%em0 U em0 ff01::%lo0/32 ::1 U lo0 ff02::/16 ::1 UGRS lo0 ff02::%em0/32 fe80::230:48ff:fef2:299c%em0 U em0 ff02::%lo0/32 ::1 U lo0 ------------------------------------------------------------------------ netstat -anA Active Internet connections (including servers) Tcpcb Proto Recv-Q Send-Q Local Address Foreign Address (state) fffffe01c28423f8 tcp4 0 1008 192.168.200.4.22 192.147.25.65.4400 ESTABLISHED fffffe01038a5000 tcp4 0 0 192.168.200.4.1391 198.20.8.246.80 CLOSE_WAIT fffffe01c2272be8 tcp4 0 0 127.0.0.1.31416 *.* LISTEN fffffe01d5855be8 tcp4 0 0 192.168.200.4.1551 192.147.25.65.22 ESTABLISHED fffffe01c28427f0 tcp4 0 0 *.9101 *.* LISTEN fffffe010370f7f0 tcp4 0 0 *.22 *.* LISTEN fffffe010370fbe8 tcp6 0 0 *.22 *.* LISTEN fffffe010381dbe8 tcp4 0 0 *.9102 *.* LISTEN fffffe01c2273000 tcp4 0 0 *.9103 *.* LISTEN fffffe0103a3bbe8 tcp4 0 0 *.631 *.* LISTEN fffffe0103a3c000 tcp6 0 0 *.631 *.* LISTEN fffffe010381a000 tcp4 0 0 127.0.0.1.587 *.* LISTEN fffffe010381a3f8 tcp4 0 0 127.0.0.1.25 *.* LISTEN fffffe010381a7f0 tcp4 0 0 192.168.200.4.587 *.* LISTEN fffffe010381abe8 tcp4 0 0 192.168.200.4.25 *.* LISTEN fffffe010370e7f0 tcp4 0 0 127.0.0.1.5432 *.* LISTEN fffffe010370e000 tcp6 0 0 ::1.5432 *.* LISTEN fffffe01038a7000 tcp6 0 0 *.2049 *.* LISTEN fffffe01038a73f8 tcp4 0 0 *.2049 *.* LISTEN fffffe010381b000 tcp4 0 0 *.843 *.* LISTEN fffffe010381b3f8 tcp6 0 0 *.843 *.* LISTEN fffffe010381d3f8 tcp4 0 0 *.111 *.* LISTEN fffffe010381d7f0 tcp6 0 0 *.111 *.* LISTEN fffffe010370e3f8 tcp4 0 0 192.168.200.4.952 192.168.200.23.204 ESTABLISHED fffffe010313bdc8 udp4 0 0 *.69 *.* fffffe0103106930 udp4 0 0 *.631 *.* fffffe0103131ab8 udp6 0 0 ::1.47400 ::1.47400 fffffe0103199930 udp4 0 0 127.0.0.1.123 *.* fffffe010319a188 udp6 0 0 fe80:3::1.123 *.* fffffe010319a620 udp6 0 0 ::1.123 *.* fffffe010319a310 udp6 0 0 fe80:1::230:48ff.1 *.* fffffe010319a000 udp4 0 0 192.168.200.4.123 *.* fffffe0103199dc8 udp6 0 0 *.123 *.* fffffe0103199ab8 udp4 0 0 *.123 *.* fffffe0103199000 udp6 0 0 *.2049 *.* fffffe0103199498 udp4 0 0 *.2049 *.* fffffe0103197930 udp4 0 0 *.843 *.* fffffe0103197620 udp6 0 0 *.843 *.* fffffe0103199c40 udp6 0 0 *.* *.* fffffe010314cab8 udp4 0 0 *.1009 *.* fffffe010314c7a8 udp4 0 0 *.111 *.* fffffe010314c498 udp6 0 0 *.998 *.* fffffe010314c188 udp6 0 0 *.111 *.* fffffe0103197310 udp4 0 0 *.514 *.* fffffe0103197498 udp6 0 0 *.514 *.* Active UNIX domain sockets Address Type Recv-Q Send-Q Inode Conn Refs Nextref Addr fffffe0103a5b870 stream 0 0 fffffe01c22c71d8 0 0 0 /var/run/cups.sock fffffe010349ed20 stream 0 0 fffffe0103e66588 0 0 0 /tmp/.s.PGSQL.5432 fffffe01034a60f0 stream 0 0 fffffe0103457938 0 0 0 /var/run/rpcbind.sock fffffe01034a63c0 stream 0 0 fffffe01034981d8 0 0 0 /var/run/devd.pipe fffffe0103a5b780 dgram 0 0 0 fffffe0103517b40 0 fffffe010349b0f0 fffffe01035174b0 dgram 0 0 0 fffffe0103517c30 0 fffffe010349dc30 fffffe010349dc30 dgram 0 0 0 fffffe0103517c30 0 fffffe0103a5b960 fffffe0103a5b960 dgram 0 0 0 fffffe0103517c30 0 0 fffffe010349b0f0 dgram 0 0 0 fffffe0103517b40 0 0 fffffe0103517b40 dgram 0 0 fffffe01037f1000 0 fffffe0103a5b780 0 /var/run/logpriv fffffe0103517c30 dgram 0 0 fffffe00c0b14ce8 0 fffffe01035174b0 0 /var/run/log ------------------------------------------------------------------------ netstat -aL Current listen queue sizes (qlen/incqlen/maxqlen) Proto Listen Local Address tcp4 0/0/128 localhost.31416 tcp4 0/0/50 *.bacula-dir tcp4 0/0/128 *.ssh tcp6 0/0/128 *.ssh tcp4 0/0/50 *.bacula-fd tcp4 0/0/50 *.bacula-sd tcp4 0/0/128 *.ipp tcp6 0/0/128 *.ipp tcp4 0/0/20 localhost.submission tcp4 0/0/20 localhost.smtp tcp4 0/0/20 borg.submission tcp4 0/0/20 borg.smtp tcp4 0/0/128 localhost.postgresql tcp6 0/0/128 localhost.postgresql tcp6 0/0/5 *.nfsd tcp4 0/0/5 *.nfsd tcp4 0/0/128 *.843 tcp6 0/0/128 *.843 tcp4 0/0/128 *.sunrpc tcp6 0/0/128 *.sunrpc unix 0/0/128 /var/run/cups.sock unix 0/0/128 /tmp/.s.PGSQL.5432 unix 0/0/128 /var/run/rpcbind.sock unix 0/0/4 /var/run/devd.pipe ------------------------------------------------------------------------ fstat fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read file 8 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read file 8 at 0x78 fstat: can't read file 10 at 0xffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read file 8 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read file 8 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read file 8 at 0x78 fstat: can't read file 10 at 0xffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read file 8 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read file 8 at 0x78 fstat: can't read file 10 at 0xffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read file 8 at 0x78 fstat: can't read file 10 at 0xffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read file 8 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read file 8 at 0x78 fstat: can't read file 10 at 0xffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read file 8 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read file 8 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read file 8 at 0x78 fstat: can't read file 10 at 0xffff fstat: can't read file 13 at 0xfffffffffffffff fstat: can't read file 14 at 0x78 fstat: can't read file 16 at 0xffff fstat: can't read file 19 at 0xfffffffffffffff fstat: can't read file 20 at 0x78 fstat: can't read file 22 at 0xffff fstat: can't read file 23 at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read file 8 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read file 8 at 0x78 fstat: can't read file 10 at 0xffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read file 7 at 0xfffffffffffffff fstat: can't read file 8 at 0x78 fstat: can't read file 10 at 0xffff fstat: can't read file 13 at 0xfffffffffffffff fstat: can't read file 14 at 0x78 fstat: can't read file 16 at 0xffff fstat: can't read file 19 at 0xfffffffffffffff fstat: can't read file 20 at 0x78 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0xfffffffffffffff fstat: can't read file 2 at 0x78 fstat: can't read file 4 at 0xffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 USER CMD PID FD MOUNT INUM MODE SZ|DV R/W root zfs 1044 root - - error - root zfs 1044 wd - - error - root zfs 1044 text - - error - root zfs 1044 0* pipe fffffe00c0d6f000 <-> fffffe00c0d6f160 65224 rw root zfs 1044 6* pipe fffffe00c0d97730 <-> fffffe00c0d975d0 0 rw root csh 1042 root - - error - root csh 1042 wd - - error - root csh 1042 text - - error - root sshd 1040 root - - error - root sshd 1040 wd - - error - root sshd 1040 text - - error - root sshd 1040 0 /dev 28 crw-rw-rw- null rw root sshd 1040 6 /dev 28 crw-rw-rw- null rw boinc wcg_hpf2_rosetta_6 1038 root - - error - boinc wcg_hpf2_rosetta_6 1038 wd - - error - boinc wcg_hpf2_rosetta_6 1038 text - - error - boinc wcg_hpf2_rosetta_6 1038 0 /dev 28 crw-rw-rw- null rw boinc wcg_hpf2_rosetta_6 1038 6 - - error - boinc wcg_hpf2_rosetta_6 1037 root - - error - boinc wcg_hpf2_rosetta_6 1037 wd - - error - boinc wcg_hpf2_rosetta_6 1037 text - - error - boinc wcg_hpf2_rosetta_6 1037 0 /dev 28 crw-rw-rw- null rw boinc wcg_hpf2_rosetta_6 1037 6 - - error - boinc wcg_hpf2_rosetta_6 1036 root - - error - boinc wcg_hpf2_rosetta_6 1036 wd - - error - boinc wcg_hpf2_rosetta_6 1036 text - - error - boinc wcg_hpf2_rosetta_6 1036 0 /dev 28 crw-rw-rw- null rw boinc wcg_hpf2_rosetta_6 1036 6 - - error - boinc wcg_hpf2_rosetta_6 1035 root - - error - boinc wcg_hpf2_rosetta_6 1035 wd - - error - boinc wcg_hpf2_rosetta_6 1035 text - - error - boinc wcg_hpf2_rosetta_6 1035 0 /dev 28 crw-rw-rw- null rw boinc wcg_hpf2_rosetta_6 1035 6 - - error - boinc wcgrid_cep2_6.40_i 1034 root - - error - boinc wcgrid_cep2_6.40_i 1034 wd - - error - boinc wcgrid_cep2_6.40_i 1034 text - - error - boinc wcgrid_cep2_6.40_i 1034 0 /dev 28 crw-rw-rw- null rw boinc wcgrid_cep2_6.40_i 1034 6 - - error - boinc wcgrid_cep2_6.40_i 1033 root - - error - boinc wcgrid_cep2_6.40_i 1033 wd - - error - boinc wcgrid_cep2_6.40_i 1033 text - - error - boinc wcgrid_cep2_6.40_i 1033 0 /dev 28 crw-rw-rw- null rw boinc wcgrid_cep2_6.40_i 1033 6 - - error - boinc setiathome-6.12.am 1032 root - - error - boinc setiathome-6.12.am 1032 wd - - error - boinc setiathome-6.12.am 1032 text - - error - boinc setiathome-6.12.am 1032 0 /dev 28 crw-rw-rw- null rw boinc setiathome-6.12.am 1032 6 /dev 28 crw-rw-rw- null rw boinc wcgrid_faah_7.15_i 1031 root - - error - boinc wcgrid_faah_7.15_i 1031 wd - - error - boinc wcgrid_faah_7.15_i 1031 text - - error - boinc wcgrid_faah_7.15_i 1031 0 /dev 28 crw-rw-rw- null rw boinc wcgrid_faah_7.15_i 1031 6 /dev 28 crw-rw-rw- null rw boinc setiathome-6.12.am 1030 root - - error - boinc setiathome-6.12.am 1030 wd - - error - boinc setiathome-6.12.am 1030 text - - error - boinc setiathome-6.12.am 1030 0 /dev 28 crw-rw-rw- null rw boinc setiathome-6.12.am 1030 6 /dev 28 crw-rw-rw- null rw boinc setiathome-6.12.am 1029 root - - error - boinc setiathome-6.12.am 1029 wd - - error - boinc setiathome-6.12.am 1029 text - - error - boinc setiathome-6.12.am 1029 0 /dev 28 crw-rw-rw- null rw boinc setiathome-6.12.am 1029 6 /dev 28 crw-rw-rw- null rw boinc setiathome-6.12.am 1028 root - - error - boinc setiathome-6.12.am 1028 wd - - error - boinc setiathome-6.12.am 1028 text - - error - boinc setiathome-6.12.am 1028 0 /dev 28 crw-rw-rw- null rw boinc setiathome-6.12.am 1028 6 /dev 28 crw-rw-rw- null rw boinc wcgrid_cep2_6.40_i 1027 root - - error - boinc wcgrid_cep2_6.40_i 1027 wd - - error - boinc wcgrid_cep2_6.40_i 1027 text - - error - boinc wcgrid_cep2_6.40_i 1027 0 /dev 28 crw-rw-rw- null rw boinc wcgrid_cep2_6.40_i 1027 6 - - error - boinc wcgrid_faah_7.15_i 1026 root - - error - boinc wcgrid_faah_7.15_i 1026 wd - - error - boinc wcgrid_faah_7.15_i 1026 text - - error - boinc wcgrid_faah_7.15_i 1026 0 /dev 28 crw-rw-rw- null rw boinc wcgrid_faah_7.15_i 1026 6 /dev 28 crw-rw-rw- null rw boinc wcg_hpf2_rosetta_6 1025 root - - error - boinc wcg_hpf2_rosetta_6 1025 wd - - error - boinc wcg_hpf2_rosetta_6 1025 text - - error - boinc wcg_hpf2_rosetta_6 1025 0 /dev 28 crw-rw-rw- null rw boinc wcg_hpf2_rosetta_6 1025 6 - - error - boinc wcg_hpf2_rosetta_6 1024 root - - error - boinc wcg_hpf2_rosetta_6 1024 wd - - error - boinc wcg_hpf2_rosetta_6 1024 text - - error - boinc wcg_hpf2_rosetta_6 1024 0 /dev 28 crw-rw-rw- null rw boinc wcg_hpf2_rosetta_6 1024 6 - - error - root ssh 1022 root - - error - root ssh 1022 wd - - error - root ssh 1022 text - - error - root ssh 1022 ctty /dev 71 crw------- ttyv0 rw root ssh 1022 0 /dev 71 crw------- ttyv0 rw root ssh 1022 6 /dev 71 crw------- ttyv0 rw root sh 1020 root - - error - root sh 1020 wd - - error - root sh 1020 text - - error - root sh 1020 ctty /dev 71 crw------- ttyv0 rw root sh 1020 0 /dev 71 crw------- ttyv0 rw root sh 1020 6 /dev 71 crw------- ttyv0 rw root getty 1017 root - - error - root getty 1017 wd - - error - root getty 1017 text - - error - root getty 1017 ctty /dev 78 crw------- ttyv7 rw root getty 1017 0 /dev 78 crw------- ttyv7 rw root getty 1016 root - - error - root getty 1016 wd - - error - root getty 1016 text - - error - root getty 1016 ctty /dev 77 crw------- ttyv6 rw root getty 1016 0 /dev 77 crw------- ttyv6 rw root getty 1015 root - - error - root getty 1015 wd - - error - root getty 1015 text - - error - root getty 1015 ctty /dev 76 crw------- ttyv5 rw root getty 1015 0 /dev 76 crw------- ttyv5 rw root getty 1014 root - - error - root getty 1014 wd - - error - root getty 1014 text - - error - root getty 1014 ctty /dev 75 crw------- ttyv4 rw root getty 1014 0 /dev 75 crw------- ttyv4 rw root getty 1013 root - - error - root getty 1013 wd - - error - root getty 1013 text - - error - root getty 1013 ctty /dev 74 crw------- ttyv3 rw root getty 1013 0 /dev 74 crw------- ttyv3 rw root getty 1012 root - - error - root getty 1012 wd - - error - root getty 1012 text - - error - root getty 1012 ctty /dev 73 crw------- ttyv2 rw root getty 1012 0 /dev 73 crw------- ttyv2 rw root getty 1011 root - - error - root getty 1011 wd - - error - root getty 1011 text - - error - root getty 1011 ctty /dev 72 crw------- ttyv1 rw root getty 1011 0 /dev 72 crw------- ttyv1 rw root login 1010 root - - error - root login 1010 wd - - error - root login 1010 text - - error - root login 1010 ctty /dev 71 crw------- ttyv0 rw root login 1010 0 /dev 71 crw------- ttyv0 rw root inetd 990 root - - error - root inetd 990 wd - - error - root inetd 990 text - - error - root inetd 990 0 /dev 28 crw-rw-rw- null rw root inetd 990 6 /dev 28 crw-rw-rw- null rw root cron 969 root - - error - root cron 969 wd - - error - root cron 969 text - - error - root cron 969 0 /dev 28 crw-rw-rw- null rw root sshd 959 root - - error - root sshd 959 wd - - error - root sshd 959 text - - error - root sshd 959 0 /dev 28 crw-rw-rw- null rw bacula bacula-dir 954 root - - error - bacula bacula-dir 954 wd - - error - bacula bacula-dir 954 text - - error - bacula bacula-dir 954 0 /dev 28 crw-rw-rw- null r root bacula-fd 951 root - - error - root bacula-fd 951 wd - - error - root bacula-fd 951 text - - error - root bacula-fd 951 0 /dev 28 crw-rw-rw- null r bacula bacula-sd 948 root - - error - bacula bacula-sd 948 wd - - error - bacula bacula-sd 948 text - - error - bacula bacula-sd 948 0 /dev 28 crw-rw-rw- null r boinc boinc_client 944 root - - error - boinc boinc_client 944 wd - - error - boinc boinc_client 944 text - - error - boinc boinc_client 944 0 /dev 28 crw-rw-rw- null rw boinc boinc_client 944 6 - - error - root cupsd 938 root - - error - root cupsd 938 wd - - error - root cupsd 938 text - - error - root cupsd 938 0 /dev 28 crw-rw-rw- null r root cupsd 938 6 /dev 28 crw-rw-rw- null w root cupsd 938 12 /dev 28 crw-rw-rw- null w mailnull exim-4.80.1-2 928 root - - error - mailnull exim-4.80.1-2 928 wd - - error - mailnull exim-4.80.1-2 928 text - - error - mailnull exim-4.80.1-2 928 0 /dev 28 crw-rw-rw- null rw mailnull exim-4.80.1-2 928 6 /dev 28 crw-rw-rw- null rw pgsql postgres 906 root - - error - pgsql postgres 906 wd - - error - pgsql postgres 906 text - - error - pgsql postgres 906 0 /dev 28 crw-rw-rw- null r pgsql postgres 906 6 - - error - pgsql postgres 905 root - - error - pgsql postgres 905 wd - - error - pgsql postgres 905 text - - error - pgsql postgres 905 0 /dev 28 crw-rw-rw- null r pgsql postgres 905 6 - - error - pgsql postgres 904 root - - error - pgsql postgres 904 wd - - error - pgsql postgres 904 text - - error - pgsql postgres 904 0 /dev 28 crw-rw-rw- null r pgsql postgres 904 6 - - error - pgsql postgres 903 root - - error - pgsql postgres 903 wd - - error - pgsql postgres 903 text - - error - pgsql postgres 903 0 /dev 28 crw-rw-rw- null r pgsql postgres 903 6 - - error - pgsql postgres 899 root - - error - pgsql postgres 899 wd - - error - pgsql postgres 899 text - - error - pgsql postgres 899 0 /dev 28 crw-rw-rw- null r pgsql postgres 899 6 - - error - root smartd 890 root - - error - root smartd 890 wd - - error - root smartd 890 text - - error - root smartd 890 0 /dev 28 crw-rw-rw- null rw root perl 886 root - - error - root perl 886 wd - - error - root perl 886 text - - error - root perl 886 0 - - error - root ng_queue 874 root - - error - root ng_queue 874 wd - - error - root ntpd 853 root - - error - root ntpd 853 wd - - error - root ntpd 853 text - - error - root ntpd 853 0 /dev 28 crw-rw-rw- null rw root ntpd 853 6 /dev 28 crw-rw-rw- null rw root ntpd 853 12 /dev 28 crw-rw-rw- null rw root ntpd 853 18* local dgram fffffe010349b0f0 <-> fffffe0103517b40 root nfsd 817 root - - error - root nfsd 817 wd - - error - root nfsd 817 text - - error - root nfsd 817 0 /dev 28 crw-rw-rw- null rw root nfsd 815 root - - error - root nfsd 815 wd - - error - root nfsd 815 text - - error - root nfsd 815 0 /dev 28 crw-rw-rw- null rw root nfsd 815 6 /dev 28 crw-rw-rw- null rw root mountd 809 root - - error - root mountd 809 wd - - error - root mountd 809 text - - error - root mountd 809 0 /dev 28 crw-rw-rw- null rw root mountd 809 6 /dev 28 crw-rw-rw- null rw root rpcbind 774 root - - error - root rpcbind 774 wd - - error - root rpcbind 774 text - - error - root rpcbind 774 0 /dev 28 crw-rw-rw- null rw root rpcbind 774 6 /dev 28 crw-rw-rw- null rw root watchdogd 760 root - - error - root watchdogd 760 wd - - error - root watchdogd 760 text - - error - root watchdogd 760 0 /dev 28 crw-rw-rw- null rw root syslogd 757 root - - error - root syslogd 757 wd - - error - root syslogd 757 text - - error - root syslogd 757 0 /dev 28 crw-rw-rw- null rw root syslogd 757 6 /dev 28 crw-rw-rw- null rw root syslogd 757 12 /dev 28 crw-rw-rw- null rw root syslogd 757 18 - - error - root devd 632 root - - error - root devd 632 wd - - error - root devd 632 text - - error - root devd 632 0 /dev 28 crw-rw-rw- null rw root init 1 root - - error - root init 1 wd - - error - root init 1 text - - error - root kernel 0 root - - error - root kernel 0 wd - - error - ------------------------------------------------------------------------ dmesg Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #129 r248695: Mon Mar 25 05:03:32 CDT 2013 root@borg.lerctr.org:/usr/obj/usr/src/sys/BORG-DTRACE amd64 FreeBSD clang version 3.2 (tags/RELEASE_32/final 170710) 20121221 CPU: Intel(R) Xeon(R) CPU E5410 @ 2.33GHz (2327.54-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x10676 Family = 0x6 Model = 0x17 Stepping = 6 Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Features2=0xce3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1> AMD Features=0x20100800<SYSCALL,NX,LM> AMD Features2=0x1<LAHF> TSC: P-state invariant, performance statistics real memory = 68719476736 (65536 MB) avail memory = 63193878528 (60266 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: <PTLTD APIC > FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 ioapic0 <Version 2.0> irqs 0-23 on motherboard ioapic1 <Version 2.0> irqs 24-47 on motherboard kbd1 at kbdmux0 netmap: loaded module cryptosoft0: <software crypto> on motherboard acpi0: <PTLTD RSDT> on motherboard acpi0: Power Button (fixed) unknown: I/O range not supported cpu0: <ACPI CPU> on acpi0 cpu1: <ACPI CPU> on acpi0 cpu2: <ACPI CPU> on acpi0 cpu3: <ACPI CPU> on acpi0 cpu4: <ACPI CPU> on acpi0 cpu5: <ACPI CPU> on acpi0 cpu6: <ACPI CPU> on acpi0 cpu7: <ACPI CPU> on acpi0 hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff irq 0,8 on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 atrtc0: <AT realtime clock> port 0x70-0x71 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: <AT timer> port 0x40-0x43,0x50-0x53 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 pci0: <ACPI PCI bus> on pcib0 pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0 pci1: <ACPI PCI bus> on pcib1 pcib2: <ACPI PCI-PCI bridge> irq 16 at device 0.0 on pci1 pci2: <ACPI PCI bus> on pcib2 pcib3: <ACPI PCI-PCI bridge> irq 16 at device 0.0 on pci2 pci3: <ACPI PCI bus> on pcib3 pcib4: <ACPI PCI-PCI bridge> at device 0.0 on pci3 pci4: <ACPI PCI bus> on pcib4 pcib5: <ACPI PCI-PCI bridge> at device 0.2 on pci3 pci5: <ACPI PCI bus> on pcib5 pcib6: <ACPI PCI-PCI bridge> irq 18 at device 2.0 on pci2 pci6: <ACPI PCI bus> on pcib6 em0: <Intel(R) PRO/1000 Network Connection 7.3.7> port 0x2000-0x201f mem 0xd8020000-0xd803ffff,0xd8000000-0xd801ffff irq 18 at device 0.0 on pci6 em0: Using an MSI interrupt em0: Ethernet address: 00:30:48:f2:29:9c 001.000008 netmap_attach [1680] success for em0 em1: <Intel(R) PRO/1000 Network Connection 7.3.7> port 0x2020-0x203f mem 0xd8060000-0xd807ffff,0xd8040000-0xd805ffff irq 19 at device 0.1 on pci6 em1: Using an MSI interrupt em1: Ethernet address: 00:30:48:f2:29:9d 001.000009 netmap_attach [1680] success for em1 pcib7: <ACPI PCI-PCI bridge> at device 0.3 on pci1 pci7: <ACPI PCI bus> on pcib7 pcib8: <ACPI PCI-PCI bridge> at device 4.0 on pci0 pci8: <ACPI PCI bus> on pcib8 pcib9: <ACPI PCI-PCI bridge> at device 6.0 on pci0 pci9: <ACPI PCI bus> on pcib9 pci0: <base peripheral> at device 8.0 (no driver attached) pcib10: <ACPI PCI-PCI bridge> irq 17 at device 28.0 on pci0 pci10: <ACPI PCI bus> on pcib10 uhci0: <Intel 631XESB/632XESB/3100 USB controller USB-1> port 0x1800-0x181f irq 17 at device 29.0 on pci0 usbus0 on uhci0 uhci1: <Intel 631XESB/632XESB/3100 USB controller USB-2> port 0x1820-0x183f irq 19 at device 29.1 on pci0 usbus1 on uhci1 uhci2: <Intel 631XESB/632XESB/3100 USB controller USB-3> port 0x1840-0x185f irq 18 at device 29.2 on pci0 usbus2 on uhci2 ehci0: <Intel 63XXESB USB 2.0 controller> mem 0xd8500400-0xd85007ff irq 17 at device 29.7 on pci0 usbus3: EHCI version 1.0 usbus3 on ehci0 pcib11: <ACPI PCI-PCI bridge> at device 30.0 on pci0 pci11: <ACPI PCI bus> on pcib11 vgapci0: <VGA-compatible display> port 0x3000-0x30ff mem 0xd0000000-0xd7ffffff,0xd8200000-0xd820ffff irq 18 at device 1.0 on pci11 isab0: <PCI-ISA bridge> at device 31.0 on pci0 isa0: <ISA bus> on isab0 atapci0: <Intel 63XXESB2 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1860-0x186f at device 31.1 on pci0 ata0: <ATA channel> at channel 0 on atapci0 ahci0: <Intel ESB2 AHCI SATA controller> port 0x18a0-0x18a7,0x1874-0x1877,0x1878-0x187f,0x1870-0x1873,0x1880-0x189f mem 0xd8500800-0xd8500bff irq 19 at device 31.2 on pci0 ahci0: AHCI v1.10 with 6 3Gbps ports, Port Multiplier supported ahcich0: <AHCI channel> at channel 0 on ahci0 ahcich1: <AHCI channel> at channel 1 on ahci0 ahcich2: <AHCI channel> at channel 2 on ahci0 ahcich3: <AHCI channel> at channel 3 on ahci0 ahcich4: <AHCI channel> at channel 4 on ahci0 ahcich5: <AHCI channel> at channel 5 on ahci0 ichsmb0: <Intel 631xESB/6321ESB (ESB2) SMBus controller> port 0x1100-0x111f irq 19 at device 31.3 on pci0 smbus0: <System Management Bus> on ichsmb0 acpi_button0: <Power Button> on acpi0 atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fd0: <1440-KB 3.5" drive> on fdc0 drive 0 ppc0: <Parallel port> port 0x378-0x37f,0x778-0x77f irq 7 drq 3 on acpi0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/9 bytes threshold ppbus0: <Parallel port bus> on ppc0 lpt0: <Printer> on ppbus0 lpt0: Interrupt-driven port ppi0: <Parallel I/O> on ppbus0 ichwd0 on isa0 orm0: <ISA Option ROM> at iomem 0xc0000-0xcafff on isa0 sc0: <System console> at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 coretemp0: <CPU On-Die Thermal Sensors> on cpu0 est0: <Enhanced SpeedStep Frequency Control> on cpu0 p4tcc0: <CPU Frequency Thermal Control> on cpu0 coretemp1: <CPU On-Die Thermal Sensors> on cpu1 est1: <Enhanced SpeedStep Frequency Control> on cpu1 p4tcc1: <CPU Frequency Thermal Control> on cpu1 coretemp2: <CPU On-Die Thermal Sensors> on cpu2 est2: <Enhanced SpeedStep Frequency Control> on cpu2 p4tcc2: <CPU Frequency Thermal Control> on cpu2 coretemp3: <CPU On-Die Thermal Sensors> on cpu3 est3: <Enhanced SpeedStep Frequency Control> on cpu3 p4tcc3: <CPU Frequency Thermal Control> on cpu3 coretemp4: <CPU On-Die Thermal Sensors> on cpu4 est4: <Enhanced SpeedStep Frequency Control> on cpu4 p4tcc4: <CPU Frequency Thermal Control> on cpu4 coretemp5: <CPU On-Die Thermal Sensors> on cpu5 est5: <Enhanced SpeedStep Frequency Control> on cpu5 p4tcc5: <CPU Frequency Thermal Control> on cpu5 coretemp6: <CPU On-Die Thermal Sensors> on cpu6 est6: <Enhanced SpeedStep Frequency Control> on cpu6 p4tcc6: <CPU Frequency Thermal Control> on cpu6 coretemp7: <CPU On-Die Thermal Sensors> on cpu7 est7: <Enhanced SpeedStep Frequency Control> on cpu7 p4tcc7: <CPU Frequency Thermal Control> on cpu7 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1.000 msec vboxdrv: fAsync=0 offMin=0x40c offMax=0x58d usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 480Mbps High Speed USB v2.0 ugen1.1: <Intel> at usbus1 uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1 ugen0.1: <Intel> at usbus0 uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0 ugen3.1: <Intel> at usbus3 uhub2: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3 ugen2.1: <Intel> at usbus2 uhub3: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2 ata0: DMA limited to UDMA33, controller found non-ATA66 cable ada0 at ahcich0 bus 0 scbus1 target 0 lun 0 ada0: <ST2000DL003-9VT166 CC3D> ATA-8 SATA 3.x device ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad4 ada1 at ahcich1 bus 0 scbus2 target 0 lun 0 ada1: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada1: Previously was known as ad6 ada2 at ahcich2 bus 0 scbus3 target 0 lun 0 ada2: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada2: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada2: Previously was known as ad8 ada3 at ahcich3 bus 0 scbus4 target 0 lun 0 ada3: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada3: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada3: Previously was known as ad10 ada4 at ahcich4 bus 0 scbus5 target 0 lun 0 ada4: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada4: Previously was known as ad12 ada5 at ahcich5 bus 0 scbus6 target 0 lun 0 ada5: <ST2000DL003-9VT166 CC3D> ATA-8 SATA 3.x device ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada5: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada5: Previously was known as ad14 cd0 at ata0 bus 0 scbus0 target 0 lun 0 cd0: <Memorex DVD+-RAM 510L v1 MWS7> Removable CD-ROM SCSI-0 device cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present SMP: AP CPU #7 Launched! SMP: AP CPU #1 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #5 Launched! uhub1: 2 ports with 2 removable, self powered uhub0: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered Root mount waiting for: usbus3 uhub2: 6 ports with 6 removable, self powered Root mount waiting for: usbus3 ugen3.2: <Peppercon AG> at usbus3 ukbd0: <Peppercon AG Multidevice, class 0/0, rev 2.00/0.01, addr 2> on usbus3 kbd2 at ukbd0 Trying to mount root from zfs:zroot/ROOT/default []... ffclock reset: HPET (14318180 Hz), time = 1364219053.500000000 Setting hostuuid: 53d19f64-d663-a017-8922-0030488e9ff3. Setting hostid: 0xf53a926e. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: Mounting local file systems:. Writing entropy file:. Setting hostname: borg.lerctr.org. Starting Network: lo0 em0 em1. lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO> ether 00:30:48:f2:29:9c inet 192.168.200.4 netmask 0xffffff00 broadcast 192.168.200.255 inet6 fe80::230:48ff:fef2:299c%em0 prefixlen 64 scopeid 0x1 nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> media: Ethernet autoselect status: no carrier em1: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO> ether 00:30:48:f2:29:9d nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> media: Ethernet autoselect status: no carrier Starting devd. Starting Network: em1. em1: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO> ether 00:30:48:f2:29:9d nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> media: Ethernet autoselect status: no carrier uhid0: <Peppercon AG Multidevice, class 0/0, rev 2.00/0.01, addr 2> on usbus3 add net default: gateway 192.168.200.11 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 Mounting NFS file systems:. ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/kde4/lib /usr/local/lib/compat /usr/local/lib/dbmail /usr/local/lib/event2 /usr/local/lib/pth /usr/local/lib/qt4 /usr/local/lib/virtualbox 32-bit compatibility ldconfig path: /usr/lib32 /usr/local/lib32/compat Creating and/or trimming log files. Starting syslogd. Starting watchdogd. No core dumps found. Additional ABI support: linux. Starting rpcbind. NFS access cache time=60 Clearing /tmp (X related). Starting mountd. NFSv4 is disabled Starting nfsd. Updating motd:. Starting ntpd. Starting sshblock. Starting smartd. Updating cpucodes... /usr/local/share/cpucontrol/m401067660F.fw: updating cpu /dev/cpuctl0 from rev 0x60c to rev 0x60f... done. /usr/local/share/cpucontrol/m401067660F.fw: updating cpu /dev/cpuctl1 from rev 0x60c to rev 0x60f... done. /usr/local/share/cpucontrol/m401067660F.fw: updating cpu /dev/cpuctl2 from rev 0x60c to rev 0x60f... done. /usr/local/share/cpucontrol/m401067660F.fw: updating cpu /dev/cpuctl3 from rev 0x60c to rev 0x60f... done. /usr/local/share/cpucontrol/m401067660F.fw: updating cpu /dev/cpuctl4 from rev 0x60c to rev 0x60f... done. /usr/local/share/cpucontrol/m401067660F.fw: updating cpu /dev/cpuctl5 from rev 0x60c to rev 0x60f... done. /usr/local/share/cpucontrol/m401067660F.fw: updating cpu /dev/cpuctl6 from rev 0x60c to rev 0x60f... done. /usr/local/share/cpucontrol/m401067660F.fw: updating cpu /dev/cpuctl7 from rev 0x60c to rev 0x60f... done. Done. Starting exim. Starting cupsd. Starting boinc_client. Starting bacula_sd. Starting bacula_fd. Starting bacula_dir. Performing sanity check on sshd configuration. Starting sshd. Configuring syscons: blanktime. Starting cron. Starting inetd. Starting background file system checks in 60 seconds. Mon Mar 25 08:44:56 CDT 2013 Mar 25 08:45:03 borg login: ROOT LOGIN (root) ON ttyv0 Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x378 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80531426 stack pointer = 0x28:0xffffff91579193d0 frame pointer = 0x28:0xffffff9157919470 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 1044 (zfs) trap number = 12 panic: page fault cpuid = 0 Uptime: 2m10s Dumping 4913 out of 64747 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% ------------------------------------------------------------------------ kernel config options CONFIG_AUTOGENERATED ident BORG-DTRACE machine amd64 cpu HAMMER makeoptions WITH_CTF=1 makeoptions DEBUG=-g options FFCLOCK options USB_DEBUG options RDRAND_RNG options PADLOCK_RNG options ATH_ENABLE_11N options AH_AR5416_INTERRUPT_MITIGATION options AH_SUPPORT_AR5416 options IEEE80211_SUPPORT_MESH options IEEE80211_AMPDU_AGE options IEEE80211_DEBUG options SC_PIXEL_MODE options VESA options CTL_DISABLE options AHD_REG_PRETTY_PRINT options AHC_REG_PRETTY_PRINT options ATA_STATIC_ID options ATA_CAM options SMP options MALLOC_DEBUG_MAXZONES=8 options INCLUDE_CONFIG_FILE options DDB_CTF options KDTRACE_HOOKS options KDTRACE_FRAME options MAC options CAPABILITIES options CAPABILITY_MODE options AUDIT options HWPMC_HOOKS options KBD_INSTALL_CDEV options PRINTF_BUFR_SIZE=128 options _KPOSIX_PRIORITY_SCHEDULING options SYSVSEM options SYSVMSG options SYSVSHM options STACK options KTRACE options SCSI_DELAY=5000 options COMPAT_FREEBSD7 options COMPAT_FREEBSD6 options COMPAT_FREEBSD5 options COMPAT_FREEBSD4 options COMPAT_FREEBSD32 options GEOM_LABEL options GEOM_RAID options GEOM_PART_GPT options PSEUDOFS options PROCFS options CD9660 options MSDOSFS options NFS_ROOT options NFSLOCKD options NFSD options NFSCL options QUOTA options SCTP options TCP_OFFLOAD options INET6 options INET options PREEMPTION options SCHED_ULE options NEW_PCIB options GEOM_PART_MBR options GEOM_PART_EBR_COMPAT options GEOM_PART_EBR options GEOM_PART_BSD device isa device mem device io device uart_ns8250 device cpufreq device acpi device pci device fdc device ahci device ata device esp device isci device scbus device ch device da device sa device cd device pass device ses device ctl device atkbdc device atkbd device psm device kbdmux device vga device splash device sc device agp device uart device ppc device ppbus device lpt device ppi device em device miibus device cas device gem device hme device nfe device nge device loop device random device ether device vlan device tun device md device gif device faith device firmware device bpf device uhci device ohci device ehci device xhci device usb device ukbd device umass device virtio device virtio_pci device vtnet device virtio_blk device virtio_scsi device virtio_balloon device netmap ------------------------------------------------------------------------ ddb capture buffer ddb: ddb_capture: kvm_nlist -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 512-248-2683 E-Mail: ler@lerctr.org US Mail: 430 Valona Loop, Round Rock, TX 78681-3893 _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.BSF.2.00.1304011305010.33680>