Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 26 Nov 2014 08:43:19 +0000
From:      Kolontai Andrej <Andrej.Kolontai@Verwaltung.Uni-Muenchen.DE>
To:        "'freebsd-questions@freebsd.org'" <freebsd-questions@freebsd.org>
Subject:   Relayd crashing Kernel on 10.1
Message-ID:  <894145A3DDBDEF4880E00D334DCD872618A8B2DE@MXS2.zuv.uni-muenchen.de>

next in thread | raw e-mail | index | archive | help

[-- Attachment #1 --]
Hello @all,

I'm new to this list and hope this is the right place to ask. 
We are using FreeBSD for our Firewalls and are actually happy with it. Since recently we use relayd (installed via pkg) to do some load balancing stuff. On a freshly installed machine running 10.0-RELEASE everything worked fine. 
On monday, I tried to upgrade to 10.1-RELEASE using freebsd-update as described in the handbook chapter 24. At first everything looked good but relayd wouldn't come up:

"Nov 24 10:50:48 flutters relayd[3300]: fatal: cannot add rule: Operation not supported by device
Nov 24 10:50:48 flutters relayd[3293]: lost child: pfe exited abnormally"

When I tried to start it with /usr/local/etc/rc.d/relayd start the kernel panicked. I had to roll back the update (which worked fine). However, I was able to reproduce this behavior on a virtual machine. 


My guess is it happens here:
#7  0xffffffff81a37954 in pfr_detach_table (kt=0x0)
    at /usr/src_10.1.0/sys/modules/pf/../../netpfil/pf/pf_table.c:2047

The corresponding code is:
void
pfr_detach_table(struct pfr_ktable *kt)
{

        PF_RULES_WASSERT();
        KASSERT(kt->pfrkt_refcnt[PFR_REFCNT_RULE] > 0, ("%s: refcount %d\n",
            __func__, kt->pfrkt_refcnt[PFR_REFCNT_RULE]));

        if (!--kt->pfrkt_refcnt[PFR_REFCNT_RULE])
                pfr_setflags_ktable(kt, kt->pfrkt_flags&~PFR_TFLAG_REFERENCED);
}

>From what I know about C programming: kt is not supposed to be 0x0. 
My guess was that some data structure has changed between 10.0 and 10.1 kernels. So a recompile of relayd should fix that. It did. I compiled it from the ports and it worked. 

Now my question ist: did I do something wrong?  Maybe compiling is the preferred method over using binaries? I'm still trying to figure out what  the "stay-out-of-trouble"-mode on FreeBSD is (like yum -y update). But I'd rather use the binaries. 


Viele Grüße 

Andrej Kolontai

Ludwig-Maximilians-Universitaet Muenchen
Ref. VI.4 (IT-Sicherheit & Verzeichnisdienste) 
Martiusstrasse 4 / 207
80802 Muenchen

phone  +49 (0)89 2180-3815
email  mailto:andrej.kolontai@verwaltung.uni-muenchen.de
web     http://www.uni-muenchen.de/zuv/it/


[-- Attachment #2 --]
bsd1 dumped core - see /var/crash/vmcore.0

Fri Nov 21 17:28:04 CET 2014

FreeBSD bsd1 10.1-RELEASE FreeBSD 10.1-RELEASE #0 r274786: Sat Nov 15 10:51:07 CET 2014     root@bsd1:/usr/obj/usr/src_10.1.0/sys/IPSEC  amd64

panic: page fault

GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...

Unread portion of the kernel message buffer:


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address	= 0x4a4
fault code		= supervisor write data, page not present
instruction pointer	= 0x20:0xffffffff81a37954
stack pointer	        = 0x28:0xfffffe001b139c30
frame pointer	        = 0x28:0xfffffe001b139c40
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 82338 (relayd)
trap number		= 12
panic: page fault
cpuid = 0
KDB: stack backtrace:
#0 0xffffffff8096b580 at kdb_backtrace+0x60
#1 0xffffffff809306a5 at panic+0x155
#2 0xffffffff80d4ebef at trap_fatal+0x38f
#3 0xffffffff80d4ef08 at trap_pfault+0x308
#4 0xffffffff80d4e56a at trap+0x47a
#5 0xffffffff80d34452 at calltrap+0x8
#6 0xffffffff81a25784 at pf_empty_pool+0x44
#7 0xffffffff81a27bb2 at pfioctl+0x7e2
#8 0xffffffff80820154 at devfs_ioctl_f+0x114
#9 0xffffffff80983035 at kern_ioctl+0x255
#10 0xffffffff80982d8c at sys_ioctl+0x13c
#11 0xffffffff80d4f521 at amd64_syscall+0x351
#12 0xffffffff80d3473b at Xfast_syscall+0xfb
Uptime: 50m53s
Dumping 83 out of 231 MB:..20%..39%..58%..77%..96%

Reading symbols from /boot/kernel/carp.ko.symbols...done.
Loaded symbols for /boot/kernel/carp.ko.symbols
Reading symbols from /boot/kernel/pf.ko.symbols...done.
Loaded symbols for /boot/kernel/pf.ko.symbols
#0  doadump (textdump=<value optimized out>) at pcpu.h:219
219	pcpu.h: No such file or directory.
	in pcpu.h
(kgdb) #0  doadump (textdump=<value optimized out>) at pcpu.h:219
#1  0xffffffff80930322 in kern_reboot (howto=260)
    at /usr/src_10.1.0/sys/kern/kern_shutdown.c:452
#2  0xffffffff809306e4 in panic (fmt=<value optimized out>)
    at /usr/src_10.1.0/sys/kern/kern_shutdown.c:759
#3  0xffffffff80d4ebef in trap_fatal (frame=<value optimized out>, 
    eva=<value optimized out>) at /usr/src_10.1.0/sys/amd64/amd64/trap.c:865
#4  0xffffffff80d4ef08 in trap_pfault (frame=0xfffffe001b139b80, 
    usermode=<value optimized out>)
    at /usr/src_10.1.0/sys/amd64/amd64/trap.c:676
#5  0xffffffff80d4e56a in trap (frame=0xfffffe001b139b80)
    at /usr/src_10.1.0/sys/amd64/amd64/trap.c:440
#6  0xffffffff80d34452 in calltrap ()
    at /usr/src_10.1.0/sys/amd64/amd64/exception.S:232
#7  0xffffffff81a37954 in pfr_detach_table (kt=0x0)
    at /usr/src_10.1.0/sys/modules/pf/../../netpfil/pf/pf_table.c:2047
#8  0xffffffff81a25784 in pf_empty_pool (poola=0xffffffff81a3bf40)
    at /usr/src_10.1.0/sys/modules/pf/../../netpfil/pf/pf_ioctl.c:351
#9  0xffffffff81a27bb2 in pfioctl (dev=<value optimized out>, 
    cmd=<value optimized out>, addr=0xfffff80002ab8000 "", 
    flags=<value optimized out>, td=<value optimized out>)
    at /usr/src_10.1.0/sys/modules/pf/../../netpfil/pf/pf_ioctl.c:2191
#10 0xffffffff80820154 in devfs_ioctl_f (fp=0xfffff800024920a0, 
    com=3295691827, data=0xfffff80002ab8000, cred=<value optimized out>, 
    td=0xfffff80002a4c490) at /usr/src_10.1.0/sys/fs/devfs/devfs_vnops.c:757
#11 0xffffffff80983035 in kern_ioctl (td=0xfffff80002a4c490, 
    fd=<value optimized out>, com=18446744071589578632) at file.h:320
#12 0xffffffff80982d8c in sys_ioctl (td=0xfffff80002a4c490, 
    uap=0xfffffe001b13ab80) at /usr/src_10.1.0/sys/kern/sys_generic.c:702
#13 0xffffffff80d4f521 in amd64_syscall (td=0xfffff80002a4c490, traced=0)
    at subr_syscall.c:134
#14 0xffffffff80d3473b in Xfast_syscall ()
    at /usr/src_10.1.0/sys/amd64/amd64/exception.S:391
#15 0x0000000801205b9a in ?? ()
Previous frame inner to this frame (corrupt stack?)
Current language:  auto; currently minimal
(kgdb) 

------------------------------------------------------------------------
ps -axlww

UID   PID  PPID CPU PRI NI   VSZ               RSS MWCHAN   STAT TT      TIME COMMAND
  0     0     0   0  -8  0     0 18005602445774944 -        DLs   -   0:29.67 [kernel]
  0     1     0   0  20  0  9472 17979214290267648 wait     DLs   -   0:00.01 [init]
  0     2     0   0 -16  0     0 18005602445774944 crypto_w DL    -   0:00.00 [crypto]
  0     3     0   0 -16  0     0 18005602445774944 crypto_r DL    -   0:00.00 [crypto returns]
  0     4     0   0 -16  0     0 18005602445774944 -        DL    -   0:02.91 [cam]
  0     5     0   0 -16  0     0 18005602445774944 waiting_ DL    -   0:00.00 [sctp_iterator]
  0     6     0   0 -16  0     0 18005602445774944 -        RL    -   0:00.29 [pagedaemon]
  0     7     0   0 -16  0     0 18005602445774944 psleep   DL    -   0:00.01 [vmdaemon]
  0     8     0   0 155  0     0 18005602445774944 pgzero   DL    -   0:00.00 [pagezero]
  0     9     0   0  20  0     0 18005602445774944 -        RL    -   0:01.14 [bufdaemon]
  0    10     0   0 -16  0     0 18005602445774944 audit_wo DL    -   0:00.00 [audit]
  0    11     0   0 155  0     0 18005602445774944 -        RL    -  43:54.30 [idle]
  0    12     0   0 -72  0     0 18005602445774944 -        WL    -   0:06.84 [intr]
  0    13     0   0  -8  0     0 18005602445774944 -        DL    -   0:00.11 [geom]
  0    14     0   0 -16  0     0 18005602445774944 -        RL    -   0:02.21 [rand_harvestq]
  0    15     0   0 -68  0     0 18005602445774944 -        DL    -   0:00.07 [usb]
  0    16     0   0 -16  0     0 18005602445774944 -        RL    -   0:00.01 [vnlru]
  0    17     0   0  16  0     0 18005602445774944 -        RL    -   0:00.17 [syncer]
  0   130     1   0  52  0 12332 17979214290248704 pause    DWs   -   0:00.00 [adjkerntz]
  0   301     1   0  20  0 13164 17979214290215424 select   Ds    -   0:00.01 [devd]
  0   310     0   0 -16  0     0 18005602445774944 -        RL    -   0:00.73 [pf purge]
  0   390     1   0  20  0 14492 17979214290264576 select   Ds    -   0:00.03 [syslogd]
  0   556     1   0  20  0 26124 17979214290281984 select   Ds    -   0:00.01 [racoon]
  0   576     1   0  20  0 61204 17979214304548864 select   Ds    -   0:00.00 [sshd]
  0   581     1   0  20  0 89244 17979214294639104 -        Rs    -   0:00.11 [httpd]
 80   588   581   0  20  0 92704 17979214310355968 lockf    D     -   0:00.01 [httpd]
 80   589   581   0  20  0 92704                 0 lockf    D     -   0:00.02 [httpd]
 80   590   581   0  20  0 92704 17979214307803648 lockf    D     -   0:00.02 [httpd]
  0   596     1   0  20  0 24104 17979214308985344 select   Ds    -   0:00.06 [sendmail]
 25   599     1   0  20  0 24104 17979214310307328 pause    Ds    -   0:00.00 [sendmail]
  0   603     1   0  20  0 16592 17979214310293504 nanslp   Ds    -   0:00.01 [cron]
  0   649     1   0  20  0 47724 17979214276939264 wait     Ds    -   0:00.02 [login]
  0   650     1   0  52  0 14488                 0 ttyin    Ds+   -   0:00.00 [getty]
  0   651     1   0  52  0 14488 17979214276906496 ttyin    Ds+   -   0:00.00 [getty]
  0   652     1   0  52  0 14488 17979214309025792 ttyin    Ds+   -   0:00.00 [getty]
  0   653     1   0  52  0 14488 17979214306560512 ttyin    Ds+   -   0:00.00 [getty]
  0   654     1   0  52  0 14488 17979214276900864 ttyin    Ds+   -   0:00.00 [getty]
  0   655     1   0  52  0 14488 17979214314293248 ttyin    Ds+   -   0:00.00 [getty]
  0   656     1   0  52  0 14488 17979214310352384 ttyin    Ds+   -   0:00.00 [getty]
 80   661   581   0  20  0 92704 17979214318919168 kqread   D     -   0:00.01 [httpd]
  0   663   576   0  20  0 86472 17979214314322432 select   Ds    -   0:05.85 [sshd]
  0   666   663   0  20  0 17648 17979214314330624 ttyin    Ds+   -   0:00.07 [bash]
  0 82265   649   0  28  0 17648 17979214556030464 ttyin    D+    -   0:00.03 [bash]
  0 82337     1   0  44  0 21344 17979214314355200 kqread   Ds    -   0:00.00 [relayd]
913 82338 82337   0 -92  0 21344 17979214276942848 -        R     -   0:00.00 [relayd]
913 82339 82337   0  20  0 21344                 0 -        R     -   0:00.00 [relayd]
913 82340 82337   0  44  0 21344 17979215009833728 kqread   D     -   0:00.00 [relayd]
913 82341 82337   0  45  0 21344 17979215009839104 kqread   D     -   0:00.00 [relayd]
913 82342 82340   0  43  0 21344                 0 kqread   D     -   0:00.00 [relayd]
913 82343 82340   0  44  0 21344 17979214276948224 kqread   D     -   0:00.00 [relayd]
913 82344 82341   0  43  0 21344 17979215009844736 kqread   D     -   0:00.00 [relayd]
913 82345 82341   0  44  0 21344 17979215009861120 kqread   D     -   0:00.00 [relayd]

------------------------------------------------------------------------
vmstat -s

  2729897 cpu context switches
   911527 device interrupts
   107132 software interrupts
 11240392 traps
 26526119 system calls
       18 kernel threads created
    59590  fork() calls
    22751 vfork() calls
        0 rfork() calls
      762 swap pager pageins
     3648 swap pager pages paged in
     2700 swap pager pageouts
    11864 swap pager pages paged out
     7686 vnode pager pageins
    54040 vnode pager pages paged in
        0 vnode pager pageouts
        0 vnode pager pages paged out
      187 page daemon wakeups
        0 pages examined by the page daemon
     1926 pages reactivated
  2100669 copy-on-write faults
     3347 copy-on-write optimized faults
  7822254 zero fill pages zeroed
     2216 zero fill pages prezeroed
      780 intransit blocking page faults
 10701558 total VM faults taken
     8857 page faults requiring I/O
        0 pages affected by kernel thread creation
  2288947 pages affected by  fork()
  1080015 pages affected by vfork()
        0 pages affected by rfork()
        0 pages cached
 12671198 pages freed
        0 pages freed by daemon
        0 pages freed by exiting processes
     3847 pages active
    22497 pages inactive
       55 pages in VM cache
    15665 pages wired down
    15499 pages free
     4096 bytes per page
  7091541 total name lookups
          cache hits (88% pos + 6% neg) system 0% per-directory
          deletions 0%, falsehits 0%, toolong 0%

------------------------------------------------------------------------
vmstat -m

         Type InUse MemUse HighUse Requests  Size(s)
       kbdmux     6    22K       -        6  16,512,1024,2048,16384
          LED     2     1K       -        2  16,128
      CAM CCB     0     0K       -    59267  2048
      entropy  1026    65K       -     9538  32,64,4096
         cdev     6     2K       -        6  256
     filedesc    65   173K       -    82493  32,2048,4096,16384
        sigio     1     1K       -        1  64
     filecaps     0     0K       -       28  16,64
      kdtrace   257    55K       -    82576  64,256
         kenv    74    11K       -       82  16,32,64,128,8192
       kqueue    35    34K       -      212  256,512,2048
    proc-args    26     2K       -    53815  16,32,64,128,256
        hhook     2     1K       -        2  256
      ithread    60    11K       -       60  32,128,256
       KTRACE   100    13K       -      100  128
       linker   170   153K       -      177  16,32,64,128,256,512,2048,4096,16384
        lockf    42     4K       -     8573  64,128
   loginclass     3     1K       -       29  64
       devbuf   408  1244K       -      466  16,32,64,128,256,512,1024,2048,4096,8192,16384
         temp    15    21K       -   119911  16,32,64,128,256,1024,2048,4096,8192,16384,32768
       ip6ndp     4     1K       -        5  64,128
       module   486    61K       -      486  128
     mtx_pool     2    16K       -        2  8192
     acpiintr     1     1K       -        1  64
     pmchooks     1     1K       -        1  128
         ac97     2     1K       -        2  16,512
         pgrp    26     4K       -      535  128
      session    21     3K       -       42  128
         proc     2     4K       -        2  2048
      subproc   125   245K       -    82433  512,4096
         cred    66    11K       -   317402  64,256
       plimit    18     5K       -       52  256
      uidinfo     5     1K       -       24  128,512
       acpica  2073   233K       -    21831  16,32,64,128,256,1024,4096
     CAM path     7     1K       -       43  32
       sysctl     0     0K       -     6138  16,32,64
    sysctloid  1918    96K       -     1954  16,32,64,128
    sysctltmp     0     0K       -     5270  16,32,64,128
       feeder    12     1K       -       14  32,128
      tidhash     1     4K       -        1  4096
      callout     2   712K       -        2  
         umtx   414    52K       -      414  128
     p1003.1b     1     1K       -        1  16
         SWAP     2    73K       -        2  64
          bus   984    80K       -     2903  16,32,64,128,256,1024
       bus-sc    31    52K       -     1422  16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536
      devstat     6    13K       -        6  32,4096
 eventhandler   106     9K       -      106  64,128
         kobj   336  1344K       -      534  4096
      Per-cpu     1     1K       -        1  32
         rman    92    11K       -      430  32,128
         sbuf     0     0K       -      630  16,32,64,128,256,512,1024,2048,4096,8192,32768
    taskqueue    17     3K       -       17  16,32,256
       Unitno    21     2K       -    67033  32,64
         vmem     4    11K       -       20  128,1024,2048,4096,8192
     ioctlops     1     2K       -    25179  16,32,64,128,256,512,1024,2048,4096
       select    39     5K       -       39  128
          iov     0     0K       -   930395  16,64,128,256,512
          msg     4    30K       -        4  2048,4096,8192,16384
          sem     4   106K       -        4  2048,4096
          shm     1    32K       -        1  32768
          tty    18    18K       -       19  1024
          pts     1     1K       -        2  256
     mbuf_tag     0     0K       -    21060  32,128
        shmfd     1     8K       -        1  8192
       soname     6     1K       -     5901  16,32,128
          pcb    24   183K       -      196  16,32,64,128,1024,2048,8192,16384
          acl     0     0K       -     5266  4096
     vfscache     1   128K       -        1  
   cl_savebuf     0     0K       -     1289  64
     vfs_hash     1    64K       -        1  65536
       vnodes     1     1K       -        1  256
       DEVFS3    97    25K       -      120  256
       DEVFS1    77    39K       -       87  512
        mount    16     1K       -       86  16,32,64,128,256
  vnodemarker     0     0K       -      204  512
          BPF     2     1K       -        2  128
   DEVFS_RULE    55    26K       -       55  64,512
        ifnet     3     5K       -        3  128,2048
       ifaddr    41    14K       -       41  32,64,128,256,512,2048,4096
  ether_multi    45     3K       -       52  16,32,64
        clone     7     1K       -        7  128
       arpcom     1     1K       -        1  16
      lltable    12     4K       -       15  256,512
        DEVFS    15     1K       -       16  16,128
     routetbl    47    11K       -      225  32,64,128,256,512
         igmp     2     1K       -        2  256
     in_multi     3     1K       -        3  256
encap_export_host     2     2K       -        2  1024
    sctp_a_it     0     0K       -       10  16
     sctp_vrf     1     1K       -        1  64
     sctp_ifa     7     1K       -       10  128
     sctp_ifn     2     1K       -        2  128
    sctp_iter     0     0K       -       10  256
    hostcache     1    32K       -        1  32768
     syncache     1    64K       -        1  65536
     fragment     0     0K       -      342  64,128
    in6_multi    25     3K       -       25  32,256
          mld     2     1K       -        2  128
  inpcbpolicy    15     1K       -      392  32
  ipsecpolicy    32     8K       -      810  256
 ipsecrequest     2     1K       -        2  128
    ipsec-reg     3     1K       -        3  32
       crypto     1     1K       -        1  512
          rpc     2    16K       -        2  8192
audit_evclass   187     6K       -      228  32
      pagedep     2    17K       -     7626  256,16384
     inodedep     2    65K       -    43009  512,65536
    bmsafemap     2     9K       -    29294  256,8192
       newblk     1   128K       -    54188  256
     indirdep     0     0K       -      320  128,32768
     freefrag     0     0K       -     5590  128
     freeblks     0     0K       -    27894  256
     freefile     0     0K       -    28572  64
       diradd     1     1K       -    33883  128
        mkdir     0     0K       -    11348  128
       dirrem     0     0K       -    33466  128
    newdirblk     0     0K       -     5720  64
     freework     1     1K       -    37029  16,128
      freedep     0     0K       -      128  64
      jaddref     0     0K       -    45231  128
      jremref     0     0K       -    44742  128
       jmvref     0     0K       -      945  128
      jnewblk     0     0K       -    54187  128
    jfreefrag     0     0K       -     5590  128
         jseg     1     1K       -     1927  128
      jsegdep     1     1K       -   149750  64
        sbdep     0     0K       -       47  64
     savedino     0     0K       -    21528  256
      jblocks     2     1K       -        2  128,256
      softdep     1     1K       -        1  512
  ufs_dirhash   103    27K       -      757  16,32,64,128,256,512,1024,8192,16384
    ufs_quota     1    64K       -        1  65536
    ufs_mount     3    13K       -        3  512,4096,8192
    vm_pgdata     2    33K       -        2  128,32768
      UMAHash     3     5K       -        6  512,1024,2048,4096
     atkbddev     2     1K       -        2  64
    pfs_nodes    21     6K       -       21  256
         GEOM    58    11K       -      665  16,32,64,128,256,512,1024,2048,8192,16384
        mixer     1     4K       -        1  4096
     acpitask     1    16K       -        1  16384
      ata_pci     1     1K       -        1  64
    raid_data     0     0K       -       96  32,128,256
      acpisem    17     3K       -       17  128
   CAM periph     6     2K       -       20  16,32,64,128,256
      acpidev    24     2K       -       24  64
    CAM queue     8     3K       -       25  16,32,512
CAM dev queue     3     1K       -        3  64
      CAM SIM     3     1K       -        3  256
md_nvidia_data     0     0K       -       15  512
      scsi_cd     0     0K       -        5  16
  md_sii_data     0     0K       -       15  512
       apmdev     1     1K       -        1  128
   madt_table     0     0K       -        1  4096
      CAM XPT    22     2K       -       82  32,64,128,256,512,1024,2048,65536
     pci_link     8     1K       -        8  16,128
      io_apic     1     2K       -        1  2048
          MCA     1     1K       -        1  128
      CAM DEV     5    10K       -       10  2048
          USB     6     5K       -        6  16,128,512,4096
     nexusdev     4     1K       -        4  16
       USBdev     5     1K       -        5  32,64,128,512
       isadev     9     2K       -        9  128
         CARP     6     3K       -        6  16,256,1024
      pf_hash     3  2880K       -        3  
     pf_ifnet     5     3K       -        6  256,2048
      pf_osfp  1130   115K       -     1130  64,128
      pf_rule     2     2K       -        2  128,1024
     pf_table     2     4K       -        4  2048

------------------------------------------------------------------------
vmstat -z

ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP

UMA Kegs:               384,      0,     115,       5,     115,   0,   0
UMA Zones:              768,      0,     115,       0,     115,   0,   0
UMA Slabs:               80,      0,    1241,     409,    2758,   0,   0
UMA RCntSlabs:           88,      0,     257,      13,     257,   0,   0
UMA Hash:               256,      0,       7,       8,      10,   0,   0
4 Bucket:                32,      0,      16,     359,   16648,   0,   0
6 Bucket:                48,      0,       0,       0,       0,   0,   0
8 Bucket:                64,      0,       5,     305,     342,  11,   0
12 Bucket:               96,      0,       3,     161,      77,   0,   0
16 Bucket:              128,      0,      13,     142,     338,   0,   0
32 Bucket:              256,      0,     282,      48,    3070,  52,   0
64 Bucket:              512,      0,     264,      32,    8192,  52,   0
128 Bucket:            1024,      0,      91,      13,    1933,   0,   0
256 Bucket:            2048,      0,      44,       4,     540,   0,   0
vmem btag:               56,      0,    1061,     714,   12674,  13,   0
VM OBJECT:              256,      0,    3911,    5239, 1098090,   0,   0
RADIX NODE:             144,      0,    3865,    2777, 2494391,   0,   0
MAP:                    240,      0,       3,      61,       3,   0,   0
KMAP ENTRY:             128,      0,       6,     149,       6,   0,   0
MAP ENTRY:              128,      0,    1501,     359, 2716176,   0,   0
VMSPACE:                448,      0,      34,      29,   82340,   0,   0
fakepg:                 104,      0,       0,       0,       0,   0,   0
mt_zone:               4112,      0,     376,       0,     376,   0,   0
16:                      16,      0,    1049,     457,   27841,   0,   0
32:                      32,      0,    1169,     206,   85772,   0,   0
64:                      64,      0,    3034,   10048, 1470386,   0,   0
128:                    128,      0,    4455,   10704,  331885,   0,   0
256:                    256,      0,     629,    4381,  305763,   0,   0
512:                    512,      0,     344,    3688,   44499,   0,   0
1024:                  1024,      0,      41,      27,   58345,   0,   0
2048:                  2048,      0,      96,       4,  141896,   0,   0
4096:                  4096,      0,     433,       4,   96873,   0,   0
8192:                  8192,      0,      16,       1,     198,   0,   0
16384:                16384,      0,      15,       2,     279,   0,   0
32768:                32768,      0,       3,       4,     228,   0,   0
65536:                65536,      0,       4,       1,      70,   0,   0
SLEEPQUEUE:              80,      0,     208,      71,     208,   0,   0
64 pcpu:                  8,      0,    2370,     190,    2370,   0,   0
Files:                   80,      0,     143,     107,  800658,   0,   0
TURNSTILE:              136,      0,     208,      52,     208,   0,   0
rl_entry:                40,      0,      60,     240,      60,   0,   0
umtx pi:                 96,      0,       0,       0,       0,   0,   0
MAC labels:              40,      0,       0,       0,       0,   0,   0
PROC:                  1216,      0,      51,      21,   82359,   0,   0
THREAD:                1168,      0,     204,       3,     215,   0,   0
cpuset:                  72,      0,      47,     118,      47,   0,   0
audit_record:          1248,      0,       0,       0,       0,   0,   0
mbuf_packet:            256,  92115,     259,     253, 1353663,   0,   0
mbuf:                   256,  92115,       3,     256,  142061,   0,   0
mbuf_cluster:          2048,  14392,     506,       4,     506,   0,   0
mbuf_jumbo_page:       4096,   7195,       0,       2,      43,   0,   0
mbuf_jumbo_9k:         9216,   6396,       0,       0,       0,   0,   0
mbuf_jumbo_16k:       16384,   4796,       0,       0,       0,   0,   0
mbuf_ext_refcnt:          4,      0,       0,       0,       0,   0,   0
DMAR_MAP_ENTRY:         120,      0,       0,       0,       0,   0,   0
g_bio:                  248,      0,       0,     432,  174930,   0,   0
ttyinq:                 160,      0,     180,      20,     375,   0,   0
ttyoutq:                256,      0,      95,      40,     198,   0,   0
ata_request:            336,      0,       1,      32,   62501,   0,   0
cryptop:                 88,      0,       0,       0,       0,   0,   0
cryptodesc:              72,      0,       0,       0,       0,   0,   0
vtnet_tx_hdr:            24,      0,       0,       0,       0,   0,   0
FPU_save_area:          512,      0,       0,       0,       0,   0,   0
VNODE:                  472,      0,    3135,    3977,   33927,   0,   0
VNODEPOLL:              112,      0,       0,       0,       0,   0,   0
BUF TRIE:               144,      0,     135,    2025,   23863,   0,   0
S VFS Cache:            108,      0,    3145,    4275,  258434,   0,   0
STS VFS Cache:          148,      0,       0,       0,       0,   0,   0
L VFS Cache:            328,      0,       3,     753,    2789,   0,   0
LTS VFS Cache:          368,      0,       0,       0,       0,   0,   0
NAMEI:                 1024,      0,       0,      16, 2303804,   0,   0
DIRHASH:               1024,      0,     165,       7,    2286,   0,   0
NCLNODE:                528,      0,       0,       0,       0,   0,   0
procdesc:               128,      0,       0,       0,       0,   0,   0
Mountpoints:            816,      0,       2,      13,       2,   0,   0
pipe:                   744,      0,       2,      28,   43893,   0,   0
ksiginfo:               112,      0,      62,      78,     188,   0,   0
itimer:                 352,      0,       0,       0,       0,   0,   0
KNOTE:                  128,      0,     112,      43,    1447,   0,   0
socket:                 696,   7415,      70,      10,    1518,   0,   0
ipq:                     56,    497,       0,     284,     302,   0,   0
udp_inpcb:              392,   7420,       3,      27,     296,   0,   0
udpcb:                   16,   7530,       3,     499,     296,   0,   0
tcp_inpcb:              392,   7420,       8,      22,      84,   0,   0
tcpcb:                 1024,   7416,       8,       8,      84,   0,   0
tcptw:                   88,   1485,       0,     135,       7,   0,   0
syncache:               160,  15375,       0,      75,      11,   0,   0
hostcache:              136,  15370,       1,      86,       1,   0,   0
tcpreass:                40,    900,       0,       0,       0,   0,   0
sackhole:                32,      0,       0,       0,       0,   0,   0
sctp_ep:               1408,   7416,       0,       0,       0,   0,   0
sctp_asoc:             2416,  40000,       0,       0,       0,   0,   0
sctp_laddr:              48,  80012,       0,     332,      12,   0,   0
sctp_raddr:             728,  80000,       0,       0,       0,   0,   0
sctp_chunk:             136, 400026,       0,       0,       0,   0,   0
sctp_readq:             104, 400026,       0,       0,       0,   0,   0
sctp_stream_msg_out:    104, 400026,       0,       0,       0,   0,   0
sctp_asconf:             40, 400000,       0,       0,       0,   0,   0
sctp_asconf_ack:         48, 400060,       0,       0,       0,   0,   0
udplite_inpcb:          392,   7420,       0,       0,       0,   0,   0
ripcb:                  392,   7420,       4,      26,      12,   0,   0
unpcb:                  240,   7424,      53,      11,    1118,   0,   0
rtentry:                200,      0,      19,      61,      24,   0,   0
selfd:                   56,      0,      60,     224,  196650,   0,   0
SWAPMETA:               288,  28782,     691,     505,    1688,   0,   0
FFS inode:              168,      0,    3113,    4707,   33890,   0,   0
FFS1 dinode:            128,      0,       0,       0,       0,   0,   0
FFS2 dinode:            256,      0,    3113,    4507,   33890,   0,   0
pf mtags:                40,      0,       0,       0,       0,   0,   0
pf states:              296,  10010,     367,      49,    8915,   0,   0
pf state keys:           88,      0,     367,     173,    8915,   0,   0
pf source nodes:        136,  10005,       0,       0,       0,   0,   0
pf table entries:       160, 200000,       2,      73,       3,   0,   0
pf table counters:       64,      0,       0,       0,       0,   0,   0
pf frags:                80,      0,       0,       0,       0,   0,   0
pf frag entries:         32,   5000,       0,       0,       0,   0,   0
pf state scrubs:         40,      0,       0,       0,       0,   0,   0


------------------------------------------------------------------------
vmstat -i

interrupt                          total       rate
irq1: atkbd0                        2639        203
irq14: ata0                        60622       4663
irq15: ata1                         2055        158
irq19: em0                        846215      65093
cpu0:timer                        133010      10231
Total                            1044541      80349

------------------------------------------------------------------------
pstat -T

143/7414 files
17M/511M swap space

------------------------------------------------------------------------
pstat -s

Device          512-blocks     Used    Avail Capacity
/dev/ada0p3        1048320    35408  1012912     3%

------------------------------------------------------------------------
iostat

iostat: kvm_read(_tk_nin): invalid address (0x0)
iostat: disabling TTY statistics
            ada0              cd0            pass0             cpu
  KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id
 24.10 4368 102.79   0.00   1  0.00   0.00   0  0.00   1  7  5  0 87

------------------------------------------------------------------------
ipcs -a

Message Queues:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP                 CBYTES                 QNUM               QBYTES        LSPID        LRPID STIME    RTIME    CTIME   

Shared Memory:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP         NATTCH        SEGSZ         CPID         LPID ATIME    DTIME    CTIME   

Semaphores:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP          NSEMS OTIME    CTIME   


------------------------------------------------------------------------
ipcs -T

msginfo:
	msgmax:        16384	(max characters in a message)
	msgmni:           40	(# of message queues)
	msgmnb:         2048	(max characters in a message queue)
	msgtql:           40	(max # of messages in system)
	msgssz:            8	(size of a message segment)
	msgseg:         2048	(# of message segments in system)

shminfo:
	shmmax:    536870912	(max shared memory segment size)
	shmmin:            1	(min shared memory segment size)
	shmmni:          192	(max number of shared memory identifiers)
	shmseg:          128	(max shared memory segments per process)
	shmall:       131072	(max amount of shared memory in pages)

seminfo:
	semmni:           50	(# of semaphore identifiers)
	semmns:          340	(# of semaphores in system)
	semmnu:          150	(# of undo structures in system)
	semmsl:          340	(max # of semaphores per id)
	semopm:          100	(max # of operations per semop call)
	semume:           50	(max # of undo entries per process)
	semusz:          632	(size in bytes of undo structure)
	semvmx:        32767	(semaphore maximum value)
	semaem:        16384	(adjust on exit max value)


------------------------------------------------------------------------
nfsstat

Client Info:
Rpc Counts:
  Getattr   Setattr    Lookup  Readlink      Read     Write    Create    Remove
        0         0         0         0         0         0         0         0
   Rename      Link   Symlink     Mkdir     Rmdir   Readdir  RdirPlus    Access
        0         0         0         0         0         0         0         0
    Mknod    Fsstat    Fsinfo  PathConf    Commit
        0         0         0         0         0
Rpc Info:
 TimedOut   Invalid X Replies   Retries  Requests
        0         0         0         0         0
Cache Info:
Attr Hits    Misses Lkup Hits    Misses BioR Hits    Misses BioW Hits    Misses
        0         0         0         0         0         0         0         0
BioRLHits    Misses BioD Hits    Misses DirE Hits    Misses Accs Hits    Misses
        0         0         0         0         0         0         0         0

Server Info:
  Getattr   Setattr    Lookup  Readlink      Read     Write    Create    Remove
        0         0         0         0         0         0         0         0
   Rename      Link   Symlink     Mkdir     Rmdir   Readdir  RdirPlus    Access
        0         0         0         0         0         0         0         0
    Mknod    Fsstat    Fsinfo  PathConf    Commit
        0         0         0         0         0
Server Ret-Failed
                0
Server Faults
            0
Server Cache Stats:
   Inprog      Idem  Non-idem    Misses
        0         0         0         0
Server Write Gathering:
 WriteOps  WriteRPC   Opsaved
        0         0         0

------------------------------------------------------------------------
netstat -s

tcp:
	40386 packets sent
		34563 data packets (2277375 bytes)
		0 data packets (0 bytes) retransmitted
		0 data packets unnecessarily retransmitted
		0 resends initiated by MTU discovery
		5654 ack-only packets (16 delayed)
		0 URG only packets
		0 window probe packets
		78 window update packets
		91 control packets
	44790 packets received
		32874 acks (for 2276733 bytes)
		43 duplicate acks
		0 acks for unsent data
		12003 packets (15834814 bytes) received in-sequence
		0 completely duplicate packets (0 bytes)
		0 old duplicate packets
		0 packets with some dup. data (0 bytes duped)
		0 out-of-order packets (0 bytes)
		0 packets (0 bytes) of data after window
		0 window probes
		0 window update packets
		0 packets received after close
		0 discarded for bad checksums
		0 discarded for bad header offset fields
		0 discarded because packet too short
		0 discarded due to memory problems
	53 connection requests
	11 connection accepts
	0 bad connection attempts
	0 listen queue overflows
	13 ignored RSTs in the windows
	50 connections established (including accepts)
	76 connections closed (including 12 drops)
		20 connections updated cached RTT on close
		20 connections updated cached RTT variance on close
		0 connections updated cached ssthresh on close
	14 embryonic connections dropped
	32874 segments updated rtt (of 30745 attempts)
	0 retransmit timeouts
		0 connections dropped by rexmit timeout
	0 persist timeouts
		0 connections dropped by persist timeout
	0 Connections (fin_wait_2) dropped because of timeout
	0 keepalive timeouts
		0 keepalive probes sent
		0 connections dropped by keepalive
	32507 correct ACK header predictions
	11818 correct data packet header predictions
	11 syncache entries added
		0 retransmitted
		0 dupsyn
		0 dropped
		11 completed
		0 bucket overflow
		0 cache overflow
		0 reset
		0 stale
		0 aborted
		0 badack
		0 unreach
		0 zone failures
	11 cookies sent
	0 cookies received
	1 hostcache entry added
		0 bucket overflow
	0 SACK recovery episodes
	0 segment rexmits in SACK recovery episodes
	0 byte rexmits in SACK recovery episodes
	0 SACK options (SACK blocks) received
	0 SACK options (SACK blocks) sent
	0 SACK scoreboard overflow
	0 packets with ECN CE bit set
	0 packets with ECN ECT(0) bit set
	0 packets with ECN ECT(1) bit set
	0 successful ECN handshakes
	0 times ECN reduced the congestion window
	0 packets with valid tcp-md5 signature received
	0 packets with invalid tcp-md5 signature received
	0 packets with tcp-md5 signature mismatch
	0 packets with unexpected tcp-md5 signature received
	0 packets without expected tcp-md5 signature received
udp:
	58973 datagrams received
	0 with incomplete header
	0 with bad data length field
	0 with bad checksum
	235 with no checksum
	15401 dropped due to no socket
	43468 broadcast/multicast datagrams undelivered
	0 dropped due to full socket buffers
	0 not for hashed pcb
	104 delivered
	103 datagrams output
	0 times multicast source filter matched
ip:
	96066 total packets received
	0 bad header checksums
	0 with size smaller than minimum
	0 with data size < data length
	0 with ip length > max ip packet size
	0 with header length < data size
	0 with data length < header length
	0 with bad options
	0 with incorrect version number
	887 fragments received
	0 fragments dropped (dup or out of space)
	0 fragments dropped after timeout
	302 packets reassembled ok
	94614 packets for this host
	0 packets for unknown/unsupported protocol
	0 packets forwarded (0 packets fast forwarded)
	574 packets not forwardable
	0 packets received for unknown multicast group
	0 redirects sent
	43057 packets sent from this host
	0 packets sent with fabricated ip header
	0 output packets dropped due to no bufs, etc.
	0 output packets discarded due to no route
	0 output datagrams fragmented
	0 fragments created
	0 datagrams that can't be fragmented
	0 tunneling packets that can't find gif
	0 datagrams with bad address in header
icmp:
	0 calls to icmp_error
	0 errors not generated in response to an icmp message
	Output histogram:
		echo reply: 2557
	0 messages with bad code fields
	0 messages less than the minimum length
	0 messages with bad checksum
	0 messages with bad length
	0 multicast echo requests ignored
	0 multicast timestamp requests ignored
	Input histogram:
		echo reply: 5
		echo: 2557
	2557 message responses generated
	0 invalid return addresses
	0 no return routes
igmp:
	0 messages received
	0 messages received with too few bytes
	0 messages received with wrong TTL
	0 messages received with bad checksum
	0 V1/V2 membership queries received
	0 V3 membership queries received
	0 membership queries received with invalid field(s)
	0 general queries received
	0 group queries received
	0 group-source queries received
	0 group-source queries dropped
	0 membership reports received
	0 membership reports received with invalid field(s)
	0 membership reports received for groups to which we belong
	0 V3 reports received without Router Alert
	1 membership report sent
ipsec:
	0 inbound packets violated process security policy
	0 inbound packets failed due to insufficient memory
	0 invalid inbound packets
	0 outbound packets violated process security policy
	0 outbound packets with no SA available
	0 outbound packets failed due to insufficient memory
	0 outbound packets with no route available
	0 invalid outbound packets
	0 outbound packets with bundled SAs
	0 mbufs coalesced during clone
	0 clusters coalesced during clone
	0 clusters copied during clone
	0 mbufs inserted during makespace
ah:
	0 packets shorter than header shows
	0 packets dropped; protocol family not supported
	0 packets dropped; no TDB
	0 packets dropped; bad KCR
	0 packets dropped; queue full
	0 packets dropped; no transform
	0 replay counter wraps
	0 packets dropped; bad authentication detected
	0 packets dropped; bad authentication length
	0 possible replay packets detected
	0 packets in
	0 packets out
	0 packets dropped; invalid TDB
	0 bytes in
	0 bytes out
	0 packets dropped; larger than IP_MAXPACKET
	0 packets blocked due to policy
	0 crypto processing failures
	0 tunnel sanity check failures
esp:
	0 packets shorter than header shows
	0 packets dropped; protocol family not supported
	0 packets dropped; no TDB
	0 packets dropped; bad KCR
	0 packets dropped; queue full
	0 packets dropped; no transform
	0 packets dropped; bad ilen
	0 replay counter wraps
	0 packets dropped; bad encryption detected
	0 packets dropped; bad authentication detected
	0 possible replay packets detected
	0 packets in
	0 packets out
	0 packets dropped; invalid TDB
	0 bytes in
	0 bytes out
	0 packets dropped; larger than IP_MAXPACKET
	0 packets blocked due to policy
	0 crypto processing failures
	0 tunnel sanity check failures
ipcomp:
	0 packets shorter than header shows
	0 packets dropped; protocol family not supported
	0 packets dropped; no TDB
	0 packets dropped; bad KCR
	0 packets dropped; queue full
	0 packets dropped; no transform
	0 replay counter wraps
	0 packets in
	0 packets out
	0 packets dropped; invalid TDB
	0 bytes in
	0 bytes out
	0 packets dropped; larger than IP_MAXPACKET
	0 packets blocked due to policy
	0 crypto processing failures
	0 packets sent uncompressed; size < compr. algo. threshold
	0 packets sent uncompressed; compression was useless
arp:
	13 ARP requests sent
	25 ARP replies sent
	195716 ARP requests received
	13 ARP replies received
	195729 ARP packets received
	2 total packets dropped due to no ARP entry
	3 ARP entrys timed out
	0 Duplicate IPs seen
ip6:
	17671 total packets received
	0 with size smaller than minimum
	0 with data size < data length
	0 with bad options
	0 with incorrect version number
	228 fragments received
	0 fragments dropped (dup or out of space)
	0 fragments dropped after timeout
	0 fragments that exceeded limit
	114 packets reassembled ok
	17661 packets for this host
	0 packets forwarded
	0 packets not forwardable
	0 redirects sent
	3 packets sent from this host
	0 packets sent with fabricated ip header
	0 output packets dropped due to no bufs, etc.
	6 output packets discarded due to no route
	0 output datagrams fragmented
	0 fragments created
	0 datagrams that can't be fragmented
	0 packets that violated scope rules
	634 multicast packets which we don't join
	Input histogram:
		hop by hop: 448
		UDP: 15288
		fragment: 228
		ICMP6: 196
		#112: 1511
	Mbuf statistics:
		0 one mbuf
		17671 one ext mbuf
		0 two or more ext mbuf
	0 packets whose headers are not contiguous
	0 tunneling packets that can't find gif
	0 packets discarded because of too many headers
	0 failures of source address selection
	Source addresses selection rule applied:
icmp6:
	0 calls to icmp6_error
	0 errors not generated in response to an icmp6 message
	0 errors not generated because of rate limitation
	Output histogram:
		neighbor solicitation: 1
		MLDv2 listener report: 2
	0 messages with bad code fields
	0 messages < minimum length
	0 bad checksums
	0 messages with bad length
	Input histogram:
		neighbor advertisement: 10
	Histogram of error messages to be generated:
		0 no route
		0 administratively prohibited
		0 beyond scope
		0 address unreachable
		0 port unreachable
		0 packet too big
		0 time exceed transit
		0 time exceed reassembly
		0 erroneous header field
		0 unrecognized next header
		0 unrecognized option
		0 redirect
		0 unknown
	0 message responses generated
	0 messages with too many ND options
	0 messages with bad ND options
	0 bad neighbor solicitation messages
	0 bad neighbor advertisement messages
	0 bad router solicitation messages
	0 bad router advertisement messages
	0 bad redirect messages
	0 path MTU changes
ipsec6:
	0 inbound packets violated process security policy
	0 inbound packets failed due to insufficient memory
	0 invalid inbound packets
	0 outbound packets violated process security policy
	0 outbound packets with no SA available
	0 outbound packets failed due to insufficient memory
	0 outbound packets with no route available
	0 invalid outbound packets
	0 outbound packets with bundled SAs
	0 mbufs coalesced during clone
	0 clusters coalesced during clone
	0 clusters copied during clone
	0 mbufs inserted during makespace
rip6:
	0 messages received
	0 checksum calculations on inbound
	0 messages with bad checksum
	0 messages dropped due to no socket
	0 multicast messages dropped due to no socket
	0 messages dropped due to full socket buffers
	0 delivered
	0 datagrams output
pfkey:
	9 requests sent from userland
	288 bytes sent from userland
	histogram by message type:
		register: 4
		flush: 1
		x_spdadd: 2
		x_spddump: 1
		x_spdflush: 1
	0 messages with invalid length field
	0 messages with invalid version field
	0 messages with invalid message type field
	0 messages too short
	0 messages with memory allocation failure
	0 messages with duplicate extension
	0 messages with invalid extension type
	0 messages with invalid sa type
	0 messages with invalid address extension
	10 requests sent to userland
	1056 bytes sent to userland
	histogram by message type:
		register: 4
		flush: 1
		x_spdadd: 2
		x_spddump: 2
		x_spdflush: 1
	0 messages toward single socket
	0 messages toward all sockets
	0 messages toward registered sockets
	0 messages with memory allocation failure

------------------------------------------------------------------------
netstat -m

netstat: invalid address (0x0)
262/509/771 mbufs in use (current/cache/total)
253/257/510/14392 mbuf clusters in use (current/cache/total/max)
259/253 mbuf+clusters out of packet secondary zone in use (current/cache)
0/2/2/7195 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/6396 9k jumbo clusters in use (current/cache/total/max)
0/0/0/4796 16k jumbo clusters in use (current/cache/total/max)
571K/649K/1220K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)

------------------------------------------------------------------------
netstat -anr

Routing tables

Internet:
Destination        Gateway            Flags      Netif Expire
default            172.23.254.254     UGS         em0
127.0.0.1          link#2             UH          lo0
172.23.0.0/16      link#1             U           em0
172.23.30.104      link#1             UHS         lo0
172.23.30.110      link#1             UHS         lo0
172.23.30.111      link#1             UHS         lo0

Internet6:
Destination                       Gateway                       Flags      Netif Expire
::/96                             ::1                           UGRS        lo0
::1                               link#2                        UH          lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%em0/64                     link#1                        U           em0
fe80::a00:27ff:fefd:1b07%em0      link#1                        UHS         lo0
fe80::%lo0/64                     link#2                        U           lo0
fe80::1%lo0                       link#2                        UHS         lo0
ff01::%em0/32                     fe80::a00:27ff:fefd:1b07%em0  U           em0
ff01::%lo0/32                     ::1                           U           lo0
ff02::/16                         ::1                           UGRS        lo0
ff02::%em0/32                     fe80::a00:27ff:fefd:1b07%em0  U           em0
ff02::%lo0/32                     ::1                           U           lo0

------------------------------------------------------------------------
netstat -anA

Active Internet connections (including servers)
Tcpcb            Proto Recv-Q Send-Q Local Address      Foreign Address    (state)
fffff80002743400 tcp4       0      0 172.23.30.104.22   172.23.3.4.58156   ESTABLISHED
fffff80002743800 tcp4       0      0 127.0.0.1.25       *.*                LISTEN
fffff80002743c00 tcp4       0      0 *.443              *.*                LISTEN
fffff80002744000 tcp6       0      0 *.443              *.*                LISTEN
fffff80002744400 tcp4       0      0 *.80               *.*                LISTEN
fffff80002744800 tcp6       0      0 *.80               *.*                LISTEN
fffff80002744c00 tcp4       0      0 *.22               *.*                LISTEN
fffff80002745000 tcp6       0      0 *.22               *.*                LISTEN
fffff8000252d930 udp4       0      0 172.23.30.104.500  *.*                
fffff8000252dab8 udp4       0      0 *.514              *.*                
fffff8000252dc40 udp6       0      0 *.514              *.*                
fffff8000bc01ab8 icm4       0      0 *.*                *.*                
fffff8000bc01c40 icm4     200      0 *.*                *.*                
fffff8000bc017a8 icm6       0      0 *.*                *.*                
fffff8000bc01930 icm6       0      0 *.*                *.*                
Active UNIX domain sockets
Address  Type   Recv-Q Send-Q    Inode     Conn     Refs  Nextref Addr
fffff80002515870 stream      0      0 fffff80003e7e938        0        0        0 /var/run/relayd.sock
fffff800025155a0 stream      0      0        0 fffff800025144b0        0        0
fffff800025144b0 stream      0      0        0 fffff800025155a0        0        0
fffff80002513960 stream      0      0        0 fffff80002513d20        0        0
fffff80002513d20 stream      0      0        0 fffff80002513960        0        0
fffff80002515e10 stream      0      0        0 fffff800025161e0        0        0
fffff800025161e0 stream      0      0        0 fffff80002515e10        0        0
fffff80002513a50 stream      0      0        0 fffff80002515b40        0        0
fffff80002515b40 stream      0      0        0 fffff80002513a50        0        0
fffff800025152d0 stream      0      0        0 fffff800025140f0        0        0
fffff800025140f0 stream      0      0        0 fffff800025152d0        0        0
fffff800025154b0 stream      0      0        0 fffff80002514780        0        0
fffff80002514780 stream      0      0        0 fffff800025154b0        0        0
fffff80002514d20 stream      0      0        0 fffff80002514c30        0        0
fffff80002514c30 stream      0      0        0 fffff80002514d20        0        0
fffff80002515a50 stream      0      0        0 fffff800025142d0        0        0
fffff800025142d0 stream      0      0        0 fffff80002515a50        0        0
fffff80002515690 stream      0      0        0 fffff80002516000        0        0
fffff80002516000 stream      0      0        0 fffff80002515690        0        0
fffff80002513b40 stream      0      0        0 fffff800025150f0        0        0
fffff800025150f0 stream      0      0        0 fffff80002513b40        0        0
fffff80002514000 stream      0      0        0 fffff800025143c0        0        0
fffff800025143c0 stream      0      0        0 fffff80002514000        0        0
fffff800025151e0 stream      0      0        0 fffff80002514690        0        0
fffff80002514690 stream      0      0        0 fffff800025151e0        0        0
fffff80002515c30 stream      0      0        0 fffff80002513c30        0        0
fffff80002513c30 stream      0      0        0 fffff80002515c30        0        0
fffff80002515780 stream      0      0        0 fffff80002515960        0        0
fffff80002515960 stream      0      0        0 fffff80002515780        0        0
fffff80002515d20 stream      0      0        0 fffff80002513780        0        0
fffff80002513780 stream      0      0        0 fffff80002515d20        0        0
fffff800025153c0 stream      0      0        0 fffff80002515000        0        0
fffff80002515000 stream      0      0        0 fffff800025153c0        0        0
fffff80002514b40 stream      0      0        0 fffff80002513870        0        0
fffff80002513870 stream      0      0        0 fffff80002514b40        0        0
fffff80002513e10 stream      0      0        0 fffff80002514960        0        0
fffff80002514960 stream      0      0        0 fffff80002513e10        0        0
fffff80002514e10 stream      0      0        0 fffff80002514a50        0        0
fffff80002514a50 stream      0      0        0 fffff80002514e10        0        0
fffff800025160f0 stream      0      0        0 fffff800025145a0        0        0
fffff800025145a0 stream      0      0        0 fffff800025160f0        0        0
fffff800025141e0 stream      0      0        0 fffff80002514870        0        0
fffff80002514870 stream      0      0        0 fffff800025141e0        0        0
fffff80002516b40 stream      0      0 fffff80002553ce8        0        0        0 /var/run/devd.pipe
fffff800025162d0 dgram       0      0        0 fffff80002516870        0 fffff800025163c0
fffff800025163c0 dgram       0      0        0 fffff80002516870        0 fffff800025165a0
fffff800025164b0 dgram       0      0        0 fffff80002516960        0        0
fffff800025165a0 dgram       0      0        0 fffff80002516870        0 fffff80002516690
fffff80002516690 dgram       0      0        0 fffff80002516870        0 fffff80002516780
fffff80002516780 dgram       0      0        0 fffff80002516870        0        0
fffff80002516870 dgram       0      0 fffff8000272b938        0 fffff800025162d0        0 /var/run/logpriv
fffff80002516960 dgram       0      0 fffff800025d4760        0 fffff800025164b0        0 /var/run/log
fffff80002516a50 seqpac      0      0 fffff80002553b10        0        0        0 /var/run/devd.seqpacket.pipe

------------------------------------------------------------------------
netstat -aL

Current listen queue sizes (qlen/incqlen/maxqlen)
Proto Listen         Local Address         
tcp4  0/0/10         localhost.smtp         
tcp4  0/0/128        *.https                
tcp6  0/0/128        *.https                
tcp4  0/0/128        *.http                 
tcp6  0/0/128        *.http                 
tcp4  0/0/128        *.ssh                  
tcp6  0/0/128        *.ssh                  
unix  0/0/5          /var/run/relayd.sock
unix  0/0/4          /var/run/devd.pipe
unix  0/0/4          /var/run/devd.seqpacket.pipe

------------------------------------------------------------------------
fstat

fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 5 at 0x2400000000
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read file 8 at 0x4000000001fffff
fstat: can't read file 10 at 0x780000ffff
fstat: can't read file 11 at 0x400000000
fstat: can't read file 13 at 0x200007fffffffff
fstat: can't read file 14 at 0x4000000001fffff
fstat: can't read file 16 at 0x780000ffff
fstat: can't read file 17 at 0x400000000
fstat: can't read file 19 at 0x200007fffffffff
fstat: can't read file 20 at 0x4000000001fffff
fstat: can't read file 22 at 0x780000ffff
fstat: can't read file 23 at 0x2a00000001
fstat: can't read file 29 at 0x800000000
fstat: can't read file 35 at 0x400000000
fstat: can't read file 41 at 0x400000000
fstat: can't read file 47 at 0x400000000
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 5 at 0x2400000000
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read file 8 at 0x4000000001fffff
fstat: can't read file 10 at 0x780000ffff
fstat: can't read file 11 at 0x400000000
fstat: can't read file 13 at 0x200007fffffffff
fstat: can't read file 14 at 0x4000000001fffff
fstat: can't read file 16 at 0x780000ffff
fstat: can't read file 17 at 0x400000000
fstat: can't read file 19 at 0x200007fffffffff
fstat: can't read file 20 at 0x4000000001fffff
fstat: can't read file 22 at 0x780000ffff
fstat: can't read file 23 at 0x2a00000001
fstat: can't read file 29 at 0x800000000
fstat: can't read file 35 at 0x400000000
fstat: can't read file 41 at 0x400000000
fstat: can't read file 47 at 0x400000000
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 5 at 0x2400000000
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read file 8 at 0x4000000001fffff
fstat: can't read file 10 at 0x780000ffff
fstat: can't read file 11 at 0x400000000
fstat: can't read file 13 at 0x200007fffffffff
fstat: can't read file 14 at 0x4000000001fffff
fstat: can't read file 16 at 0x780000ffff
fstat: can't read file 17 at 0x400000000
fstat: can't read file 19 at 0x200007fffffffff
fstat: can't read file 20 at 0x4000000001fffff
fstat: can't read file 22 at 0x780000ffff
fstat: can't read file 23 at 0x2a00000001
fstat: can't read file 29 at 0x800000000
fstat: can't read file 35 at 0x400000000
fstat: can't read file 41 at 0x400000000
fstat: can't read file 47 at 0x400000000
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 5 at 0x2400000000
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read file 8 at 0x4000000001fffff
fstat: can't read file 10 at 0x780000ffff
fstat: can't read file 11 at 0x400000000
fstat: can't read file 13 at 0x200007fffffffff
fstat: can't read file 14 at 0x4000000001fffff
fstat: can't read file 16 at 0x780000ffff
fstat: can't read file 17 at 0x400000000
fstat: can't read file 19 at 0x200007fffffffff
fstat: can't read file 20 at 0x4000000001fffff
fstat: can't read file 22 at 0x780000ffff
fstat: can't read file 23 at 0x2a00000001
fstat: can't read file 29 at 0x800000000
fstat: can't read file 35 at 0x400000000
fstat: can't read file 41 at 0x400000000
fstat: can't read file 47 at 0x400000000
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 5 at 0x2400000000
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read file 8 at 0x4000000001fffff
fstat: can't read file 10 at 0x780000ffff
fstat: can't read file 11 at 0x400000000
fstat: can't read file 13 at 0x200007fffffffff
fstat: can't read file 14 at 0x4000000001fffff
fstat: can't read file 16 at 0x780000ffff
fstat: can't read file 17 at 0x400000000
fstat: can't read file 19 at 0x200007fffffffff
fstat: can't read file 20 at 0x4000000001fffff
fstat: can't read file 22 at 0x780000ffff
fstat: can't read file 23 at 0x2a00000001
fstat: can't read file 29 at 0x800000000
fstat: can't read file 35 at 0x400000000
fstat: can't read file 41 at 0x400000000
fstat: can't read file 47 at 0x400000000
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 5 at 0x2400000000
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read file 8 at 0x4000000001fffff
fstat: can't read file 10 at 0x780000ffff
fstat: can't read file 11 at 0x400000000
fstat: can't read file 13 at 0x200007fffffffff
fstat: can't read file 14 at 0x4000000001fffff
fstat: can't read file 16 at 0x780000ffff
fstat: can't read file 17 at 0x400000000
fstat: can't read file 19 at 0x200007fffffffff
fstat: can't read file 20 at 0x4000000001fffff
fstat: can't read file 22 at 0x780000ffff
fstat: can't read file 23 at 0x2a00000001
fstat: can't read file 29 at 0x800000000
fstat: can't read file 35 at 0x400000000
fstat: can't read file 41 at 0x400000000
fstat: can't read file 47 at 0x400000000
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 5 at 0x2400000000
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read file 8 at 0x4000000001fffff
fstat: can't read file 10 at 0x780000ffff
fstat: can't read file 11 at 0x400000000
fstat: can't read file 13 at 0x200007fffffffff
fstat: can't read file 14 at 0x4000000001fffff
fstat: can't read file 16 at 0x780000ffff
fstat: can't read file 17 at 0x400000000
fstat: can't read file 19 at 0x200007fffffffff
fstat: can't read file 20 at 0x4000000001fffff
fstat: can't read file 22 at 0x780000ffff
fstat: can't read file 23 at 0x2a00000001
fstat: can't read file 29 at 0x800000000
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 5 at 0x2400000000
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read file 8 at 0x4000000001fffff
fstat: can't read file 10 at 0x780000ffff
fstat: can't read file 11 at 0x400000000
fstat: can't read file 13 at 0x200007fffffffff
fstat: can't read file 14 at 0x4000000001fffff
fstat: can't read file 16 at 0x780000ffff
fstat: can't read file 17 at 0x400000000
fstat: can't read file 19 at 0x200007fffffffff
fstat: can't read file 20 at 0x4000000001fffff
fstat: can't read file 22 at 0x780000ffff
fstat: can't read file 23 at 0x2a00000001
fstat: can't read file 25 at 0x200007fffffffff
fstat: can't read file 26 at 0x4000000001fffff
fstat: can't read file 28 at 0x780000ffff
fstat: can't read file 29 at 0x800000000
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 5 at 0x2400000000
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read file 8 at 0x4000000001fffff
fstat: can't read file 10 at 0x780000ffff
fstat: can't read file 11 at 0x400000000
fstat: can't read file 13 at 0x200007fffffffff
fstat: can't read file 14 at 0x4000000001fffff
fstat: can't read file 16 at 0x780000ffff
fstat: can't read file 17 at 0x400000000
fstat: can't read file 19 at 0x200007fffffffff
fstat: can't read file 20 at 0x4000000001fffff
fstat: can't read file 22 at 0x780000ffff
fstat: can't read file 23 at 0x2a00000001
fstat: can't read file 25 at 0x200007fffffffff
fstat: can't read file 26 at 0x4000000001fffff
fstat: can't read file 28 at 0x780000ffff
fstat: can't read file 29 at 0x600000000
fstat: can't read file 35 at 0xc00000000
fstat: can't read file 37 at 0x200007fffffffff
fstat: can't read file 38 at 0x4000000001fffff
fstat: can't read file 40 at 0x780000ffff
fstat: can't read file 41 at 0x200000000
fstat: can't read file 47 at 0x800000000
fstat: can't read file 49 at 0x200007fffffffff
fstat: can't read file 50 at 0x4000000001fffff
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 5 at 0x2200000000
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read file 8 at 0x4000000001fffff
fstat: can't read file 10 at 0x780000ffff
fstat: can't read file 11 at 0x200000000
fstat: can't read file 13 at 0x200007fffffffff
fstat: can't read file 14 at 0x4000000001fffff
fstat: can't read file 16 at 0x780000ffff
fstat: can't read file 17 at 0x200000000
fstat: can't read file 23 at 0x15c00000000
fstat: can't read file 29 at 0x800000000
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 5 at 0xa00000000
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read file 8 at 0x4000000001fffff
fstat: can't read file 10 at 0x780000ffff
fstat: can't read file 11 at 0xa00000000
fstat: can't read file 13 at 0x200007fffffffff
fstat: can't read file 14 at 0x4000000001fffff
fstat: can't read file 16 at 0x780000ffff
fstat: can't read file 17 at 0x800000000
fstat: can't read file 23 at 0x1f400000000
fstat: can't read file 29 at 0x7000000000
fstat: can't read file 35 at 0x1a00000000
fstat: can't read file 41 at 0x4800000000
fstat: can't read file 47 at 0x1c00000000
fstat: can't read file 53 at 0x4400000000
fstat: can't read file 59 at 0x3400000000
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 5 at 0x800000000
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read file 8 at 0x4000000001fffff
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 5 at 0x400000000
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read file 8 at 0x4000000001fffff
fstat: can't read file 10 at 0x780000ffff
fstat: can't read file 11 at 0x800000000
fstat: can't read file 13 at 0x200007fffffffff
fstat: can't read file 14 at 0x4000000001fffff
fstat: can't read file 16 at 0x780000ffff
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 5 at 0x600000000
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 5 at 0x400000000
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read file 8 at 0x4000000001fffff
fstat: can't read file 10 at 0x780000ffff
fstat: can't read file 11 at 0x800000000
fstat: can't read file 13 at 0x200007fffffffff
fstat: can't read file 14 at 0x4000000001fffff
fstat: can't read file 16 at 0x780000ffff
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 5 at 0x400000000
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read file 8 at 0x4000000001fffff
fstat: can't read file 10 at 0x780000ffff
fstat: can't read file 11 at 0x800000000
fstat: can't read file 13 at 0x200007fffffffff
fstat: can't read file 14 at 0x4000000001fffff
fstat: can't read file 16 at 0x780000ffff
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 5 at 0x400000000
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read file 8 at 0x4000000001fffff
fstat: can't read file 10 at 0x780000ffff
fstat: can't read file 11 at 0x800000000
fstat: can't read file 13 at 0x200007fffffffff
fstat: can't read file 14 at 0x4000000001fffff
fstat: can't read file 16 at 0x780000ffff
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 5 at 0x400000000
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read file 8 at 0x4000000001fffff
fstat: can't read file 10 at 0x780000ffff
fstat: can't read file 11 at 0x800000000
fstat: can't read file 13 at 0x200007fffffffff
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 5 at 0x400000000
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 5 at 0x400000000
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read file 8 at 0x4000000001fffff
fstat: can't read file 10 at 0x780000ffff
fstat: can't read file 11 at 0x400000000
fstat: can't read file 13 at 0x200007fffffffff
fstat: can't read file 14 at 0x4000000001fffff
fstat: can't read file 16 at 0x780000ffff
fstat: can't read file 17 at 0x400000000
fstat: can't read file 19 at 0x200007fffffffff
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 5 at 0x800000000
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
USER     CMD          PID   FD MOUNT      INUM MODE         SZ|DV R/W
_relayd  relayd     82345 root /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82345   wd /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82345 jail /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82345 text /        403512 -r-xr-xr-x  228360  r
_relayd  relayd     82345    0 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82345    6 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82345   12 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82345   18* local dgram fffff800025162d0 <-> fffff80002516870
_relayd  relayd     82344 root /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82344   wd /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82344 jail /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82344 text /        403512 -r-xr-xr-x  228360  r
_relayd  relayd     82344    0 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82344    6 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82344   12 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82344   18* local dgram fffff800025162d0 <-> fffff80002516870
_relayd  relayd     82343 root /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82343   wd /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82343 jail /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82343 text /        403512 -r-xr-xr-x  228360  r
_relayd  relayd     82343    0 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82343    6 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82343   12 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82343   18* local dgram fffff800025162d0 <-> fffff80002516870
_relayd  relayd     82342 root /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82342   wd /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82342 jail /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82342 text /        403512 -r-xr-xr-x  228360  r
_relayd  relayd     82342    0 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82342    6 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82342   12 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82342   18* local dgram fffff800025162d0 <-> fffff80002516870
_relayd  relayd     82341 root /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82341   wd /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82341 jail /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82341 text /        403512 -r-xr-xr-x  228360  r
_relayd  relayd     82341    0 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82341    6 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82341   12 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82341   18* local dgram fffff800025162d0 <-> fffff80002516870
_relayd  relayd     82340 root /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82340   wd /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82340 jail /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82340 text /        403512 -r-xr-xr-x  228360  r
_relayd  relayd     82340    0 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82340    6 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82340   12 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82340   18* local dgram fffff800025162d0 <-> fffff80002516870
_relayd  relayd     82339 root /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82339   wd /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82339 jail /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82339 text /        403512 -r-xr-xr-x  228360  r
_relayd  relayd     82339    0 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82339    6 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82339   12 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82339   18* local dgram fffff800025162d0 <-> fffff80002516870
_relayd  relayd     82338 root /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82338   wd /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82338 jail /        963089 dr-xr-xr-x     512  r
_relayd  relayd     82338 text /        403512 -r-xr-xr-x  228360  r
_relayd  relayd     82338    0 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82338    6 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82338   12 /dev         13 crw-rw-rw-    null rw
_relayd  relayd     82338   18* local dgram fffff800025162d0 <-> fffff80002516870
_relayd  relayd     82338   24 /dev         73 crw-------      pf rw
root     relayd     82337 root /             2 drwxr-xr-x    1024  r
root     relayd     82337   wd /             2 drwxr-xr-x    1024  r
root     relayd     82337 text /        403512 -r-xr-xr-x  228360  r
root     relayd     82337    0 /dev         13 crw-rw-rw-    null rw
root     relayd     82337    6 /dev         13 crw-rw-rw-    null rw
root     relayd     82337   12 /dev         13 crw-rw-rw-    null rw
root     relayd     82337   18* local dgram fffff800025162d0 <-> fffff80002516870
root     relayd     82337   24* local stream fffff80002514870 <-> fffff800025141e0
root     relayd     82337   36* local stream fffff800025145a0 <-> fffff800025160f0
root     relayd     82337   48* local stream fffff80002514a50 <-> fffff80002514e10
root     bash       82265 root /             2 drwxr-xr-x    1024  r
root     bash       82265   wd /        642048 drwxr-xr-x     512  r
root     bash       82265 text /        429297 -rwxr-xr-x  869232  r
root     bash       82265 ctty /dev         41 crw-------   ttyv0 rw
root     bash       82265    0 /dev         41 crw-------   ttyv0 rw
root     bash       82265    6 /dev         41 crw-------   ttyv0 rw
root     bash       82265   12 /dev         41 crw-------   ttyv0 rw
root     bash         666 root /             2 drwxr-xr-x    1024  r
root     bash         666   wd /        642048 drwxr-xr-x     512  r
root     bash         666 text /        429297 -rwxr-xr-x  869232  r
root     bash         666 ctty /dev         81 crw--w----   pts/0 rw
root     bash         666    0 /dev         81 crw--w----   pts/0 rw
root     bash         666    6 /dev         81 crw--w----   pts/0 rw
root     bash         666   12 /dev         81 crw--w----   pts/0 rw
root     sshd         663 root /             2 drwxr-xr-x    1024  r
root     sshd         663   wd /             2 drwxr-xr-x    1024  r
root     sshd         663 text /        390240 -r-xr-xr-x  297792  r
root     sshd         663    0 /dev         13 crw-rw-rw-    null rw
root     sshd         663    6 /dev         13 crw-rw-rw-    null rw
www      httpd        661 root /             2 drwxr-xr-x    1024  r
www      httpd        661   wd /             2 drwxr-xr-x    1024  r
www      httpd        661 text /        418044 -rwxr-xr-x  579920  r
www      httpd        661    0 /dev         13 crw-rw-rw-    null  r
www      httpd        661    6 /dev         13 crw-rw-rw-    null  w
www      httpd        661   12 /        963125 -rw-r--r--    1453  w
root     getty        656 root /             2 drwxr-xr-x    1024  r
root     getty        656   wd /             2 drwxr-xr-x    1024  r
root     getty        656 text /        242199 -r-xr-xr-x   28360  r
root     getty        656 ctty /dev         48 crw-------   ttyv7 rw
root     getty        656    0 /dev         48 crw-------   ttyv7 rw
root     getty        655 root /             2 drwxr-xr-x    1024  r
root     getty        655   wd /             2 drwxr-xr-x    1024  r
root     getty        655 text /        242199 -r-xr-xr-x   28360  r
root     getty        655 ctty /dev         47 crw-------   ttyv6 rw
root     getty        655    0 /dev         47 crw-------   ttyv6 rw
root     getty        654 root /             2 drwxr-xr-x    1024  r
root     getty        654   wd /             2 drwxr-xr-x    1024  r
root     getty        654 text /        242199 -r-xr-xr-x   28360  r
root     getty        654 ctty /dev         46 crw-------   ttyv5 rw
root     getty        654    0 /dev         46 crw-------   ttyv5 rw
root     getty        653 root /             2 drwxr-xr-x    1024  r
root     getty        653   wd /             2 drwxr-xr-x    1024  r
root     getty        653 text /        242199 -r-xr-xr-x   28360  r
root     getty        653 ctty /dev         45 crw-------   ttyv4 rw
root     getty        653    0 /dev         45 crw-------   ttyv4 rw
root     getty        652 root /             2 drwxr-xr-x    1024  r
root     getty        652   wd /             2 drwxr-xr-x    1024  r
root     getty        652 text /        242199 -r-xr-xr-x   28360  r
root     getty        652 ctty /dev         44 crw-------   ttyv3 rw
root     getty        652    0 /dev         44 crw-------   ttyv3 rw
root     getty        651 root /             2 drwxr-xr-x    1024  r
root     getty        651   wd /             2 drwxr-xr-x    1024  r
root     getty        651 text /        242199 -r-xr-xr-x   28360  r
root     getty        651 ctty /dev         43 crw-------   ttyv2 rw
root     getty        651    0 /dev         43 crw-------   ttyv2 rw
root     getty        650 root /             2 drwxr-xr-x    1024  r
root     getty        650   wd /             2 drwxr-xr-x    1024  r
root     getty        650 text /        242199 -r-xr-xr-x   28360  r
root     getty        650 ctty /dev         42 crw-------   ttyv1 rw
root     getty        650    0 /dev         42 crw-------   ttyv1 rw
root     login        649 root /             2 drwxr-xr-x    1024  r
root     login        649   wd /        642048 drwxr-xr-x     512  r
root     login        649 text /        388667 -r-sr-xr-x   25896  r
root     login        649 ctty /dev         41 crw-------   ttyv0 rw
root     login        649    0 /dev         41 crw-------   ttyv0 rw
root     cron         603 root /             2 drwxr-xr-x    1024  r
root     cron         603   wd /        963093 drwxr-x---     512  r
root     cron         603 text /        243094 -r-xr-xr-x   41352  r
root     cron         603    0 /dev         13 crw-rw-rw-    null rw
smmsp    sendmail     599 root /             2 drwxr-xr-x    1024  r
smmsp    sendmail     599   wd /        963112 drwxrwx---     512  r
smmsp    sendmail     599 text /        390104 -r-xr-sr-x  692520  r
smmsp    sendmail     599    0 /dev         13 crw-rw-rw-    null  r
root     sendmail     596 root /             2 drwxr-xr-x    1024  r
root     sendmail     596   wd /        963111 drwxr-xr-x     512  r
root     sendmail     596 text /        390104 -r-xr-sr-x  692520  r
root     sendmail     596    0 /dev         13 crw-rw-rw-    null  r
www      httpd        590 root /             2 drwxr-xr-x    1024  r
www      httpd        590   wd /             2 drwxr-xr-x    1024  r
www      httpd        590 text /        418044 -rwxr-xr-x  579920  r
www      httpd        590    0 /dev         13 crw-rw-rw-    null  r
www      httpd        590    6 /dev         13 crw-rw-rw-    null  w
www      httpd        590   12 /        963125 -rw-r--r--    1453  w
www      httpd        589 root /             2 drwxr-xr-x    1024  r
www      httpd        589   wd /             2 drwxr-xr-x    1024  r
www      httpd        589 text /        418044 -rwxr-xr-x  579920  r
www      httpd        589    0 /dev         13 crw-rw-rw-    null  r
www      httpd        589    6 /dev         13 crw-rw-rw-    null  w
www      httpd        589   12 /        963125 -rw-r--r--    1453  w
www      httpd        588 root /             2 drwxr-xr-x    1024  r
www      httpd        588   wd /             2 drwxr-xr-x    1024  r
www      httpd        588 text /        418044 -rwxr-xr-x  579920  r
www      httpd        588    0 /dev         13 crw-rw-rw-    null  r
www      httpd        588    6 /dev         13 crw-rw-rw-    null  w
www      httpd        588   12 /        963125 -rw-r--r--    1453  w
root     httpd        581 root /             2 drwxr-xr-x    1024  r
root     httpd        581   wd /             2 drwxr-xr-x    1024  r
root     httpd        581 text /        418044 -rwxr-xr-x  579920  r
root     httpd        581    0 /dev         13 crw-rw-rw-    null  r
root     httpd        581    6 /dev         13 crw-rw-rw-    null  w
root     httpd        581   12 /        963125 -rw-r--r--    1453  w
root     sshd         576 root /             2 drwxr-xr-x    1024  r
root     sshd         576   wd /             2 drwxr-xr-x    1024  r
root     sshd         576 text /        390240 -r-xr-xr-x  297792  r
root     sshd         576    0 /dev         13 crw-rw-rw-    null rw
root     racoon       556 root /             2 drwxr-xr-x    1024  r
root     racoon       556   wd /             2 drwxr-xr-x    1024  r
root     racoon       556 text /        431086 -r-xr-xr-x  1186082  r
root     racoon       556    0 /dev         13 crw-rw-rw-    null rw
root     racoon       556    6 /dev         13 crw-rw-rw-    null rw
root     syslogd      390 root /             2 drwxr-xr-x    1024  r
root     syslogd      390   wd /             2 drwxr-xr-x    1024  r
root     syslogd      390 text /        390241 -r-xr-xr-x   40352  r
root     syslogd      390    0 /dev         13 crw-rw-rw-    null rw
root     syslogd      390    6 /dev         13 crw-rw-rw-    null rw
root     syslogd      390   12 /dev         13 crw-rw-rw-    null rw
root     syslogd      390   18 /        963139 -rw-------       3  w
root     pf purge     310 root /             2 drwxr-xr-x    1024  r
root     pf purge     310   wd /             2 drwxr-xr-x    1024  r
root     devd         301 root /             2 drwxr-xr-x    1024  r
root     devd         301   wd /             2 drwxr-xr-x    1024  r
root     devd         301 text /        815951 -r-xr-xr-x  634824  r
root     devd         301    0 /dev         13 crw-rw-rw-    null rw
root     devd         301    6 /dev         13 crw-rw-rw-    null rw
root     adjkerntz    130 root /             2 drwxr-xr-x    1024  r
root     adjkerntz    130   wd /             2 drwxr-xr-x    1024  r
root     adjkerntz    130 text /        815941 -r-xr-xr-x    9280  r
root     adjkerntz    130    0 /dev         13 crw-rw-rw-    null rw
root     init           1 root /             2 drwxr-xr-x    1024  r
root     init           1   wd /             2 drwxr-xr-x    1024  r
root     init           1 text /        815978 -r-xr-xr-x  981064  r
root     kernel         0 root /             2 drwxr-xr-x    1024  r
root     kernel         0   wd /             2 drwxr-xr-x    1024  r

------------------------------------------------------------------------
dmesg

Copyright (c) 1992-2014 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 10.1-RELEASE #0 r274786: Sat Nov 15 10:51:07 CET 2014
    root@bsd1:/usr/obj/usr/src_10.1.0/sys/IPSEC amd64
FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512
CPU: Intel(R) Core(TM) i3-2120 CPU @ 3.30GHz (3279.50-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x206a7  Family = 0x6  Model = 0x2a  Stepping = 7
  Features=0x783fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2>
  Features2=0x209<SSE3,MON,SSSE3>
  AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
real memory  = 268369920 (255 MB)
avail memory = 230010880 (219 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <VBOX   VBOXAPIC>
ioapic0 <Version 1.1> irqs 0-23 on motherboard
random: <Software, Yarrow> initialized
kbd1 at kbdmux0
cryptosoft0: <software crypto> on motherboard
acpi0: <VBOX VBOXXSDT> on motherboard
acpi0: Power Button (fixed)
acpi0: Sleep Button (fixed)
cpu0: <ACPI CPU> on acpi0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <32-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
isab0: <PCI-ISA bridge> at device 1.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel PIIX4 UDMA33 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xd000-0xd00f at device 1.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ata1: <ATA channel> at channel 1 on atapci0
vgapci0: <VGA-compatible display> mem 0xe0000000-0xe0ffffff irq 18 at device 2.0 on pci0
vgapci0: Boot video device
em0: <Intel(R) PRO/1000 Legacy Network Connection 1.0.6> port 0xd010-0xd017 mem 0xf0000000-0xf001ffff irq 19 at device 3.0 on pci0
em0: Ethernet address: 08:00:27:fd:1b:07
pcm0: <Intel ICH (82801AA)> port 0xd100-0xd1ff,0xd200-0xd23f irq 21 at device 5.0 on pci0
pcm0: <SigmaTel STAC9700/83/84 AC97 Codec>
ohci0: <OHCI (generic) USB controller> mem 0xf0804000-0xf0804fff irq 22 at device 6.0 on pci0
usbus0 on ohci0
pci0: <bridge> at device 7.0 (no driver attached)
acpi_acad0: <AC Adapter> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model IntelliMouse Explorer, device ID 4
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xe2000-0xeefff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
atrtc0: <AT realtime clock> at port 0x70 irq 8 on isa0
Event timer "RTC" frequency 32768 Hz quality 0
ppc0: cannot reserve I/O port range
Timecounters tick every 10.000 msec
IPsec: Initialized Security Association Processing.
pcm0: measured ac97 link rate at 58314 Hz
usbus0: 12Mbps Full Speed USB v1.0
ugen0.1: <Apple> at usbus0
uhub0: <Apple OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ada0 at ata0 bus 0 scbus0 target 0 lun 0
ada0: <VBOX HARDDISK 1.0> ATA-6 device
ada0: Serial Number VB788bbdf1-83f7f0bc
ada0: 33.300MB/s transferscd0 at ata1 bus 0 scbus1 target 0 lun 0
cd0: <VBOX CD-ROM 1.0> Removable CD-ROM SCSI-0 device 
cd0: Serial Number VB2-01700376
cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present
 (UDMA2, PIO 65536bytes)
ada0: 10240MB (20971520 512 byte sectors: 16H 63S/T 16383C)
ada0: Previously was known as ad0
random: unblocking device.
Timecounter "TSC-low" frequency 1639748116 Hz quality 800
Root mount waiting for: usbus0
uhub0: 8 ports with 8 removable, self powered
Trying to mount root from ufs:/dev/ada0p2 [rw]...
Setting hostuuid: da156d6a-fcdb-4909-8bfe-ae7ea8298a04.
Setting hostid: 0xc4ef3a87.
Entropy harvesting: interrupts ethernet point_to_point swi.
Starting file system checks:
/dev/ada0p2: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/ada0p2: clean, 893672 free (18984 frags, 109336 blocks, 0.8% fragmentation)
Mounting local file systems:.
Writing entropy file:.
Setting hostname: bsd1.
/etc/rc: WARNING: $ipv6_enable is obsolete.  Use $ipv6_activate_all_interfaces instead.
/etc/rc: WARNING: $ipv6_enable is obsolete.  Use $ipv6_activate_all_interfaces instead.
Starting Network: lo0 em0.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128 
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 
	inet 127.0.0.1 netmask 0xff000000 
	nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
	ether 08:00:27:fd:1b:07
	inet 172.23.30.104 netmask 0xffff0000 broadcast 172.23.255.255 
	inet6 fe80::a00:27ff:fefd:1b07%em0 prefixlen 64 scopeid 0x1 
	inet 172.23.30.110 netmask 0xffff0000 broadcast 172.23.255.255 vhid 17 
	nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	carp: BACKUP vhid 17 advbase 1 advskew 0
Starting devd.
Installing ipsec manual keys/policies.
Enabling pfNo ALTQ support in kernel
ALTQ related functions disabled
No ALTQ support in kernel
ALTQ related functions disabled
.
add net default: gateway 172.23.254.254
add net fe80::: gateway ::1
add net ff02::: gateway ::1
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib
32-bit compatibility ldconfig path: /usr/lib32
Creating and/or trimming log files.
Starting syslogd.
No core dumps found.
Clearing /tmp (X related).
Updating motd:.
Mounting late file systems:.
Starting racoon.
carp: VHID 17@em0: BACKUP -> MASTER (master down)
Performing sanity check on sshd configuration.
Starting sshd.
Performing sanity check on apache24 configuration:
AH00557: httpd: apr_sockaddr_info_get() failed for bsd1
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1. Set the 'ServerName' directive globally to suppress this message
Syntax OK
Starting apache24.
AH00557: httpd: apr_sockaddr_info_get() failed for bsd1
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1. Set the 'ServerName' directive globally to suppress this message
Configuring syscons: keymap blanktime.
Starting sendmail_submit.
Starting sendmail_msp_queue.
Starting cron.
Starting background file system checks in 60 seconds.

Fri Nov 21 16:36:50 CET 2014
carp: demoted by 240 to 240 (sysctl)
carp: VHID 17@em0: MASTER -> BACKUP (more frequent advertisement received)
arp: 172.23.30.110 moved from 08:00:27:eb:46:99 to 00:00:5e:00:01:11 on em0
carp: demoted by -240 to 0 (sysctl)
carp: VHID 17@em0: BACKUP -> MASTER (preempting a slower master)
arp: 172.23.30.110 moved from 00:00:5e:00:01:11 to 08:00:27:eb:46:99 on em0
carp: VHID 18@em0: INIT -> BACKUP
carp: VHID 18@em0: BACKUP -> MASTER (master down)
carp: VHID 18@em0: MASTER -> BACKUP (more frequent advertisement received)
carp: demoted by 240 to 240 (sysctl)
carp: VHID 17@em0: MASTER -> BACKUP (more frequent advertisement received)
arp: 172.23.30.110 moved from 08:00:27:eb:46:99 to 00:00:5e:00:01:11 on em0
carp: demoted by -240 to 0 (sysctl)
carp: VHID 17@em0: BACKUP -> MASTER (preempting a slower master)
arp: 172.23.30.110 moved from 00:00:5e:00:01:11 to 08:00:27:eb:46:99 on em0
carp: VHID 18@em0: BACKUP -> MASTER (master down)
carp: demoted by 240 to 240 (interface down)
carp: demoted by 240 to 480 (interface down)
em0: link state changed to DOWN
Nov 21 17:19:47 bsd1 login: ROOT LOGIN (root) ON ttyv0
carp: VHID 17@em0: INIT -> BACKUP
carp: demoted by -240 to 240 (interface up)
carp: VHID 18@em0: INIT -> BACKUP
carp: demoted by -240 to 0 (interface up)
em0: link state changed to UP
carp: VHID 17@em0: BACKUP -> MASTER (master down)
ifa_add_loopback_route: insertion failed: 17
carp: VHID 18@em0: BACKUP -> MASTER (master down)
ifa_add_loopback_route: insertion failed: 17
Nov 21 17:23:01 bsd1 relayd[82300]: /usr/local/etc/relayd.conf:21: syntax error
Nov 21 17:23:01 bsd1 relayd[82300]: no actions, nothing to do
Nov 21 17:23:30 bsd1 relayd[82318]: fatal: cannot add rule: Operation not supported by device
Nov 21 17:23:30 bsd1 relayd[82317]: lost child: pfe exited abnormally


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address	= 0x4a4
fault code		= supervisor write data, page not present
instruction pointer	= 0x20:0xffffffff81a37954
stack pointer	        = 0x28:0xfffffe001b139c30
frame pointer	        = 0x28:0xfffffe001b139c40
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 82338 (relayd)
trap number		= 12
panic: page fault
cpuid = 0
KDB: stack backtrace:
#0 0xffffffff8096b580 at kdb_backtrace+0x60
#1 0xffffffff809306a5 at panic+0x155
#2 0xffffffff80d4ebef at trap_fatal+0x38f
#3 0xffffffff80d4ef08 at trap_pfault+0x308
#4 0xffffffff80d4e56a at trap+0x47a
#5 0xffffffff80d34452 at calltrap+0x8
#6 0xffffffff81a25784 at pf_empty_pool+0x44
#7 0xffffffff81a27bb2 at pfioctl+0x7e2
#8 0xffffffff80820154 at devfs_ioctl_f+0x114
#9 0xffffffff80983035 at kern_ioctl+0x255
#10 0xffffffff80982d8c at sys_ioctl+0x13c
#11 0xffffffff80d4f521 at amd64_syscall+0x351
#12 0xffffffff80d3473b at Xfast_syscall+0xfb
Uptime: 50m53s
Dumping 83 out of 231 MB:..20%..39%..58%..77%..96%

------------------------------------------------------------------------
kernel config

options	CONFIG_AUTOGENERATED
ident	IPSEC_KERNEL
machine	amd64
cpu	HAMMER
makeoptions	WITH_CTF=1
makeoptions	DEBUG=-g
options	IPSEC_DEBUG
options	IPSEC
options	XENHVM
options	USB_DEBUG
options	ATH_ENABLE_11N
options	AH_AR5416_INTERRUPT_MITIGATION
options	AH_SUPPORT_AR5416
options	IEEE80211_SUPPORT_MESH
options	IEEE80211_AMPDU_AGE
options	IEEE80211_DEBUG
options	SC_PIXEL_MODE
options	VESA
options	AHD_REG_PRETTY_PRINT
options	AHC_REG_PRETTY_PRINT
options	ATA_STATIC_ID
options	ACPI_DMAR
options	SMP
options	KDB_TRACE
options	KDB
options	INCLUDE_CONFIG_FILE
options	DDB_CTF
options	KDTRACE_HOOKS
options	KDTRACE_FRAME
options	MAC
options	PROCDESC
options	CAPABILITIES
options	CAPABILITY_MODE
options	AUDIT
options	HWPMC_HOOKS
options	KBD_INSTALL_CDEV
options	PRINTF_BUFR_SIZE=128
options	_KPOSIX_PRIORITY_SCHEDULING
options	SYSVSEM
options	SYSVMSG
options	SYSVSHM
options	STACK
options	KTRACE
options	SCSI_DELAY=5000
options	COMPAT_FREEBSD7
options	COMPAT_FREEBSD6
options	COMPAT_FREEBSD5
options	COMPAT_FREEBSD4
options	COMPAT_FREEBSD32
options	GEOM_LABEL
options	GEOM_RAID
options	GEOM_PART_GPT
options	PSEUDOFS
options	PROCFS
options	CD9660
options	MSDOSFS
options	NFS_ROOT
options	NFSLOCKD
options	NFSD
options	NFSCL
options	MD_ROOT
options	QUOTA
options	UFS_GJOURNAL
options	UFS_DIRHASH
options	UFS_ACL
options	SOFTUPDATES
options	FFS
options	SCTP
options	TCP_OFFLOAD
options	INET6
options	INET
options	PREEMPTION
options	SCHED_ULE
options	NEW_PCIB
options	GEOM_PART_MBR
options	GEOM_PART_EBR_COMPAT
options	GEOM_PART_EBR
options	GEOM_PART_BSD
device	isa
device	mem
device	io
device	uart_ns8250
device	cpufreq
device	acpi
device	pci
device	fdc
device	ahci
device	ata
device	mvs
device	siis
device	ahc
device	ahd
device	esp
device	hptiop
device	isp
device	mpt
device	mps
device	mpr
device	sym
device	trm
device	adv
device	adw
device	aic
device	bt
device	isci
device	scbus
device	ch
device	da
device	sa
device	cd
device	pass
device	ses
device	amr
device	arcmsr
device	ciss
device	dpt
device	hptmv
device	hptnr
device	hptrr
device	hpt27xx
device	iir
device	ips
device	mly
device	twa
device	tws
device	aac
device	aacp
device	aacraid
device	ida
device	mfi
device	mlx
device	mrsas
device	twe
device	atkbdc
device	atkbd
device	psm
device	kbdmux
device	vga
device	splash
device	sc
device	vt
device	vt_vga
device	vt_efifb
device	agp
device	cbb
device	pccard
device	cardbus
device	uart
device	ppc
device	ppbus
device	lpt
device	ppi
device	puc
device	bxe
device	de
device	em
device	igb
device	ixgbe
device	ixl
device	ixlv
device	le
device	ti
device	txp
device	vx
device	miibus
device	ae
device	age
device	alc
device	ale
device	bce
device	bfe
device	bge
device	cas
device	dc
device	et
device	fxp
device	gem
device	hme
device	jme
device	lge
device	msk
device	nfe
device	nge
device	pcn
device	re
device	rl
device	sf
device	sge
device	sis
device	sk
device	ste
device	stge
device	tl
device	tx
device	vge
device	vr
device	wb
device	xl
device	cs
device	ed
device	ex
device	ep
device	fe
device	sn
device	xe
device	wlan
device	wlan_wep
device	wlan_ccmp
device	wlan_tkip
device	wlan_amrr
device	an
device	ath
device	ath_pci
device	ath_hal
device	ath_rate_sample
device	ipw
device	iwi
device	iwn
device	malo
device	mwl
device	ral
device	wi
device	wpi
device	loop
device	random
device	padlock_rng
device	rdrand_rng
device	ether
device	vlan
device	tun
device	md
device	gif
device	faith
device	firmware
device	bpf
device	uhci
device	ohci
device	ehci
device	xhci
device	usb
device	ukbd
device	umass
device	sound
device	snd_cmi
device	snd_csa
device	snd_emu10kx
device	snd_es137x
device	snd_hda
device	snd_ich
device	snd_via8233
device	mmc
device	mmcsd
device	sdhci
device	virtio
device	virtio_pci
device	vtnet
device	virtio_blk
device	virtio_scsi
device	virtio_balloon
device	hyperv
device	xenpci
device	vmx
device	crypto

------------------------------------------------------------------------
ddb capture buffer

ddb: ddb_capture: kvm_nlist

Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?894145A3DDBDEF4880E00D334DCD872618A8B2DE>