From owner-freebsd-x11@freebsd.org Tue Apr 9 14:49:23 2019 Return-Path: Delivered-To: freebsd-x11@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C8B581582C51 for ; Tue, 9 Apr 2019 14:49:22 +0000 (UTC) (envelope-from andersbo87@icloud.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 0284068079 for ; Tue, 9 Apr 2019 14:49:22 +0000 (UTC) (envelope-from andersbo87@icloud.com) Received: by mailman.ysv.freebsd.org (Postfix) id B731F1582C50; Tue, 9 Apr 2019 14:49:21 +0000 (UTC) Delivered-To: x11@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4A1541582C4F for ; Tue, 9 Apr 2019 14:49:21 +0000 (UTC) (envelope-from andersbo87@icloud.com) Received: from mr85p00im-hyfv06021301.me.com (mr85p00im-hyfv06021301.me.com [17.58.23.188]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EC26D68078 for ; Tue, 9 Apr 2019 14:49:19 +0000 (UTC) (envelope-from andersbo87@icloud.com) Received: from freebsd-hp.local (ti0025a400-6061.bb.online.no [85.167.194.192]) by mr85p00im-hyfv06021301.me.com (Postfix) with ESMTPSA id D8CE6401F8 for ; Tue, 9 Apr 2019 14:49:09 +0000 (UTC) From: Anders Bolt-Evensen To: x11@freebsd.org Subject: Kernel panic in /boot/modules/nvidia.ko Message-ID: <6122272f-4d73-18f9-ea0b-95e1e88682bf@icloud.com> Date: Tue, 9 Apr 2019 16:49:06 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-04-09_06:, , signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1812120000 definitions=main-1904090094 X-Rspamd-Queue-Id: EC26D68078 X-Spamd-Bar: --------- X-Spamd-Result: default: False [-9.45 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; RBL_COMPOSITE_RCVD_IN_DNSWL_MED_DWL_DNSWL_LOW(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:17.58.0.0/16]; FREEMAIL_FROM(0.00)[icloud.com]; TO_DN_NONE(0.00)[]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; MX_GOOD(-0.01)[cached: mx1.mail.icloud.com]; DKIM_TRACE(0.00)[icloud.com:+]; DMARC_POLICY_ALLOW(-0.50)[icloud.com,quarantine]; RCVD_IN_DNSWL_MED(-0.20)[188.23.58.17.list.dnswl.org : 127.0.5.2]; NEURAL_HAM_SHORT(-0.99)[-0.990,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[icloud.com]; ASN(0.00)[asn:714, ipnet:17.58.16.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[icloud.com:s=04042017]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[x11@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(0.00)[icloud.com.dwl.dnswl.org : 127.0.5.1]; IP_SCORE(-1.75)[ip: (-4.76), ipnet: 17.58.16.0/20(-1.83), asn: 714(-2.12), country: US(-0.06)]; WHITELIST_SPF_DKIM(-3.00)[icloud.com:d:+,icloud.com:s:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-x11@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: X11 on FreeBSD -- maintaining and support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Apr 2019 14:49:23 -0000 Hello! Since I've upgraded to 12.0, the system has occasionally panicked. After taking a look at the crash log, it seems like the nvidia kernel module is causing these panics. When the panic occurred, the system was idling, I had just completed an upgrade process of outdated ports (the NVIDIA ports were already "up-to-date" as far as the Ports collection is concerned - though the newest version available on the NVIDIA site is version 418.56). I am using version 390.77, which should be the newest version available in the Ports collection. Error message: Fatal trap 12: page fault while in kernel mode Here's the output of freebsd-version: 12.0-RELEASE-p3 Here's the output of uname -a: FreeBSD freebsd-hp.local 12.0-RELEASE-p3 FreeBSD 12.0-RELEASE-p3 GENERIC  amd64 The full content of /var/crash/core.txt.7 can be downloaded here: https://my.owndrive.com/index.php/s/KKyrwx2CgwKZPwg Here's the content of /var/crash/core.txt.7, up until the output of netstat -aL: freebsd-hp.local dumped core - see /var/crash/vmcore.7 Tue Apr  9 11:23:31 CEST 2019 FreeBSD freebsd-hp.local 12.0-RELEASE-p3 FreeBSD 12.0-RELEASE-p3 GENERIC  amd64 panic: page fault GNU gdb (GDB) 8.2.1 [GDB v8.2.1 for FreeBSD] Copyright (C) 2018 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-portbld-freebsd12.0". Type "show configuration" for configuration details. For bug reporting instructions, please see: . Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /boot/kernel/kernel...Reading symbols from /usr/lib/debug//boot/kernel/kernel.debug...done. done. Unread portion of the kernel message buffer: NVRM: GPU at PCI:0000:01:00: GPU-9efca1be-6c8a-1b17-2ae0-233df4817b45 NVRM: GPU Board Serial Number: PGBXA0D9V7407R NVRM: Xid (PCI:0000:01:00): 8, Channel 00000010 Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address    = 0x0 fault code        = supervisor read data, page not present instruction pointer    = 0x20:0xffffffff82e2b3a4 stack pointer            = 0x28:0xfffffe003fb4d5b0 frame pointer            = 0x28:0xfffffe004cc22c80 code segment        = base 0x0, limit 0xfffff, type 0x1b             = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags    = interrupt enabled, resume, IOPL = 0 current process        = 12 (swi4: clock (0)) trap number        = 12 panic: page fault cpuid = 0 time = 1554801177 KDB: stack backtrace: #0 0xffffffff80be7977 at kdb_backtrace+0x67 #1 0xffffffff80b9b563 at vpanic+0x1a3 #2 0xffffffff80b9b3b3 at panic+0x43 #3 0xffffffff8107496f at trap_fatal+0x35f #4 0xffffffff810749c9 at trap_pfault+0x49 #5 0xffffffff81073fee at trap+0x29e #6 0xffffffff8104f435 at calltrap+0x8 Uptime: 2h19m27s (ada0:ahcich0:0:0:0): spin-down (ada1:ahcich2:0:0:0): spin-down Dumping 1381 out of 8062 MB:..2%..11%..21%..31%..41%..51%..61%..71%..82%..91% __curthread () at ./machine/pcpu.h:230 230    ./machine/pcpu.h: No such file or directory. (kgdb) #0  __curthread () at ./machine/pcpu.h:230 #1  doadump (textdump=)     at /usr/src/sys/kern/kern_shutdown.c:366 #2  0xffffffff80b9b14b in kern_reboot (howto=260)     at /usr/src/sys/kern/kern_shutdown.c:446 #3  0xffffffff80b9b5c3 in vpanic (fmt=, ap=0xfffffe003fb4d300)     at /usr/src/sys/kern/kern_shutdown.c:872 #4  0xffffffff80b9b3b3 in panic (fmt=)     at /usr/src/sys/kern/kern_shutdown.c:799 #5  0xffffffff8107496f in trap_fatal (frame=0xfffffe003fb4d4f0, eva=0)     at /usr/src/sys/amd64/amd64/trap.c:929 #6  0xffffffff810749c9 in trap_pfault (frame=0xfffffe003fb4d4f0, usermode=0)     at /usr/src/sys/amd64/amd64/trap.c:765 #7  0xffffffff81073fee in trap (frame=0xfffffe003fb4d4f0)     at /usr/src/sys/amd64/amd64/trap.c:441 #8  #9  0xffffffff82e2b3a4 in _nv008794rm () from /boot/modules/nvidia.ko #10 0xfffff800535d4000 in ?? () #11 0xffffffff82e2b3b9 in _nv008790rm () from /boot/modules/nvidia.ko #12 0xfffff80077f79880 in ?? () #13 0xffffffff82e2b237 in _nv008791rm () from /boot/modules/nvidia.ko #14 0xfffffe004cc22c80 in ?? () #15 0xfffffe004cc22cf8 in ?? () #16 0x0000000000000000 in ?? () (kgdb) ------------------------------------------------------------------------ ps -axlww  UID   PID  PPID CPU PRI NI      VSZ    RSS MWCHAN   STAT TT TIME COMMAND    0     0     0   0 -16  0        0      0 swapin   DLs   - 0:00.02 [kernel]    0     1     0   0  20  0     9952    656 wait     DLs   - 0:00.00 [init]    0     2     0   0 -16  0        0      0 crypto_w DL    - 0:00.00 [crypto]    0     3     0   0 -16  0        0      0 crypto_r DL    - 0:00.00 [crypto returns 0]    0     4     0   0 -16  0        0      0 crypto_r DL    - 0:00.00 [crypto returns 1]    0     5     0   0 -16  0        0      0 crypto_r DL    - 0:00.00 [crypto returns 2]    0     6     0   0 -16  0        0      0 crypto_r DL    - 0:00.00 [crypto returns 3]    0     7     0   0 -16  0        0      0 -        RL    - 0:00.05 [cam]    0     8     0   0 -16  0        0      0 -        DL    - 0:00.00 [soaiod1]    0     9     0   0 -16  0        0      0 -        DL    - 0:00.00 [soaiod2]    0    10     0   0 -16  0        0      0 audit_wo DL    - 0:00.00 [audit]    0    11     0   0 155  0        0      0 -        RL    - 4:40.30 [idle]    0    12     0   0 -52  0        0      0 -        WL    - 0:00.68 [intr]    0    13     0   0  -8  0        0      0 -        DL    - 0:01.39 [geom]    0    14     0   0 -16  0        0      0 seqstate DL    - 0:00.00 [sequencer 00]    0    15     0   0 -68  0        0      0 -        DL    - 0:00.44 [usb]    0    16     0   0 -16  0        0      0 -        DL    - 0:00.00 [soaiod3]    0    17     0   0 -16  0        0      0 -        DL    - 0:00.00 [soaiod4]    0    18     0   0 -20  0        0      0 VBoxIS   DL    - 0:00.00 [Timer]    0    19     0   0 -20  0        0      0 VBoxIS   DL    - 0:00.00 [Timer]    0    20     0   0 -16  0        0      0 waiting_ DL    - 0:00.00 [sctp_iterator]    0    21     0   0 -16  0        0      0 tzpoll   DL    - 0:01.35 [acpi_thermal]    0    22     0   0 -16  0        0      0 -        DL    - 0:00.02 [rand_harvestq]    0    23     0   0 -16  0        0      0 idle     DL    - 0:00.00 [enc_daemon0]    0    24     0   0 -16  0        0      0 psleep   DL    - 0:00.00 [pagedaemon]    0    25     0   0 -16  0        0      0 psleep   DL    - 0:00.00 [vmdaemon]    0    26     0   0  20  0        0      0 qsleep   DL    - 0:00.01 [bufdaemon]    0    27     0   0  20  0        0      0 vlruwt   DL    - 0:00.00 [vnlru]    0    28     0   0  16  0        0      0 syncer   DL    - 0:00.00 [syncer]  922 14343     1   0  38  0   825532  38400 uwait    D     - 0:10.78 [mongod]    0 17898     1   0  20  0    29328   7248 select   D     - 0:00.02 [gdm]    0 19045     1   0  20  0    15408   3892 select   Ds    - 0:00.71 [dsbmd]    0 19383     1   0  20  0    37884   6788 kqread   Ds    - 0:00.02 [cupsd]  970 20304     1   0  20  0    30916  11268 select   D     - 0:00.08 [colord]    0 20794     1   0  20  0    17456   4164 select   Ds    - 0:00.00 [wpa_supplicant]    0 21100     1   0  20  0    11384   2088 nanslp   Ds    - 0:00.00 [cron]    0 29800     1   0  21  0    10848   1804 ttyin    Ds+   - 0:00.00 [getty]    0 30063 17898   0  23  0    26348  10072 select   D     - 0:00.08 [gdm-session-worker] 1001 30574     1   0  20  0    26600   9264 select   D     - 0:00.02 [gnome-keyring-daem] 1001 31704 30063   0  52  0    11740   2460 wait     Ds    - 0:00.00 [sh]    0 33416     1   0  52  0    10848   1732 ttyin    Ds+   - 0:00.00 [getty] 1001 33742 31704   0  20  0    16620   5196 select   Ds    - 0:00.00 [ssh-agent]    0 33760     1   0  52  0    10848   1732 ttyin    Ds+   - 0:00.00 [getty]    0 33992     1   0  52  0    10848   1732 ttyin    Ds+   - 0:00.00 [getty]    0 34233     1   0  52  0    10848   1732 ttyin    Ds+   - 0:00.00 [getty]    0 34565     1   0  52  0    10848   1732 ttyin    Ds+   - 0:00.00 [getty]    0 34973     1   0  52  0    10848   1732 ttyin    Ds+   - 0:00.00 [getty]    0 34979     1   0  52  0    10848   1732 ttyin    Ds+   - 0:00.00 [getty] 1001 35039 31704   0  20  0    64112  32332 select   D     - 0:00.06 [xfce4-session] 1001 35710     1   0  49  0    13856   3764 select   D     - 0:00.00 [dbus-launch] 1001 36209     1   0  20  0    13328   4124 select   Ds    - 0:00.00 [dbus-daemon] 1001 36675     1   0  20  0    17808   6216 select   D     - 0:00.00 [xfconfd]    0 36870     1   0  20  0    25400   6840 select   D     - 0:00.02 [accounts-daemon] 1001 37200     1   0  20  0    18496   3768 select   Ds    - 0:00.01 [gpg-agent] 1001 37328     1   0  20  0    67476  37476 select   D     - 0:00.00 [xfwm4]  560 37516     1   0  20  0    22540   8356 select   Ds    - 0:00.04 [hald] 1001 37594     1   0  20  0    74196  39592 select   D     - 0:00.00 [xfce4-panel]    0 37617 37516   0  52  0    19440   5428 select   D     - 0:00.06 [hald-runner]    0 37964 17898   0  20  0 25266128  41864 select   D     - 0:00.95 [Xorg] 1001 38085     1   0  20  0    61940  32160 select   D     - 0:00.00 [thunar] 1001 38723     1   0  20  0   167844 125480 select   D     - 0:00.00 [xfdesktop] 1001 38793     1   0  20  0    72284  38520 select   Ds    - 0:00.00 [xfsettingsd] 1001 39072 35039   0  20  0   242032 103332 select   D     - 0:01.43 [owncloud] 1001 39166 35039   0  20  0  1519772 251076 select   D     - 0:04.95 [quiterss] 1001 39543 35039   0  52  0    32876  16704 select   D     - 0:00.00 [python3.6] 1001 39956 35039   0  20  0   232592  88836 select   D     - 0:00.00 [evolution-alarm-no]    0 40367 37617   0  52  0    15204   3924 kqread   D     - 0:00.06 [hald-addon-mouse-s] 1001 40428     1   0  20  0    84952  33572 select   D     - 0:00.00 [tracker-store] 1001 40783 35039   0  20  0    77348  39772 select   D     - 0:00.00 [zeitgeist-datahub] 1001 41112     1   0  20  0    31204   8764 select   D     - 0:00.00 [zeitgeist-daemon] 1001 41148 35039   0  41 19   140820  61180 select   DN    - 0:00.00 [tracker-extract] 1001 41406     1   0  20  0    27896   8040 select   D     - 0:00.00 [gvfsd] 1001 41725 35039   0  33  0    34248   8992 select   D     - 0:00.00 [ibus-daemon] 1001 42006 35039   0  43 19    83988  27504 select   DN    - 0:00.01 [tracker-miner-apps] 1001 42535     1   0  21  0   100548  10556 select   D     - 0:00.00 [pulseaudio] 1001 42890     1   0  28  0    29692   6780 usem     D     - 0:00.00 [gvfsd-fuse] 1001 43686     1   0  20  0    25976   7972 select   D     - 0:00.00 [gvfs-hal-volume-mo] 1001 44434     1   0  20  0    26624   7984 select   D     - 0:00.00 [gvfs-gphoto2-volum] 1001 44479 35039   0  24  0    80452  26452 select   D     - 0:00.00 [tracker-miner-rss] 1001 45334 41725   0  21  0    26880   8228 select   D     - 0:00.00 [ibus-dconf] 1001 45681 41725   0  20  0    82744  45260 select   D     - 0:00.00 [ibus-ui-gtk3] 1001 46367     1   0  20  0    74232  39828 select   Ds    - 0:00.02 [xfce4-power-manage] 1001 46817     1   0  20  0    28040   7340 select   D     - 0:00.00 [at-spi-bus-launche] 1001 47176 35039   0  20  0    15140   4836 select   D     - 0:00.00 [xscreensaver] 1001 47479 46817   0  20  0    12464   3036 select   D     - 0:00.00 [dbus-daemon] 1001 47616 41725   0  20  0    78720  41972 select   D     - 0:00.00 [ibus-extension-gtk] 1001 47761     1   0  20  0    73992  39308 select   D     - 0:00.00 [ibus-x11]    0 48053 37617   0  20  0    12040   2532 select   D     - 0:00.00 [hald-addon-storage] 1001 48292     1   0  20  0    23856   7536 select   D     - 0:00.00 [at-spi2-registryd] 1001 48926     1   0  20  0    26644   8144 select   D     - 0:00.00 [ibus-portal] 1001 49501     1   0  20  0    76996  40200 select   D     - 0:00.03 [mate-screensaver] 1001 49816 41725   0  23  0    27092   8588 select   D     - 0:00.00 [ibus-engine-simple]    0 50088 37617   0  20  0    12040   2532 select   D     - 0:00.00 [hald-addon-storage] 1001 51049 37594   0  20  0    64200  34968 select   D     - 0:00.00 [wrapper-1.0] 1001 51176 37594   0  20  0    61832  32364 select   D     - 0:00.00 [wrapper-1.0] 1001 51314 37594   0  20  0    65364  35624 select   D     - 0:00.00 [wrapper-1.0] 1001 51433 37594   0  20  0    75404  40552 select   D     - 0:00.00 [wrapper-1.0] 1001 51716     1   0  20  0    31660   9112 select   D     - 0:00.00 [gvfsd-trash] 1001 51963     1   0  20  0    47140  26856 select   D     - 0:00.00 [python2.7] 1001 52273 51963   0  20  0    43740  24840 select   D     - 0:00.00 [python2.7] 1001 52977     1   0  25  0    21632   6148 select   D     - 0:00.00 [gvfsd-metadata] 1001 53384     1   0  21  0   113112  61524 select   D     - 0:00.00 [evolution-source-r] 1001 53439     1   0  22  0   166748  71160 select   D     - 0:00.00 [goa-daemon]    0 53649     1   0  20  0    24108   5896 select   D     - 0:00.00 [upowerd] 1001 53885     1   0  52  0   119804  62688 select   D     - 0:00.01 [evolution-calendar] 1001 54111 53885   0  52  0   133288  63000 select   D     - 0:00.00 [evolution-calendar] 1001 54568     1   0  21  0    21572   6032 select   D     - 0:00.00 [dconf-service] 1001 55080     1   0  28  0   118740  62148 select   D     - 0:00.00 [evolution-addressb] 1001 55388 55080   0  29  0   149692  62632 select   D     - 0:00.00 [evolution-addressb] 1001 55671     1   0  20  0    27880   7828 select   D     - 0:00.00 [gnome-keyring-daem]    0 57716     0   0 -16  0        0      0 sleep    DL    - 0:00.00 [ng_queue]    0 60442     1   0  52  0    11352   2056 select   Ds    - 0:00.00 [dhclient]    0 60702     1   0  52  0    11548   2164 select   Ds    - 0:00.00 [dhclient]   65 64115     1   0  20  0    11716   2280 select   DCs   - 0:00.00 [dhclient]    0 64580     1   0  20  0    11016   1784 select   Ds    - 0:00.00 [moused]    0 69808     1   0  20  0    10480    992 select   Ds    - 0:00.00 [devd]    0 76312     1   0  20  0    11372   2108 select   Ds    - 0:00.01 [syslogd] 1001 79416 47176   0  52 10    68544  39268 os.sema_ DN    - 0:00.00 [circuit]  556 87224     1   0  20  0    13480   3828 select   Ds    - 0:00.01 [dbus-daemon]    0 89445     1   0  20  0    10824   1692 select   Ds    - 0:00.01 [powerd]    0 92942     1   0  20  0    52200   6760 select   D     - 0:00.02 [console-kit-daemon]  565 93135     1   0  20  0  1143920  20840 select   D     - 0:00.09 [polkitd]    0 94250     1   0  52  0    12404   1988 select   Ds    - 0:00.00 [rsync]    0 95608     1   0  52  0    19580   5144 select   Ds    - 0:00.00 [sshd]  770 96502     1   0  20  0   174036  17932 select   D     - 0:00.01 [postgres]  770 96718 96502   0  20  0   174088  17988 select   Ds    - 0:00.00 [postgres]  770 96821 96502   0  20  0   174088  17932 select   Ds    - 0:00.00 [postgres]  770 96961 96502   0  20  0   174088  17928 select   Ds    - 0:00.00 [postgres]  770 96972 96502   0  20  0   174572  18292 select   Ds    - 0:00.00 [postgres]  770 97025 96502   0  20  0    29084   7380 select   Ds    - 0:00.00 [postgres]  770 97102 96502   0  20  0   174436  18248 select   Ds    - 0:00.00 [postgres] ------------------------------------------------------------------------ vmstat -s 101583681 cpu context switches  17651570 device interrupts    123994 software interrupts  91732903 traps 239716789 system calls        29 kernel threads created     98370  fork() calls     35177 vfork() calls         4 rfork() calls         0 swap pager pageins         0 swap pager pages paged in         0 swap pager pageouts         0 swap pager pages paged out     25979 vnode pager pageins    178015 vnode pager pages paged in      3191 vnode pager pageouts     88070 vnode pager pages paged out         0 page daemon wakeups         0 pages examined by the page daemon         0 clean page reclamation shortfalls         0 pages reactivated by the page daemon  10745541 copy-on-write faults      2129 copy-on-write optimized faults  77620961 zero fill pages zeroed      1334 zero fill pages prezeroed      1511 intransit blocking page faults  91450468 total VM faults taken    315482 page faults requiring I/O         0 pages affected by kernel thread creation   8812814 pages affected by  fork()   8823634 pages affected by vfork()       228 pages affected by rfork()  95679361 pages freed         0 pages freed by daemon         0 pages freed by exiting processes         0 pages active         0 pages inactive         0 pages in the laundry queue         0 pages wired down         0 pages free         0 bytes per page         0 total name lookups           cache hits (0% pos + 0% neg) system 0% per-directory           deletions 0%, falsehits 0%, toolong 0% ------------------------------------------------------------------------ vmstat -m          Type InUse MemUse HighUse Requests  Size(s)       CAM CCB     0     0K       -   642947  2048      CAM path    22     1K       -      103  32    CAM periph    16     4K       -     2838  16,32,64,128,256     acpi_perf     4     2K       -        4  512       acpipwr     5     1K       -        5  64          hdaa    10    62K       -       10 128,1024,2048,4096,16384,32768          hdac     2     3K       -        2  1024,2048         hdacc     2     1K       -        2  32        feeder    45     5K       -     6175  32,128         mixer     6    24K       -        6  4096 CAM I/O Scheduler     5     1K       -        5  128     CAM queue    26     8K       -       60  16,32,512       scsi_cd     0     0K       -       14  16           USB    61   177K       -       69 16,32,128,256,512,1024,2048,4096,8192,16384,32768        USBdev    81    26K       -      395 32,64,128,256,512,1024,4096,8192 CAM dev queue    11     1K       -       11  64       scsi_da     0     0K       -      207  16,32,64,256         vtbuf    24  7584K       -       46  4096            vt    11     6K       -       11  512      sysmouse     1     2K       -        4  2048      SCSI ENC    25   100K       -     1417  16,64,256,2048,32768        DEVFS3   225    57K       -      259  256        DEVFS1   187    94K       -      217  512    DEVFS_RULE    56    27K       -       56  64,512         DEVFS    18     1K       -       19  16,128        DEVFSP    36     3K       -     2829  64 NFSD V4client     1     1K       -        1  256  NFSD lckfile     1     1K       -        1  256  NFSD session     1     1K       -        1  1024     pfs_nodes   323    81K       -      323  256   pfs_vncache   118     8K       -      236  64          GEOM   443    72K       -    12811 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768     raid_data     0     0K       -      420  32,128,256        isadev     8     1K       -        8  128          cdev     3     1K       -        3  256      filedesc    77   797K       -      229 16,32,64,128,256,512,4096,8192,16384,32768,65536         sigio     1     1K       -        3  64      filecaps     1     1K       -     5167  16,32,64       kdtrace   548   113K       -   272623  64,256          kenv   100    12K       -      105  16,32,64,128,8192        kqueue   194   107K       -   133880 64,256,512,2048,4096,8192,16384,32768     proc-args   133    10K       -    73965  16,32,64,128,256         hhook    13     4K       -       13  256       ithread   186    40K       -      186  32,128,256        prison    14     1K       -       14  32        KTRACE   100    13K       -      100  128        linker   542   769K       -      672 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536         lockf    99    11K       -    84402  64,128    loginclass     2     1K       -        2  64         cache     1     1K       -        1  32        devbuf 26806 51681K       -    35385 16,32,64,128,256,512,1024,2048,4096,8192,16384,65536          temp    45    50K       -   835583 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536        module   530    67K       -      531  128      mtx_pool     2    16K       -        2  8192           osd     3     1K       -       10  16,32,64,128,256      pmchooks     1     1K       -        1  128           pmc     1     1K       -        1  64          pgrp    44     6K       -      288  128       session    41     6K       -      234  128          proc     2    32K       -        2  16384       subproc   303   610K       -   133753  512,4096          cred    48    12K       -     7718  256      acpiintr     1     1K       -        1  64        acpica 18836  1958K       - 13953006 16,32,64,128,256,512,1024,2048,4096,8192        plimit    50    13K       -     1162  256       uidinfo    11     6K       -      320  128,4096        dumper     1     1K       -        1  512        sysctl     0     0K       -       13  64     sysctloid  5034   258K       -     5133  16,32,64,128     sysctltmp     0     0K       -    42382  16,64,256,1024      acpitask     1    64K       -        1  65536       tidhash     1    32K       -        1  32768       callout     5  2184K       -        5          umtx   992   124K       -      992  128      p1003.1b     1     1K       -        1  16          SWAP     1  1096K       -        1           bus  1403   132K       -    26929 16,32,64,128,256,512,1024,4096        bus-sc    87  2253K       -     4018 16,32,128,256,512,1024,2048,4096,8192,16384,32768,65536       acpisem   128    16K       -      128  128       devstat    20    41K       -       20  32,4096         epoch     4     1K       -        4  128  eventhandler   122    11K       -      122  64,128    gtaskqueue    34    35K       -       34  16,32,256,8192          kobj   354  1416K       -     1058  4096       Per-cpu     1     1K       -        1  32        kbdmux     7    22K       -        7 16,512,1024,2048,16384          rman   236    28K       -      753  32,128          sbuf     0     0K       -  1487735 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536        sglist    53   208K       -     2330 32,128,512,1024,2048,4096,8192,16384,32768,65536     toponodes    18     3K       -       18  128           LED    24     2K       -       24  16,128     taskqueue    39     5K       -       39  16,32,256      terminal    11     3K       -       11  256        Unitno    30     2K       -   116338  32,64          vmem     3   520K       -       16 2048,4096,8192,16384,32768,65536      ioctlops     0     0K       -    63368 256,512,1024,2048,4096        select   332    42K       -      332  128           iov     0     0K       -  5474343 16,32,64,128,256,512,4096           msg     4    30K       -        4  2048,4096,8192,16384           sem     4   106K       -        4  2048,4096           shm    13    56K       -      154  2048,32768           tty    12    12K       -       12  1024      mbuf_tag     0     0K       -    10078  32,256         shmfd    13    14K       -       22  64,256,1024,8192        soname   182    22K       -    41041  16,32,64,128           pcb    29  1691K       -      160 16,32,64,128,1024,2048,8192           acl     0     0K       -      902  4096      vfscache     4  2097K       -        4  256,16384,32768    cl_savebuf     0     0K       -     2908  64      vfs_hash     1  1024K       -        1        vnodes     1     1K       -        1  256         mount   116     6K       -      186  16,32,64,128,256        statfs     0     0K       -   389885  4096   vnodemarker     0     0K       -     2564  512       fadvise     7     1K       -       33  32 chacha20random     1     1K       -        1  1024           BPF    16  1043K       -       17  16,128,512,4096         ifnet     4     7K       -        4  128,2048        ifaddr    90    16K       -       90 16,32,64,128,256,512,2048,4096   ether_multi    40     4K       -       45  16,32,64,128         clone     7     1K       -        7  128         ipsec     1     1K       -        1  256       lltable    15     5K       -       15  256,512      routetbl    28     4K       -       33  32,64,128,512          vnet     1     1K       -        1  64     vnet_data     1   240K       -        1 vnet_data_free     1     1K       -        1  32      80211vap     1     4K       -        1  4096     80211node     1    16K       -      558  16,16384   80211nodeie     6     3K       -     4128  256,512  80211ratectl     2     1K       -        2  16,64     80211scan     8    21K       -      769  512,2048,16384          igmp     3     1K       -        3  128      in_multi     2     1K       -        3  256 encap_export_host     8     1K       -        8  64     sctp_a_it     0     0K       -        3  16      sctp_vrf     1     1K       -        1  64      sctp_ifa     5     1K       -        5  128      sctp_ifn     3     1K       -        3  128     sctp_iter     0     0K       -        3  256    tfo_ccache     1   128K       -        1     hostcache     1    32K       -        1  32768       tcpfunc     1     1K       -        1  64      syncache     1    68K       -        1     in6_multi    25     3K       -       25  32,256        ip6opt     0     0K       -        3  32           mld     3     1K       -        3  128        ip6ndp     5     1K       -        6  64,256   inpcbpolicy    15     1K       -     7683  32      secasvar     1     1K       -        1  1024        sahead     1     1K       -        1  1024   ipsecpolicy     2     2K       -        2  256,1024     ipsec-saq     2     2K       -        2  1024        crypto     2     2K       -        2  512,1024           rpc     2     8K       -        2  4096 audit_evclass   230     8K       -      285  32       pagedep     5   513K       -    53788  256      inodedep    30  2062K       -   669876  512     bmsafemap     4    17K       -   344323  256,8192        newblk     7  4098K       -   482106  256      indirdep     0     0K       -     2417  128,32768      freefrag     0     0K       -    14275  128      freeblks    11     3K       -   329262  256      freefile     7     1K       -   340772  64        diradd     5     1K       -   349186  128         mkdir     0     0K       -    54888  128        dirrem     6     1K       -   349123  128     newdirblk     0     0K       -    28564  64      freework    13     2K       -   368999  32,128       freedep     0     0K       -     1162  64       jaddref     0     0K       -   404074  128       jremref     0     0K       -   403995  128        jmvref     0     0K       -     2756  128       jnewblk     0     0K       -   482104  128     jfreefrag     0     0K       -    14275  128          jseg   160    20K       -    37044  128       jsegdep    29     2K       -  1304448  64         sbdep     0     0K       -      409  64      savedino     0     0K       -    70412  256       jblocks     4     1K       -        4  128,256       softdep     2     1K       -        2  512   ufs_dirhash   855   175K       -     6997 16,32,64,128,256,512,1024,8192,16384     ufs_quota     1  1024K       -        1     ufs_mount     6    33K       -        8  512,4096,8192     vm_pgdata     1     1K       -        1  128       UMAHash     6    68K       -       32 512,1024,2048,4096,8192,16384       memdesc     1     4K       -        1  4096      pci_link    16     2K       -       16  64,128       acpidev   130     9K       -      130  64       CAM SIM    11     3K       -       11  256       CAM XPT    66     6K       -    24264 16,32,64,128,256,512,1024,2048,65536        apmdev     1     1K       -        1  128    madt_table     0     0K       -        2  64,4096          intr     4   400K       -        4  65536       io_apic     1    16K       -        1  16384    local_apic     1     8K       -        1  8192           MCA    15     3K       -       15  32,128,256          cpus     2     1K       -        2  32           msi    15     2K       -       15  128      nexusdev     5     1K       -        5  16       CAM DEV    15    30K       -       25  2048       entropy     1     1K       -    10868  32,4096    fuse_vnode     1     1K       -        1  512   fuse_msgbuf   511   130K       -      511  256,2048         linux    36     3K       -       36  64        nvidia 13710 34302K       -   918377 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536 nvidia-modeset   194   307K       -   204941 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536      iprtheap    31    69K       -       38 32,64,128,256,2048,4096,65536   fdesc_mount     1     1K       -        1  16      netgraph     2     1K       -        2  64  netgraph_msg     0     0K       -       40  64,128,256,1024,32768 netgraph_hook    10     2K       -       10  128 netgraph_node    11     3K       -       22  128,256  netgraph_hci     1     1K       -        1  256 netgraph_l2cap     1     1K       -        1  128 netgraph_btsocks_hci_raw     1     8K       -       23  128,8192 netgraph_btsocks_l2cap_raw     1     1K       -        1  32 netgraph_btsocks_l2cap     1     1K       -        1  32 netgraph_path     0     0K       -       11  16 netgraph_sock     0     0K       -       18  128 ------------------------------------------------------------------------ vmstat -z ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP UMA Kegs:               224,      0,     127,       9,     127, 0,   0 UMA Zones:              904,      0,     129,       3,     129, 0,   0 UMA Slabs:               80,      0,   44858,      42,   54832, 0,   0 UMA Hash:               256,      0,       7,       8,      13, 0,   0 4 Bucket:                32,      0,      52,    1573,  463691, 0,   0 6 Bucket:                48,      0,      47,    1530,   59775, 0,   0 8 Bucket:                64,      0,      34,    1516,   44801, 21,   0 12 Bucket:               96,      0,      26,     507,    2292, 0,   0 16 Bucket:              128,      0,     307,     468,   14092, 1,   0 32 Bucket:              256,      0,   18150,    1740,  100260, 6,   0 64 Bucket:              512,      0,   13035,    1045, 113667,2290,   0 128 Bucket:            1024,      0,    6550,     510, 68763,11749,   0 256 Bucket:            2048,      0,    1909,      45, 253956,15396,   0 vmem:                  1792,      0,       3,       1,       3, 0,   0 vmem btag:               56,      0,   35644,   10435,  301262, 325,   0 VM OBJECT:              256,      0,   32179,  172721, 2720799, 0,   0 RADIX NODE:             144,      0,   63595,   85094, 9222749, 0,   0 MAP:                    232,      0,       3,      65,       3, 0,   0 KMAP ENTRY:             120,      0,      39,     489,      78, 0,   0 MAP ENTRY:              120,      0,   23000,    3400, 8480977, 0,   0 VMSPACE:               2512,      0,     102,      18,  133543, 0,   0 fakepg:                 104,      0,    5538,    4114,  292116, 0,   0 64 pcpu:                  8,      0,    2730,     598,    2730, 0,   0 mt_stats_zone:           64,      0,     456,     824,     456, 0,   0 mt_zone:                 24,      0,     456,     880,     456, 0,   0 16:                      16,      0,    3681,    2343,15573245, 0,   0 32:                      32,      0,    4713,    2037,  350743, 0,   0 64:                      64,      0,   20402,  450116, 6743765, 0,   0 128:                    128,      0,   21587,  556067, 3707370, 0,   0 256:                    256,      0,    3435,  283710, 1520797, 0,   0 512:                    512,      0,    2294,  287778,  791220, 0,   0 1024:                  1024,      0,     726,      86,  186230, 0,   0 2048:                  2048,      0,     582,      52,  707670, 0,   0 4096:                  4096,      0,     818,      10,  557006, 0,   0 8192:                  8192,      0,     110,       7,    2503, 0,   0 16384:                16384,      0,      77,      37,  609873, 0,   0 32768:                32768,      0,      33,       8,   77316, 0,   0 65536:                65536,      0,      47,       8,   28420, 0,   0 SLEEPQUEUE:              80,      0,     497,     278,     497, 0,   0 Files:                   80,      0,    5060,     440,11191690, 0,   0 filedesc0:             1104,      0,     131,      49,  133581, 0,   0 TURNSTILE:              136,      0,     497,     123,     497, 0,   0 rl_entry:                40,      0,     241,     859,     241, 0,   0 umtx pi:                 96,      0,       0,       0,       0, 0,   0 umtx_shm:                88,      0,       0,       0,       0, 0,   0 MAC labels:              40,      0,       0,       0,       0, 0,   0 PROC:                  1320,      0,     130,      41,  133580, 0,   0 THREAD:                1400,      0,     457,      39,    5632, 0,   0 cpuset:                 104,      0,      11,     516,      11, 0,   0 domainset:               40,      0,       0,       0,       0, 0,   0 audit_record:          1280,      0,       0,       0,       0, 0,   0 mbuf_packet:            256, 3200355,     256,    1528,   59994, 0,   0 mbuf:                   256, 3200355,     258,    1526,  466508, 0,   0 mbuf_cluster:          2048, 500054,    1771,       9,    2286, 0,   0 mbuf_jumbo_page:       4096, 250027,     256,      47,  211658, 0,   0 mbuf_jumbo_9k:         9216, 222246,       0,       0,       0, 0,   0 mbuf_jumbo_16k:       16384, 166684,       0,       0,       0, 0,   0 epoch_record pcpu:      256,      0,       4,      60,       4, 0,   0 g_bio:                  376,      0,       0,    5570, 1772129, 0,   0 DMAR_MAP_ENTRY:         120,      0,       0,       0,       0, 0,   0 ttyinq:                 160,      0,     135,     115,     465, 0,   0 ttyoutq:                256,      0,      72,     183,     248, 0,   0 FPU_save_area:         1088,      0,       0,       0,       0, 0,   0 nvme_request:           128,      0,       0,       0,       0, 0,   0 cryptop:                128,      0,       0,       0,       0, 0,   0 cryptodesc:             120,      0,       0,       0,       0, 0,   0 crypto_session:          24,      0,       0,       0,       0, 0,   0 vtnet_tx_hdr:            24,      0,       0,       0,       0, 0,   0 nvidia_stack_t:       12288,      0,       8,       7,   22967, 0,   0 VNODE:                  480,      0,   26393,  186007, 1334770, 0,   0 VNODEPOLL:              120,      0,    3402,     228,    3404, 0,   0 BUF TRIE:               144,      0,    3206,   49066,  149868, 0,   0 NAMEI:                 1024,      0,       0,      80,33470293, 0,   0 rentr:                   24,      0,       0,       0,       0, 0,   0 S VFS Cache:            108,      0,   25896,  172239, 2692360, 0,   0 STS VFS Cache:          148,      0,       0,       0,       0, 0,   0 L VFS Cache:            328,      0,    1758,   35826,  225639, 0,   0 LTS VFS Cache:          368,      0,       0,       0,       0, 0,   0 NCLNODE:                592,      0,       0,       0,       0, 0,   0 DIRHASH:               1024,      0,    1548,    2820,   10611, 0,   0 fuse_ticket:            224,      0,       0,     255,    2886, 0,   0 Mountpoints:           2744,      0,       8,       4,       8, 0,   0 procdesc:               136,      0,       1,     144,       4, 0,   0 AIO:                    208,      0,       0,       0,       0, 0,   0 AIOP:                    32,      0,       0,       0,       0, 0,   0 AIOCB:                  752,      0,       0,       0,       0, 0,   0 AIOLIO:                 280,      0,       0,       0,       0, 0,   0 pipe:                   760,      0,     306,      49,   81822, 0,   0 ksiginfo:               112,      0,     154,     616,    1728, 0,   0 itimer:                 352,      0,       0,       0,       0, 0,   0 KNOTE:                  160,      0,    3848,      77,    4734, 0,   0 socket:                 872, 257996,     378,      66,   16311, 0,   0 unpcb:                  256, 258000,     361,     134,    8575, 0,   0 ipq:                     56,  15691,       0,       0,       0, 0,   0 udp_inpcb:              488, 258000,       2,     118,    7649, 0,   0 udpcb:                   32, 258000,       2,    1123,    7649, 0,   0 tcp_inpcb:              488, 258000,      12,     108,      31, 0,   0 tcpcb:                  976, 257996,      12,      40,      31, 0,   0 tcptw:                   88,  27810,       0,     135,       4, 0,   0 syncache:               168,  15364,       0,       0,       0, 0,   0 hostcache:               96,  15375,       1,     163,       4, 0,   0 sackhole:                32,      0,       0,       0,       0, 0,   0 tfo:                      4,      0,       0,       0,       0, 0,   0 tfo_ccache_entries:      80,      0,       0,       0,       0, 0,   0 tcpreass:                48,  31291,       0,       0,       0, 0,   0 tcp_log:                400, 1000000,       0,       0,       0, 0,   0 tcp_log_bucket:         144,      0,       0,       0,       0, 0,   0 tcp_log_node:           120,      0,       0,       0,       0, 0,   0 sctp_ep:               1280, 257997,       0,       0,       0, 0,   0 sctp_asoc:             2408,  40000,       0,       0,       0, 0,   0 sctp_laddr:              48,  80012,       0,     581,       4, 0,   0 sctp_raddr:             736,  80000,       0,       0,       0, 0,   0 sctp_chunk:             152, 400010,       0,       0,       0, 0,   0 sctp_readq:             152, 400010,       0,       0,       0, 0,   0 sctp_stream_msg_out:    112, 400015,       0,       0,       0, 0,   0 sctp_asconf:             40, 400000,       0,       0,       0, 0,   0 sctp_asconf_ack:         48, 400060,       0,       0,       0, 0,   0 udplite_inpcb:          488, 258000,       0,       0,       0, 0,   0 ripcb:                  488, 258000,       1,      63,       3, 0,   0 IPsec SA lft_c:          16,      0,       0,       0,       0, 0,   0 rtentry:                208,      0,      13,     234,      16, 0,   0 selfd:                   64,      0,     956,    1338,12173512, 0,   0 swpctrie:               144, 1000134,       0,       0,       0, 0,   0 swblk:                  136, 1000123,       0,       0,       0, 0,   0 FFS inode:              160,      0,   26215,  186335, 1334339, 0,   0 FFS1 dinode:            128,      0,       0,       0,       0, 0,   0 FFS2 dinode:            256,      0,   26215,  186200, 1334311, 0,   0 NetGraph items:          72,   4123,       0,     527,      60, 0,   0 NetGraph data items:     72,   4123,       0,     527,      54, 0,   0 ------------------------------------------------------------------------ vmstat -i interrupt                          total       rate cpu0:timer                       2711433      30670 cpu1:timer                       2555695      28908 cpu2:timer                       2605375      29470 cpu3:timer                       2490769      28174 irq264: vgapci0                   315489       3569 irq265: hdac0                       2383         27 irq266: xhci0                   16686288     188744 irq267: ahci0                     598665       6772 irq268: iwm0                       24306        275 irq269: re0                        22310        252 irq270: hdac1                       2129         24 Total                           28014842     316885 ------------------------------------------------------------------------ pstat -T 5060/257995 files 0M/0M swap space ------------------------------------------------------------------------ pstat -s Device          512-blocks     Used    Avail Capacity ------------------------------------------------------------------------ iostat        tty            ada0             ada1 da0             cpu  tin  tout  KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id    0  1301  3.22   0  0.00  17.25  70  1.18   0.00   0  0.00   0 21  2  0 76 ------------------------------------------------------------------------ ipcs -a Message Queues: T           ID          KEY MODE        OWNER    GROUP    CREATOR CGROUP                 CBYTES                 QNUM QBYTES LSPID        LRPID STIME    RTIME    CTIME Shared Memory: T           ID          KEY MODE        OWNER    GROUP    CREATOR CGROUP         NATTCH        SEGSZ         CPID         LPID ATIME    DTIME    CTIME m        65536      5432001 --rw------- postgres postgres postgres postgres            6           56        96502        96502 8:54:12 11:12:16  8:54:12 m       131073            0 --rw------- andersbo andersbo andersbo andersbo            2       393216        37328        37964 10:24:18 10:24:20 10:24:18 m        65538            0 --rw------- andersbo andersbo andersbo andersbo            2       393216        37328        37964 10:24:24 no-entry 10:24:24 m       131075   1365697808 --rw------- andersbo andersbo andersbo andersbo            1         1024        39072        39072 10:24:26 no-entry 10:24:26 m       131076            0 --rw------- andersbo andersbo andersbo andersbo            2       393216        37594        37964 10:24:25 no-entry 10:24:25 Semaphores: T           ID          KEY MODE        OWNER    GROUP    CREATOR CGROUP          NSEMS OTIME    CTIME s        65536   1365697807 --rw------- andersbo andersbo andersbo andersbo            1 10:24:26 10:24:26 ------------------------------------------------------------------------ ipcs -T msginfo:     msgmax:        16384    (max characters in a message)     msgmni:           40    (# of message queues)     msgmnb:         2048    (max characters in a message queue)     msgtql:           40    (max # of messages in system)     msgssz:            8    (size of a message segment)     msgseg:         2048    (# of message segments in system) shminfo:     shmmax:    536870912    (max shared memory segment size)     shmmin:            1    (min shared memory segment size)     shmmni:          192    (max number of shared memory identifiers)     shmseg:          128    (max shared memory segments per process)     shmall:       131072    (max amount of shared memory in pages) seminfo:     semmni:           50    (# of semaphore identifiers)     semmns:          340    (# of semaphores in system)     semmnu:          150    (# of undo structures in system)     semmsl:          340    (max # of semaphores per id)     semopm:          100    (max # of operations per semop call)     semume:           50    (max # of undo entries per process)     semusz:          632    (size in bytes of undo structure)     semvmx:        32767    (semaphore maximum value)     semaem:        16384    (adjust on exit max value) ------------------------------------------------------------------------ nfsstat Rpc Counts:       Getattr      Setattr       Lookup     Readlink Read Write       Create       Remove             0            0            0            0 0 0            0            0        Rename         Link      Symlink        Mkdir Rmdir Readdir     RdirPlus       Access             0            0            0            0 0 0            0            0         Mknod       Fsstat       Fsinfo     PathConf       Commit             0            0            0            0            0 Rpc Info:      TimedOut      Invalid    X Replies      Retries     Requests             0            0            0            0            0 Cache Info:     Attr Hits  Attr Misses    Lkup Hits  Lkup Misses    BioR Hits BioR Misses    BioW Hits  BioW Misses             0            0            0            0 0 0            0            0    BioRL Hits BioRL Misses    BioD Hits  BioD Misses    DirE Hits DirE Misses    Accs Hits  Accs Misses             0            0            0            0 0 0            0            0 Server Info:       Getattr      Setattr       Lookup     Readlink Read Write       Create       Remove             0            0            0            0 0 0            0            0        Rename         Link      Symlink        Mkdir Rmdir Readdir     RdirPlus       Access             0            0            0            0 0 0            0            0         Mknod       Fsstat       Fsinfo     PathConf       Commit             0            0            0            0            0 Server Re-Failed:   140737488346800 Server Faults: 140737488346800 Server Write      WriteOps     WriteRPC      Opsaved             0            0            0 Server Cache        Inprog         Idem     Non-Idem       Misses             0            0            0            0 ------------------------------------------------------------------------ netstat -s tcp:     5349 packets sent         389 data packets (123934 bytes)         0 data packets (0 bytes) retransmitted         0 data packets unnecessarily retransmitted         0 resends initiated by MTU discovery         4932 ack-only packets (180 delayed)         0 URG only packets         0 window probe packets         1 window update packet         27 control packets     10058 packets received         407 acks (for 123921 bytes)         0 duplicate acks         0 acks for unsent data         9682 packets (12419231 bytes) received in-sequence         1 completely duplicate packet (0 bytes)         0 old duplicate packets         0 packets with some dup. data (0 bytes duped)         0 out-of-order packets (0 bytes)         0 packets (0 bytes) of data after window         0 window probes         0 window update packets         0 packets received after close         0 discarded for bad checksums         0 discarded for bad header offset fields         0 discarded because packet too short         0 discarded due to memory problems     15 connection requests     0 connection accepts     0 bad connection attempts     0 listen queue overflows     0 ignored RSTs in the windows     15 connections established (including accepts)         3 times used RTT from hostcache         3 times used RTT variance from hostcache         0 times used slow-start threshold from hostcache     19 connections closed (including 1 drop)         7 connections updated cached RTT on close         7 connections updated cached RTT variance on close         0 connections updated cached ssthresh on close     0 embryonic connections dropped     407 segments updated rtt (of 400 attempts)     0 retransmit timeouts         0 connections dropped by rexmit timeout     0 persist timeouts         0 connections dropped by persist timeout     0 Connections (fin_wait_2) dropped because of timeout     0 keepalive timeouts         0 keepalive probes sent         0 connections dropped by keepalive     200 correct ACK header predictions     9637 correct data packet header predictions     0 syncache entries added         0 retransmitted         0 dupsyn         0 dropped         0 completed         0 bucket overflow         0 cache overflow         0 reset         0 stale         0 aborted         0 badack         0 unreach         0 zone failures     0 cookies sent     0 cookies received     4 hostcache entries added         0 bucket overflow     0 SACK recovery episodes     0 segment rexmits in SACK recovery episodes     0 byte rexmits in SACK recovery episodes     0 SACK options (SACK blocks) received     0 SACK options (SACK blocks) sent     0 SACK scoreboard overflow     0 packets with ECN CE bit set     0 packets with ECN ECT(0) bit set     0 packets with ECN ECT(1) bit set     0 successful ECN handshakes     0 times ECN reduced the congestion window     0 packets with matching signature received     0 packets with bad signature received     0 times failed to make signature due to no SA     0 times unexpected signature received     0 times no signature provided by segment     0 Path MTU discovery black hole detection activations     0 Path MTU discovery black hole detection min MSS activations     0 Path MTU discovery black hole detection failures TCP connection count by state:     0 connections in CLOSED state     9 connections in LISTEN state     0 connections in SYN_SENT state     0 connections in SYN_RCVD state     3 connections in ESTABLISHED state     0 connections in CLOSE_WAIT state     0 connections in FIN_WAIT_1 state     0 connections in CLOSING state     0 connections in LAST_ACK state     0 connections in FIN_WAIT_2 state     0 connections in TIME_WAIT state udp:     935 datagrams received     0 with incomplete header     0 with bad data length field     0 with bad checksum     0 with no checksum     0 dropped due to no socket     1 broadcast/multicast datagram undelivered     0 dropped due to full socket buffers     0 not for hashed pcb     934 delivered     934 datagrams output     0 times multicast source filter matched ip:     10223 total packets received     0 bad header checksums     0 with size smaller than minimum     0 with data size < data length     0 with ip length > max ip packet size     0 with header length < data size     0 with data length < header length     0 with bad options     0 with incorrect version number     0 fragments received     0 fragments dropped (dup or out of space)     0 fragments dropped after timeout     0 packets reassembled ok     10222 packets for this host     0 packets for unknown/unsupported protocol     0 packets forwarded (0 packets fast forwarded)     1 packet not forwardable     0 packets received for unknown multicast group     0 redirects sent     5422 packets sent from this host     0 packets sent with fabricated ip header     0 output packets dropped due to no bufs, etc.     0 output packets discarded due to no route     0 output datagrams fragmented     0 fragments created     0 datagrams that can't be fragmented     0 tunneling packets that can't find gif     0 datagrams with bad address in header icmp:     0 calls to icmp_error     0 errors not generated in response to an icmp message     0 messages with bad code fields     0 messages less than the minimum length     0 messages with bad checksum     0 messages with bad length     0 multicast echo requests ignored     0 multicast timestamp requests ignored     0 message responses generated     0 invalid return addresses     0 no return routes ipsec:     0 inbound packets violated process security policy     0 inbound packets failed due to insufficient memory     0 invalid inbound packets     0 outbound packets violated process security policy     0 outbound packets with no SA available     0 outbound packets failed due to insufficient memory     0 outbound packets with no route available     0 invalid outbound packets     0 outbound packets with bundled SAs     0 spd cache hits     0 spd cache misses     0 clusters copied during clone     0 mbufs inserted during makespace ah:     0 packets shorter than header shows     0 packets dropped; protocol family not supported     0 packets dropped; no TDB     0 packets dropped; bad KCR     0 packets dropped; queue full     0 packets dropped; no transform     0 replay counter wraps     0 packets dropped; bad authentication detected     0 packets dropped; bad authentication length     0 possible replay packets detected     0 packets in     0 packets out     0 packets dropped; invalid TDB     0 bytes in     0 bytes out     0 packets dropped; larger than IP_MAXPACKET     0 packets blocked due to policy     0 crypto processing failures     0 tunnel sanity check failures esp:     0 packets shorter than header shows     0 packets dropped; protocol family not supported     0 packets dropped; no TDB     0 packets dropped; bad KCR     0 packets dropped; queue full     0 packets dropped; no transform     0 packets dropped; bad ilen     0 replay counter wraps     0 packets dropped; bad encryption detected     0 packets dropped; bad authentication detected     0 possible replay packets detected     0 packets in     0 packets out     0 packets dropped; invalid TDB     0 bytes in     0 bytes out     0 packets dropped; larger than IP_MAXPACKET     0 packets blocked due to policy     0 crypto processing failures     0 tunnel sanity check failures ipcomp:     0 packets shorter than header shows     0 packets dropped; protocol family not supported     0 packets dropped; no TDB     0 packets dropped; bad KCR     0 packets dropped; queue full     0 packets dropped; no transform     0 replay counter wraps     0 packets in     0 packets out     0 packets dropped; invalid TDB     0 bytes in     0 bytes out     0 packets dropped; larger than IP_MAXPACKET     0 packets blocked due to policy     0 crypto processing failures     0 packets sent uncompressed; size < compr. algo. threshold     0 packets sent uncompressed; compression was useless arp:     2 ARP requests sent     18 ARP replies sent     337 ARP requests received     1 ARP reply received     338 ARP packets received     0 total packets dropped due to no ARP entry     0 ARP entrys timed out     0 Duplicate IPs seen ip6:     864 total packets received     0 with size smaller than minimum     0 with data size < data length     0 with bad options     0 with incorrect version number     0 fragments received     0 fragments dropped (dup or out of space)     0 fragments dropped after timeout     0 fragments that exceeded limit     0 packets reassembled ok     864 packets for this host     0 packets forwarded     0 packets not forwardable     0 redirects sent     869 packets sent from this host     0 packets sent with fabricated ip header     0 output packets dropped due to no bufs, etc.     13 output packets discarded due to no route     0 output datagrams fragmented     0 fragments created     0 datagrams that can't be fragmented     0 packets that violated scope rules     3 multicast packets which we don't join     Input histogram:         UDP: 861         ICMP6: 3     Mbuf statistics:         419 one mbuf         445 one ext mbuf         0 two or more ext mbuf     0 packets whose headers are not contiguous     0 tunneling packets that can't find gif     0 packets discarded because of too many headers     0 failures of source address selection     source addresses on an outgoing I/F         3 link-locals     source addresses of a different scope         3 link-locals     Source addresses selection rule applied:         3 first candidate icmp6:     0 calls to icmp6_error     0 errors not generated in response to an icmp6 message     0 errors not generated because of rate limitation     Output histogram:         router solicitation: 3         neighbor solicitation: 1     0 messages with bad code fields     0 messages < minimum length     0 bad checksums     0 messages with bad length     Histogram of error messages to be generated:         0 no route         0 administratively prohibited         0 beyond scope         0 address unreachable         0 port unreachable         0 packet too big         0 time exceed transit         0 time exceed reassembly         0 erroneous header field         0 unrecognized next header         0 unrecognized option         0 redirect         0 unknown     0 message responses generated     0 messages with too many ND options     0 messages with bad ND options     0 bad neighbor solicitation messages     0 bad neighbor advertisement messages     0 bad router solicitation messages     0 bad router advertisement messages     0 bad redirect messages     0 path MTU changes ipsec6:     0 inbound packets violated process security policy     0 inbound packets failed due to insufficient memory     0 invalid inbound packets     0 outbound packets violated process security policy     0 outbound packets with no SA available     0 outbound packets failed due to insufficient memory     0 outbound packets with no route available     0 invalid outbound packets     0 outbound packets with bundled SAs     0 spd cache hits     0 spd cache misses     0 clusters copied during clone     0 mbufs inserted during makespace rip6:     0 messages received     0 checksum calculations on inbound     0 messages with bad checksum     0 messages dropped due to no socket     0 multicast messages dropped due to no socket     0 messages dropped due to full socket buffers     0 delivered     0 datagrams output pfkey:     0 requests sent from userland     0 bytes sent from userland     0 messages with invalid length field     0 messages with invalid version field     0 messages with invalid message type field     0 messages too short     0 messages with memory allocation failure     0 messages with duplicate extension     0 messages with invalid extension type     0 messages with invalid sa type     0 messages with invalid address extension     0 requests sent to userland     0 bytes sent to userland     0 messages toward single socket     0 messages toward all sockets     0 messages toward registered sockets     0 messages with memory allocation failure ------------------------------------------------------------------------ netstat -m 514/3054/3568 mbufs in use (current/cache/total) 243/1537/1780/500054 mbuf clusters in use (current/cache/total/max) 256/1528 mbuf+clusters out of packet secondary zone in use (current/cache) 256/47/303/250027 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/222246 9k jumbo clusters in use (current/cache/total/max) 0/0/0/166684 16k jumbo clusters in use (current/cache/total/max) 1638K/4025K/5664K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0 sendfile syscalls 0 sendfile syscalls completed without I/O request 0 requests for I/O initiated by sendfile 0 pages read by sendfile as part of a request 0 pages were valid at time of a sendfile request 0 pages were valid and substituted to bogus page 0 pages were requested for read ahead by applications 0 pages were read ahead by sendfile 0 times sendfile encountered an already busy page 0 requests for sfbufs denied 0 requests for sfbufs delayed ------------------------------------------------------------------------ netstat -anA Active Internet connections (including servers) Tcpcb            Proto Recv-Q Send-Q Local Address      Foreign Address    (state) fffff800103827a0 udp4       0      0 *.* *.* ------------------------------------------------------------------------ netstat -aL Current listen queue sizes (qlen/incqlen/maxqlen) Proto Listen                           Local Address