From owner-freebsd-current@freebsd.org Fri Sep 29 09:18:44 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0B42BE2801E for ; Fri, 29 Sep 2017 09:18:44 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 579BC844CE; Fri, 29 Sep 2017 09:18:43 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from freyja.zeit4.iv.bundesimmobilien.de ([87.138.105.249]) by mail.gmx.com (mrgmx003 [212.227.17.190]) with ESMTPSA (Nemesis) id 0MXqaN-1dsCNt3C5f-00Wpfg; Fri, 29 Sep 2017 11:18:39 +0200 Date: Fri, 29 Sep 2017 11:18:37 +0200 From: "O. Hartmann" To: "O. Hartmann" Cc: Hans Petter Selasky , Guido Falsi , freebsd-current Subject: [SOLVED] Re: net/asterisk13: memory leak under 12-CURRENT? Message-ID: <20170929111837.79bb7c32@freyja.zeit4.iv.bundesimmobilien.de> In-Reply-To: <20170928075701.23ca8b10@freyja.zeit4.iv.bundesimmobilien.de> References: <20170926144522.21e59cfe@freyja.zeit4.iv.bundesimmobilien.de> <979b6cfe-0e38-5df3-7bb5-cdb8de6677bf@FreeBSD.org> <20170926154155.28deb2e1@freyja.zeit4.iv.bundesimmobilien.de> <31faf367-69e2-b7e3-dd14-67bf69a67ec2@selasky.org> <20170928075701.23ca8b10@freyja.zeit4.iv.bundesimmobilien.de> Organization: Walstatt MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:11CDzZo09CD2eyE4kd1loSOuMkJcpJx1txwfDGrYN0e1ZdolaKZ NukyKMukwuruxgHREO4dURh7+VFWdvGu2hFDepThZtlTXn1eXyOJww0t96aOmdEOUW7av5l rIREb+yPDRtty59L3AjfYxRX07PgFC01bDP22WreYmJ6hUJRYn68pz9sHjSypia7R57xFQZ 5c4m4E/pF1cndq4wBHFEA== X-UI-Out-Filterresults: notjunk:1;V01:K0:UtEYn3p7rz0=:guooH+oQKs/xCn7l1bV1fY N66H8GIW/HW91DL0C6tYt+vG0xogQ33s7VnHlPdw2dLn7nFwp+hEN91ZKce8zOxBEXBVkqmfG 3zD0Im4eYh80VYOCR/yCjQqtBLs2F7mLxfoUeCMuqcCS96a1KnHwjqf9nP3utdJ237aEgv9YP UYXiBIushEoD3CZkxPMZDRGq8WVxzJP71G2qLHkar76xODb1HDOYvlXxIU27uMEyXQv4seCJ7 +7MqRgip5vQJP5wqRHC+HfJTaSsU6IxxapNsM/jsxh+rYuDp6KOsg0ZXbtpoAixnsYsamg9Ib ft+E1teHmxzl1OH3b0QXqctyDiCkuqKi+wcVROw55T/wGCwRz3uVpVacrQU+vLm0og+M4NgRY wJu4L5MXzbkTm5HSDAiXp+xFhd2zZjfaeyeSzHrZevuCF49LSb2+5SlplGCBtDV7ZZ7clS8Bc h/w0JBRVPW9ZnWCEGBCXylECeu3fah+Co+1FPrTJfEoWSYGvXGwrOEhx0gAhBbUIrSV+L2eE6 wTHAKrr4L6hbbzXKmmxRRRZmDyPElPN9MfRmCbk7ls47c4TaKGCPhvxyxVHtBVfa5BBSzCjfV hEL98yyThZkANjXWpp0XX67LEoZJVReqmoiiGr5a7pKAitGU8CVFQplHmViFx8rRT3eIuOdIC OuLfnKZRB6WtODC1o56Vo/GWV/spv1263gKCsejUtZ1/LMPCRd4HzvFty0sHkGIVx0G9iD/a5 Rqx9dXv/UV6gcIMKNWPF+F1I+LXq8qmsT0CevvVdTn2utzGDFkp4HCxJ6tleVGy7HQC20w4od rTREPRrbEc7s88Niriho2Exuz13+MwKdHZn9eAZoipdNW/bfjA= X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 09:18:44 -0000 On Thu, 28 Sep 2017 07:57:01 +0200 "O. Hartmann" wrote: > On Wed, 27 Sep 2017 12:51:18 +0200 > Hans Petter Selasky wrote: > > > On 09/27/17 09:05, Guido Falsi wrote: > > > On 09/26/2017 15:41, O. Hartmann wrote: > > >> On Tue, 26 Sep 2017 15:06:23 +0200 > > >> Guido Falsi wrote: > > > > > >> Since I run net/asterisk with automatic module loading (I'm new to > > >> asterisk), this is very likely and might cause the problem somehow. > > >> > > > > > > You can exclude single modules from autoloading via modules.conf. > > > > > >>> Not sure, restarting the daemon should free any leaked memory the daemon > > >>> has. If a killed process leaves memory locked at the system level there > > >>> should be some other cause. > > >> > > >> Even with no runnidng asterisk, memory level drops after the last > > >> shutdown of asterisk and keeps that low. Even for weeks! My router never > > >> shows that high memory consumption, even under load. > > > > > > But while asterisk is running does the memory usage increase unbounded > > > till filling all available memory or does it stabilize at some point? > > > > > > Asterisk is relatively memory hungry, especially with all modules > > > enabled. It also caches and logs various information in RAM, even doing > > > "nothing" it will cache and log that "nothing" activity. If memory does > > > stabilize after some point it's not really a leak but it's standard > > > memory usage. To reduce it you should disable all unused modules. > > > > > >> > > >> The question would be: how to use vmstat to give hints for those familiar > > >> with memory subsystems to indicate a real bug? > > >> > > >> I tried to find some advices, but maybe my English isn't good enough to > > >> make google help. > > > > > > I'm not able to give you a correct indication, but if the memory usage > > > is not increasing indefinitely but is stabilizing I'd say it's not > > > really a leak. > > > > > > > Did you look at the output from "vmstat -m" and "vmstat -z" ? > > > > --HPS > > I did not, but now I will ;-) > > Thanks for the hint! > > Kind regards, > Oliver For the completeness. Guido already explained that things I considered "stange' could be normal. I think I missed the point, that in some cases, caching data can also persist even when the process, which I considered to be the consumer, died. The vmstat output is taken after a couple of restarts of asterisk and while asterisk was still running. I had a very simplistic, outdated memory model in mind and Guido made that clear that it isn't. I was concerned about if it comes to the use of a small SoC with only 1 or 2 GB of RAM. Sorry for the noise :-( Kind regards, Oliver [...] last pid: 19557; load averages: 0.22, 0.18, 0.14 up 1+14:17:56 11:14:44 25 processes: 1 running, 24 sleeping CPU: 0.3% user, 0.0% nice, 0.8% system, 0.0% interrupt, 98.9% idle Mem: 27M Active, 104M Inact, 392M Wired, 146M Buf, 3421M Free Swap: 8192M Total, 8192M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 36617 asterisk 70 20 0 147M 74236K piperd 2 3:38 2.42% asterisk 19557 root 1 20 0 13180K 3508K CPU3 3 0:00 0.13% top 93400 root 1 20 0 11444K 2100K select 3 1:34 0.06% syslogd 12569 root 1 20 0 15136K 3576K select 1 4:35 0.04% ppp 88182 administrator 1 20 0 18768K 7980K select 0 0:00 0.02% sshd 24901 root 1 20 0 10840K 1704K select 1 0:19 0.01% powerd 17169 root 1 20 0 18192K 18296K select 2 0:10 0.01% ntpd 10719 root 1 20 0 14724K 4712K bpf 1 0:03 0.00% arpwatch 9691 bind 7 52 0 64552K 38192K sigwai 3 0:19 0.00% named 98424 root 1 20 0 32100K 20948K nanslp 2 0:05 0.00% perl 39431 asterisk 1 52 0 13848K 5072K select 0 0:01 0.00% mpg123 73938 root 1 20 0 11332K 2036K nanslp 3 0:01 0.00% cron 45343 daemon 1 20 0 11316K 1940K select 2 0:00 0.00% rpcbind 41607 root 1 20 0 10976K 1848K kqread 1 0:00 0.00% autounmountd 82722 root 1 22 0 18396K 7920K select 1 0:00 0.00% sshd 97178 root 1 21 0 16216K 6780K select 2 0:00 0.00% sudo 97625 root 1 20 0 13604K 4356K pause 2 0:00 0.00% csh 46309 root 1 20 0 10880K 1784K autofs 3 0:00 0.00% automountd 90914 administrator 1 20 0 13172K 4184K pause 0 0:00 0.00% csh 40858 asterisk 1 52 0 13616K 3040K pipewr 3 0:00 0.00% mpg123 97392 root 1 23 0 11776K 3096K wait 3 0:00 0.00% su 77979 dhcpd 1 20 0 21896K 8340K select 0 0:00 0.00% dhcpd 69003 root 1 20 0 18000K 4904K select 0 0:00 0.00% sshd 52102 root 1 52 0 10880K 1788K ttyin 2 0:00 0.00% getty 97105 root 1 20 0 10868K 2444K ttyin 3 0:00 0.00% getty [vmstat -m] root@gate:~ # vmstat -m Type InUse MemUse HighUse Requests Size(s) acpiintr 1 1K - 1 64 acpica 2689 269K - 53492 16,32,64,128,256,512,1024 vtbuf 8 656K - 14 4096 vt 3 2K - 3 512 autofs 6 9K - 8219485 16,128,8192 acpitask 1 64K - 1 65536 acpisem 24 3K - 24 128 acpidev 32 2K - 32 64 CAM SIM 3 1K - 3 256 CAM XPT 18 2K - 112 32,64,128,256,512,1024,2048,65536 CAM DEV 4 8K - 10 2048 DEVFS3 140 35K - 520 256 DEVFS1 109 55K - 349 512 DEVFS_RULE 55 26K - 55 64,512 DEVFS 11 1K - 12 16,128 DEVFSP 7 1K - 7 64 NFSD V4client 1 1K - 1 256 NFSD lckfile 1 1K - 1 256 NFSD session 1 1K - 1 1024 pfs_nodes 20 5K - 20 256 tmpfs mount 2 1K - 2 128 tmpfs name 1500 30K - 2098 16,32,64 eli data 7 2K - 29 64,256,512,1024 CAM CCB 0 0K - 117024 2048 geom_flashmap 0 0K - 24 256 GEOM 278 42K - 12066 16,32,64,128,256,512,1024,2048,4096,8192,16384 raid_data 0 0K - 2088 32,128,256 isadev 6 1K - 6 128 CAM path 5 1K - 38 32 CAM periph 4 1K - 19 16,32,64,128,256 acpi_perf 4 1K - 4 128 cdev 4 1K - 4 256 filedesc 7 85K - 276 128,4096,65536 sigio 0 0K - 1 64 filecaps 0 0K - 25082 16,32,64 kenv 119 12K - 122 16,32,64,128,8192 kqueue 61 17K - 16840 64,256,512,2048,8192 proc-args 56 3K - 29790 16,32,64,128,256 hhook 13 3K - 13 32,256 ithread 130 26K - 130 32,128,256 linker 127 9K - 129 16,32,64,512 lockf 23 3K - 5019 64,128 loginclass 3 1K - 3 64 cache 1 1K - 1 32 devbuf 16929 33991K - 17350 16,32,64,128,256,512,1024,2048,4096,8192 temp 75 5K - 79184 16,32,64,128,256,512,1024,2048,4096,8192,16384,65536 CAM I/O Scheduler 1 1K - 1 1024 module 327 41K - 327 128 mtx_pool 2 16K - 2 8192 osd 3 1K - 9 16,32,64,128,256 pmchooks 1 1K - 1 128 pgrp 23 3K - 922 128 session 19 3K - 359 128 proc 2 32K - 2 16384 subproc 162 267K - 16726 512,4096 cred 111 28K - 16500 256 kbdmux 5 22K - 5 512,1024,2048,16384 plimit 22 6K - 5489 256 uidinfo 9 5K - 287 128,4096 sysctl 0 0K - 21212 16,32,64 sysctloid 5895 303K - 5975 16,32,64,128 sysctltmp 0 0K - 1183 16,32,64,256,1024 tidhash 1 32K - 1 32768 callout 5 2184K - 5 umtx 560 70K - 560 128 p1003.1b 1 1K - 1 16 SWAP 2 549K - 2 32 bus 2646 191K - 5020 16,32,64,128,256,1024 bus-sc 61 1729K - 1214 16,32,64,128,256,512,1024,2048,4096,8192,16384,65536 devstat 16 33K - 16 32,4096 eventhandler 107 9K - 107 64,128 gtaskqueue 30 27K - 30 16,32,256,8192 kobj 131 524K - 3199 4096 Per-cpu 1 1K - 1 32 rman 164 19K - 553 32,128 sbuf 1 1K - 9894 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536 toponodes 14 2K - 14 128 taskqueue 36 4K - 48 16,32,256 terminal 3 1K - 3 256 Unitno 41 3K - 7477 32,64 vmem 3 200K - 5 4096,8192,65536 ioctlops 1 8K - 297 256,1024,8192 select 181 23K - 280 128,8192 iov 0 0K - 2135314 16,64,128,256 msg 4 30K - 4 2048,4096,8192,16384 sem 4 106K - 4 2048,4096 shm 1 32K - 1 32768 tty 7 7K - 10 1024 pts 1 1K - 4 256 mbuf_tag 175 11K - 84049978 32,64,256 shmfd 1 8K - 1 8192 soname 4 1K - 2413713 16,32,128 pcb 37 663K - 33716 16,32,64,128,1024,2048,8192 acl 0 0K - 3696 4096 vfscache 4 2097K - 4 256,16384,32768 cl_savebuf 0 0K - 956 64,128,256 vfs_hash 1 1024K - 1 vnodes 1 1K - 1 256 mount 98 5K - 539 16,32,64,128,256 statfs 0 0K - 32609 4096 vnodemarker 0 0K - 33467 512 GSS-API 1 1K - 1 64 chacha20random 1 1K - 1 1024 iconv 2 1K - 2 128 BPF 42 1079K - 43 16,64,128,512,4096 BPF_JIT 7 1K - 15 16,32,64 ifnet 15 29K - 15 128,2048 ifaddr 69 13K - 70 16,32,256,512,4096 ether_multi 36 2K - 36 16,64 clone 17 3K - 17 128 ipsec 1 1K - 1 256 lltable 38 12K - 624 256,512 sppp 1 2K - 1 2048 tun 1 1K - 1 256 vlan 14 2K - 14 64,128 entropy 0 0K - 15571 4096 iflib 132 816K - 144 32,64,128,1024,2048,8192,16384,32768 routetbl 96 19K - 123 32,128,256,512,4096 vnet 1 1K - 1 64 vnet_data 1 64K - 1 65536 vnet_data_free 1 1K - 1 32 netgraph 9 1K - 9 64 netgraph_msg 0 0K - 22 64,128,256,512,1024 netgraph_hook 4 1K - 6 128 netgraph_node 14 4K - 17 128,256 netgraph_pppoe 2 17K - 5 64,512,16384 netgraph_path 0 0K - 9 16 netgraph_sock 3 1K - 6 32,128 igmp 14 2K - 14 128 ipid 2 24K - 2 8192,16384 in_mfilter 1 1K - 1 1024 in_multi 8 2K - 8 256 ip_moptions 2 1K - 2 64,256 encap_export_host 2 2K - 2 1024 mroutetbl 1 1K - 1 256 sctp_a_it 0 0K - 8 16 sctp_vrf 1 1K - 1 64 sctp_ifa 8 1K - 9 128 sctp_ifn 8 1K - 8 128 sctp_iter 0 0K - 8 256 hostcache 1 32K - 1 32768 LRO 24 480K - 24 8192,32768 tcpfunc 1 1K - 1 64 syncache 1 68K - 1 libalias 66 136K - 31040 128,65536 sctpnat 6 80K - 6 8192,16384 inpcbpolicy 32 1K - 37745 32 secasvar 1 1K - 1 1024 sahead 1 1K - 1 1024 ipsecpolicy 2 2K - 2 256,1024 ipsec-saq 2 2K - 2 1024 dummynet 4 4K - 4 512,1024 dummynet 13 6K - 278817 256,512 UART 6 5K - 6 16,1024 IpFw/IpAcct 96 59K - 163 16,32,64,128,256,512,1024,2048,4096,8192,16384 ipfw_tbl 10 2K - 10 128 crypto 1 1K - 1 512 xform 0 0K - 986 16,32,256 rpc 2 8K - 2 4096 audit_evclass 230 8K - 286 32 mactemp 0 0K - 194 8192 pagedep 2 256K - 416 256 inodedep 2 2048K - 4035 512 bmsafemap 2 16K - 3283 256,8192 newblk 2 4096K - 5152 256 indirdep 0 0K - 2368 128,32768 freefrag 0 0K - 183 128 freeblks 0 0K - 367 256 freefile 0 0K - 368 64 diradd 0 0K - 710 128 mkdir 0 0K - 6 128 dirrem 0 0K - 706 128 newdirblk 0 0K - 3 64 freework 2 1K - 772 16,128 freedep 0 0K - 4 64 jaddref 0 0K - 693 128 jremref 0 0K - 684 128 jmvref 0 0K - 42 128 jnewblk 0 0K - 5097 128 jfreefrag 0 0K - 181 128 jseg 0 0K - 4829 128 jsegdep 0 0K - 6655 64 sbdep 0 0K - 1182 64 savedino 0 0K - 1317 256 jblocks 4 1K - 12 128,256 softdep 2 1K - 27 512 ufs_dirhash 213 44K - 243 16,32,64,128,256,512,1024 ufs_quota 1 1024K - 1 ufs_mount 9 30K - 87 512,2048,4096,8192 vm_pgdata 1 1K - 1 128 UMAHash 7 38K - 18 512,1024,2048,4096,8192,16384,32768 CAM queue 7 3K - 26 16,32,512 USB 20 23K - 22 16,128,512,4096 fpukern_ctx 4 4K - 4 1024 memdesc 1 4K - 1 4096 aesni_data 7 6K - 7 32,256,1024 pci_link 12 1K - 12 32,128 cpuctl 1 1K - 1 32 USBdev 15 2K - 15 32,64,128,256 apmdev 1 1K - 1 128 madt_table 0 0K - 2 32,4096 CAM dev queue 3 1K - 3 64 io_apic 2 6K - 2 2048,4096 local_apic 1 4K - 1 4096 MCA 11 2K - 11 32,128 cpus 2 1K - 2 16 msi 16 2K - 16 128 nexusdev 6 1K - 6 16 root@gate:~ # [vmstat -z] root@gate:~ # vmstat -z ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP UMA Kegs: 256, 0, 221, 4, 221, 0, 0 UMA Zones: 704, 0, 222, 3, 222, 0, 0 UMA Slabs: 80, 0, 5287, 13, 5512, 0, 0 UMA Hash: 256, 0, 5, 10, 12, 0, 0 4 Bucket: 32, 0, 221, 904, 2102, 0, 0 6 Bucket: 48, 0, 23, 1056, 570, 0, 0 8 Bucket: 64, 0, 42, 1260, 37128, 11, 0 12 Bucket: 96, 0, 74, 705, 469791, 0, 0 16 Bucket: 128, 0, 126, 1207, 70315, 0, 0 32 Bucket: 256, 0, 198, 672, 27016, 135, 0 64 Bucket: 512, 0, 207, 425, 4310, 167, 0 128 Bucket: 1024, 0, 178, 318, 160115, 0, 0 256 Bucket: 2048, 0, 278, 38, 94068, 0, 0 vmem btag: 56, 0, 15673, 586, 17122, 115, 0 VM OBJECT: 240, 0, 7368, 696, 194473, 0, 0 RADIX NODE: 144, 0, 9059, 1147, 409047, 0, 0 MAP: 232, 0, 3, 65, 3, 0, 0 KMAP ENTRY: 120, 0, 5, 259, 5, 0, 0 MAP ENTRY: 120, 0, 2904, 2607, 808977, 0, 0 VMSPACE: 2496, 0, 26, 28, 16590, 0, 0 fakepg: 104, 0, 0, 0, 0, 0, 0 mt_zone: 16400, 0, 397, 0, 397, 0, 0 16: 16, 0, 23, 1232, 36289, 0, 0 16: 16, 0, 1211, 797, 25573, 0, 0 16: 16, 0, 25, 1230, 34, 0, 0 16: 16, 0, 222, 1033, 8220542, 0, 0 16: 16, 0, 1905, 856, 20512, 0, 0 16: 16, 0, 122, 1886, 2411743, 0, 0 16: 16, 0, 92, 1163, 70381, 0, 0 16: 16, 0, 0, 1255, 385, 0, 0 32: 32, 0, 105, 1270, 38699, 0, 0 32: 32, 0, 460, 665, 4210, 0, 0 32: 32, 0, 59, 1316, 33644, 0, 0 32: 32, 0, 1107, 768, 27259, 0, 0 32: 32, 0, 2444, 681, 5616, 0, 0 32: 32, 0, 66, 1059, 719, 0, 0 32: 32, 0, 182, 943, 1510, 0, 0 32: 32, 0, 11, 1114, 7866, 0, 0 64: 64, 0, 84, 970, 86, 0, 0 64: 64, 0, 919, 879, 19134, 0, 0 64: 64, 0, 207, 847, 1124, 0, 0 64: 64, 0, 688, 614, 1734, 0, 0 64: 64, 0, 351, 1199, 1545, 0, 0 64: 64, 0, 136, 1414, 24244, 0, 0 64: 64, 0, 8325, 789, 362569, 0, 0 64: 64, 0, 211, 1339,20905924, 0, 0 128: 128, 0, 662, 857, 23856, 0, 0 128: 128, 0, 1690, 573, 33432, 0, 0 128: 128, 0, 37, 490, 85, 0, 0 128: 128, 0, 835, 1304, 37383, 0, 0 128: 128, 0, 1749, 762, 3792, 0, 0 128: 128, 0, 392, 755, 10647, 0, 0 128: 128, 0, 343, 432, 6647, 0, 0 128: 128, 0, 33, 742, 5036, 0, 0 256: 256, 0, 82, 173, 1413, 0, 0 256: 256, 0, 19, 356, 2226, 0, 0 256: 256, 0, 28, 227, 30, 0, 0 256: 256, 0, 250, 365, 6275, 0, 0 256: 256, 0, 181, 314, 24437, 0, 0 256: 256, 0, 137, 358, 5862, 0, 0 256: 256, 0, 106, 329, 1713374, 0, 0 256: 256, 0, 32, 403,63057061, 0, 0 512: 512, 0, 1, 319, 4017, 0, 0 512: 512, 0, 114, 118, 369, 0, 0 512: 512, 0, 121, 55, 122, 0, 0 512: 512, 0, 76, 76, 143, 0, 0 512: 512, 0, 27, 613, 279136, 0, 0 512: 512, 0, 56, 96, 3584, 0, 0 512: 512, 0, 41, 111, 100, 0, 0 512: 512, 0, 4, 148, 33451, 0, 0 1024: 1024, 0, 1, 51, 16875, 0, 0 1024: 1024, 0, 18, 34, 909, 0, 0 1024: 1024, 0, 5, 11, 5, 0, 0 1024: 1024, 0, 13, 143, 600, 0, 0 1024: 1024, 0, 48, 44, 152, 0, 0 1024: 1024, 0, 13, 55, 922, 0, 0 1024: 1024, 0, 19, 49, 28, 0, 0 1024: 1024, 0, 0, 28, 2, 0, 0 2048: 2048, 0, 4, 22, 1622, 0, 0 2048: 2048, 0, 1, 9, 2, 0, 0 2048: 2048, 0, 1, 5, 1, 0, 0 2048: 2048, 0, 2, 4, 25, 0, 0 2048: 2048, 0, 3, 3, 3, 0, 0 2048: 2048, 0, 7, 33, 117156, 0, 0 2048: 2048, 0, 31, 15, 37, 0, 0 2048: 2048, 0, 4, 22, 48, 0, 0 4096: 4096, 0, 1, 2, 9, 0, 0 4096: 4096, 0, 0, 1, 1, 0, 0 4096: 4096, 0, 194, 16, 19831, 0, 0 4096: 4096, 0, 9, 8, 32834, 0, 0 4096: 4096, 0, 2, 7, 19229, 0, 0 4096: 4096, 0, 3, 3, 18, 0, 0 4096: 4096, 0, 28, 3, 34, 0, 0 4096: 4096, 0, 4, 1, 8, 0, 0 8192: 8192, 0, 0, 1, 1, 0, 0 8192: 8192, 0, 0, 1, 1, 0, 0 8192: 8192, 0, 2, 0, 2, 0, 0 8192: 8192, 0, 4, 8, 340, 0, 0 8192: 8192, 0, 41, 5, 140, 0, 0 8192: 8192, 0, 8, 6, 788, 0, 0 8192: 8192, 0, 3, 3, 9, 0, 0 8192: 8192, 0, 15, 1, 19, 0, 0 16384: 16384, 0, 2, 2, 11, 0, 0 16384: 16384, 0, 0, 1, 1, 0, 0 16384: 16384, 0, 0, 0, 0, 0, 0 16384: 16384, 0, 2, 1, 97, 0, 0 16384: 16384, 0, 11, 0, 11, 0, 0 16384: 16384, 0, 1, 9, 164, 0, 0 16384: 16384, 0, 1, 6, 13, 0, 0 16384: 16384, 0, 1, 0, 1, 0, 0 32768: 32768, 0, 0, 4, 8, 0, 0 32768: 32768, 0, 2, 0, 2, 0, 0 32768: 32768, 0, 0, 0, 0, 0, 0 32768: 32768, 0, 0, 0, 0, 0, 0 32768: 32768, 0, 13, 0, 13, 0, 0 32768: 32768, 0, 2, 0, 2, 0, 0 32768: 32768, 0, 0, 6, 30, 0, 0 32768: 32768, 0, 12, 0, 12, 0, 0 65536: 65536, 0, 0, 6, 2062, 0, 0 65536: 65536, 0, 0, 0, 0, 0, 0 65536: 65536, 0, 0, 0, 0, 0, 0 65536: 65536, 0, 6, 1, 104, 0, 0 65536: 65536, 0, 1, 0, 1, 0, 0 65536: 65536, 0, 1, 2, 13, 0, 0 65536: 65536, 0, 0, 4, 10, 0, 0 65536: 65536, 0, 0, 0, 0, 0, 0 SLEEPQUEUE: 80, 0, 281, 246, 281, 0, 0 64 pcpu: 8, 0, 1842, 462, 1846, 0, 0 Files: 80, 0, 149, 751, 264053, 0, 0 filedesc0: 1104, 0, 53, 64, 16616, 0, 0 rl_entry: 40, 0, 168, 932, 168, 0, 0 TURNSTILE: 136, 0, 281, 159, 281, 0, 0 umtx pi: 96, 0, 0, 0, 0, 0, 0 umtx_shm: 88, 0, 0, 0, 0, 0, 0 MAC labels: 40, 0, 52353, 1347, 200398, 0, 0 mac_mls: 112, 0, 52528, 1372,21106277, 0, 0 mac_biba: 112, 0, 52528, 847,21106277, 0, 0 PROC: 1344, 0, 52, 56, 16615, 0, 0 THREAD: 1360, 0, 239, 41, 3200, 0, 0 cpuset: 96, 0, 84, 490, 121, 0, 0 audit_record: 1280, 0, 0, 0, 0, 0, 0 mbuf_packet: 256, 1615545, 22, 1243, 1054258, 0, 0 mbuf: 256, 1615545, 6228, 1627,20476925, 0, 0 mbuf_cluster: 2048, 757350, 7309, 109, 7108913, 0, 0 mbuf_jumbo_page: 4096, 126213, 0, 46, 300, 0, 0 mbuf_jumbo_9k: 9216, 37396, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 21035, 0, 0, 0, 0, 0 NetGraph items: 72, 4123, 0, 279, 26, 0, 0 NetGraph data items: 72, 4123, 0, 527, 1775955, 0, 0 g_bio: 376, 0, 0, 1720, 600366, 0, 0 DMAR_MAP_ENTRY: 120, 0, 0, 0, 0, 0, 0 ttyinq: 160, 0, 255, 195, 1410, 0, 0 ttyoutq: 256, 0, 132, 303, 730, 0, 0 nvme_request: 128, 0, 0, 0, 0, 0, 0 cryptop: 88, 0, 0, 0, 0, 0, 0 cryptodesc: 72, 0, 0, 0, 0, 0, 0 vtnet_tx_hdr: 24, 0, 0, 0, 0, 0, 0 FPU_save_area: 832, 0, 0, 0, 0, 0, 0 VNODE: 472, 0, 51606, 282, 64870, 0, 0 VNODEPOLL: 120, 0, 4, 392, 5, 0, 0 BUF TRIE: 144, 0, 3578, 23098, 28233, 0, 0 NAMEI: 1024, 0, 0, 80, 9028599, 0, 0 rentr: 24, 0, 0, 0, 0, 0, 0 S VFS Cache: 108, 0, 51645, 855, 75910, 0, 0 STS VFS Cache: 148, 0, 0, 0, 0, 0, 0 L VFS Cache: 328, 0, 1418, 94, 1538, 0, 0 LTS VFS Cache: 368, 0, 0, 0, 0, 0, 0 NCLNODE: 592, 0, 0, 0, 0, 0, 0 nandfs node zone: 312, 0, 0, 0, 0, 0, 0 DIRHASH: 1024, 0, 398, 46, 519, 0, 0 fuse_ticket: 224, 0, 0, 0, 0, 0, 0 autofs_request: 5280, 0, 0, 7, 74, 0, 0 autofs_node: 208, 0, 4, 129, 4, 0, 0 Mountpoints: 2744, 0, 7, 14, 33, 0, 0 AIO: 224, 0, 0, 0, 0, 0, 0 AIOP: 32, 0, 0, 0, 0, 0, 0 AIOCB: 1520, 0, 0, 0, 0, 0, 0 AIOL: 128, 0, 0, 0, 0, 0, 0 AIOLIO: 280, 0, 0, 0, 0, 0, 0 pipe: 760, 0, 8, 72, 2352, 0, 0 procdesc: 136, 0, 0, 0, 0, 0, 0 ksiginfo: 112, 0, 94, 956, 416385, 0, 0 itimer: 352, 0, 0, 0, 0, 0, 0 bridge_rtnode: 64, 0, 0, 0, 0, 0, 0 KNOTE: 160, 0, 20, 305, 83900, 0, 0 socket: 856, 129972, 53, 79, 38782, 0, 0 IPsec SA lft_c: 16, 0, 0, 0, 0, 0, 0 ipq: 56, 23714, 0, 781, 29, 0, 0 udp_inpcb: 424, 129978, 20, 160, 37423, 0, 0 udpcb: 32, 130000, 20, 1355, 37423, 0, 0 tcp_inpcb: 424, 129978, 11, 106, 138, 0, 0 tcpcb: 920, 129972, 11, 41, 138, 0, 0 tcptw: 88, 26010, 0, 540, 55, 0, 0 syncache: 168, 15364, 0, 253, 6, 0, 0 hostcache: 96, 15375, 0, 287, 7, 0, 0 sackhole: 32, 0, 0, 0, 0, 0, 0 tcpreass: 40, 47400, 0, 0, 0, 0, 0 sctp_ep: 1208, 129972, 0, 0, 0, 0, 0 sctp_asoc: 2400, 40000, 0, 0, 0, 0, 0 sctp_laddr: 48, 80012, 0, 830, 9, 0, 0 sctp_raddr: 728, 80000, 0, 0, 0, 0, 0 sctp_chunk: 152, 400010, 0, 0, 0, 0, 0 sctp_readq: 152, 400010, 0, 0, 0, 0, 0 sctp_stream_msg_out: 112, 400015, 0, 0, 0, 0, 0 sctp_asconf: 40, 400000, 0, 0, 0, 0, 0 sctp_asconf_ack: 48, 400060, 0, 0, 0, 0, 0 udplite_inpcb: 424, 129978, 0, 0, 0, 0, 0 ripcb: 424, 129978, 1, 116, 88, 0, 0 unpcb: 240, 129984, 17, 239, 1114, 0, 0 rtentry: 208, 0, 56, 191, 70, 0, 0 IPFW counters: 16, 0, 57, 967, 57, 0, 0 IPFW dynamic rule: 128, 16399, 12, 1631, 48491, 0, 0 divcb: 424, 129978, 0, 0, 0, 0, 0 selfd: 64, 0, 233, 1317,23093991, 0, 0 swpctrie: 144, 504873, 0, 0, 0, 0, 0 swblk: 136, 504861, 0, 0, 0, 0, 0 FFS inode: 160, 0, 50104, 421, 62783, 0, 0 FFS1 dinode: 128, 0, 0, 0, 0, 0, 0 FFS2 dinode: 256, 0, 50104, 416, 62782, 0, 0 TMPFS dirent: 64, 0, 1298, 500, 1298, 0, 0 TMPFS node: 240, 0, 1299, 173, 1299, 0, 0 TMPFS dirent: 64, 0, 158, 896, 716, 0, 0 TMPFS node: 240, 0, 159, 273, 713, 0, 0