From owner-freebsd-bugs@freebsd.org Sun Mar 28 07:07:18 2021 Return-Path: Delivered-To: freebsd-bugs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AAB385B7767 for ; Sun, 28 Mar 2021 07:07:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4F7Rbp45QNz3mgq for ; Sun, 28 Mar 2021 07:07:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 8C6D65B7566; Sun, 28 Mar 2021 07:07:18 +0000 (UTC) Delivered-To: bugs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8C21E5B7A10 for ; Sun, 28 Mar 2021 07:07:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F7Rbp3GD1z3mbW for ; Sun, 28 Mar 2021 07:07:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 638681BF84 for ; Sun, 28 Mar 2021 07:07:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 12S77Ig6042553 for ; Sun, 28 Mar 2021 07:07:18 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 12S77I2e042552 for bugs@FreeBSD.org; Sun, 28 Mar 2021 07:07:18 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 254453] [zone: mbuf] kern.ipc.mbufs limit reached - network memory leaks? Date: Sun, 28 Mar 2021 07:07:18 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: viaprog@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Mar 2021 07:07:18 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D254453 --- Comment #3 from Igor A. Valkov --- Bytes allocated to network in netstat -m is constantly growing. After approx 1 day uptime network is hang with: dmesg: [zone: mbuf] kern.ipc.nmbufs limit reached [zone: mbuf] kern.ipc.nmbufs limit reached [zone: mbuf] kern.ipc.nmbufs limit reached [zone: mbuf] kern.ipc.nmbufs limit reached [zone: mbuf] kern.ipc.nmbufs limit reached [zone: mbuf] kern.ipc.nmbufs limit reached netstat -m: 31994144/57556/32051700 mbufs in use (current/cache/total) 32898/14358/47256/5000000 mbuf clusters in use (current/cache/total/max) 8/10270 mbuf+clusters out of packet secondary zone in use (current/cache) 34079/298/34377/2000000 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/900000 9k jumbo clusters in use (current/cache/total/max) 0/0/0/600000 16k jumbo clusters in use (current/cache/total/max) 8200648K/44297K/8244945K bytes allocated to network (current/cache/total) 383111218/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 27833/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 1681210 sendfile syscalls 1678853 sendfile syscalls completed without I/O request 0 requests for I/O initiated by sendfile 0 pages read by sendfile as part of a request 12525327 pages were valid at time of a sendfile request 0 pages were valid and substituted to bogus page 0 pages were requested for read ahead by applications 0 pages were read ahead by sendfile 2703 times sendfile encountered an already busy page 0 requests for sfbufs denied 0 requests for sfbufs delayed net.inet.tcp.functions_available: Stack D Alias PCB count freebsd freebsd 10 bbr * bbr 44987 rack rack 0 vmstat -m: Type InUse MemUse Requests Size(s) filedesc 64 17654K 473=20 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536 pwddesc 111 7K 23949 64 sigio 31 2K 33 64 filecaps 0 0K 2314 16,32,64 kdtrace 1048 241K 24983 64,256 kenv 117 13K 117 16,32,64,128,8192 kqueue 204 2260K 24965=20 64,256,512,2048,4096,8192,16384,32768,65536 proc-args 311 12K 48278 16,32,64,128,256 Fail Points 0 0K 20 1024 hhook 13 4K 13 256 ithread 448 74K 448 32,128,256 prison 8 1K 8 16,32 KTRACE 100 13K 100 128 linker 213 2001K 257=20 16,32,64,128,256,384,512,1024,2048,4096,8192,16384 lockf 23 3K 2515 64,128 loginclass 3 1K 7 64 devbuf 18525 117840K 20947=20 16,32,64,128,256,384,512,1024,2048,4096,8192,65536 temp 195 20K 2142283=20 16,32,64,128,256,512,1024,2048,4096,8192,16384,65536 acpiintr 1 1K 1 64 module 234 30K 234 128 mtx_pool 2 72K 2 8192,65536 osd 134 3K 145 16,32,64,128,256 pmchooks 1 1K 1 128 pmc 2 1K 2 64 session 19 3K 7515 128 proc 3 132K 3 4096,65536 subproc 614 637K 24451 384,4096 cred 53 14K 74359 256 acpica 32007 3114K 4383696 16,32,64,128,256,384,512,1024,2048 acpitask 1 64K 1 65536 acpisem 33 5K 33 128 plimit 15 4K 28343 256 uidinfo 5 17K 6847 128,16384 sendfile 0 0K 4179384=20 64,128,256,384,512,1024,2048,4096,8192,16384 dumper 2 1K 2 128,512 acpidev 232 15K 232 64 sysctl 1 1K 401 32,64 sysctloid 50151 2825K 50666 16,32,64,128,256,384 sysctltmp 0 0K 34983 16,32,64,256,1024 CAM SIM 14 4K 14 256 CAM XPT 43 5K 97 16,32,128,256,512,1024,2048 tidhash 3 260K 3 8192 callout 32 8192K 32 umtx 2124 266K 2124 128 p1003.1b 1 1K 1 16 SWAP 1 4164K 1 bus 1555 207K 264553 16,32,64,128,256,384,4096 bus-sc 225 2460K 20627=20 16,32,64,128,256,384,512,1024,2048,4096,8192,16384,32768,65536 CAM DEV 16 32K 38 2048 devstat 12 25K 12 32,4096 eventhandler 142 12K 142 64,128 gtaskqueue 198 44K 198 16,32,256,8192 kobj 132 528K 484 4096 Per-cpu 1 1K 1 32 CAM CCB 0 0K 63419 2048 rman 378 43K 769 16,32,128 sbuf 1 1K 7285=20 16,32,64,128,256,512,1024,2048,4096,8192,16384 toponodes 84 11K 84 128 CAM path 17 1K 110 32 CAM periph 6 2K 39 16,32,64,128,256 acpi_perf 32 12K 32 384 stack 0 0K 201 256 taskqueue 54 7K 54 16,32,256 terminal 11 3K 11 256 Unitno 22 2K 4932 32,64 vmem 4 448K 37 2048,4096,8192,16384,32768,65536 ioctlops 0 0K 11948 256,384,512,1024,2048,4096 select 12 2K 12853 128 iov 0 0K 1542747788 16,64,128,256,384,512,1024,2048 selfd 41 3K 2383115 64 procdesc 4 1K 21 128 msg 4 30K 4 2048,4096,8192,16384 sem 4 106K 4 2048,4096 shm 1 32K 1 32768 tty 14 14K 18 1024 pts 0 0K 4 256 ktls 1 4K 1 4096 mbuf_tag 0 0K 1013618 32,64 shmfd 1 8K 1 8192 soname 4 1K 256798386 16,32,64,128 pcb 2309 3224K 48094843 16,32,64,1024,2048 ix 22 74K 22 1024,2048,4096 kbdmux 7 22K 7 16,384,1024,2048,16384 LED 40 3K 40 16,128 mfibuf 133 51K 13638=20 16,32,128,256,384,1024,2048,4096,8192,32768 sr_iov 2 1K 2 384 entropy 1 1K 29218 32,64,4096 UART 6 5K 6 16,1024 CAM queue 30 9K 98 16,32,512 USB 61 97K 69 16,128,256,384,1024,2048,4096,8192,163= 84 vfscache 3 8384K 3 65536 USBdev 47 10K 47 32,64,128,256,384,512,4096 vtbuf 24 1968K 46 4096 vt 13 41K 13 384,4096,32768 cl_savebuf 0 0K 168291 64,128,256,384 vfs_hash 1 4096K 1 vnodes 1 1K 2 32,256 mount 76 4K 153 16,32,64,128,256,1024 statfs 0 0K 319 4096 vnodemarker 3 2K 60944 512 fadvise 1 1K 5 32 chacha20random 1 8K 1 8192 BPF 12 1026K 14 32,128,384,512 ifnet 9 17K 9 128,2048 ifaddr 90 19K 90 16,32,64,128,256,384,2048,4096 ether_multi 64 6K 104 16,32,64,128 clone 7 1K 7 128 lltable 47 14K 1280 256,384 tun 3 1K 3 32 vlan 13 2K 33 64,128 iflib 402 3951K 418 64,128,256,512,1024,8192,16384,32768 pfil 7 1K 15 64,128 nhops 6 1K 6 128 routetbl 19 4K 19 32,64,128,384,1024 vnet 1 1K 1 64 vnet_data 1 180K 1 vnet_data_free 1 1K 1 32 igmp 8 1K 8 128 in_multi 3 1K 3 256 ip_moptions 2 1K 2 64 tfo_ccache 1 128K 1 hostcache 1 32K 1 32768 LRO 64 1280K 64 8192,32768 tcp_hpts 65 51209K 65 256 DEVFS3 168 42K 205 256 tcpfunc 3 1K 3 64 syncache 1 136K 1 in6_multi 31 4K 31 32,256 ip6_moptions 3 1K 3 32 ip6opt 5 2K 11 32,256 mld 7 1K 7 128 ip6ndp 10 2K 12 64,256 inpcbpolicy 45039 1408K 253441302 32 secasvar 1 1K 1 1024 sahead 1 1K 1 1024 ipsecpolicy 1 1K 1 1024 ipsec-saq 2 2K 2 1024 pf_temp 0 0K 3381 32 pf_hash 5 11524K 5 2048 pf_ifnet 13 5K 57 256,2048 pf_rule 38 24K 38 128,1024 pf_osfp 1191 123K 1191 64,128 crypto 3 5K 3 64,256,4096 DEVFS1 139 139K 166 1024 audit_evclass 236 8K 294 32 pagedep 4 2048K 293568 256 inodedep 4 16384K 1319452 384 bmsafemap 4 32K 469650 256,8192 newblk 4 32768K 4749893 256 indirdep 0 0K 24729 128,32768 freefrag 0 0K 496137 128 freeblks 0 0K 529155 256 freefile 0 0K 534502 64 diradd 0 0K 795038 128 mkdir 0 0K 18 128 dirrem 0 0K 932077 128 newdirblk 0 0K 297 64 freework 4 1K 934622 64,128 jaddref 0 0K 6 128 jremref 0 0K 6 128 jnewblk 0 0K 30 128 jfreefrag 0 0K 3 128 jseg 0 0K 6 128 jsegdep 0 0K 45 64 sbdep 0 0K 6177 64 savedino 0 0K 393583 256 jblocks 2 1K 2 128,256 softdep 4 2K 4 512 ufs_dirhash 1768 738K 380492 16,32,64,128,256,384,512,1024,2048 ufs_quota 1 4096K 1 ufs_mount 25 631K 42 32,64,512,4096,8192,16384,32768 vm_pgdata 1 1K 1 128 DEVFS_RULE 59 21K 59 64,384 UMA 240 5K 243 16,32 DEVFS 19 1K 20 16,128 DEVFSP 2 1K 4 64 pfs_nodes 20 8K 20 384 pfs_vncache 1 64K 1 65536 GEOM 126 22K 1445=20 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768 raid_data 0 0K 222 32,128,256 pci_link 16 2K 16 64,128 apmdev 1 1K 1 128 intr 4 216K 4 32768,65536 MCA 65 17K 65 128,256,384 cpus 2 1K 2 128 nexusdev 6 1K 6 16 memdesc 1 4K 1 4096 ioat 16 7168K 16 isadev 6 1K 6 128 scsi_cd 0 0K 12 16 CAM dev queue 14 1K 14 64 nvlist 94 9K 94 16,64,128 madt_table 0 0K 2 256,4096 cdev 1 1K 1 256 io_apic 3 6K 3 2048 local_apic 1 32K 1 32768 msi 65 9K 65 128 acpiwmi 2 1K 4 32,64 kstat_data 17 17K 17 1024 solaris 131 65839K 248 16,32,64,128,256,1024,2048,4096,8192,6= 5536 taskq 2 3K 2 1024,2048 --=20 You are receiving this mail because: You are the assignee for the bug.=