Date: Mon, 29 Jun 2015 11:02:59 +0200 From: Csaba Banhalmi <bimmer@field.hu> To: Adrian Chadd <adrian@freebsd.org>, Christopher Forgeron <csforgeron@gmail.com> Cc: FreeBSD Net <freebsd-net@freebsd.org> Subject: Re: FreeBSD 10.1-REL - network unaccessible after high traffic Message-ID: <559109C3.7070900@field.hu> In-Reply-To: <CAJ-Vmo=hSE%2Bk1q_JrX9wKOshSRa_WJ78hbL54ZaXMH03PrNFdg@mail.gmail.com> References: <374339249.53058039.1433681874571.JavaMail.root@uoguelph.ca> <55744F28.5000402@field.hu> <CAB2_NwA-D7bH47=Qkf9QLF3=mZOQBVo81bUsQzQr02W9U4vHMA@mail.gmail.com> <557AB1BB.60502@field.hu> <CAB2_NwA9i-wMXGH2%2BcP9SWxDMNomFRjoVP25hsGWaTDGjBxFTw@mail.gmail.com> <557AD10D.5070205@field.hu> <CAB2_NwAeD43tSwWO3LGuniRMNZ3TVupOuLWj3aUm228jLT2y1A@mail.gmail.com> <557AD2FA.103@field.hu> <CAB2_NwCgEvmMxqmAotO1USsipXOSaGkwK3Uu%2BiVbKd9_bn%2BLWg@mail.gmail.com> <CAJ-VmokXk69V_YURWOjLOQmKrW%2B2-YAiFQnhLOA7SKO6ipw_KQ@mail.gmail.com> <CAB2_NwANWRB2SJY0rO7n%2B_8RK61dyGJ5FCphH_ewQG-E7eOAUg@mail.gmail.com> <CAJ-Vmo=hSE%2Bk1q_JrX9wKOshSRa_WJ78hbL54ZaXMH03PrNFdg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi All, "vmstat 5" output when system freezes: procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad0 ad1 in sy cs us sy id 0 0 0 8752M 126M 5663 0 0 0 4042 445 66 0 1219 7148 4870 3 2 95 0 0 0 8650M 145M 2167 0 0 0 3501 447 79 0 974 4042 3578 1 1 98 0 0 0 8374M 201M 3113 0 0 0 6790 441 5 0 1130 6670 3729 3 1 96 0 0 0 8252M 220M 2632 0 0 0 4014 435 4 0 726 11653 2401 2 1 97 0 0 0 8188M 224M 1625 0 0 0 2189 434 5 0 713 6714 2376 1 1 98 0 0 0 7992M 233M 1504 0 0 0 2254 433 2 0 867 2890 2868 1 1 98 4 0 0 8032M 216M 2145 0 0 0 1995 435 18 0 526 3769 2048 1 1 98 0 0 0 8180M 195M 1949 0 0 0 1741 435 50 0 593 3441 2363 1 1 98 0 0 0 8186M 178M 2859 0 0 0 2525 436 6 0 499 3313 1733 2 1 97 1 0 0 8410M 146M 2521 0 0 0 1764 440 11 0 736 67271 2121 4 2 94 0 0 0 8182M 205M 2910 0 0 0 6378 927 8 0 495 16043 1775 1 1 98 1 1 0 7944M 210M 3009 0 0 0 3696 438 8 0 522 4247 1963 2 1 97 0 0 0 8091M 169M 7529 0 0 0 3601 436 105 0 1359 75290 4400 9 3 88 0 0 0 8121M 141M 4607 0 0 0 3288 444 62 0 949 12169 3268 5 1 94 0 0 0 8044M 201M 1782 0 0 0 4954 1795 9 0 446 3025 1927 1 1 99 0 0 0 7916M 222M 1296 0 0 0 2671 438 5 0 525 2984 1920 1 1 98 1 0 0 7870M 230M 888 0 0 0 1677 432 8 0 473 6424 2126 1 1 99 0 0 0 7968M 228M 3375 0 0 0 2625 433 51 0 768 4100 2852 3 1 96 0 0 0 8238M 194M 7586 0 0 0 4758 436 88 0 1026 9631 3908 4 2 94 0 0 0 8293M 185M 3253 0 0 0 2362 437 52 0 747 4475 3105 2 1 97 I increased the vm.v_free_min, but did not help. It was a different froze, the system was unreacheable even through IPMI, needed a hard reset. Regards, Csaba 2015.06.12. 20:17 keltezéssel, Adrian Chadd írta: > On 12 June 2015 at 10:57, Christopher Forgeron <csforgeron@gmail.com> wrote: >> I agree it shouldn't run out of memory. Here's what mine does under network >> load, or rsync load: >> >> 2 0 9 1822M 1834M 0 0 0 0 14 8 0 0 22750 724 136119 >> 0 23 77 >> >> 0 0 9 1822M 1823M 0 0 0 0 0 8 0 0 44317 347 138151 >> 0 16 84 >> >> 0 0 9 1822M 1761M 0 0 0 0 17 8 0 0 23818 820 92198 0 >> 12 88 >> >> 0 0 9 1822M 1727M 0 0 0 0 14 8 0 0 40768 634 126688 >> 0 17 83 >> >> 0 0 9 1822M 8192B 0 8 0 0 15 3 3 0 9236 305 57149 0 >> 33 67 >> >> >> That's with a 5 second vmstat output. After the 8KiB, the system is nearly >> completely brain-dead and needs a hard power-off. >> >> >> I've seen it go from 6 GiB free to 8KiB in 5 sec as well. Currently my large >> machines are set to 12 GiB free to keep them from crashing, from what I >> presume is just network load due to lots of iSCSI / NFS traffic on my 10GiB >> network. >> >> >> I haven't had time to type this up for the list yet, but I'm putting it here >> just to make sure people know it's real. >> > Hi, > > Then something is leaking or holding onto memory when it shouldn't be. > > Try doing vmstat -z and vmstat -m in a one second loop, post the data > just before it falls over. > > > -adrian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?559109C3.7070900>