Date: Mon, 11 Aug 2014 21:27:45 +0200 From: Michael Tuexen <Michael.Tuexen@lurchi.franken.de> To: "Eggert, Lars" <lars@netapp.com> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>, John-Mark Gurney <jmg@funkthat.com>, Niu Zhixiong <kaiaixi@gmail.com>, Bill Yuan <bycn82@gmail.com> Subject: Re: A problem on TCP in High RTT Environment. Message-ID: <5E8A6382-7096-495A-907C-86CE26A163A2@lurchi.franken.de> In-Reply-To: <7A4120EE-60F3-4D32-89C4-C694B8DFEAE4@netapp.com> References: <CAOENNMA_CiBDJc0kchzUbTcf_JBwTJPF=PdBAUB6FPo-KzYkeQ@mail.gmail.com> <20140809184232.GF83475@funkthat.com> <8AE1AC56-D52F-4F13-AAA3-BB96042B37DD@lurchi.franken.de> <20140809204500.GG83475@funkthat.com> <3F6BC212-4223-4AAC-8668-A27075DC55C2@lurchi.franken.de> <CAOENNMCPuiYS7LHwMfOczhZ4yisjGkpOmWzv2pcAoi9Hhzb7dw@mail.gmail.com> <20140810022350.GI83475@funkthat.com> <CAOENNMB3=FZx5kSHVPDPBTtMKbmYJ=c_XNMcuYuoLPe=6U%2Bkxg@mail.gmail.com> <CAOENNMARg36KH1Y%2B0wG8pd7sSf8XKnMf6g790_KiKaj3Mdwyjw@mail.gmail.com> <20140810033212.GL83475@funkthat.com> <CAOENNMA-dwPQr53bM4rzC=1eitoi-JAB4mCGx4zybFwUC=GMNg@mail.gmail.com> <17A804F3-BEA6-46F4-887F-B68750618FD9@netapp.com> <0CF85443-26AC-4931-9D00-3396C18C7690@lurchi.franken.de> <7A4120EE-60F3-4D32-89C4-C694B8DFEAE4@netapp.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 11 Aug 2014, at 14:12, Eggert, Lars <lars@netapp.com> wrote: > On 2014-8-11, at 9:17, Michael Tuexen = <Michael.Tuexen@lurchi.franken.de> wrote: >> Was there any suspicious output provided by netstat -m when the = problems occur? >=20 > root@laurel:~ # netstat -m > 8186/2179/10365 mbufs in use (current/cache/total) > 8184/1214/9398/2036224 mbuf clusters in use (current/cache/total/max) > 8184/885 mbuf+clusters out of packet secondary zone in use = (current/cache) > 0/5/5/1018111 4k (page size) jumbo clusters in use = (current/cache/total/max) > 0/0/0/301662 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/169685 16k jumbo clusters in use (current/cache/total/max) > 18414K/2992K/21407K bytes allocated to network (current/cache/total) > 544/57/8194 requests for mbufs denied (mbufs/clusters/mbuf+clusters) I guess the above is the problem. The card wants a lot of mbufs... So the problem should go away if you increase the number of = mbufs/clusters, which means no requests are denied and you don't experience any = performance issue. I ran into this on machines having several igb and ixgbe cards, each = wanting a lot of clusters for each of the receive rings per receive queue... Best regards Michael > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile >=20 > root@laurel:~ # uptime > 2:12PM up 37 mins, 3 users, load averages: 0.20, 0.25, 0.15 >=20 > Lars
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5E8A6382-7096-495A-907C-86CE26A163A2>