Date: Tue, 30 Oct 2012 23:09:09 +0700 From: Adam Strohl <adams-freebsd@ateamsystems.com> To: Adrian Chadd <adrian@freebsd.org> Cc: FreeBSD-Stable ML <freebsd-stable@freebsd.org> Subject: Re: No buffer space available / tcp_inpcb value Message-ID: <508FFBA5.5000600@ateamsystems.com> In-Reply-To: <CAJ-VmonvY6hau%2BLpaa5b0hi%2Bssx_Y%2BdQ5JheDretc1jY9erpAg@mail.gmail.com> References: <508FD46D.3040608@ateamsystems.com> <CAJ-VmonvY6hau%2BLpaa5b0hi%2Bssx_Y%2BdQ5JheDretc1jY9erpAg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 10/30/2012 23:05, Adrian Chadd wrote: > Check the output of 'netstat -mb', maybe you're also running out of mbufs? There was nothing denied there that I can see: 35696/4039/39735 mbufs in use (current/cache/total) 2069/3797/5866/32768 mbuf clusters in use (current/cache/total/max) 2069/2077 mbuf+clusters out of packet secondary zone in use (current/cache) 4/3283/3287/16384 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/8192 9k jumbo clusters in use (current/cache/total/max) 0/0/0/4096 16k jumbo clusters in use (current/cache/total/max) 13078K/21735K/34813K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines > > Adrian > > > On 30 October 2012 06:21, Adam Strohl <adams-freebsd@ateamsystems.com> wrote: >> Hey -STABLE, >> >> I've got a client who we've setup a FreeBSD cluster for with about a dozens >> servers, all behind two front end proxies/LBs/firewalls which also act as >> NAT gateways for the internal servers. >> >> On the active front end proxy we've started seeing "fatal: socket: No buffer >> space available" errors during high-peak times. I can see in vmstat -z >> that this is what is getting denied: >> >> ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP >> tcp_inpcb: 392, 32770, 19398, 13372,1449734621,6312858, 0 >> >> We've got a lot of the other values bumped, and it appears to be this input >> limit that is getting hit. There are no other non-zero FAILed counters >> except 64 and 128 buckets which I believe are normal. >> >> I cannot seem to find the sysctl (or equiv) that controls this limit though, >> or even what it is. Anyone know? >> >> I'm obviously in need of this specific answer, but overall is there a codex >> of vmstat -z's items that explains this that I have just not found in my >> searches? This isn't the first time I've had to dig into a value like this >> to increase it's limit, but this time I'm not turning anything up. >> >> Any thoughts/ideas appreciated! >> >> -- >> Adam Strohl >> http://www.ateamsystems.com/ >> >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" -- Adam Strohl http://www.ateamsystems.com/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?508FFBA5.5000600>