From owner-freebsd-questions@FreeBSD.ORG Mon Apr 23 11:16:14 2007 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4CAFB16A403; Mon, 23 Apr 2007 11:16:14 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from rwcrmhc14.comcast.net (rwcrmhc14.comcast.net [216.148.227.154]) by mx1.freebsd.org (Postfix) with ESMTP id 3055013C4DE; Mon, 23 Apr 2007 11:16:14 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from icarus.home.lan (c-71-198-0-135.hsd1.ca.comcast.net[71.198.0.135]) by comcast.net (rwcrmhc14) with ESMTP id <20070423110611m1400ilk04e>; Mon, 23 Apr 2007 11:06:11 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id DE85C1FA03D; Mon, 23 Apr 2007 04:06:10 -0700 (PDT) Date: Mon, 23 Apr 2007 04:06:10 -0700 From: Jeremy Chadwick To: "Marc G. Fournier" Message-ID: <20070423110610.GA13461@icarus.home.lan> Mail-Followup-To: "Marc G. Fournier" , Chris , freebsd-stable@freebsd.org, Thiago Esteves de Oliveira , "Brian A. Seklecki" , freebsd-questions@freebsd.org References: <3aaaa3a0704052217g67281716ge8a2d1a888025d0e@mail.gmail.com> <0D3D8201F40120DCE73AD00B@ganymede.hub.org> <3aaaa3a0704071212o3b4dce4dgd8def804e5b1156c@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.14 (2007-02-12) X-Mailman-Approved-At: Mon, 23 Apr 2007 11:33:12 +0000 Cc: Chris , "Brian A. Seklecki" , Thiago Esteves de Oliveira , freebsd-stable@freebsd.org, freebsd-questions@freebsd.org Subject: Re: No buffer space available X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Apr 2007 11:16:14 -0000 On Sat, Apr 07, 2007 at 08:20:58PM -0300, Marc G. Fournier wrote: > - --On Saturday, April 07, 2007 20:12:00 +0100 Chris wrote: > > Also to add I now have a 2nd box using 6.2 STABLE few days old code, > > had to use it because of broadcom 5755 nic card, I plan to use large > > tcp window sizes so will be interesting to see if this also suffers > > from the problem. > > I've got 8 servers on the same network, 3 are almost identical, but one of them > (the one with the problem) is using software RAID vs hardware ... but, if you > are seeing it without using software RAID, then that is obviously not the > culprit :( May be a red herring... I'm able to reproduce the "No buffer space available" message when setting net.inet.tcp.(send|recv)space to non-default values. All I've tried is the following, with a kernel dated 2007/04/22: # sysctl net.inet.tcp.sendspace=131072 # sysctl net.inet.tcp.recvspace=262144 Example session: $ su2 # sysctl net.inet.tcp.sendspace=131072 net.inet.tcp.sendspace: 32768 -> 131072 # sysctl net.inet.tcp.recvspace=262144 net.inet.tcp.recvspace: 65536 -> 262144 # logout $ ssh medusa socket: No buffer space available ssh: connect to host medusa port 22: No buffer space available $ su2 # sysctl net.inet.tcp.sendspace=32768 net.inet.tcp.sendspace: 131072 -> 32768 # sysctl net.inet.tcp.recvspace=65536 net.inet.tcp.recvspace: 262144 -> 65536 # logout $ ssh medusa Last login: Mon Apr 23 03:45:45 2007 from ... I assume this is because the maximum size of a TCP datagram is 65536 bytes, but as I'm not familiar enough with TCP on such a low level, this may be speculation on my part. Just something worth checking/tinkering with. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |