From owner-freebsd-net@FreeBSD.ORG Mon Oct 6 12:33:24 2003 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D4FB016A4B3 for ; Mon, 6 Oct 2003 12:33:24 -0700 (PDT) Received: from mtiwmhc13.worldnet.att.net (mtiwmhc13.worldnet.att.net [204.127.131.117]) by mx1.FreeBSD.org (Postfix) with ESMTP id EC11043F93 for ; Mon, 6 Oct 2003 12:33:23 -0700 (PDT) (envelope-from cmascott@att.net) Received: from callisto.local (133.cambridge-01rh15rt.ma.dial-access.att.net[12.91.16.133]) by mtiwmhc13.worldnet.att.net (mtiwmhc13) with ESMTP id <20031006193321113007rs89e>; Mon, 6 Oct 2003 19:33:22 +0000 Received: from callisto.local (localhost.local [127.0.0.1]) by callisto.local (8.12.8p1/8.12.8) with ESMTP id h96JXGmf000506 for ; Mon, 6 Oct 2003 15:33:16 -0400 (EDT) (envelope-from cmascott@callisto.local) Received: (from cmascott@localhost) by callisto.local (8.12.8p1/8.12.8/Submit) id h96JXFY8000505 for freebsd-net@freebsd.org; Mon, 6 Oct 2003 15:33:15 -0400 (EDT) Date: Mon, 6 Oct 2003 15:33:15 -0400 (EDT) From: Carl Mascott Message-Id: <200310061933.h96JXFY8000505@callisto.local> To: freebsd-net@freebsd.org Subject: 4.8-R: problem when tcp.recvspace=56K (default) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2003 19:33:25 -0000 FreeBSD 4.8-R kernel.GENERIC user ppp P II - 400 128 MB RAM 56K ext. modem, 45.3K connection Something in the socket/proto/network interface area doesn't work correctly when tcp.recvspace=56K, the default value in 4.8-R. It DOES work correctly when tcp.recvspace= 16K, 32K, 48K. I see the following repeatable problem. At the same point in the ftp reception of a 218K .gz file, received data stops getting delivered to the ftp client and starts stacking up in mbufs. The ftp client reports "stalled". When the recvspace limit is reached, the entire socket receive buffer is delivered to the ftp client as fast as the client can take it. For the remainder of the file transfer there are no further ftp client stalls. Note that the stall occurs at approx. 2 * tcp.recvspace. Please note that this stall would not be perceptible on a LAN: the 56K socket receive buffer would fill up too quickly. This would be mostly harmless were it not for the extra mbufs being consumed. On a system with many TCP connections the supply of mbufs might be exhausted. Following is a log of my activities in trying to track down this problem. I haven't been able to pinpoint it. Does anyone have any idea what it might be or how to further track it down? 10/05/03 Start: tcp.recvspace=48K ftp receive 218K .gz file no ftp client stalls netstat -m 191/224/6016 mbufs in use (current/peak/max): 189 mbufs allocated to data 2 mbufs allocated to packet headers 130/146/1504 mbuf clusters in use (current/peak/max) 348 Kbytes allocated to network (7% of mb_map in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines Change tcp.recvspace=56K (default) ftp receive 218K .gz file ftp client stalls @ 113K for ~10 seconds, then jumps to 170K ftp client reports "stalled" no modem RxD stall netstat -w 2 -I tun0 shows no stall in tun0 input. netstat -m 192/464/6016 mbufs in use (current/peak/max): 190 mbufs allocated to data 2 mbufs allocated to packet headers 130/146/1504 mbuf clusters in use (current/peak/max) 408 Kbytes allocated to network (9% of mb_map in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines Consumed 464-224=240 extra mbufs, no extra mbuf clusters 408-348=60 KB additional RAM allocated to network Note: MSIZE=256 (machine/param.h) Note: No stall w/ tcp.recvspace = 16K, 32K, 48K. Tried ftp -d: echoed all commands sent to host, but didn't appear to produce any socket/proto debug output.