From owner-freebsd-net  Tue Apr 30 10:29:45 2002
Delivered-To: freebsd-net@freebsd.org
Received: from salmon.maths.tcd.ie (salmon.maths.tcd.ie [134.226.81.11])
	by hub.freebsd.org (Postfix) with SMTP id 05B0637B417
	for <net@freebsd.org>; Tue, 30 Apr 2002 10:29:37 -0700 (PDT)
Received: from walton.maths.tcd.ie by salmon.maths.tcd.ie with SMTP
          id <aa56944@salmon>; 30 Apr 2002 18:29:36 +0100 (BST)
To: Terry Lambert <tlambert2@mindspring.com>
Cc: net@freebsd.org
Subject: Re: soisdisconnected tweak. 
In-reply-to: Your message of "Mon, 29 Apr 2002 03:54:49 PDT."
             <3CCD2679.B5D5A709@mindspring.com> 
X-Request-Do: 
Date: Tue, 30 Apr 2002 18:29:35 +0100
From: David Malone <dwmalone@maths.tcd.ie>
Message-ID:  <200204301829.aa56944@salmon.maths.tcd.ie>
Sender: owner-freebsd-net@FreeBSD.ORG
Precedence: bulk
List-ID: <freebsd-net.FreeBSD.ORG>
List-Archive: <http://docs.freebsd.org/mail/> (Web Archive)
List-Help: <mailto:majordomo@FreeBSD.ORG?subject=help> (List Instructions)
List-Subscribe: <mailto:majordomo@FreeBSD.ORG?subject=subscribe%20freebsd-net>
List-Unsubscribe: <mailto:majordomo@FreeBSD.ORG?subject=unsubscribe%20freebsd-net>
X-Loop: FreeBSD.org

> PPS: Ask yourself: how is it that it's *EVER* possible for the function
> soisdisconnected() to be called without the socket buffer having been
> emptied *FIRST*.

I think it is quite simple. Say that the machine to which you are
talking is powerd down while you are sending data to it. When you
stop getting ACKs back data will begin to build up in the socket
socket buffer. After TCP_MAXRXTSHIFT retransmits the connection
will be timed out because you get no responses to your ACKs and
soisdisconnected() will be called.

You should be able to reproduce the problem by enabeling discard
in inetd.conf and then adding firewall rules which block non-syn
packets to discard and running the program below (which opens lots
of connections to discard, fills the socket buffers, waits until
the tcp connection is timed out and then closes the connections).
You can see that the mbufs used for data are not freed until the
sockets are actually closed.

(The code is based on some Andreas Persson sent me to tickle a
sendfile bug about 18 months ago).

	David.


#include <sys/param.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/uio.h>

#include <netinet/in.h>
#include <arpa/inet.h>

#include <err.h>
#include <errno.h>
#include <fcntl.h>
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>

#define FDS             64
#define SENDS           63
#define HOST            "127.0.0.1"
#define PORT            9

int fds[FDS];
char buf[1024];

int main()
{
        int i, j, k;
        struct sockaddr_in sin;

        for (i = 0; i < FDS; i++) {
                fds[i] = socket(AF_INET, SOCK_STREAM, 0);
                if (fds[i] == -1) 
                        err(1, "socket");

                memset(&sin, 0, sizeof(sin));
                if (bind(fds[i], (struct sockaddr *)&sin, sizeof(sin)) 
                    == -1) 
                        err(1, "bind");
                
                memset(&sin, 0, sizeof(sin));
                sin.sin_addr.s_addr = inet_addr(HOST);
                sin.sin_port = htons(PORT);
                sin.sin_family = AF_INET;
                if (connect(fds[i], (struct sockaddr *)&sin, sizeof(sin))
                    == -1) 
                        err(1, "connect");
                
                if (fcntl(fds[i], F_SETFL, O_NONBLOCK) == -1) 
                        err(1, "fcntl");
        }

	printf("Bound and ready\n");
	system("netstat -m");
                
        for (j = 0; j < FDS; j++) {
                for(k = 0; k < SENDS; k++) {
                        if (write(fds[j], buf, sizeof(buf)) != sizeof(buf)) {
				if (errno != EWOULDBLOCK)
					warn("write");
				else
					break;
			}
		}
		if (errno != EWOULDBLOCK)
			printf("Didn't fill fd=%d\n", fds[j]);
	}

	printf("Data written\n");
	system("netstat -m");
	printf("Waiting 10 minutes for retransmits to timeout\n");
	sleep(600);
	system("netstat -m");

	for (j = 0; j < FDS; j++)
		if (read(fds[j], buf, sizeof(buf)) > 0)
			printf("Odd, read worked for fd=%d\n", fds[j]);
		else if (errno != ETIMEDOUT)
			warn("read");
		

	printf("Before close\n");
	system("netstat -m");

	for (j = 0; j < FDS; j++)
		close(fds[j]);

	printf("After close\n");
	system("netstat -m");

        exit(0);
}


To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-net" in the body of the message