From owner-freebsd-net@FreeBSD.ORG Tue Dec 30 00:28:58 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CEBE5A13 for ; Tue, 30 Dec 2014 00:28:58 +0000 (UTC) Received: from mail-wg0-x22e.google.com (mail-wg0-x22e.google.com [IPv6:2a00:1450:400c:c00::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5745B1607 for ; Tue, 30 Dec 2014 00:28:58 +0000 (UTC) Received: by mail-wg0-f46.google.com with SMTP id x13so20092872wgg.33 for ; Mon, 29 Dec 2014 16:28:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=xz3fGDIhlQsGwBFpzqfqL/jqaCWiKFIf0axA1uKB2wI=; b=rPbHHaJW+Z6knBFg57Z8YZieBTjSCUgKeNpvsvSzwC8L495aPCCz3z/4LFgYx3Rhsh iAW8LS6ZBdTkI0jgCUpAb33zhgwDnZ+yLr5ptFWDdB8NxCgF9RNcAMiKAedzdoFUYeSN 0A55wSoBNMiacSwwFRSq9/uVVTsPFrZaTIL6KVUYG66DzqFxLTPKRyASQY2H9JsS8SR4 lmR9iYwcrBV+bfSZS8HlIsFk4TpD8a0C7shx/tU7Rdq0Wj7fRlajpVrQndadhvwfj7ea 3VjGOmLVXCwjCfHu3dPJH0gz1/+JPQJyZLY0zU0cn+VJaYWuHf+pMgYa0qgW1FkfjUaw VRgw== MIME-Version: 1.0 X-Received: by 10.194.200.1 with SMTP id jo1mr118086119wjc.64.1419899336800; Mon, 29 Dec 2014 16:28:56 -0800 (PST) Received: by 10.27.177.218 with HTTP; Mon, 29 Dec 2014 16:28:56 -0800 (PST) In-Reply-To: <81C17F4A-B0AA-48C9-ABFB-6B14B7223643@lists.zabbadoz.net> References: <54A0FDD9.4090009@freebsd.org> <81C17F4A-B0AA-48C9-ABFB-6B14B7223643@lists.zabbadoz.net> Date: Tue, 30 Dec 2014 01:28:56 +0100 Message-ID: Subject: Re: setfib and RSTs From: Nikolay Denev To: "Bjoern A. Zeeb" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2014 00:28:58 -0000 On Tue, Dec 30, 2014 at 12:51 AM, Bjoern A. Zeeb < bzeeb-lists@lists.zabbadoz.net> wrote: > > > On 29 Dec 2014, at 21:03 , Nikolay Denev wrote: > > > > No, no PR yet, but I will file one. I wanted to collect some more data > > first. > > > > So, I've did some dtrace digging : > > > > [20:54][root@nas:~]#cat reset.d > > #!/usr/sbin/dtrace -s > > > > fbt:kernel:tcp_dropwithreset:entry > > { > > printf("reason %d fib %d src_port %d dst_port %d", args[4], args[2] = ? > > args[2]->t_inpcb->inp_inc.inc_fibnum : -1, ntohs(args[1]->th_sport), > > ntohs(args[1]->th_dport)); > > /* stack(); */ > > } > > =E2=80=A6 > > > The port numbers here match RST packets that I'm seeing with tcpdump in > > another window. > > reason 3 is BANDLIM_RST_CLOSEDPORT (from icmp_var.h) > > Looking at tcp_input.c I see that there are cases where the INPCB does > not > > exists, and from what I see this is how the FIB gets determined. > > Also here I see that tcp_dropwithreset() is called with tcpcb set to > NULL, > > so probably this is why the FIB is not found. > > > > Why this is happening, I have no idea yet. > > Could you also check for the mbuf *m and the fibnum from the pkthdr there= ? > > It might be even more interesting to see this for tcp_respond and the > following ip_output as well, in case you want to keep state in the d > script; otherwise just tcp_dropwithreset and/or tcp_respond should be fi= ne. > > Usually I would expect for the tcp_dropwithreset case that inp will be > NULL in tcp_respond, the mbuf *m and th will be valid and thus the FIB > number from the incoming mbuf would be re-used as the mbuf will be re-use= d, > but for that the mbuf needs to be properly tagged on receive. > > /bz > > =E2=80=94 > Bjoern A. Zeeb Charles Haddon Spurgeon: > "Friendship is one of the sweetest joys of life. Many might have failed > beneath the bitterness of their trial had they not found a friend." > > If I got this right i see FIB 0 here: fbt:kernel:tcp_dropwithreset:entry { this->mbuf =3D (struct mbuf *)args[0]; printf("reason %d fib %d src_port %d dst_port %d", args[4], this->mbuf->M_dat.MH.MH_pkthdr.fibnum, ntohs(args[1]->th_sport), ntohs(args[1]->th_dport)); } I've did some tracing of the the transmission-daemon, and I saw that after it calls connect() on the sockets (which are non-blocking), the connect returns with EINPROGRESS (as expected), but in most cases after about 20-60us close() is called. (not sure why it's doing this, but this is what I saw) So, by that time the SYN-ACK from the remote end is received the socket is already gone and RST is sent using default FIB, which does seem to be the correct behaviour. I've now set the fib for the tun interface with "ifconfig tun2 fib 1", and I see the RSTs sent back over the tunnel. --Nikolay