From owner-freebsd-current@FreeBSD.ORG Thu Nov 5 23:49:13 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 093BE1065692 for ; Thu, 5 Nov 2009 23:49:13 +0000 (UTC) (envelope-from rpaulo@gmail.com) Received: from ey-out-2122.google.com (ey-out-2122.google.com [74.125.78.24]) by mx1.freebsd.org (Postfix) with ESMTP id 8A7AC8FC0A for ; Thu, 5 Nov 2009 23:49:12 +0000 (UTC) Received: by ey-out-2122.google.com with SMTP id 25so147326eya.9 for ; Thu, 05 Nov 2009 15:49:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:references:in-reply-to :mime-version:content-type:message-id:content-transfer-encoding:cc :from:subject:date:to:x-mailer; bh=Ij/Vh9mDZ073OcAcMaVYezenCb2B6dAL9jHJ2A0Fo+c=; b=W/Xt7YSwLD97eid+ZnZ54rjEufRAowc7CwSXiFxBvTtFexFFJb3mHOen1vW/sgjreW e7NCZ5oG/fVF074EgKNwykC40qcJiJQBcXH/L0HEva6M7kFobR84DsTeGPl2oPTzKmYj UkD85jlT9g2DRPmaNxlkXjmDilvkQCtefatcw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:references:in-reply-to:mime-version:content-type:message-id :content-transfer-encoding:cc:from:subject:date:to:x-mailer; b=noQqNYB0jqSDMH1SNpybElunHtauFr62MfaS6WWQId/zW6OZ0QEYAtWdrCa9sRdM17 Q9rkqiXaH3yN9KjwhmGGJlv/TbbDexTWlhRlApLSTEJgw+O75f+FBk63GaazYSMF80/w FeaeQrwETKMnJdpgIlGpLmaHwgWMUd2F9t7vU= Received: by 10.213.24.20 with SMTP id t20mr2169356ebb.49.1257464951343; Thu, 05 Nov 2009 15:49:11 -0800 (PST) Received: from rui-macbook.lan (bl7-119-248.dsl.telepac.pt [85.240.119.248]) by mx.google.com with ESMTPS id 5sm501419eyf.39.2009.11.05.15.49.09 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 05 Nov 2009 15:49:09 -0800 (PST) Sender: Rui Paulo References: <4AF0B7DF.9030405@freebsd.org> In-Reply-To: Mime-Version: 1.0 (Apple Message framework v1076) Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes Message-Id: <030A8229-9707-4F70-B4BE-584F1BF9ECEC@FreeBSD.org> Content-Transfer-Encoding: 7bit From: Rui Paulo Date: Thu, 5 Nov 2009 23:49:08 +0000 To: Rick Macklem X-Mailer: Apple Mail (2.1076) Cc: freebsd-current@freebsd.org Subject: Re: Help needed: TCP Wizards (was 8.0-RC1 NFS client timeout issue) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2009 23:49:13 -0000 On 5 Nov 2009, at 16:36, Rick Macklem wrote: > > > Rick Macklem wrote: >> I can now reproduce what I think others were seeing as slow >> reconnects >> when using NFSv3 over TCP against a server that disconnects inactive >> TCP connections. I have had some luck figuring out what is going on >> and can reproduce it fairly easily, but I really need help from >> someone >> who understands the FreeBSD TCP stack. >> > Ok, I haven't made much progress on this, but here's what little I > currently know about it. > > The problem occurs after a server has dropped an inactive TCP > connection > for an NFS over TCP mount (in my case a Solaris10 server). When the > client > does a new connection it, for some reason, sends a RST at almost > exactly > the same time as the first RPC request on the new TCP connection, > causing > the server to shut it down. > > Ok, things I now know don't affect this are: > - doing the soshutdown(), soclose() on the old connection. I commented > them out and it still happened. > - Avoiding the sobind() on the new connection, done before the > soconnect(). > - Using a non-reserved port#. > (The above tests shot down pretty well all the "theories" I could > come up > with.) > > The only thing I've found that avoids the problem: > - putting a 2sec delay right before the soconnect() call. (A 1sec > delay > made it hard to reproduce and I've never reproduced it yet with a > 2sec > delay.) > Not much of a fix, though. > > Now, here's where someone may be able to help? > > Grep'ng around, I found 4 places where the TCP stack called ip_output > () > (one in each of tcp_output.c, tcp_subr.c, tcp_timewait.c and > tcp_syncache.c) and I put a printf like this just before them: > if (flags & TH_RST) > printf("sent a reset\n"); > > (The exact format varies for each, because of where the TCP > header flags are and have different printf messages.) > > Now, the weird part is, that when the extraneous RST is sent to the > server, I don't get any printf. (I do get a few of these, but at other > times for what appear to be legitimate RSTs.) > > I can't see anywhere else that the TCP stack would send an RST and, > so, > I'm stuck w.r.t. figuring out what is sending them? > > Anyone know of another place the TCP stack would make the send happen? > (Or is it queued earlier when I see the printf message, and then the > send is "triggered" inside the ip layer when the first data is sent on > the new connection?) Are you running TSO? -- Rui Paulo