From owner-freebsd-stable@FreeBSD.ORG Thu May 29 19:12:17 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E63B4106564A for ; Thu, 29 May 2008 19:12:17 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id 870E68FC1B for ; Thu, 29 May 2008 19:12:17 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.14.1/8.14.1) with ESMTP id m4TJCGiF025526; Thu, 29 May 2008 12:12:16 -0700 (PDT) Received: (from dillon@localhost) by apollo.backplane.com (8.14.1/8.13.4/Submit) id m4TJCG56025525; Thu, 29 May 2008 12:12:16 -0700 (PDT) Date: Thu, 29 May 2008 12:12:16 -0700 (PDT) From: Matthew Dillon Message-Id: <200805291912.m4TJCG56025525@apollo.backplane.com> To: Robert Blayzor References: <1A19ABA2-61CD-4D92-A08D-5D9650D69768@mac.com> <23C02C8B-281A-4ABD-8144-3E25E36EDAB4@inoc.net> <483DE2E0.90003@FreeBSD.org> <483E36CE.3060400@FreeBSD.org> <483E3C26.3060103@paradise.net.nz> <483E4657.9060906@FreeBSD.org> <483EA513.4070409@earthlink.net> <96AFE8D3-7EAC-4A4A-8EFF-35A5DCEC6426@inoc.net> <483EAED1.2050404@FreeBSD.org> Cc: freebsd-stable@freebsd.org Subject: Re: Sockets stuck in FIN_WAIT_1 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2008 19:12:18 -0000 I guess nobody mentioned the obvious thing to check: Make sure TCP keepalive is turned on. sysctl net.inet.tcp.always_keepalive=1 If you don't do this then dead TCP connections can build up, particularly on busy servers, due to the other end simply disappearing. Without this option the TCP protocol can get stuck, because it does not usually send packets to the other end of an idle connection unless (1) its window has closed completely or (2) it has unackncowledged data or state pending. The keepalive forces a probe to occur every so often on an idle connection (like once every 30min-2hrs, I forget what the default is), to check that the connection still exists. It is possible to get stuck during normal data operation and while in a half-closed state. The 2MSL timeout does not activate until you go into a fully closed state (FIN2/TIME_WAIT). Pretty much if you are running any sort of service on the internet, and even if you aren't, you need to make sure keepalive is turned on for the long term health of your system. -Matt