From owner-freebsd-net@freebsd.org Mon Sep 28 09:08:56 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 31099A0B184 for ; Mon, 28 Sep 2015 09:08:56 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-wi0-f176.google.com (mail-wi0-f176.google.com [209.85.212.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C32FC1F32; Mon, 28 Sep 2015 09:08:55 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by wicgb1 with SMTP id gb1so94255073wic.1; Mon, 28 Sep 2015 02:08:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=from:subject:to:references:cc:message-id:date:user-agent :mime-version:in-reply-to:content-type; bh=QDk2oDQoi0qAbfN75+dsqtX8XcAQN00MWqSG23Wru5A=; b=HCY5VaH5jrz2HgfN1UyruazlLZUN1ZvxNVHDpZR/A9kECxPA38okA1S8PUtPx20trb dKs3bkYN7s1PnnljC6vnQQQRrtOhZ72zzCy6+kompmk5hvWUHMZJcGrv3Fo5rlN6bHvG mh4IzRHHkp+Otedkjagc4n+40fghITW/a838oFPv0JFDhZdZeAta0PclqXruLk2guhvg llv1hEZRwdgnJLSZNuyVzPLzlcongWN4C8/61RGYlNPSIkWC4d0RAtcyMBY9WhoSXEOo pzalufYXYTxDprM5AhtmXoGBPstolqdc2qlvh/9PkpOfwZpaFx+h1YOfaRdyV4P/VogZ ljiA== X-Received: by 10.194.121.40 with SMTP id lh8mr23271383wjb.115.1443431327763; Mon, 28 Sep 2015 02:08:47 -0700 (PDT) Received: from fri2jcharbon-m1.vcorp.ad.vrsn.com ([217.30.88.7]) by smtp.googlemail.com with ESMTPSA id lv4sm16467305wjb.43.2015.09.28.02.08.47 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Sep 2015 02:08:47 -0700 (PDT) From: Julien Charbon Subject: Re: Can tcp_close() be called in INP_TIMEWAIT case To: John Baldwin References: <26B0FF93-8AE3-4514-BDA1-B966230AAB65@FreeBSD.org> <5602BB7A.9010504@freebsd.org> <5603E8E4.5030406@freebsd.org> <2216936.QIvWsOndvU@ralph.baldwin.cx> Cc: Palle Girgensohn , freebsd-net@freebsd.org X-Enigmail-Draft-Status: N1110 Message-ID: <56090398.2070309@freebsd.org> Date: Mon, 28 Sep 2015 11:08:40 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <2216936.QIvWsOndvU@ralph.baldwin.cx> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="vwr74Sb8IPl79CrqbqNTJRdLVuT0ekrIh" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Sep 2015 09:08:56 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --vwr74Sb8IPl79CrqbqNTJRdLVuT0ekrIh Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi John, On 25/09/15 18:42, John Baldwin wrote: > On Thursday, September 24, 2015 02:13:24 PM Julien Charbon wrote: >> So the issue is: >> >> - tcp_close() is called for some reasons with an inp in INP_TIMEWAIT >> state and sets the INP_DROPPED flag, >> - tcp_detach() is called when the last reference on socket is dropped= >> >> then now in_pcbfree() can be called twice instead of once: >> >> 1. First in tcp_detach(): >> >> static void >> tcp_detach(struct socket *so, struct inpcb *inp) >> { >> struct tcpcb *tp; >> tp =3D intotcpcb(inp); >> >> if (inp->inp_flags & INP_TIMEWAIT) { >> if (inp->inp_flags & INP_DROPPED) { >> in_pcbdetach(inp); >> in_pcbfree(inp); <-- >> } >> >> 2. Second when tcptw expires here: >> >> void >> tcp_twclose(struct tcptw *tw, int reuse) >> { >> struct socket *so; >> struct inpcb *inp; >> >> inp =3D tw->tw_inpcb; >> >> tcp_tw_2msl_stop(tw, reuse); >> inp->inp_ppcb =3D NULL; >> in_pcbdrop(inp); >> >> so =3D inp->inp_socket; >> if (so !=3D NULL) { >> ... >> } else { >> in_pcbfree(inp); <-- >> } >> >> This behavior is backed by Palle kernel panic backstraces and coredum= ps. >> >> o Solutions: >> >> Long: Forbid to call tcp_close() when inp is in INP_TIMEWAIT state, >> the TCP stack rule being: >> >> - if !INP_TIMEWAIT: Call tcp_close() >> - if INP_TIMEWAIT: Call tcp_twclose() (or call nothing, the tcptw wil= l >> expire/be recycled anyway) >> >> Short: >> if INP_TIMEWAIT & INP_DROPPED: >> Do not call in_pcbfree(inp) in tcp_detach() unless tcptw is alread= y >> discarded. >> >> The long solution seems cleaner, backed by tcp_detach() old comments >> and most of current tcp_close() calls but I won't take that longer pat= h >> without -net approval first. >=20 > I prefer the longer solution if it keeps tcp_detach() simpler by avoidi= ng > an extra condition. Please just document it via assertions in tcp_clos= e() > (or is this the assertion that fired and triggered the reported panic? = If so, > then you obviously don't need to add it. :-P) Thanks for your answer. And indeed tcp_detach() will be kept simpler with the longer solution, I will introduce the new assertion in tcp_close(), something like : struct tcpcb * tcp_close(struct tcpcb *tp) { ... /* * tcp_close() should not called on TIME_WAIT connections. * These connections should be either teardown using * tcp_twclose(), or left alone waiting for TIME_WAIT timeout * expiration. */ KASSERT((inpb->inp_flags & INP_TIMEWAIT) =3D=3D 0, ("tcp_close cannot be called on TIME_WAIT connections")); And I will fix all paths that could lead to calling tcp_close() on TCP TIME_WAIT connections (that is why this solution will be longer). I found three potential paths so far. -- Julien --vwr74Sb8IPl79CrqbqNTJRdLVuT0ekrIh Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJWCQOeAAoJEKVlQ5Je6dhxn7cIAJbwLeCyeewBpBaHNaHMuNUl n1M9mpFG9EpzwP2TFjsSaTGq3UinsVnVAGOweNWGv31oM8izBw3QVB6BKXB5WyPI aVAin0xF4deEFljGdFYjUgFbu4eZ4Ce8TFXnsnhk7PPdrpNWdUwYA/8XBfyzFGkK h3mviAT+IZuJSQA32WuNe4B/7z4TEeh3D/o7HKKz7aL1NkvSNd1bWvL2v2trWHun HoZoiiW6iehUXOHGjtMyTw8s3Q2EDXwCwKnzlKYCW3vNorWR+hNzZlwBxIS3c5nf Ic8jm1skKN0sELzb9gD90GXFSsbBURWPxFdfMtK65eLumoF+ld9VNbSIsfb4TUs= =ELxz -----END PGP SIGNATURE----- --vwr74Sb8IPl79CrqbqNTJRdLVuT0ekrIh--