From nobody Mon Mar 31 17:31:38 2025 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ZRJ8W3BnPz5sL18; Mon, 31 Mar 2025 17:31:39 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ZRJ8V74jZz3cnj; Mon, 31 Mar 2025 17:31:38 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1743442299; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=PFdfmXUcA8ddZwk8HlfJyd76zGmx73VE1hyL2n7JaIw=; b=WqLQsa9oc+d41mPm+fD9ZlLNhqoOzJtV8u1D7DDYie8x7UsQ7cyVRAsR95XEOf3lf6PvCV iSHPcrjj9Mw2hsu9+22zDNy18/YVtCPehy+JNrhI2eKE8adM0fkgT79zFaLARvATSjIpVF DK3nFuCci9A35Yqoj4s0WyKgQULq29zSYegG++BXJp2ZShPxwSw4r4dKJX4av5xxt2IaT/ k3g3cjVUOfAjRJQwl9Pc2aWPx5BGkSNrjoPiL4cBpzE/LFbtlwsDNxmtLbtkikGsPlSggM tq2KHzSAxDxhFfD0j6cywe2hmyvw7qSxP3b9CuSXREPVxTmRBHjev8x5uA1UqA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1743442299; a=rsa-sha256; cv=none; b=ftNo9PfwwXW0NB9+oN7Ix9AUE+kvFBK1R8U3y39UTfErWOCrJBLrSftd9vq6Wqg1Cc+VS6 VGfmEMoHMn5lCP2yqgS2//bUqTTF0zwIUhcXISwiF4lOtO45EmHO3OA6yELi7VrBPF5pKZ lzcnGclkEtrfK7FMWlHTR+SGq66HA3AliASbg/GufjIXCTiPb4kEkJ+Xxqnm3B0uBxSU2M 8ELqu4+bZNL0hgRujxi3ZhMzNQqcjD+f3IP/j6MrTRBDQ5x1MElYfy21h3qAzONq2u3KV4 yULTp4uogH026/Z1PPEFvTNsn9a7S97xs2sOvS3N2+mmmTnry5anCJ9xFt5fNw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1743442299; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=PFdfmXUcA8ddZwk8HlfJyd76zGmx73VE1hyL2n7JaIw=; b=dioj6p3cwbjgGFkekUIL9tDmruIAZGClIYbmTjXMBUXcTCH4so0Q7TfKcVzfFAGKGiuHZ5 A9w3sJza89ApcxNv/GVAq7lZSvdYJGAthOGlTn8IJ1We7BY6yeEOf6hIQwpZw+OFgCqp70 nn7mqVgsEq3iEwpQi9hA6AvA5XRohW2Zn2IDhnhCGmvKLt/UGhj0w305KwMRLuhAlo21jf qtV2EAKZ292OrqLKSYsqguQE8ud7LAbpddN3LkqKSJlCKvAmw+KydBWQzoOLYCLLAu3TwE W8K5n20blRKG4nVq5cJkDpZEcjDg+ZUM2P+omHHoOekl3P+irys7k1fa7vRdwA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4ZRJ8V4kmwzvLM; Mon, 31 Mar 2025 17:31:38 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 52VHVc2C036628; Mon, 31 Mar 2025 17:31:38 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 52VHVcBk036625; Mon, 31 Mar 2025 17:31:38 GMT (envelope-from git) Date: Mon, 31 Mar 2025 17:31:38 GMT Message-Id: <202503311731.52VHVcBk036625@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Gleb Smirnoff Subject: git: 1bda3fae784e - stable/14 - tcp: don't ever return ECONNRESET on close(2) List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: glebius X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 1bda3fae784ecb6e227220d20088cfde5f19eed6 Auto-Submitted: auto-generated The branch stable/14 has been updated by glebius: URL: https://cgit.FreeBSD.org/src/commit/?id=1bda3fae784ecb6e227220d20088cfde5f19eed6 commit 1bda3fae784ecb6e227220d20088cfde5f19eed6 Author: Gleb Smirnoff AuthorDate: 2024-12-23 18:35:49 +0000 Commit: Gleb Smirnoff CommitDate: 2025-03-31 17:31:21 +0000 tcp: don't ever return ECONNRESET on close(2) The SUS doesn't mention this error code as a possible one [1]. The FreeBSD manual page specifies a possible ECONNRESET for close(2): [ECONNRESET] The underlying object was a stream socket that was shut down by the peer before all pending data was delivered. In the past it had been EINVAL (see 21367f630d72), and this EINVAL was added as a safety measure in 623dce13c64ef. After conversion to ECONNRESET it had been documented in the manual page in 78e3a7fdd51e6, but I bet wasn't ever tested to actually be ever returned, cause the tcp-testsuite[2] didn't exist back then. So documentation is incorrect since 2006, if my bet wins. Anyway, in the modern FreeBSD the condition described above doesn't end up with ECONNRESET error code from close(2). The error condition is reported via SO_ERROR socket option, though. This can be checked using the tcp-testsuite, temporarily disabling the getsockopt(SO_ERROR) lines using sed command [3]. Most of these getsockopt(2)s are followed by '+0.00 close(3) = 0', which will confirm that close(2) doesn't return ECONNRESET even on a socket that has the error stored, neither it is returned in the case described in the manual page. The latter case is covered by multiple tests residing in tcp- testsuite/state-event-engine/rcv-rst-*. However, the deleted block of code could be entered in a race condition between close(2) and processing of incoming packet, when connection had already been half-closed with shutdown(SHUT_WR) and sits in TCPS_LAST_ACK. This was reported in the bug 146845. With the block deleted, we will continue into tcp_disconnect() which has proper handling of INP_DROPPED. The race explanation follows. The connection is in TCPS_LAST_ACK. The network input thread acquires the tcpcb lock first, sets INP_DROPPED, acquires the socket lock in soisdisconnected() and clears SS_ISCONNECTED. Meanwhile, the syscall thread goes through sodisconnect() which checks for SS_ISCONNECTED locklessly(!). The check passes and the thread blocks on the tcpcb lock in tcp_usr_disconnect(). Once input thread releases the lock, the syscall thread observes INP_DROPPED and returns ECONNRESET. - Thread 1: tcp_do_segment()->tcp_close()->in_pcbdrop(),soisdisconnected() - Thread 2: sys_close()...->soclose()->sodisconnect()->tcp_usr_disconnect() Note that the lockless operation in sodisconnect() isn't correct, but enforcing the socket lock there will not fix the problem. [1] https://pubs.opengroup.org/onlinepubs/9799919799/ [2] https://github.com/freebsd-net/tcp-testsuite [3] sed -i "" -Ee '/\+0\.00 getsockopt\(3, SOL_SOCKET, SO_ERROR, \[ECONNRESET\]/d' $(grep -lr ECONNRESET tcp-testsuite) PR: 146845 Reviewed by: tuexen, rrs, imp Differential Revision: https://reviews.freebsd.org/D48148 (cherry picked from commit 053a988497342a6fd0a717cc097d09c23f83e103) --- lib/libc/sys/close.2 | 5 +---- sys/netinet/tcp_usrreq.c | 5 ----- 2 files changed, 1 insertion(+), 9 deletions(-) diff --git a/lib/libc/sys/close.2 b/lib/libc/sys/close.2 index aba9cac5ae8a..1392541d91c4 100644 --- a/lib/libc/sys/close.2 +++ b/lib/libc/sys/close.2 @@ -27,7 +27,7 @@ .\" .\" @(#)close.2 8.2 (Berkeley) 4/19/94 .\" -.Dd December 1, 2017 +.Dd December 18, 2024 .Dt CLOSE 2 .Os .Sh NAME @@ -113,9 +113,6 @@ is not an active descriptor. An interrupt was received. .It Bq Er ENOSPC The underlying object did not fit, cached data was lost. -.It Bq Er ECONNRESET -The underlying object was a stream socket that was shut down by the peer -before all pending data was delivered. .El .Pp In case of any error except diff --git a/sys/netinet/tcp_usrreq.c b/sys/netinet/tcp_usrreq.c index 67645827cb58..3bdbd968216f 100644 --- a/sys/netinet/tcp_usrreq.c +++ b/sys/netinet/tcp_usrreq.c @@ -697,11 +697,6 @@ tcp_usr_disconnect(struct socket *so) inp = sotoinpcb(so); KASSERT(inp != NULL, ("tcp_usr_disconnect: inp == NULL")); INP_WLOCK(inp); - if (inp->inp_flags & INP_DROPPED) { - INP_WUNLOCK(inp); - NET_EPOCH_EXIT(et); - return (ECONNRESET); - } tp = intotcpcb(inp); if (tp->t_state == TCPS_TIME_WAIT)