From nobody Wed Apr 3 12:14:34 2024 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V8kFp5XR9z5Gh8X for ; Wed, 3 Apr 2024 12:14:38 +0000 (UTC) (envelope-from cryintothebluesky@gmail.com) Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4V8kFn3KFbz4N9Q for ; Wed, 3 Apr 2024 12:14:37 +0000 (UTC) (envelope-from cryintothebluesky@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=hNaps4kp; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of cryintothebluesky@gmail.com designates 2a00:1450:4864:20::433 as permitted sender) smtp.mailfrom=cryintothebluesky@gmail.com Received: by mail-wr1-x433.google.com with SMTP id ffacd0b85a97d-343a5b667a8so125345f8f.2 for ; Wed, 03 Apr 2024 05:14:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712146476; x=1712751276; darn=freebsd.org; h=content-transfer-encoding:mime-version:message-id:subject:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=jdDax2WOSLe8N2GirX2b6MU1VuHNZhOau0LNJU+Xjt4=; b=hNaps4kpI4yu3dygLrlDtfKa60ON+Ct5OgkIoldIuPuLlsyXM85DdDq7BYClg1YttM UDW0fmlsKagAn2BQn0ObGtSNT0RxtUuP79N+jaPIEpIGjvqBFFeO2l4WDkyupGHXH0cP amurxGhkKhIVaRCyyRsj+onIIxRY9WmFoG8hFkzsLbct08absYCIy3VMvM1d2dlku/qX 1WdiwAdtvF9chQ3o+srDX0UMhQDqiZ3qAp2R1IZ5mBuDELp0ELh3V3aXH42mj97dfzwK 4ApaZrbxGqKZH0q+YM6ydntU2MLYLVnOJZWuiTQG+DIsvRoqN9GYexxWzNHeA6EwG0j1 IHAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712146476; x=1712751276; h=content-transfer-encoding:mime-version:message-id:subject:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=jdDax2WOSLe8N2GirX2b6MU1VuHNZhOau0LNJU+Xjt4=; b=bojkhs7hrMh7sZtMu1kBAqUZwQs5GJG21/8O936UrQ60JaKdXvtMRLMrui6ILR7v4t rfgRt68wVerRm064EJxYLX96Lod8ambmP+9S6rTFUwheMnxygBsxgC8aI2se50Yi3Uny NRSd2gI/w2uRllx2gcakyJPJTdWc6lR28d+kZ7KlIels7WQyFcpzt7peJZun76/QBxeZ CKj2aEgilsdgPNTXTOgy+AHBlDW46coLrZ9lOZFmnV7F5MdcjQJKh+zCH8UXByZ082OB rUTGWuLD0/7CgrkcPre5s54tHhQpCZuh6bFSqtxWaGvfPvxBO+go+J38w3Pmee0roJFy +Ifg== X-Gm-Message-State: AOJu0YxWXFUKww4hAfZcT5OgEt29erBe0XQycBrYV5Y0HpsVRbmrDEiq ZB5PMPZuOwvqSWykEuTZ63n7GDR5CQ5U3pEGAEf4y7mpEkDwart+MQ2QvMq4 X-Google-Smtp-Source: AGHT+IFzdV8a+ESTvp4XIDzXtX/XT/2gz6ddGPKpiTQm5T2pp7jrCpAPk72pGyT7CnTrMYXcAZ+Zbg== X-Received: by 2002:adf:cd8b:0:b0:343:aa7e:4b0b with SMTP id q11-20020adfcd8b000000b00343aa7e4b0bmr37442wrj.2.1712146475645; Wed, 03 Apr 2024 05:14:35 -0700 (PDT) Received: from z600.home.lan (193.85.199.146.dyn.plus.net. [146.199.85.193]) by smtp.gmail.com with ESMTPSA id dj13-20020a0560000b0d00b0033e9fca1e49sm16984937wrb.60.2024.04.03.05.14.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Apr 2024 05:14:35 -0700 (PDT) Date: Wed, 3 Apr 2024 13:14:34 +0100 From: Sad Clouds To: freebsd-net@FreeBSD.org Subject: TCP socket handling errors Message-Id: <20240403131434.c3bfa64c726505d842408c80@gmail.com> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.32; x86_64-pc-linux-gnu) List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.37 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.87)[-0.866]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCPT_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; FREEMAIL_FROM(0.00)[gmail.com]; FROM_HAS_DN(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TO_DN_NONE(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-net@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-net@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::433:from] X-Rspamd-Queue-Id: 4V8kFn3KFbz4N9Q Hello, I have a client/server networking application that exhibits TCP socket handling errors. This only happens on FreeBSD, while NetBSD, Linux, Solaris, etc. all seem to work correctly. I was hoping to get some advice on what could be the root cause. I have two processes - client and server, sending and receiving data to/from each other on 127.0.0.1 Client connects to server and calls send(2)/recv(2) in a loop. This is a bidirectional data exchange. When all send data is transferred, client calls shutdown(sockfd, SHUT_WR) and continues receiving data on the same socket until recv(2) returns 0 bytes, which signals end of receive data. At this stage client calls close(sockfd) and terminates. Server has the same data transfer loop as the client. I frequently get ECONNRESET when calling close(2), sometimes from the server and sometimes from the client process. This should not be happening, but I'm not sure what could be causing it. The client logic is as follows: 1. Set sockfd nonblocking. 2. Call send(2)/recv(2) in a loop until N bytes have been transferred in each direction. 3. Set sockfd blocking. 4. Call send_buf() to send control handshake to server. 5. Call shutdown(sockfd, SHUT_WR) to signal end of send data from client. 6. Call recv_buf() to receive control handshake from server. 7. Call recv_buf() and verify it returned 0 bytes to indicate end of data from server. 8. Call close(sockfd) and verify success. Step 8 sometimes fails and returns ECONNRESET. Functions send_buf() and recv_buf() are wrappers around send(2) and recv(2) which restart those system calls until the specified number of buffer bytes have been fully transferred or 0 is returned in the case of recv_buf() indicating end of data. They are designed to work with blocking file descriptors and avoid short reads/writes. I don't understand why close(2) sometimes returns ECONNRESET when the previous recv(2) call at step 7 returned 0 bytes, indicating the remote TCP end sent us a FIN. I don't set SO_LINGER socket option and when I checked the default on FreeBSD it reports l_onoff=0, l_linger=0 so there should be no immediate RST on socket close(2). Does anyone have any suggestions or ideas? Thanks.