Date: Sun, 30 May 2010 11:05:45 +0300 From: Mikolaj Golub <to.my.trociny@gmail.com> To: freebsd-net@FreeBSD.org Cc: "Robert N. M. Watson" <rwatson@FreeBSD.org>, "Lavrentiev, Anton \(NIH/NLM/NCBI\) \[C\]" <lavr@ncbi.nlm.nih.gov>, bug-followup@FreeBSD.org Subject: Re: kern/146845: [libc] close(2) returns error 54 (connection reset by peer) wrongly Message-ID: <86eigt4xwm.fsf@kopusha.home.net> In-Reply-To: <86mxvkqsvq.fsf@zhuzha.ua1> (Mikolaj Golub's message of "Fri, 28 May 2010 12:26:33 %2B0300") References: <201005280440.o4S4e3sM052201@freefall.freebsd.org> <86mxvkqsvq.fsf@zhuzha.ua1>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --]
> IMHO, it is not, unfortunately, a solution: it seems to clear ECONNRESET
> blindly and w/o distinguishing the situation when the remote end closes the
> connection prematurely (i.e. before acknowledging all data written from the
> local end) -- and that qualifies for the true "connection reset by peer"
> from close()...
I did some experiments the results I would like to share here. The idea is
following: the client sends data in one write() more then a win, while the
server closes the connection without reading (sending RST on close). I also
played with LINGER option. I have managed to get ECONNRESET only on write(),
if the server sends RST before the client calls write(). In all other cases
write()/close() returned without error. See the attachment for details.
So I think that with the workaround (ignore ECONNRESET returned by
sodisconnect() in soclose()) we would not make the situation worse (while it
fixed the issue with applications getting unexpectedly ECONNRESET after
shutdown()/close() sequence).
--
Mikolaj Golub
[-- Attachment #2 --]
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <signal.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <err.h>
#define BUFSIZE 409600
#define PORT 23481
#define SLEEP1 0
#define SLEEP2 1
#undef LINGER_IN_CLIENT
#undef LINGER_IN_SERVER
int
main(int argc, char **argv)
{
struct sockaddr_in sin;
int listenfd, connfd, pid;
char buf[BUFSIZE];
#ifdef LINGER_IN_CLIENT
struct linger ling;
#else
#ifdef LINGER_IN_SERVER
struct linger ling;
#endif
#endif /* LINGER_IN_CLIENT || LINGER_IN_SERVER */
if ((listenfd = socket(AF_INET, SOCK_STREAM, 0)) < 0)
err(1, "socket error");
memset(&sin, 0, sizeof(sin));
sin.sin_family = AF_INET;
sin.sin_port = htons(PORT);
if (bind(listenfd, (struct sockaddr *) &sin,
sizeof(sin)) < 0)
err(1, "bind error");
if (listen(listenfd, 1024) < 0)
err(1, "listen error");
pid = fork();
if (pid == -1)
err(1, "fork error");
if (pid != 0) {
close(listenfd);
sleep(1);
if ((connfd = socket(AF_INET, SOCK_STREAM, 0)) < 0) {
(void)kill(pid, SIGTERM);
err(1, "parent: socket error");
}
if (connect(connfd, (struct sockaddr *)&sin,
sizeof(sin)) < 0) {
(void)kill(pid, SIGTERM);
err(1, "parent: connect error");
}
#ifdef LINGER_IN_CLIENT
ling.l_onoff = 1;
ling.l_linger = 10;
if (setsockopt(connfd, SOL_SOCKET, SO_LINGER,
&ling, sizeof(ling)) < 0)
err(1, "parent: setsockopt error");
#endif /* LINGER_IN_CLIENT */
sleep(SLEEP1);
if (write(connfd, buf, BUFSIZE) < 0) {
(void)kill(pid, SIGTERM);
err(1, "parent: write error");
}
if (close(connfd) < 0) {
(void)kill(pid, SIGTERM);
err(1, "parent: close error");
}
} else {
if ((connfd = accept(listenfd, (struct sockaddr *)NULL,
NULL)) < 0)
err(1, "child: accept error");
#ifdef LINGER_IN_SERVER
/*
* Send RST on close.
*/
ling.l_onoff = 1;
ling.l_linger = 0;
if (setsockopt(connfd, SOL_SOCKET, SO_LINGER,
&ling, sizeof(ling)) < 0)
err(1, "child: setsockopt error");
#endif /* LINGER_IN_SERVER */
sleep(SLEEP2);
if (close(connfd) < 0)
err(1, "child: close error");
}
exit(0);
}
#if 0
SLEEP1 = 0; SLEEP2 = 0: LINGER_IN_SERVER: parent: write error: Connection reset by peer
00:00:00.000000 IP 127.0.0.1.23851 > 127.0.0.1.23481: Flags [S], seq 3233765993, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val 286058 ecr 0], length 0
00:00:00.000044 IP 127.0.0.1.23481 > 127.0.0.1.23851: Flags [S.], seq 917049087, ack 3233765994, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val 3490085004 ecr 286058], length 0
00:00:00.000034 IP 127.0.0.1.23851 > 127.0.0.1.23481: Flags [.], ack 1, win 8960, options [nop,nop,TS val 286058 ecr 3490085004], length 0
00:00:00.000191 IP 127.0.0.1.23481 > 127.0.0.1.23851: Flags [R.], seq 1, ack 1, win 8960, options [nop,nop,TS val 3490085004 ecr 286058], length 0
SLEEP1 = 1; SLEEP2 = 0: LINGER_IN_SERVER: no error
00:00:00.000000 IP 127.0.0.1.27558 > 127.0.0.1.23481: Flags [S], seq 4042060821, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val 291465 ecr 0], length 0
00:00:00.000042 IP 127.0.0.1.23481 > 127.0.0.1.27558: Flags [S.], seq 2789342577, ack 4042060822, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val 1959417611 ecr 291465], length 0
00:00:00.000034 IP 127.0.0.1.27558 > 127.0.0.1.23481: Flags [.], ack 1, win 8960, options [nop,nop,TS val 291465 ecr 1959417611], length 0
00:00:00.000173 IP 127.0.0.1.23481 > 127.0.0.1.27558: Flags [R.], seq 1, ack 1, win 8960, options [nop,nop,TS val 1959417611 ecr 291465], length 0
SLEEP1 = 0; SLEEP2 = 1: LINGER_IN_SERVER: no error
00:00:00.000000 IP 127.0.0.1.64184 > 127.0.0.1.23481: Flags [S], seq 735761411, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val 299792 ecr 0], length 0
00:00:00.000049 IP 127.0.0.1.23481 > 127.0.0.1.64184: Flags [S.], seq 1761768055, ack 735761412, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val 1182089667 ecr 299792], length 0
00:00:00.000035 IP 127.0.0.1.64184 > 127.0.0.1.23481: Flags [.], ack 1, win 8960, options [nop,nop,TS val 299792 ecr 1182089667], length 0
00:00:00.000222 IP 127.0.0.1.64184 > 127.0.0.1.23481: Flags [.], ack 1, win 8960, options [nop,nop,TS val 299792 ecr 1182089667], length 14336
00:00:00.099396 IP 127.0.0.1.23481 > 127.0.0.1.64184: Flags [.], ack 14337, win 7168, options [nop,nop,TS val 1182089677 ecr 299792], length 0
00:00:00.000047 IP 127.0.0.1.64184 > 127.0.0.1.23481: Flags [.], ack 1, win 8960, options [nop,nop,TS val 299802 ecr 1182089677], length 14336
00:00:00.000014 IP 127.0.0.1.64184 > 127.0.0.1.23481: Flags [P.], ack 1, win 8960, options [nop,nop,TS val 299802 ecr 1182089677], length 14336
00:00:00.000047 IP 127.0.0.1.23481 > 127.0.0.1.64184: Flags [.], ack 43009, win 3584, options [nop,nop,TS val 1182089677 ecr 299802], length 0
00:00:00.000115 IP 127.0.0.1.64184 > 127.0.0.1.23481: Flags [P.], ack 1, win 8960, options [nop,nop,TS val 299802 ecr 1182089677], length 14336
00:00:00.000155 IP 127.0.0.1.64184 > 127.0.0.1.23481: Flags [.], ack 1, win 8960, options [nop,nop,TS val 299802 ecr 1182089677], length 14336
00:00:00.000019 IP 127.0.0.1.23481 > 127.0.0.1.64184: Flags [.], ack 71681, win 0, options [nop,nop,TS val 1182089677 ecr 299802], length 0
00:00:00.909648 IP 127.0.0.1.23481 > 127.0.0.1.64184: Flags [R.], seq 1, ack 71681, win 0, options [nop,nop,TS val 1182089768 ecr 299802], length 0
SLEEP1 = 0; SLEEP2 = 1: LINGER_IN_SERVER: LINGER_IN_CLIENT: no error
00:00:00.000000 IP 127.0.0.1.62535 > 127.0.0.1.23481: Flags [S], seq 2416311658, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val 305567 ecr 0], length 0
00:00:00.000063 IP 127.0.0.1.23481 > 127.0.0.1.62535: Flags [S.], seq 723241073, ack 2416311659, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val 95346429 ecr 305567], length 0
00:00:00.000033 IP 127.0.0.1.62535 > 127.0.0.1.23481: Flags [.], ack 1, win 8960, options [nop,nop,TS val 305567 ecr 95346429], length 0
00:00:00.000266 IP 127.0.0.1.62535 > 127.0.0.1.23481: Flags [.], ack 1, win 8960, options [nop,nop,TS val 305567 ecr 95346429], length 14336
00:00:00.000019 IP 127.0.0.1.62535 > 127.0.0.1.23481: Flags [.], ack 1, win 8960, options [nop,nop,TS val 305567 ecr 95346429], length 14336
00:00:00.000015 IP 127.0.0.1.62535 > 127.0.0.1.23481: Flags [P.], ack 1, win 8960, options [nop,nop,TS val 305567 ecr 95346429], length 14336
00:00:00.000004 IP 127.0.0.1.23481 > 127.0.0.1.62535: Flags [.], ack 28673, win 5376, options [nop,nop,TS val 95346429 ecr 305567], length 0
00:00:00.000179 IP 127.0.0.1.62535 > 127.0.0.1.23481: Flags [.], ack 1, win 8960, options [nop,nop,TS val 305567 ecr 95346429], length 14336
00:00:00.000015 IP 127.0.0.1.62535 > 127.0.0.1.23481: Flags [.], ack 1, win 8960, options [nop,nop,TS val 305567 ecr 95346429], length 14336
00:00:00.000008 IP 127.0.0.1.23481 > 127.0.0.1.62535: Flags [.], ack 57345, win 1792, options [nop,nop,TS val 95346429 ecr 305567], length 0
00:00:00.099317 IP 127.0.0.1.23481 > 127.0.0.1.62535: Flags [.], ack 71681, win 0, options [nop,nop,TS val 95346439 ecr 305567], length 0
00:00:00.910039 IP 127.0.0.1.23481 > 127.0.0.1.62535: Flags [R.], seq 1, ack 71681, win 0, options [nop,nop,TS val 95346530 ecr 305567], length 0
SLEEP1 = 0; SLEEP2 = 1: LINGER_IN_CLIENT: no error
00:00:00.000000 IP 127.0.0.1.52232 > 127.0.0.1.23481: Flags [S], seq 837748429, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val 309639 ecr 0], length 0
00:00:00.000044 IP 127.0.0.1.23481 > 127.0.0.1.52232: Flags [S.], seq 1261614707, ack 837748430, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val 3126106957 ecr 309639], length 0
00:00:00.000034 IP 127.0.0.1.52232 > 127.0.0.1.23481: Flags [.], ack 1, win 8960, options [nop,nop,TS val 309639 ecr 3126106957], length 0
00:00:00.000249 IP 127.0.0.1.52232 > 127.0.0.1.23481: Flags [.], ack 1, win 8960, options [nop,nop,TS val 309639 ecr 3126106957], length 14336
00:00:00.000018 IP 127.0.0.1.52232 > 127.0.0.1.23481: Flags [.], ack 1, win 8960, options [nop,nop,TS val 309639 ecr 3126106957], length 14336
00:00:00.000019 IP 127.0.0.1.23481 > 127.0.0.1.52232: Flags [.], ack 28673, win 5376, options [nop,nop,TS val 3126106957 ecr 309639], length 0
00:00:00.000028 IP 127.0.0.1.52232 > 127.0.0.1.23481: Flags [P.], ack 1, win 8960, options [nop,nop,TS val 309639 ecr 3126106957], length 14336
00:00:00.000143 IP 127.0.0.1.52232 > 127.0.0.1.23481: Flags [.], ack 1, win 8960, options [nop,nop,TS val 309639 ecr 3126106957], length 14336
00:00:00.000015 IP 127.0.0.1.52232 > 127.0.0.1.23481: Flags [.], ack 1, win 8960, options [nop,nop,TS val 309639 ecr 3126106957], length 14336
00:00:00.000007 IP 127.0.0.1.23481 > 127.0.0.1.52232: Flags [.], ack 57345, win 1792, options [nop,nop,TS val 3126106957 ecr 309639], length 0
00:00:00.099371 IP 127.0.0.1.23481 > 127.0.0.1.52232: Flags [.], ack 71681, win 0, options [nop,nop,TS val 3126106967 ecr 309639], length 0
00:00:00.910114 IP 127.0.0.1.23481 > 127.0.0.1.52232: Flags [F.], seq 1, ack 71681, win 8960, options [nop,nop,TS val 3126107058 ecr 309639], length 0
00:00:00.000044 IP 127.0.0.1.52232 > 127.0.0.1.23481: Flags [.], ack 2, win 8960, options [nop,nop,TS val 309740 ecr 3126107058], length 14336
00:00:00.000014 IP 127.0.0.1.52232 > 127.0.0.1.23481: Flags [.], ack 2, win 8960, options [nop,nop,TS val 309740 ecr 3126107058], length 14336
00:00:00.000035 IP 127.0.0.1.23481 > 127.0.0.1.52232: Flags [R], seq 1261614709, win 0, length 0
00:00:00.000019 IP 127.0.0.1.23481 > 127.0.0.1.52232: Flags [R], seq 1261614709, win 0, length 0
SLEEP1 = 0; SLEEP2 = 1: no error
00:00:00.000000 IP 127.0.0.1.25865 > 127.0.0.1.23481: Flags [S], seq 386318710, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val 313828 ecr 0], length 0
00:00:00.000048 IP 127.0.0.1.23481 > 127.0.0.1.25865: Flags [S.], seq 3012495999, ack 386318711, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val 629145490 ecr 313828], length 0
00:00:00.000034 IP 127.0.0.1.25865 > 127.0.0.1.23481: Flags [.], ack 1, win 8960, options [nop,nop,TS val 313828 ecr 629145490], length 0
00:00:00.000219 IP 127.0.0.1.25865 > 127.0.0.1.23481: Flags [.], ack 1, win 8960, options [nop,nop,TS val 313828 ecr 629145490], length 14336
00:00:00.000017 IP 127.0.0.1.25865 > 127.0.0.1.23481: Flags [.], ack 1, win 8960, options [nop,nop,TS val 313828 ecr 629145490], length 14336
00:00:00.000015 IP 127.0.0.1.25865 > 127.0.0.1.23481: Flags [P.], ack 1, win 8960, options [nop,nop,TS val 313828 ecr 629145490], length 14336
00:00:00.000005 IP 127.0.0.1.23481 > 127.0.0.1.25865: Flags [.], ack 28673, win 5376, options [nop,nop,TS val 629145490 ecr 313828], length 0
00:00:00.000175 IP 127.0.0.1.25865 > 127.0.0.1.23481: Flags [.], ack 1, win 8960, options [nop,nop,TS val 313828 ecr 629145490], length 14336
00:00:00.000016 IP 127.0.0.1.25865 > 127.0.0.1.23481: Flags [.], ack 1, win 8960, options [nop,nop,TS val 313828 ecr 629145490], length 14336
00:00:00.000007 IP 127.0.0.1.23481 > 127.0.0.1.25865: Flags [.], ack 57345, win 1792, options [nop,nop,TS val 629145490 ecr 313828], length 0
00:00:00.099394 IP 127.0.0.1.23481 > 127.0.0.1.25865: Flags [.], ack 71681, win 0, options [nop,nop,TS val 629145500 ecr 313828], length 0
00:00:00.910135 IP 127.0.0.1.23481 > 127.0.0.1.25865: Flags [F.], seq 1, ack 71681, win 8960, options [nop,nop,TS val 629145591 ecr 313828], length 0
00:00:00.000069 IP 127.0.0.1.25865 > 127.0.0.1.23481: Flags [.], ack 2, win 8960, options [nop,nop,TS val 313929 ecr 629145591], length 14336
00:00:00.000014 IP 127.0.0.1.25865 > 127.0.0.1.23481: Flags [.], ack 2, win 8960, options [nop,nop,TS val 313929 ecr 629145591], length 14336
00:00:00.000035 IP 127.0.0.1.23481 > 127.0.0.1.25865: Flags [R], seq 3012496001, win 0, length 0
00:00:00.000019 IP 127.0.0.1.23481 > 127.0.0.1.25865: Flags [R], seq 3012496001, win 0, length 0
#endif /* 0 */
help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?86eigt4xwm.fsf>