Date: Thu, 24 Jul 1997 10:22:13 -0400 (EDT) From: Garrett Wollman <wollman@khavrinen.lcs.mit.edu> To: =?KOI8-R?B?4c7E0sXKIP7F0s7P1w==?= <ache@nagual.pp.ru> Cc: current@freebsd.org Subject: Re: 'fetch' error with http, fix wanted Message-ID: <199707241422.KAA29714@khavrinen.lcs.mit.edu> In-Reply-To: <Pine.BSF.3.96.970724144434.641B-100000@nagual.pp.ru> References: <199707240007.UAA27406@khavrinen.lcs.mit.edu> <Pine.BSF.3.96.970724144434.641B-100000@nagual.pp.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
<<On Thu, 24 Jul 1997 14:45:33 +0400 (MSD), =?KOI8-R?B?4c7E0sXKIP7F0s7P1w==?= <ache@nagual.pp.ru> said: Something is broken on their end. Note: > 869740951.001002 lothlorien.net.http > lsd.relcom.eu.net.relief: . 1049447366:1049448826(1460) ack 2045965091 win 31744 Here they sent us a data packet. > 869740951.001285 lsd.relcom.eu.net.relief > lothlorien.net.http: . ack 1049448826 win 16060 (DF) We ack it. > 869740951.611809 lothlorien.net.http > lsd.relcom.eu.net.relief: . 1049448826:1049450286(1460) ack 2045965091 win 31744 They send us another full packet of data. > 869740951.611882 lothlorien.net.http > lsd.relcom.eu.net.relief: R 1049447366:1049447366(0) win 0 Oops. All of a sudden, they send us a reset. Near as I can tell from the rest of the trace, we have done nothing to provoke this reset; they have already acked our FIN (and, of course, all of the data we sent before) and are in a half-closed state, and we simply keep on acking their data packets. > 869740951.612068 lsd.relcom.eu.net.relief > lothlorien.net.http: . ack 1049450286 win 16060 (DF) We send an ack to that last data packet (TCP hasn't, at this point, seen the RST yet). > 869740951.618377 lothlorien.net.http > lsd.relcom.eu.net.relief: R 1049448826:1049448826(0) win 0 They send another reset in response to one of our acks which was in transit (which was an ack to their sequence number 1049445906). > 869740952.144714 lothlorien.net.http > lsd.relcom.eu.net.relief: R 1049450286:1049450286(0) win 0 And again. > 869740953.716743 lothlorien.net.http > lsd.relcom.eu.net.relief: . 1049445906:1049447366(1460) ack 2045965091 win 31744 Somehow, this old segment has floated out of the network. We acked it, but that ack is one of the ones they replied to with a reset. > 869740953.716975 lsd.relcom.eu.net.relief > lothlorien.net.http: R 2045965091:2045965091(0) win 0 Finally, they reply with a reset to our last ack. I would say that either their TCP or their Web server is busted. This is actually not terribly surprising: ;; ANSWERS: lothlorien.net. 85459 HINFO "Custom Cyrix 686-166+" "Linux" The specific problem seems to be that their system (or their Web server, but that's unlikely) does not implement half-closed states correctly. You can verify this by replacing the references to MSG_EOF in fetch with zeros; the file then transfers correctly. Here is a patch which adds a `-b' (for ``bug-compatible'') flag to fetch: Index: fetch.1 =================================================================== RCS file: /homes/cvs/src/usr.bin/fetch/fetch.1,v retrieving revision 1.17 diff -u -r1.17 fetch.1 --- fetch.1 1997/03/05 18:57:15 1.17 +++ fetch.1 1997/07/24 14:20:47 @@ -7,7 +7,7 @@ .Nd retrieve a file by Uniform Resource Locator .Sh SYNOPSIS .Nm fetch -.Op Fl MPamnpqr +.Op Fl MPabmnpqr .Op Fl o Ar file .Ar URL .Op Ar ... @@ -48,6 +48,12 @@ .Bl -tag -width Fl .It Fl a Automatically retry the transfer upon soft failures. +.It Fl b +Work around a bug in some +.Tn HTTP +servers which fail to correctly implement the +.Tn TCP +protocol. .It Fl c Ar dir The file to retrieve is in directory .Ar dir @@ -301,3 +307,7 @@ authentication mode is implemented for .Tn HTTP . This should be replaced by digest authentication. +.Pp +The +.Fl b +flag should not be necessary. Index: fetch.h =================================================================== RCS file: /homes/cvs/src/usr.bin/fetch/fetch.h,v retrieving revision 1.3 diff -u -r1.3 fetch.h --- fetch.h 1997/02/05 19:59:10 1.3 +++ fetch.h 1997/07/24 14:10:43 @@ -49,6 +49,7 @@ int fs_linkfile; /* -l option */ int fs_precious; /* -R option */ int fs_auto_retry; /* -a option */ + int fs_linux_bug; /* -b option */ time_t fs_modtime; void *fs_proto; int (*fs_retrieve)(struct fetch_state *); Index: http.c =================================================================== RCS file: /homes/cvs/src/usr.bin/fetch/http.c,v retrieving revision 1.6 diff -u -r1.6 http.c --- http.c 1997/03/06 20:01:32 1.6 +++ http.c 1997/07/24 14:12:28 @@ -483,7 +483,7 @@ n = 0; msg.msg_control = 0; msg.msg_controllen = 0; - msg.msg_flags = MSG_EOF; + msg.msg_flags = fs->fs_linux_bug ? 0 : MSG_EOF; #define addstr(Iov, N, Str) \ do { \ @@ -575,7 +575,7 @@ fs->fs_status = "sending request message"; setup_sigalrm(); alarm(timo); - if (sendmsg(s, &msg, MSG_EOF) < 0) { + if (sendmsg(s, &msg, fs->fs_linux_bug ? 0 : MSG_EOF) < 0) { warn("sendmsg: %s", https->http_hostname); fclose(remote); return EX_OSERR; Index: main.c =================================================================== RCS file: /homes/cvs/src/usr.bin/fetch/main.c,v retrieving revision 1.40 diff -u -r1.40 main.c --- main.c 1997/07/02 06:28:32 1.40 +++ main.c 1997/07/24 14:15:36 @@ -73,7 +73,7 @@ fs.fs_verbose = 1; change_to_dir = file_to_get = hostname = 0; - while ((c = getopt(argc, argv, "ac:D:f:h:HilLmMnNo:pPqRrT:vV:")) != -1) { + while ((c = getopt(argc, argv, "abc:D:f:h:HilLmMnNo:pPqRrT:vV:")) != -1) { switch (c) { case 'D': case 'H': case 'I': case 'N': case 'L': case 'V': break; /* ncftp compatibility */ @@ -81,10 +81,15 @@ case 'a': fs.fs_auto_retry = 1; break; + + case 'b': + fs.fs_linux_bug = 1; + break; + case 'c': change_to_dir = optarg; break; - + case 'f': file_to_get = optarg; break; -GAWollman -- Garrett A. Wollman | O Siem / We are all family / O Siem / We're all the same wollman@lcs.mit.edu | O Siem / The fires of freedom Opinions not those of| Dance in the burning flame MIT, LCS, CRS, or NSA| - Susan Aglukark and Chad Irschick
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199707241422.KAA29714>