From owner-freebsd-current  Thu Aug 14 17:46:41 1997
Return-Path: <owner-freebsd-current>
Received: (from root@localhost)
          by hub.freebsd.org (8.8.5/8.8.5) id RAA25470
          for current-outgoing; Thu, 14 Aug 1997 17:46:41 -0700 (PDT)
Received: from alpha.xerox.com (alpha.Xerox.COM [13.1.64.93])
          by hub.freebsd.org (8.8.5/8.8.5) with SMTP id RAA25463;
          Thu, 14 Aug 1997 17:46:37 -0700 (PDT)
Received: from crevenia.parc.xerox.com ([13.2.116.11]) by alpha.xerox.com with SMTP id <53165(5)>; Thu, 14 Aug 1997 17:45:48 PDT
Received: from localhost by crevenia.parc.xerox.com with SMTP id <177512>; Thu, 14 Aug 1997 17:45:08 -0700
To: "Jordan K. Hubbard" <jkh@time.cdrom.com>
cc: wollman@freebsd.org, current@freebsd.org
Subject: Re: Well, I guess it's about time I mentioned this little problem... 
In-reply-to: Your message of "Thu, 14 Aug 97 11:32:30 PDT."
             <1433.871583550@time.cdrom.com> 
Date: Thu, 14 Aug 1997 17:45:01 PDT
From: Bill Fenner <fenner@parc.xerox.com>
Message-Id: <97Aug14.174508pdt.177512@crevenia.parc.xerox.com>
Sender: owner-freebsd-current@freebsd.org
X-Loop: FreeBSD.org
Precedence: bulk

My bet is broken path mtu discovery in the opposite direction.

>11:30:53.276132 time.cdrom.com.1216 > rocky.sunlabs.com.http: S 2243214515:2243214515(0) win 16384 <ms
>  s 1460,nop,wscale 0,nop,nop,timestamp[|tcp]> (DF)
>11:30:53.501585 rocky.sunlabs.com.http > time.cdrom.com.1216: S 294104724:294104724(0) ack 2243214516 
>  win 8760 <mss 1460> (DF)
>11:30:53.501696 time.cdrom.com.1216 > rocky.sunlabs.com.http: . ack 1 win 16384 (DF)
>11:30:53.503656 time.cdrom.com.1216 > rocky.sunlabs.com.http: . 1:513(512) ack 1 win 16384 (DF)
>11:30:53.797419 rocky.sunlabs.com.http > time.cdrom.com.1216: . ack 513 win 8760 (DF)
>11:30:53.797508 time.cdrom.com.1216 > rocky.sunlabs.com.http: P 513:667(154) ack 1 win 16384 (DF)

For some reason, we send only 512 bytes in the first packet, but that's
probably not interesting in this case since we send a small first packet
the second time.  What's interesting is that in the one that works,
the return ACK for byte #667 is piggybacked on the other end's
MSS-sized data packet -- so, if in this one, the ACK for byte #667
is also piggybacked on the MSS-sized data packet, it's a 1500-byte
packet, which is apparently getting dropped without proper notification
on the way back to us.

>11:30:56.480098 time.cdrom.com.1216 > rocky.sunlabs.com.http: P 513:667(154) ack 1 win 16384 (DF)
>11:30:56.698495 rocky.sunlabs.com.http > time.cdrom.com.1216: . ack 667 win 8760 (DF)

We retransmit because we haven't gotten an ACK, and rocky replies with
a dataless ACK.  It doesn't include any data since it's still doing
slow-start towards us.

rocky keeps retransmitting that first MSS(MTU)-sized packet, it keeps
getting dropped and MTU discovery isn't working so we never hear from it
again.

  Bill