Date: Wed, 26 Sep 2001 09:50:40 -0700 From: jayanth <jayanth@yahoo-inc.com> To: tsuchiya@flab.fujitsu.co.jp Cc: silby@silby.com, net@FreeBSD.ORG Subject: Re: TCP performance question Message-ID: <20010926095040.A17953@yahoo-inc.com> In-Reply-To: <200109250520.OAA00963@const.kawasaki.flab.fujitsu.co.jp>; from tsuchiya@flab.fujitsu.co.jp on Tue, Sep 25, 2001 at 02:20:10PM %2B0900 References: <200109250520.OAA00963@const.kawasaki.flab.fujitsu.co.jp>
next in thread | previous in thread | raw e-mail | index | archive | help
I think this is a performance issue and it would be nice to send the 589 bytes
of data immediately rather than wait for the delayed ack from the
other end.
This issue is a combination of mbuf cluster size and the
TF_MORETOCOME flag.
2049 is one byte more than the cluster size, so the first 1460 bytes
are sent and the remaining 588 is delayed because of the
TF_MORETOCOME flag. The remaining 1 byte is now copied to the socket buffer,
but in tcp_output.c there are 4 conditionals which I am not sure should
be true together. This prevents the next transmission of data,i.e
the next 589 bytes.
In tcp_output.c
/*
* Sender silly window avoidance. If connection is idle
* and can send all data, a maximum segment,
* at least a maximum default-size segment do it,
* or are forced, do it; otherwise don't bother.
* If peer's buffer is tiny, then send
* when window is at least half open.
* If retransmitting (possibly after persist timer forced us
* to send into a small window), then must resend.
*/
if (len) {
if (len == tp->t_maxseg)
goto send;
if (!(tp->t_flags & TF_MORETOCOME) &&
(idle || tp->t_flags & TF_NODELAY) &&
(tp->t_flags & TF_NOPUSH) == 0 &&
len + off >= so->so_snd.sb_cc)
goto send;
^^^^^^^^^^^^^^^^^^^^^^^^^^
In this particular case idle and
TF_NODELAY are off and hence the conditional is false.
If len + off >= so->so_snd.sb_cc then why not send all the data ?
Probably needs a little more thought.
^^^^^^^^^^^^^^^^^^^^^^^^^^
if (tp->t_force)
goto send;
if (len >= tp->max_sndwnd / 2 && tp->max_sndwnd > 0)
goto send;
if (SEQ_LT(tp->snd_nxt, tp->snd_max))
goto send;
}
The issue being that if I can send 588 why not anything between
589 and 1460. This assumption is based on the fact that the congestion
window is large enough for us to send data.
I dont think TF_NODELAY needs to be set by the application here.
jayanth
tsuchiya@flab.fujitsu.co.jp (tsuchiya@flab.fujitsu.co.jp) wrote:
> > >
> > > I tried to say that it had no effect between FreeBSD4.3 and Solaris, on my
> > > problem. That's what I did previously.
> > >
> > > I found discussion on "delayed ack problem"(January 24 and 25) in this
> > > mailing list. Though still do not understand why delayed_ack=0 does not
> > > work for FreeBSD and Solaris.
> > >
> > > Anyway I appreciate your helping me.
> > > Thank you,
> > > Yoshi
> >
> > If you can reproduce the problem, please take some traces with tcpdump and
> > post them somewhere for public consumption. Perhaps then the problem can
> > be found and fixed.
> >
> > Mike "Silby" Silbersack
>
>
> Ok, "yokan" is FreeBSD4.1.1 and I set delayed_ack=0, and "purcell" is
> a Solaris 2.7 box. The sending size at first is 1460+588byte and 1460+589byte
> later. When the size gets more than 1460+589bytes, the performance goes down.
> At that time, purcell begins to send acks, possibly because yokan behaves
> very slow.
>
> ==========================
> 04:54:40.136343 yokan.1238 > purcell.12354: . ack 609417 win 16060 (DF)
> 04:54:40.136362 purcell.12354 > yokan.1238: P 609417:610005(588) ack 610005 win 8760 (DF)
> 04:54:40.136386 yokan.1238 > purcell.12354: . ack 610005 win 15472 (DF)
> 04:54:40.136418 yokan.1238 > purcell.12354: . 610005:611465(1460) ack 610005 win 17520 (DF)
> 04:54:40.136430 yokan.1238 > purcell.12354: P 611465:612053(588) ack 610005 win 17520 (DF)
> 04:54:40.137228 purcell.12354 > yokan.1238: . 610005:611465(1460) ack 612053 win 8760 (DF)
> 04:54:40.137276 yokan.1238 > purcell.12354: . ack 611465 win 16060 (DF)
> 04:54:40.137287 purcell.12354 > yokan.1238: P 611465:612053(588) ack 612053 win 8760 (DF)
> 04:54:40.137311 yokan.1238 > purcell.12354: . ack 612053 win 15472 (DF)
> 04:54:40.137359 yokan.1238 > purcell.12354: . 612053:613513(1460) ack 612053 win 17520 (DF)
> 04:54:40.137370 yokan.1238 > purcell.12354: P 613513:614101(588) ack 612053 win 17520 (DF)
> 04:54:40.138168 purcell.12354 > yokan.1238: . 612053:613513(1460) ack 614101 win 8760 (DF)
> 04:54:40.138203 yokan.1238 > purcell.12354: . ack 613513 win 16060 (DF)
> 04:54:40.138223 purcell.12354 > yokan.1238: P 613513:614101(588) ack 614101 win 8760 (DF)
> 04:54:40.138245 yokan.1238 > purcell.12354: . ack 614101 win 15472 (DF)
> 04:54:40.138278 yokan.1238 > purcell.12354: . 614101:615561(1460) ack 614101 win 17520 (DF)
> 04:54:40.208029 purcell.12354 > yokan.1238: . ack 615561 win 8760 (DF)
> 04:54:40.208055 yokan.1238 > purcell.12354: P 615561:616150(589) ack 614101 win 17520 (DF)
> 04:54:40.208675 purcell.12354 > yokan.1238: . 614101:615561(1460) ack 616150 win 8760 (DF)
> 04:54:40.208708 yokan.1238 > purcell.12354: . ack 615561 win 16060 (DF)
> 04:54:40.208729 purcell.12354 > yokan.1238: P 615561:616150(589) ack 616150 win 8760 (DF)
> 04:54:40.208754 yokan.1238 > purcell.12354: . ack 616150 win 15471 (DF)
> 04:54:40.208786 yokan.1238 > purcell.12354: . 616150:617610(1460) ack 616150 win 17520 (DF)
> 04:54:40.278035 purcell.12354 > yokan.1238: . ack 617610 win 8760 (DF)
> 04:54:40.278060 yokan.1238 > purcell.12354: P 617610:618199(589) ack 616150 win 17520 (DF)
> 04:54:40.278678 purcell.12354 > yokan.1238: . 616150:617610(1460) ack 618199 win 8760 (DF)
> 04:54:40.278709 yokan.1238 > purcell.12354: . ack 617610 win 16060 (DF)
> 04:54:40.278731 purcell.12354 > yokan.1238: P 617610:618199(589) ack 618199 win 8760 (DF)
> 04:54:40.278755 yokan.1238 > purcell.12354: . ack 618199 win 16931 (DF)
> 04:54:40.278796 yokan.1238 > purcell.12354: . 618199:619659(1460) ack 618199 win 17520 (DF)
> 04:54:40.348028 purcell.12354 > yokan.1238: . ack 619659 win 8760 (DF)
> 04:54:40.348059 yokan.1238 > purcell.12354: P 619659:620248(589) ack 618199 win 17520 (DF)
> 04:54:40.348680 purcell.12354 > yokan.1238: . 618199:619659(1460) ack 620248 win 8760 (DF)
> 04:54:40.348711 yokan.1238 > purcell.12354: . ack 619659 win 16060 (DF)
> 04:54:40.348736 purcell.12354 > yokan.1238: P 619659:620248(589) ack 620248 win 8760 (DF)
> 04:54:40.348758 yokan.1238 > purcell.12354: . ack 620248 win 16931 (DF)
> 04:54:40.348795 yokan.1238 > purcell.12354: . 620248:621708(1460) ack 620248 win 17520 (DF)
> 04:54:40.418025 purcell.12354 > yokan.1238: . ack 621708 win 8760 (DF)
> ==========================
>
> And here, it's between "yokan" and another FreeBSD machine. "regulus" is
> a FreeBSD4.3 box, and delayed_ack=0 has been done on both of the machines.
> It seems they send acks normally and there is no performance slow down.
>
> ==========================
> 04:55:38.774912 yokan.1239 > regulus.12354: P 613513:614101(588) ack 612053 win 17520 (DF)
> 04:55:38.775723 regulus.12354 > yokan.1239: . ack 613513 win 16060 (DF)
> 04:55:38.775765 regulus.12354 > yokan.1239: . ack 614101 win 16932 (DF)
> 04:55:38.776424 regulus.12354 > yokan.1239: . 612053:613513(1460) ack 614101 win 17520 (DF)
> 04:55:38.776459 yokan.1239 > regulus.12354: . ack 613513 win 16060 (DF)
> 04:55:38.776479 regulus.12354 > yokan.1239: P 613513:614101(588) ack 614101 win 17520 (DF)
> 04:55:38.776504 yokan.1239 > regulus.12354: . ack 614101 win 15472 (DF)
> 04:55:38.776541 yokan.1239 > regulus.12354: . 614101:615561(1460) ack 614101 win 17520 (DF)
> 04:55:38.777358 regulus.12354 > yokan.1239: . ack 615561 win 16060 (DF)
> 04:55:38.777384 yokan.1239 > regulus.12354: P 615561:616150(589) ack 614101 win 17520 (DF)
> 04:55:38.777828 regulus.12354 > yokan.1239: . ack 616150 win 16931 (DF)
> 04:55:38.778482 regulus.12354 > yokan.1239: . 614101:615561(1460) ack 616150 win 17520 (DF)
> 04:55:38.778510 yokan.1239 > regulus.12354: . ack 615561 win 16060 (DF)
> 04:55:38.778958 regulus.12354 > yokan.1239: P 615561:616150(589) ack 616150 win 17520 (DF)
> 04:55:38.778984 yokan.1239 > regulus.12354: . ack 616150 win 16931 (DF)
> 04:55:38.779020 yokan.1239 > regulus.12354: . 616150:617610(1460) ack 616150 win 17520 (DF)
> 04:55:38.779835 regulus.12354 > yokan.1239: . ack 617610 win 16060 (DF)
> 04:55:38.779861 yokan.1239 > regulus.12354: P 617610:618199(589) ack 616150 win 17520 (DF)
> 04:55:38.780305 regulus.12354 > yokan.1239: . ack 618199 win 16931 (DF)
> 04:55:38.780959 regulus.12354 > yokan.1239: . 616150:617610(1460) ack 618199 win 17520 (DF)
> 04:55:38.780986 yokan.1239 > regulus.12354: . ack 617610 win 16060 (DF)
> 04:55:38.781435 regulus.12354 > yokan.1239: P 617610:618199(589) ack 618199 win 17520 (DF)
> 04:55:38.781459 yokan.1239 > regulus.12354: . ack 618199 win 16931 (DF)
> ==========================
> Thanks,
> Yoshi
>
> To Unsubscribe: send mail to majordomo@FreeBSD.org
> with "unsubscribe freebsd-net" in the body of the message
>
>
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-net" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20010926095040.A17953>