Date: Tue, 27 May 2003 11:57:20 +0400 (MSD) From: Igor Sysoev <is@rambler-co.ru> To: Peter Jeremy <peterjeremy@optushome.com.au> Cc: arch@freebsd.org Subject: Re: sendfile(2) SF_NOPUSH flag proposal Message-ID: <Pine.BSF.4.21.0305271126470.46491-100000@is> In-Reply-To: <20030526201740.GA22178@cirb503493.alcatel.com.au>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 27 May 2003, Peter Jeremy wrote:
> On Mon, May 26, 2003 at 09:41:50PM +0400, Igor Sysoev wrote:
> >sendfile(2) now has two drawbacks:
> [IP frames are not always full]
> ...
> >When I turn TCP_NOPUSH on just before sendfile() then it sends the header
> >and the first part of the file in one 1460 bytes packet.
> >Besides it sends file pages in the full ethernet 1460 bytes packets.
> >When sendfile() completed or returned EAGAIN (I use non-blocking sockets)
> >I turn TCP_NOPUSH off and the remaining file part is flushed to client.
> >Without turing off the remaining file part is delayed for 5 seconds.
> ...
> >So here is a proposal. We can introduce a sendfile(2) flag, i.e. SF_NOPUSH
> >that will turn TF_NOPUSH on before the sending and turn it off just
> >before return. It allows to save two syscalls on each sendfile() call
> >and it's especially useful with non-blocking sockets - they can cause many
> >sendfile() calls.
>
> I'm less certain of the benefits of this - particularly in the non-
> blocking case. As I understand your proposal, your patch would turn
> off TF_NOPUSH just before returning EAGAIN. At this point, the TCP
> send buffer is full so packets should start being sent immediately.
> The last data in the send buffer may not comprise a complete frame so
> it should not be sent, but left queued to be merged with the next
> sendfile(2). Once SO_SNDLOWAT bytes are available in the send buffer,
> the socket will become writable, allowing a further sendfile(2) call.
> As long as SO_SNDLOWAT is at least one frame smaller than SO_SNDBUF,
> there should not be any send delay caused by TF_NOPUSH being set.
>
> I believe TF_NOPUSH should be set at the beginning of a transaction
> (or when the socket is opened) and cleared at the end of a transaction
> (or implicitly by close()ing the socket).
I thought about it more and I agree with you. TF_NOPUSH should be turned on
at the start of a transaction and turned off at the end of a transaction.
So I think there should be two flags:
SF_NOPUSH - it turns TF_NOPUSH on before the sending. It's cheap:
s = splnet();
inp = sotoinpcb(so);
if (inp != NULL) {
tp = intotcpcb(inp);
tp->t_flags |= TF_NOPUSH;
}
splx(s);
SF_PUSH - it turns TF_NOPUSH off after the sending has been completed.
If the sending returned EAGAIN then TF_NOPUSH would not be touched.
It's cheap too especially if the send buffer has enough data to fill
one MSS:
s = splnet();
inp = sotoinpcb(so);
if (inp != NULL) {
tp = intotcpcb(inp);
tp->t_flags &= ~TF_NOPUSH;
if (so->so_snd.sb_cc < tp->t_maxseg) {
error = tcp_output(tp);
}
}
splx(s);
Igor Sysoev
http://sysoev.ru/en/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.4.21.0305271126470.46491-100000>