From owner-freebsd-arch@FreeBSD.ORG Mon May 26 13:17:50 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2CB0037B401 for ; Mon, 26 May 2003 13:17:50 -0700 (PDT) Received: from cirb503493.alcatel.com.au (c18609.belrs1.nsw.optusnet.com.au [210.49.80.204]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9236843F85 for ; Mon, 26 May 2003 13:17:48 -0700 (PDT) (envelope-from peterjeremy@optushome.com.au) Received: from cirb503493.alcatel.com.au (localhost.alcatel.com.au [127.0.0.1])h4QKHkp9043994; Tue, 27 May 2003 06:17:46 +1000 (EST) (envelope-from jeremyp@cirb503493.alcatel.com.au) Received: (from jeremyp@localhost) by cirb503493.alcatel.com.au (8.12.8/8.12.8/Submit) id h4QKHf4c043993; Tue, 27 May 2003 06:17:41 +1000 (EST) Date: Tue, 27 May 2003 06:17:41 +1000 From: Peter Jeremy To: Igor Sysoev Message-ID: <20030526201740.GA22178@cirb503493.alcatel.com.au> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i cc: arch@freebsd.org Subject: Re: sendfile(2) SF_NOPUSH flag proposal X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 May 2003 20:17:50 -0000 On Mon, May 26, 2003 at 09:41:50PM +0400, Igor Sysoev wrote: >sendfile(2) now has two drawbacks: [IP frames are not always full] ... >When I turn TCP_NOPUSH on just before sendfile() then it sends the header >and the first part of the file in one 1460 bytes packet. >Besides it sends file pages in the full ethernet 1460 bytes packets. >When sendfile() completed or returned EAGAIN (I use non-blocking sockets) >I turn TCP_NOPUSH off and the remaining file part is flushed to client. >Without turing off the remaining file part is delayed for 5 seconds. ... >So here is a proposal. We can introduce a sendfile(2) flag, i.e. SF_NOPUSH >that will turn TF_NOPUSH on before the sending and turn it off just >before return. It allows to save two syscalls on each sendfile() call >and it's especially useful with non-blocking sockets - they can cause many >sendfile() calls. I'm less certain of the benefits of this - particularly in the non- blocking case. As I understand your proposal, your patch would turn off TF_NOPUSH just before returning EAGAIN. At this point, the TCP send buffer is full so packets should start being sent immediately. The last data in the send buffer may not comprise a complete frame so it should not be sent, but left queued to be merged with the next sendfile(2). Once SO_SNDLOWAT bytes are available in the send buffer, the socket will become writable, allowing a further sendfile(2) call. As long as SO_SNDLOWAT is at least one frame smaller than SO_SNDBUF, there should not be any send delay caused by TF_NOPUSH being set. I believe TF_NOPUSH should be set at the beginning of a transaction (or when the socket is opened) and cleared at the end of a transaction (or implicitly by close()ing the socket). Peter