From owner-freebsd-arch@FreeBSD.ORG Mon May 26 10:41:53 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 361F237B401 for ; Mon, 26 May 2003 10:41:53 -0700 (PDT) Received: from park.rambler.ru (park.rambler.ru [81.19.64.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id AC4DF43F85 for ; Mon, 26 May 2003 10:41:51 -0700 (PDT) (envelope-from is@rambler-co.ru) Received: from is.park.rambler.ru (is.park.rambler.ru [81.19.64.102]) by park.rambler.ru (8.12.6/8.12.6) with ESMTP id h4QHfomF098110 for ; Mon, 26 May 2003 21:41:50 +0400 (MSD) Date: Mon, 26 May 2003 21:41:50 +0400 (MSD) From: Igor Sysoev X-Sender: is@is To: arch@freebsd.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: sendfile(2) SF_NOPUSH flag proposal X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 May 2003 17:41:53 -0000 sendfile(2) now has two drawbacks: 1) it always sends the header, the file and the trailer in the separate packets even their sizes allow to place all them in one packet. For example the typical HTTP response header is less then an ethernet packet and sendfile() sends it in first small packet. 2) often enough it sends 4K page in three packets: 1460, 1460 and 1176 bytes. When I turn TCP_NOPUSH on just before sendfile() then it sends the header and the first part of the file in one 1460 bytes packet. Besides it sends file pages in the full ethernet 1460 bytes packets. When sendfile() completed or returned EAGAIN (I use non-blocking sockets) I turn TCP_NOPUSH off and the remaining file part is flushed to client. Without turing off the remaining file part is delayed for 5 seconds. Surprisingly that the turning TCP_NOPUSH off flushes the file part (I did not try the trailer) on FreeBSD 4.2 and 4.3 without src/sys/netinet/tcp_usrreq.c fixes 1.53 and 1.51.2.11. I looked in src/sys/netinet/tcp_usrreq.c to learn how to turn TCP_NOPUSH on/off. It's seems it's as simple as: struct inpcb *inp; struct tcpcb *tp; inp = sotoinpcb(so); tp = intotcpcb(inp); turn on: tp->t_flags |= TF_NOPUSH; turn off: tp->t_flags &= ~TF_NOPUSH; error = tcp_output(tp); So here is a proposal. We can introduce a sendfile(2) flag, i.e. SF_NOPUSH that will turn TF_NOPUSH on before the sending and turn it off just before return. It allows to save two syscalls on each sendfile() call and it's especially useful with non-blocking sockets - they can cause many sendfile() calls. Igor Sysoev http://sysoev.ru/en/