From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 31 12:15:35 2003 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E9CC116A4CE for ; Fri, 31 Oct 2003 12:15:35 -0800 (PST) Received: from bluebox.CS.Princeton.EDU (bluebox.CS.Princeton.EDU [128.112.136.38]) by mx1.FreeBSD.org (Postfix) with ESMTP id E887543F75 for ; Fri, 31 Oct 2003 12:15:34 -0800 (PST) (envelope-from vivek@CS.Princeton.EDU) Received: from cs.princeton.edu (oakley [128.112.139.27]) (authenticated bits=0)h9VKFDFn022667 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NOT); Fri, 31 Oct 2003 15:15:13 -0500 (EST) Message-ID: <3FA2C2D1.7000406@cs.princeton.edu> Date: Fri, 31 Oct 2003 15:15:13 -0500 From: Vivek Pai User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.0.1) Gecko/20020920 Netscape/7.0 X-Accept-Language: en-us, en MIME-Version: 1.0 To: John-Mark Gurney References: <1066789354.21430.39.camel@boxster.onthenet.com.au> <20031022082953.GA69506@rot13.obsecurity.org> <1066816287.25609.34.camel@boxster.onthenet.com.au> <20031022095754.GA70026@rot13.obsecurity.org> <1066820436.25609.93.camel@boxster.onthenet.com.au> <1067183332.3f9bece4c0cf4@webmail.cs.princeton.edu> <20031026175726.GC558@funkthat.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: =?ISO-8859-1?Q?Dag-Erling_Sm=F8rgrav?= cc: Q cc: freebsd-hackers@freebsd.org cc: Kris Kennaway Subject: Re: Some mmap observations compared to Linux 2.6/OpenBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 31 Oct 2003 20:15:36 -0000 There are two scenarios - sendfile can't proceed immediately because all sfbuf space is exhausted, and sendfile can't proceed immediately because the page to be sent isn't in physical memory. In both cases, we can have another process/thread call sendfile with the flag cleared, allowing sendfile to block as needed. This work is being done in the context of the Flash Web Server, which already had helper processes to perform all of the disk-related operations. These helpers use pipes/sockets to communicate back to the main process that a page has been loaded into memory (or that the page has been sent directly). We don't currently initiate async I/O's using a nonportable interface, but that's also a possibility. -Vivek John-Mark Gurney wrote: > vivek@CS.Princeton.EDU wrote this message on Sun, Oct 26, 2003 at 10:48 -0500: > >>We decided to avoid all mmap/mincore operations, and opted to instead >>add a flag to sendfile such that we can try sendfile and have it return >>an error (instead of blocking) when a page isn't in memory. > > > I was thinking about this myself, but how do you prevent spinning on > the sendfile waiting for the page to become available? You have the > send space to wait for on the socket, but there is no equivalent kqueue > event to wait for when a page becomes available. Do you simply always > spin on sendfile since the socket keeps returning available for writing? > > I have a similar web server design that I wrote at the end of last > year, and your paper brought to light to problems of sendfile, so I have > been thinking of switching to async io, since there you can get > completion status when the io completes. >