From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 4 22:25:48 2003 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3833616A4CE for ; Tue, 4 Nov 2003 22:25:48 -0800 (PST) Received: from bluebox.CS.Princeton.EDU (bluebox.CS.Princeton.EDU [128.112.136.38]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1432543FCB for ; Tue, 4 Nov 2003 22:25:47 -0800 (PST) (envelope-from vivek@CS.Princeton.EDU) Received: from cs.princeton.edu (oakley [128.112.139.27]) (authenticated bits=0)hA56PhNt005171 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NOT); Wed, 5 Nov 2003 01:25:44 -0500 (EST) Message-ID: <3FA897E7.5020103@cs.princeton.edu> Date: Wed, 05 Nov 2003 01:25:43 -0500 From: Vivek Pai User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.0.1) Gecko/20020920 Netscape/7.0 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Mike Silbersack References: <1066789354.21430.39.camel@boxster.onthenet.com.au> <20031022082953.GA69506@rot13.obsecurity.org> <1066816287.25609.34.camel@boxster.onthenet.com.au> <20031022095754.GA70026@rot13.obsecurity.org> <1067183332.3f9bece4c0cf4@webmail.cs.princeton.edu> <3FA2C43E.3030204@cs.princeton.edu> <3FA7EF77.5010808@cs.princeton.edu> <20031104221455.I9997@odysseus.silby.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-hackers@freebsd.org cc: Alan Cox Subject: Re: Update: Debox sendfile modifications X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2003 06:25:48 -0000 Mike Silbersack wrote: > On Tue, 4 Nov 2003, Vivek Pai wrote: >>The one other aspect of this is that sf_bufs mappings are maintained for >>a configurable amount of time, reducing the number of TLB ops. You can >>specify the parameter for how long, ranging from -1 (no coalescing at >>all), 0 (coalesce, but free immediately after last holder release), to >>any other time. Obviously, any value above 0 will increase the amount of >>wired memory at any given point in time, but it's configurable. > > Ah, I missed that point. Did your testing show the caching part of the > functionality to be significant? I think it buys us a small gain (a few percent) under static-content workloads, and a little less under SpecWeb99, where more time is spent in dynamic content. However, it's almost free - the additional complexity beyond just coalescing is hooking into the timer to free unused mappings. >>How about something that lets you choose what happens - we've got a >>flag field anyway, so why not have options to control the behavior on >>missing pages? Applications like Flash might just want the error >>message so that they can handle it themselves, while other applications >>may be happy with the default that you're suggesting. > > > Yeah, I guess you're right; as John-Mark Gurney also pointed out, it would > be extremely difficult to hide the asychronous implementation. Assuming > that we came up with an extra flag which told sendfile to use asynchronous > mode (and raised the maximum number of such threads), wouldn't it be even > more efficient than Flash's helper threads? We use the helper processes for mostly historical portability reasons, and if we wanted to really customize Flash for FreeBSD, we'd issue some async disk ops and get their completion status via kevent. So, it's just a tradeoff of where the additional complexity goes - kernel or application. I don't know that we'll be breaking away from our helper approach just yet, mostly because we're still interested in some level of portability. If you were to have sendfile issue the disk reads, how would you signal completion? I guess one approach is to make the socket buffer appear to have no space while the sendfile-initiated read is in progress, but it seems to me that such an approach would be considered too ugly. It would cause the least modification to applications, because otherwise apps need to disable interest on the socket having space, and re-enable it after getting notified that the sendfile-initiated read (and transfer) completed. Am I missing something? -Vivek