From owner-freebsd-hackers@FreeBSD.ORG Mon Jun 16 17:22:48 2003 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E0E1137B401 for ; Mon, 16 Jun 2003 17:22:48 -0700 (PDT) Received: from angelica.unixdaemons.com (angelica.unixdaemons.com [209.148.64.135]) by mx1.FreeBSD.org (Postfix) with ESMTP id C0DBE43F85 for ; Mon, 16 Jun 2003 17:22:47 -0700 (PDT) (envelope-from bmilekic@technokratis.com) Received: from angelica.unixdaemons.com (bmilekic@localhost.unixdaemons.com [127.0.0.1])h5H0Mh8N016069; Mon, 16 Jun 2003 20:22:43 -0400 (EDT) Received: (from bmilekic@localhost) by angelica.unixdaemons.com (8.12.9/8.12.1/Submit) id h5H0MhED016068; Mon, 16 Jun 2003 20:22:43 -0400 (EDT) (envelope-from bmilekic@technokratis.com) X-Authentication-Warning: angelica.unixdaemons.com: bmilekic set sender to bmilekic@technokratis.com using -f Date: Mon, 16 Jun 2003 20:22:43 -0400 From: Bosko Milekic To: Yaoping Ruan Message-ID: <20030617002243.GA10983@technokratis.com> References: <3EEDFC43.BD80B098@cs.princeton.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3EEDFC43.BD80B098@cs.princeton.edu> User-Agent: Mutt/1.4.1i cc: freebsd-hackers@freebsd.org Subject: Re: Kernel Support for System Call Performance Monitoring X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2003 00:22:49 -0000 On Mon, Jun 16, 2003 at 01:20:03PM -0400, Yaoping Ruan wrote: > We have been working on improving Web server performance on > FreeBSD, and think you may be interested in the results and > techniques we used. Specifically, we focus on the SpecWeb99 > benchmark and the Flash Web Server, and have roughly quadrupled > its performance. We did this by adding support for a very > low-cost kernel performance monitoring system, which allowed > us to find and fix a number of bad interactions between the > server and the OS. We additionally augmented one of the system > calls, sendfile, to be more useful for this kind of server. > We think that our observations may be useful for other servers, > and may present opportunities for performance improvement in > FreeBSD. > > A paper describing our system can be found at > http://www.cs.princeton.edu/~yruan/DeBox and we can provide the > patches we made if anyone's interested. We welcome any comments > and feedback that you have. First off, thank you for choosing FreeBSD for your research. The more effort is put into doing this sort of research, the better it is for both the academic community and the industry. I've read your paper and have a few brief notes: - On DeBox implementation. I understand that the DeBox implementation is primarily a tool used for tracking down potential application bottlenecks and so the relative importance of the crudeness of the implementation is not so high. However, I'm looking at this from the perspective of introducing DeBox as a permanent option in FreeBSD, and two immediate problems are: 1) User-visible DeBoxInfo structure has the magic number "5" PerSleepInfo structs and the magic number "200" CallTrace structs. It seems that it would be somewhat less crude to turn the struct arrays in DeBoxInfo into pointers in which case you have several options. You could provide a library to link applications compiled for DeBox use with that would take care of allocating the space in which to store maxSleeps and maxTrace-worth of memory and hooking the data into resultBuf or providing the addresses as separate arguments to the DeBoxControl() system call. For what concerns the kernel, you could take a similar approach and dynamically pre-allocate the PerSleepInfo and CallTrace structures, based on the requirements given by the DeBoxControl system call. 2) The problem of modifying entry-exit paths in function calls. Admittedly, this is hard, but crudely modifying a select number of functions to Do The Right Thing for what concerns call tracing is hard to justify from a general perspective. I don't mean to spread FUD here; the change you made is totally OK from a measurement perspective and serves great for the paper, it's just tougher to integrate this stuff into the mainline code. - On the Case Study. I was most interested in the sendfile modifications you talk about and would be interested in seeing patches. I know that some of the modifications you mention have already been done in 5.x; Notably, if you have not already, you'll want to glance at: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/uipc_syscalls.c? \ rev=1.144&content-type=text/x-cvsweb-markup (regarding your mapping caching in sf_bufs) and this [gigantic] thread: http://www.freebsd.org/cgi/getmsg.cgi?fetch=12432+15802+ \ /usr/local/www/db/text/2003/freebsd-arch/20030601.freebsd-arch (subject: sendfile(2) SF_NOPUSH flag proposal on freebsd-arch@, at least). You may want to contact Igor Sysoev or other concerned parties in that thread to show them that you actually have performance results resulting from such a change. Finally, I'd like to sort of make a longshot proposal; more of a "if you have the time" follow-up to your work that someone could be able to perform, and that would certainly be interesting to see: how all this works out when forward-ported to FreeBSD 5.x. > Sincerely > > - Yaoping > yruan@cs.princeton.edu Regards, -- Bosko Milekic * bmilekic@technokratis.com * bmilekic@FreeBSD.org TECHNOkRATIS Consulting Services * http://www.technokratis.com/