From owner-freebsd-current@FreeBSD.ORG Wed Apr 16 18:35:13 2008 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BB35E1065739 for ; Wed, 16 Apr 2008 18:35:13 +0000 (UTC) (envelope-from peter@wemm.org) Received: from wx-out-0506.google.com (wx-out-0506.google.com [66.249.82.229]) by mx1.freebsd.org (Postfix) with ESMTP id 79BEA8FC22 for ; Wed, 16 Apr 2008 18:35:13 +0000 (UTC) (envelope-from peter@wemm.org) Received: by wx-out-0506.google.com with SMTP id i29so2172199wxd.7 for ; Wed, 16 Apr 2008 11:35:12 -0700 (PDT) Received: by 10.100.91.17 with SMTP id o17mr472813anb.145.1208370910960; Wed, 16 Apr 2008 11:35:10 -0700 (PDT) Received: by 10.100.8.6 with HTTP; Wed, 16 Apr 2008 11:35:10 -0700 (PDT) Message-ID: Date: Wed, 16 Apr 2008 11:35:10 -0700 From: "Peter Wemm" To: "Jeff Roberson" In-Reply-To: <20080414213656.Q959@desktop> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <48002444.4030505@elischer.org> <20080412191300.E7693@fledge.watson.org> <20080412181601.GA14472@freebsd.org> <20080415034343.GB87024@duncan.reilly.home> <20080414213656.Q959@desktop> Cc: Andrew Reilly , Roman Divacky , Robert Watson , Julian Elischer , FreeBSD Current Subject: Re: stack hogs in kernel X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2008 18:35:13 -0000 On Tue, Apr 15, 2008 at 12:38 AM, Jeff Roberson wrote: > On Tue, 15 Apr 2008, Andrew Reilly wrote: > > On Sat, Apr 12, 2008 at 08:16:01PM +0200, Roman Divacky wrote: > > > On Sat, Apr 12, 2008 at 07:14:21PM +0100, Robert Watson wrote: > > > > On Fri, 11 Apr 2008, Julian Elischer wrote: > > > > > 0xc05667e3 kldstat [kernel]: 2100 > > > > > 0xc07214f8 sendsig [kernel]: 1416 > > > > > 0xc04fb426 ugenread [kernel]: 1200 > > > > > 0xc070616b ipmi_smbios_identify [kernel]: 1136 > > > > > 0xc050bd26 usbd_new_device [kernel]: 1128 > > > > > 0xc0525a83 pfs_readlink [kernel]: 1092 > > > > > 0xc04fb407 ugenwrite [kernel]: 1056 > > > > > 0xc055ea33 prison_enforce_statfs [kernel]: 1044 > > > > > > > > > > > > > This one, at least, is due to an issue Roman pointed out on hackers@ > in the > > > > last 24 hours -- a MAXPATHLEN sized buffer on the stack. Looks like > > > > pfs_readlink() has the same issue. > > > > > > > > > > I plan to look at some of the MAXPATHLEN usage... I guess we can shave a > few > > > tens of KBs from the kernel (static size and runtime size). > > > > > > > Why are single-digit kilobytes of memory space interesting, in this > > context? Is the concern about L1 data cache footprint, for performance > > reasons? If that is the case, the MAXPATHLEN bufffer will only really > > occupy the amount of cache actually touched. > > > > I've long wondered about the seemingly fanatical stack size concern in > > kernel space. In other domains (where I have more experience) you can > > get good performance benefits from the essentially free memory management > > and good cache re-use that comes from putting as much into the > > stack/call-frame as possible. > > > > There is a small fixed kernel stack per-thread. It has to be allocated > up-front out of kernel memory. There isn't really enough KVA to just allow > kernel stacks to grow unbounded. Also consider that most of the time this > memory is just unused. > > Right now on amd64 we allocate 4 pages for kernel stacks! This is just > huge. It makes allocation slower and more likely to fail since we have to > find 5 contiguous pages (one for a guard page). I'd like to see this reduced by one or two. We should be able to get away with the same stack size as i386 - the pcb size isn't that much different. At the very least, 3 pages + 1 guard page would get the size down to a power of two. I don't know if that'll help the kva allocator, but it might. I originally chose a KSTACK_PAGES of 4, simply out of conservatism - not by measurement. I just didn't want to have to worry about it at the time. It is fairly likely that shrinking it by a page will Just Work since i386 is running with KSTACK_PAGES = 2. Also, kernel stacks are allocated out of paged kva, so on amd64 this means they come out of the 2GB kernel area, not direct map. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell **WANTED TO BUY: Garmin Streetpilot 2650 or 2660. Not later model! **