From owner-freebsd-current@FreeBSD.ORG Wed Apr 16 13:36:38 2008 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E1ECE1065672; Wed, 16 Apr 2008 13:36:38 +0000 (UTC) (envelope-from rrs@cisco.com) Received: from sj-iport-2.cisco.com (sj-iport-2.cisco.com [171.71.176.71]) by mx1.freebsd.org (Postfix) with ESMTP id AED9E8FC1B; Wed, 16 Apr 2008 13:36:38 +0000 (UTC) (envelope-from rrs@cisco.com) Received: from sj-dkim-3.cisco.com ([171.71.179.195]) by sj-iport-2.cisco.com with ESMTP; 16 Apr 2008 06:27:15 -0700 Received: from sj-core-5.cisco.com (sj-core-5.cisco.com [171.71.177.238]) by sj-dkim-3.cisco.com (8.12.11/8.12.11) with ESMTP id m3GDRFJi008164; Wed, 16 Apr 2008 06:27:15 -0700 Received: from xbh-sjc-231.amer.cisco.com (xbh-sjc-231.cisco.com [128.107.191.100]) by sj-core-5.cisco.com (8.13.8/8.13.8) with ESMTP id m3GDRFPQ029614; Wed, 16 Apr 2008 13:27:15 GMT Received: from xfe-sjc-212.amer.cisco.com ([171.70.151.187]) by xbh-sjc-231.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 16 Apr 2008 06:27:15 -0700 Received: from rrs.local ([171.68.225.134]) by xfe-sjc-212.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 16 Apr 2008 06:27:14 -0700 Message-ID: <4805FEB1.4000904@cisco.com> Date: Wed, 16 Apr 2008 09:27:13 -0400 From: Randall Stewart User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.12) Gecko/20080201 SeaMonkey/1.1.8 MIME-Version: 1.0 To: Jeff Roberson References: <48002444.4030505@elischer.org> <20080412191300.E7693@fledge.watson.org> <20080412181601.GA14472@freebsd.org> <20080415034343.GB87024@duncan.reilly.home> <20080414213656.Q959@desktop> In-Reply-To: <20080414213656.Q959@desktop> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 16 Apr 2008 13:27:14.0622 (UTC) FILETIME=[95890DE0:01C89FC5] DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; l=2429; t=1208352435; x=1209216435; c=relaxed/simple; s=sjdkim3002; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=rrs@cisco.com; z=From:=20Randall=20Stewart=20 |Subject:=20Re=3A=20stack=20hogs=20in=20kernel |Sender:=20; bh=kftyiS7cxSd6th9k/zViggbD0yuHiLek1vM43J4GIdI=; b=AoFTV49yLrYXQyGjxNGg3vdr3Pov2aYAHd7LotYC/mExrSrqi4ghWJaSSx E1f1S/LtzmTgsVpkSES72Wn/voEEshLYsImUHAgp6o/UGKfCbUmkpEnSO+IQ q5lW8eRa6k; Authentication-Results: sj-dkim-3; header.From=rrs@cisco.com; dkim=pass ( sig from cisco.com/sjdkim3002 verified; ); Cc: Andrew Reilly , Roman Divacky , Robert Watson , Julian Elischer , FreeBSD Current Subject: Re: stack hogs in kernel X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2008 13:36:39 -0000 Jeff Roberson wrote: > On Tue, 15 Apr 2008, Andrew Reilly wrote: > >> On Sat, Apr 12, 2008 at 08:16:01PM +0200, Roman Divacky wrote: >>> On Sat, Apr 12, 2008 at 07:14:21PM +0100, Robert Watson wrote: >>>> >>>> On Fri, 11 Apr 2008, Julian Elischer wrote: >>>> >>>>> 0xc05667e3 kldstat [kernel]: 2100 >>>>> 0xc07214f8 sendsig [kernel]: 1416 >>>>> 0xc04fb426 ugenread [kernel]: 1200 >>>>> 0xc070616b ipmi_smbios_identify [kernel]: 1136 >>>>> 0xc050bd26 usbd_new_device [kernel]: 1128 >>>>> 0xc0525a83 pfs_readlink [kernel]: 1092 >>>>> 0xc04fb407 ugenwrite [kernel]: 1056 >>>>> 0xc055ea33 prison_enforce_statfs [kernel]: 1044 >>>> >>>> This one, at least, is due to an issue Roman pointed out on hackers@ >>>> in the >>>> last 24 hours -- a MAXPATHLEN sized buffer on the stack. Looks like >>>> pfs_readlink() has the same issue. >>> >>> I plan to look at some of the MAXPATHLEN usage... I guess we can >>> shave a few >>> tens of KBs from the kernel (static size and runtime size). >> >> Why are single-digit kilobytes of memory space interesting, in this >> context? Is the concern about L1 data cache footprint, for performance >> reasons? If that is the case, the MAXPATHLEN bufffer will only really >> occupy the amount of cache actually touched. >> >> I've long wondered about the seemingly fanatical stack size concern in >> kernel space. In other domains (where I have more experience) you can >> get good performance benefits from the essentially free memory management >> and good cache re-use that comes from putting as much into the >> stack/call-frame as possible. > > There is a small fixed kernel stack per-thread. It has to be allocated > up-front out of kernel memory. There isn't really enough KVA to just > allow kernel stacks to grow unbounded. Also consider that most of the > time this memory is just unused. > > Right now on amd64 we allocate 4 pages for kernel stacks! This is just > huge. It makes allocation slower and more likely to fail since we have > to find 5 contiguous pages (one for a guard page). > Ahh, so we are different depending on the arch ... interesting.. I guess that makes sense.. probably in mips we should have more too :-) R -- Randall Stewart NSSTG - Cisco Systems Inc. 803-345-0369 803-317-4952 (cell)