From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 12 17:53:19 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5B2961065799 for ; Thu, 12 Feb 2009 17:53:19 +0000 (UTC) (envelope-from sam@freebsd.org) Received: from ebb.errno.com (ebb.errno.com [69.12.149.25]) by mx1.freebsd.org (Postfix) with ESMTP id 1DFA38FC2D for ; Thu, 12 Feb 2009 17:53:19 +0000 (UTC) (envelope-from sam@freebsd.org) Received: from trouble.errno.com (trouble.errno.com [10.0.0.248]) (authenticated bits=0) by ebb.errno.com (8.13.6/8.12.6) with ESMTP id n1CHrI67043230 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 12 Feb 2009 09:53:18 -0800 (PST) (envelope-from sam@freebsd.org) Message-ID: <4994620E.5000804@freebsd.org> Date: Thu, 12 Feb 2009 09:53:18 -0800 From: Sam Leffler Organization: FreeBSD Project User-Agent: Thunderbird 2.0.0.18 (X11/20081209) MIME-Version: 1.0 To: Max Laier References: <200902121717.47841.max@love2party.net> <4994516B.8060703@freebsd.org> <200902121754.24536.max@love2party.net> In-Reply-To: <200902121754.24536.max@love2party.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-DCC-sonic.net-Metrics: ebb.errno.com; whitelist Cc: freebsd-hackers@freebsd.org, Andrew Brampton Subject: Re: pahole - Finding holes in kernel structs X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Feb 2009 17:53:19 -0000 Max Laier wrote: > On Thursday 12 February 2009 17:42:19 Sam Leffler wrote: > >> Max Laier wrote: >> >>> On Thursday 12 February 2009 15:08:22 Andrew Brampton wrote: >>> >>>> So I ran the tool pahole over a 7.1 FreeBSD Kernel, and found that >>>> many of the struct had holes, and some of which could be rearranged to >>>> fill the gap. >>>> >>> Interesting tool ... >>> >> Someone should be able to do the same thing with coverity but it's >> obviously less effort to use something that exists. If I recall this >> and related tools like sparse use dwarf symbols which we don't generate >> by default. But with dtrace support I think we now can in fact generate >> the symbols easily so maybe someone can look at porting the tools... >> >> >>>> I've made the list available here[2]. So my questions >>>> are two fold: >>>> >>>> 1) Is it worth my time trying to rearrange structs? If so do you think >>>> many of my patches would be accepted? >>>> >>>> 2) Is there a way to find out the most heavily used structs? There are >>>> ~3600 structs, and ~2000 holes, it might be a waste of my time fixing >>>> the structs which are only used once. >>>> >>> That's the tricky part. Rearranging the structs itself is not that >>> difficult, but identifying which should be rearranged and if, how ... >>> that's the problem. The fact that gaps might be different for 64 vs. 32 >>> bit architectures has already been mentioned. >>> >>> In addition one needs to keep in mind that changing a struct layout is a >>> ABI change. So if we do identify structs that we want to change we >>> should do them all at once to keep the different versions down to a >>> minimum. >>> >>> So to answer your first question, submitting 101 patches to rearrange 101 >>> structs is certainly a wasted effort. However, if you take a good look >>> at the 2000 holes, identify an interesting subset and submit a patch to >>> fix that subset ... that would be a worthwhile effort ... IMHO. >>> >> The other thing to keep in mind is that structure layout can have a >> noticeable effect on cache locality. Arbitrarily rearranging structure >> members can generate many more cache misses so one should sanity check >> changes w/ something like hwpmc. However as noted because layout may be >> platform-dependent even if something shows no change on x86 it may be a >> loss on another architecture and finding that performance drop may be >> really hard. >> > > Let's not be too "glass half empty" about it, though. The same is true in the > opposite direction. If we can identify and eliminate an unnecessary hole in > an important structure we might gain that same performance just by reshuffling > a few lines. > > Certainly plugging holes can also be beneficial but just cautioning that changes of this sort need to be checked if made to critical data structures. OTOH there aren't that many that matter in practice. Sam