From owner-freebsd-current@FreeBSD.ORG Thu Jun 4 12:22:23 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E02C6106564A; Thu, 4 Jun 2009 12:22:22 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id AE03D8FC12; Thu, 4 Jun 2009 12:22:22 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 6185A46B06; Thu, 4 Jun 2009 08:22:22 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 424528A02E; Thu, 4 Jun 2009 08:22:21 -0400 (EDT) From: John Baldwin To: freebsd-current@freebsd.org Date: Thu, 4 Jun 2009 08:16:50 -0400 User-Agent: KMail/1.9.7 References: <20090602222445.2F6017302F@freebsd-current.sentex.ca> <15550775-6B8A-414E-A579-8B518D62E06E@mac.com> In-Reply-To: <15550775-6B8A-414E-A579-8B518D62E06E@mac.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200906040816.51244.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Thu, 04 Jun 2009 08:22:21 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=4.2 tests=AWL,BAYES_00,RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: kmacy@freebsd.org, marius@freebsd.org, Marcel Moolenaar , Robert Watson , FreeBSD Tinderbox , sparc64@freebsd.org, Eygene Ryabinkin , current@freebsd.org Subject: Re: [head tinderbox] failure on sparc64/sun4v X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jun 2009 12:22:23 -0000 On Wednesday 03 June 2009 7:14:38 pm Marcel Moolenaar wrote: > > On Jun 3, 2009, at 1:06 PM, Robert Watson wrote: > > > > Is there a reason not just to use __aligned(64) or the like on the > > first entry of the MD PCPU structure for sun4v to avoid future MI > > pcpu changes from causing similar discomfort for the MD pcpu parts? > > Also, do we know why these alignment/sizing requirements exist for > > struct pcpu on sun4v but not other platforms? If this is about > > packing pcpu structures into properly aligned cache lines, again > > __aligned() might be the right approach to take... > > Adding __aligned(xx) doesn't make it aligned. For example, > malloc(3) only aligns at 16-byte boundaries, so any > user-space structure that has __aligned(x>16) must manually > make sure that this is actually the case by over-allocating > and then adjusting the pointer to an x>16 aligned address. > Likewise for the kernel, though it's easier in the kernel > to get something that's page-aligned... > FYI, Yes, but struct pcpu is a bit special I think. The MD code is responsible for allocating it (and in at least some cases it just allocates a complete page). As long as sun4v allocates struct pcpu on a 64 byte boundary, simply throwing __aligned() in will fix it. Also, since the existing code is just computing explicit padding space, it is already assuming the alignment of struct pcpu. It is simply implementing __aligned() in a harder-to-maintain way. It also doesn't make any sense the way it is doing it now since it is simply adding padding to the end of the structure. Perhaps it is attempting to round up the size of the entire structure? If so, that can easily be fixed in the MD code that allocates the structures by doing 'roundup(sizeof(struct pcpu), N)' when allocating the structure. The comments in pcpu.h seem to imply that it wants a section of the fields in the middle to be aligned to a certain boundary in which case I think the proper solution is to stick an __aligned() on the first such member and then to allocate the structures on a suitable alignment when allocating PCPU structures in the MD code. Presumably it would need the special work when allocating these structures even with the current hard-to-maintain padding hack. Again, that hack is no better than __aligned(), just a much bigger pain to maintain AFAICT. -- John Baldwin