From owner-freebsd-arch@FreeBSD.ORG Sun Nov 27 03:16:31 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6AA9D106566B; Sun, 27 Nov 2011 03:16:31 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail06.syd.optusnet.com.au (mail06.syd.optusnet.com.au [211.29.132.187]) by mx1.freebsd.org (Postfix) with ESMTP id E2BF48FC0C; Sun, 27 Nov 2011 03:16:30 +0000 (UTC) Received: from c211-28-227-231.carlnfd1.nsw.optusnet.com.au (c211-28-227-231.carlnfd1.nsw.optusnet.com.au [211.28.227.231]) by mail06.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id pAR3GKwE021619 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 27 Nov 2011 14:16:21 +1100 Date: Sun, 27 Nov 2011 14:16:20 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Robert Millan In-Reply-To: <20111126200749.GA72056@thorin> Message-ID: <20111127131422.D802@besplex.bde.org> References: <20111123070036.GA29952@thorin> <20111124141821.O932@besplex.bde.org> <20111125150324.G1018@besplex.bde.org> <20111126200749.GA72056@thorin> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Adrian Chadd , freebsd-current@freebsd.org, freebsd-arch@freebsd.org, Kostik Belousov Subject: Re: [PATCH] Detect GNU/kFreeBSD in user-visible kernel headers (v2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Nov 2011 03:16:31 -0000 On Sat, 26 Nov 2011, Robert Millan wrote: > On Fri, Nov 25, 2011 at 11:16:15AM -0700, Warner Losh wrote: >> Hey Bruce, >> >> These sound like good suggestions, but I'd hoped to actually go through all these files with a fine-toothed comb to see which ones were still relevant. You've found a bunch of good areas to clean up, but I'd like to humbly suggest they be done in a follow-on commit. > > Hi, > > I'm sending a new patch. Thanks Bruce for your input. TTBOMK this corrects > all the problems you spotted that were introduced by my patch. It doesn't > fix pre-existing problems in the files however, except in cases where I had > to modify that line anyway. > > I think it's a good compromise between my initial patch and an exhaustive > cleanup of those headers (which I'm probably not the most indicate for). It fixes most style bugs, but not some-pre-existing problems, even in cases where you had to modify the line anyway. % Index: sys/cam/scsi/scsi_low.h % =================================================================== % --- sys/cam/scsi/scsi_low.h (revision 227956) % +++ sys/cam/scsi/scsi_low.h (working copy) % @@ -53,10 +53,10 @@ % #define SCSI_LOW_INTERFACE_XS % #endif /* __NetBSD__ */ % % -#ifdef __FreeBSD__ % +#if defined(__FreeBSD__) || defined(__FreeBSD_kernel__) % #define SCSI_LOW_INTERFACE_CAM % #define CAM % -#endif /* __FreeBSD__ */ % +#endif /* __FreeBSD__ || __FreeBSD_kernel__ */ It still has the whitespace-after tab style change for cam. % Index: sys/dev/firewire/firewirereg.h % =================================================================== % --- sys/dev/firewire/firewirereg.h (revision 227956) % +++ sys/dev/firewire/firewirereg.h (working copy) % @@ -75,7 +75,8 @@ % }; % % struct firewire_softc { % -#if defined(__FreeBSD__) && __FreeBSD_version >= 500000 % +#if (defined(__FreeBSD__) || defined(__FreeBSD_kernel__)) && \ % + __FreeBSD_version >= 500000 % struct cdev *dev; % #endif % struct firewire_comm *fc; Here is a pre-existing problem that you didn't fix on a line that you changed. The __FreeBSD__ ifdef is nonsense here, since __FreeBSD__ being defined has nothing to do with either whether __FreeBSD_version is defined or whether there is a struct cdev * in the data structure. Previously: - defined(__FreeBSD__) means that the compiler is for FreeBSD - __FreeBSD_version >= 500000 means that FreeBSD has been included and has defined __FreeBSD_version to a value that satisifes this. It would be a bug for anything else to define __FreeBSD_version. Unfortunately, there is a bogus #undef of __FreeBSD_version that breaks detection of other things defining it. - the __FreeBSD__ part of the test has no effect except to break compiling this file with a non-gcc compiler. In particular, it doesn't prevent errors for -Wundef -Werror. But other ifdefs in this file use an unguarded __FreeBSD_version. Thus this file never worked with -Wundef -Werror, and the __FreeBSD__ part has no effect except the breakage. Now: as above, except: - defined(__FreeBSD_kernel__) means that FreeBSD been included and that this header is new enough to define __FreeBSD_kernel__. This has the same bug with the #undef, which I pointed out before (I noticed it for this but not for __FreeBSD_version). And it has a style bug in its name which I pointed out before -- 2 underscores in its name. __FreeBSD_version doesn't have this style bug. The definition of __FreeBSD_kernel__ has already been committed. Is it too late to fix its name? - when is new enough to define __FreeBSD_kernel__, it must be new enough to define __FreeBSD_version >= 500000. Thus there is now no -Wundef error. - the __FreeBSD__ ifdef remains nonsense. If you just removed it, then you wouldn't need the __FreeBSD_kernel__ ifdef (modulo the -Wundef error). You didn't add the __FreeBSD_kernel__ ifdef to any of the other lines with the __FreeBSD_kernel__ ifdef in this file, apparently because the others don't have the nonsensical __FreeBSD__ ifdef. The nonsense and changes to work around it make the logic for this ifdef even more convoluted and broken than might first appear. In a previous patchset, you included to ensure that __FreeBSD_kernel__ is defined for newer kernel sources (instead of testing if it is defined). Ifdefs like the above make a prerequsite for this file anyway, since without knowing __FreeBSD_version it is impossible to determine if the data structure has new fields like the cdev in it. is a prerequisite for almost all kernel .c files, so this prerequisite should be satisfied automatically for them, but it isn't clear what happens for user .c files. I think the ifdef should be something like the following to enforce the prerequisite: #ifndef _SYS_PARAM_H_ /* * Here I don't support __FreeBSD_version__ to be set outside of * to hack around a missing include of . * The case where the kernel is so old that __FreeBSD_version__ is * not defined should be handled by including to see * if __FreeBSD_version__ is in fact not defined. */ #error " is a prerequisite for this file" #endif #if __FreeBSD_version >= 500000 /* * Add defined(__FreeBSD_version) to the ifdef if you want to fully * support -Wundef. This is unlikely to have any effect here, since * has been included, and it defines __FreeBSD_version * except under very old versions of FreeBSD where -Wundef was even * more unusable than now. */ struct cdev *dev; #endif Similarly in most places that test __FreeBSD__ and __FreeBSD_version, or __FreeBSD__ and DEVICE_POLLING (the latter might need to ensure that opt_device_polling.h instead of is included, so in userland it reduces to an unconditional #error or hacks since opt_device_polling.h is a kernel-only non-module-only header). Bruce From owner-freebsd-arch@FreeBSD.ORG Sun Nov 27 11:35:29 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 663BE106564A; Sun, 27 Nov 2011 11:35:29 +0000 (UTC) (envelope-from rmh.aybabtu@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 115FF8FC0A; Sun, 27 Nov 2011 11:35:28 +0000 (UTC) Received: by iakl21 with SMTP id l21so10951011iak.13 for ; Sun, 27 Nov 2011 03:35:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=+yY3N2ymx2zaISbP+f6MFMz4q+AGw5FxJe5zDUeDasI=; b=WN0QiqTyj49MGE60XTqlxiM2xxnDU1U83NMjUpEUyvRziGLB1cDIJdvwdyexf1pawm 6BYoR0d4kMz7QTiegBKKkm2Q75mcVfaTbk9DGbq0qydjxEM5Xi4mew8Yym+jDEZ220JV 6jaIAJYTh3kzPnATd3JEZ1D01CFBtXeM/Ge+Q= MIME-Version: 1.0 Received: by 10.50.149.165 with SMTP id ub5mr45264785igb.23.1322393728440; Sun, 27 Nov 2011 03:35:28 -0800 (PST) Sender: rmh.aybabtu@gmail.com Received: by 10.42.222.200 with HTTP; Sun, 27 Nov 2011 03:35:28 -0800 (PST) In-Reply-To: <20111127131422.D802@besplex.bde.org> References: <20111123070036.GA29952@thorin> <20111124141821.O932@besplex.bde.org> <20111125150324.G1018@besplex.bde.org> <20111126200749.GA72056@thorin> <20111127131422.D802@besplex.bde.org> Date: Sun, 27 Nov 2011 12:35:28 +0100 X-Google-Sender-Auth: 9XIihjs8RyFaVJU_yXBf9-MEQiw Message-ID: From: Robert Millan To: Bruce Evans Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: Kostik Belousov , Adrian Chadd , freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Detect GNU/kFreeBSD in user-visible kernel headers (v2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Nov 2011 11:35:29 -0000 Hi Bruce, 2011/11/27 Bruce Evans : > % Index: sys/cam/scsi/scsi_low.h > % =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > % --- sys/cam/scsi/scsi_low.h =C2=A0 (revision 227956) > % +++ sys/cam/scsi/scsi_low.h =C2=A0 (working copy) > % @@ -53,10 +53,10 @@ > % =C2=A0#define =C2=A0 =C2=A0 =C2=A0SCSI_LOW_INTERFACE_XS > % =C2=A0#endif =C2=A0 =C2=A0 =C2=A0 /* __NetBSD__ */ > % % -#ifdef =C2=A0 =C2=A0 __FreeBSD__ > % +#if defined(__FreeBSD__) || defined(__FreeBSD_kernel__) > % =C2=A0#define =C2=A0 =C2=A0 =C2=A0SCSI_LOW_INTERFACE_CAM > % =C2=A0#define =C2=A0 =C2=A0 =C2=A0CAM > % -#endif =C2=A0 =C2=A0 =C2=A0 /* __FreeBSD__ */ > % +#endif /* __FreeBSD__ || __FreeBSD_kernel__ */ > > It still has the whitespace-after tab style change for cam. My initial patch didn't have it. Precisely I thought that you requested this in your first mail. Did I missunderstand? > % Index: sys/dev/firewire/firewirereg.h > % =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > % --- sys/dev/firewire/firewirereg.h =C2=A0 =C2=A0(revision 227956) > % +++ sys/dev/firewire/firewirereg.h =C2=A0 =C2=A0(working copy) > % @@ -75,7 +75,8 @@ > % =C2=A0}; > % % =C2=A0struct firewire_softc { > % -#if defined(__FreeBSD__) && __FreeBSD_version >=3D 500000 > % +#if (defined(__FreeBSD__) || defined(__FreeBSD_kernel__)) && \ > % + =C2=A0 =C2=A0__FreeBSD_version >=3D 500000 > % =C2=A0 =C2=A0 =C2=A0 struct cdev *dev; > % =C2=A0#endif > % =C2=A0 =C2=A0 =C2=A0 struct firewire_comm *fc; > > Here is a pre-existing problem that you didn't fix on a line that you > changed. Yes. I didn't say I would fix all pre-existing problems in lines that need to be modified. In some cases I did, and in some cases I opted to preserve the status quo. > __FreeBSD_version it is impossible to determine if the data structure > has new fields like the cdev in it. =C2=A0 is a prerequisite > for almost all kernel .c files, so this prerequisite should be satisfied > automatically for them, Earlier you asked not to include . If is a prerequisite, why not just include it then? From owner-freebsd-arch@FreeBSD.ORG Tue Nov 29 18:09:28 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1BC291065672 for ; Tue, 29 Nov 2011 18:09:28 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id D24588FC0A for ; Tue, 29 Nov 2011 18:09:27 +0000 (UTC) Received: by qyg36 with SMTP id 36so5991029qyg.13 for ; Tue, 29 Nov 2011 10:09:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=FiTRPUOcn8ewWLReftHws0Bo11GLcN02A9ZQybgjPak=; b=lybh6NL4bO/588yeHCr9z43iW4SUtcDXr3eyw2Pf4RwW+KRvoE7vXLLRrAQP9aFWth MyOuIz7TnSGcf0Wn+WkfqmLqlHCtth96eG9dTeV74NozWeUfl57qDmUqajoBtiquI3u2 ceSmyPGNwiShk1ht87c64WnG+l4N8OK9kGdxg= MIME-Version: 1.0 Received: by 10.68.30.193 with SMTP id u1mr64601338pbh.93.1322590166986; Tue, 29 Nov 2011 10:09:26 -0800 (PST) Sender: mdf356@gmail.com Received: by 10.68.56.97 with HTTP; Tue, 29 Nov 2011 10:09:26 -0800 (PST) Date: Tue, 29 Nov 2011 10:09:26 -0800 X-Google-Sender-Auth: 4fM7Zh9CJLuFbx2-1d58_jmDOUY Message-ID: From: mdf@FreeBSD.org To: FreeBSD Arch Content-Type: text/plain; charset=ISO-8859-1 Cc: Zack Kirsch Subject: Use of bool / stdbool.h in kernel X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Nov 2011 18:09:28 -0000 At $WORK we have a hack in one of the *.mk files to allow including stdbool.h in the kernel and we use it extensively. This is not allowed by style(9), as far as I can tell, because the file is in include/stdbool.h and those files are not allowed to be included in kernel sources. What I want to check on is, would it be acceptable to move stdbool.h from include/stdbool.h to sys/sys/stdbool.h (i.e. like errno.h) and then include it in the kernel as ? That is, is the objection / non-use because of where the file is located in the repository, or is there some other reason? Note that the pre-C99 boolean_t and TRUE/FALSE are spread over the kernel, mostly in sys/vm where I assume they come from old AT&T sources. Thanks, matthew From owner-freebsd-arch@FreeBSD.ORG Wed Nov 30 05:13:59 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C3C3E106566B; Wed, 30 Nov 2011 05:13:59 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail04.syd.optusnet.com.au (mail04.syd.optusnet.com.au [211.29.132.185]) by mx1.freebsd.org (Postfix) with ESMTP id 615168FC0C; Wed, 30 Nov 2011 05:13:58 +0000 (UTC) Received: from c211-28-227-231.carlnfd1.nsw.optusnet.com.au (c211-28-227-231.carlnfd1.nsw.optusnet.com.au [211.28.227.231]) by mail04.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id pAU5Drfa005532 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 30 Nov 2011 16:13:54 +1100 Date: Wed, 30 Nov 2011 16:13:53 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: mdf@freebsd.org In-Reply-To: Message-ID: <20111130154604.B949@besplex.bde.org> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Zack Kirsch , FreeBSD Arch Subject: Re: Use of bool / stdbool.h in kernel X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Nov 2011 05:13:59 -0000 On Tue, 29 Nov 2011 mdf@freebsd.org wrote: > At $WORK we have a hack in one of the *.mk files to allow including > stdbool.h in the kernel and we use it extensively. This is not > allowed by style(9), as far as I can tell, because the file is in > include/stdbool.h and those files are not allowed to be included in > kernel sources. Including stdbool.h in the kernel is not a style bug, but unsupported. > What I want to check on is, would it be acceptable to move stdbool.h > from include/stdbool.h to sys/sys/stdbool.h (i.e. like errno.h) and > then include it in the kernel as ? That is, is the Would be a larger style bug, especially if it were actually used. Even its spellings of TRUE and FALSE are strange. Even in userland stdbool.h is considered so useful that it is never used in src/bin and is only used a few times on other src/*bin. src/bin never uses TRUE of FALSE either. > objection / non-use because of where the file is located in the > repository, or is there some other reason? Note that the pre-C99 > boolean_t and TRUE/FALSE are spread over the kernel, mostly in sys/vm > where I assume they come from old AT&T sources. Thes came from mach vm. Pure BSD code never uses them. For example, in 4.4BSD-Lite2 /sys/kern, FALSE is used 3 times, all to call vm functions whose API uses it, while TRUE is used 3 times, once for a vm call and twice in shell scripts where it is an unquoted string. Pure BSD code also never uses booleans. It uses explicit comparisons with 0, which is equivalent to always spelling FALSE as 0 and both !FALSE and TRUE as either 1 or another literal more-magic number or "!= 0" (never !0 or !(expr) since those are booleans !!). In 1995, I made the mistake of moving the mach definitions of FALSE and TRUE and the declaration of boolean_t from a vm header to , because I wanted to use them more globally and I didn't know BSD style very well at the time. Especially that it never uses booleans. 4.4Lite2 could not have used C99's since it is older than C99, and it didn't use its own since it preferred explicit 0's and 1's. You may wish to change this style, but then all old BSD code would not conform to the new style. Bruce From owner-freebsd-arch@FreeBSD.ORG Wed Nov 30 05:40:59 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 52405106564A for ; Wed, 30 Nov 2011 05:40:59 +0000 (UTC) (envelope-from andreast-list@fgznet.ch) Received: from smtp.fgznet.ch (mail.fgznet.ch [81.92.96.47]) by mx1.freebsd.org (Postfix) with ESMTP id E380B8FC08 for ; Wed, 30 Nov 2011 05:40:58 +0000 (UTC) Received: from deuterium.andreas.nets (dhclient-91-190-14-19.flashcable.ch [91.190.14.19]) by smtp.fgznet.ch (8.13.8/8.13.8/Submit_SMTPAUTH) with ESMTP id pAU5Nap0017237 for ; Wed, 30 Nov 2011 06:23:38 +0100 (CET) (envelope-from andreast-list@fgznet.ch) Message-ID: <4ED5BE19.70805@fgznet.ch> Date: Wed, 30 Nov 2011 06:24:41 +0100 From: Andreas Tobler User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: FreeBSD Arch Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.64 on 81.92.96.47 Subject: powerpc64 malloc limit? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Nov 2011 05:40:59 -0000 All, while working on gcc I found a very strange situation which renders my powerpc64 machine unusable. The test case below tries to allocate that much memory as 'wanted'. The same test case on amd64 returns w/o trying to allocate mem because the size is far to big. I couldn't find the reason so far, that's why I'm here. As Nathan pointed out the VM_MAXUSER_SIZE is the biggest on powerpc64: #define VM_MAXUSER_ADDRESS (0x7ffffffffffff000UL) So, I'd expect a system to return an allocation error when a user tries to allocate too much memory and not really trying it and going to be unusable. Iow, I'd exepect the situation on powerpc64 as I see on amd64. Can anybody explain me the situation, why do I not have a working limit on powerpc64? The machine itself has 7GB RAM and 12GB swap. The amd64 where I compared has around 4GB/4GB RAM/swap. TIA, Andreas include #include int main() { void *p; p = (void*) malloc (1152921504606846968ULL); if (p != NULL) printf("p = %p\n", p); printf("p = %p\n", p); return (0); } From owner-freebsd-arch@FreeBSD.ORG Wed Nov 30 15:32:05 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7927C106564A; Wed, 30 Nov 2011 15:32:05 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 509C18FC17; Wed, 30 Nov 2011 15:32:05 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 05C6F46B09; Wed, 30 Nov 2011 10:32:05 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 8406FB945; Wed, 30 Nov 2011 10:32:04 -0500 (EST) From: John Baldwin To: freebsd-arch@freebsd.org Date: Wed, 30 Nov 2011 10:32:03 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) References: <20111130154604.B949@besplex.bde.org> In-Reply-To: <20111130154604.B949@besplex.bde.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201111301032.04102.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 30 Nov 2011 10:32:04 -0500 (EST) Cc: Zack Kirsch , mdf@freebsd.org Subject: Re: Use of bool / stdbool.h in kernel X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Nov 2011 15:32:05 -0000 On Wednesday, November 30, 2011 12:13:53 am Bruce Evans wrote: > On Tue, 29 Nov 2011 mdf@freebsd.org wrote: > > > At $WORK we have a hack in one of the *.mk files to allow including > > stdbool.h in the kernel and we use it extensively. This is not > > allowed by style(9), as far as I can tell, because the file is in > > include/stdbool.h and those files are not allowed to be included in > > kernel sources. > > Including stdbool.h in the kernel is not a style bug, but unsupported. > > > What I want to check on is, would it be acceptable to move stdbool.h > > from include/stdbool.h to sys/sys/stdbool.h (i.e. like errno.h) and > > then include it in the kernel as ? That is, is the > > Would be a larger style bug, especially if it were actually used. > Even its spellings of TRUE and FALSE are strange. Even in userland > stdbool.h is considered so useful that it is never used in src/bin > and is only used a few times on other src/*bin. src/bin never uses > TRUE of FALSE either. I suspect there is some bias here though due to the fact that there wasn't a standard bool type when most of this code was written. :) I don't think that means we have to forgo use of the new type now that it is in fact standardized in C99. I would be happy to have 'bool' available and the lowercase 'true' and 'false' are fine with me. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Nov 30 16:24:28 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 429F0106566B for ; Wed, 30 Nov 2011 16:24:28 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 3FA258FC08 for ; Wed, 30 Nov 2011 16:24:26 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pAUGMa98015948 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 30 Nov 2011 18:22:36 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pAUGMa8V079107; Wed, 30 Nov 2011 18:22:36 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pAUGMaAK079106; Wed, 30 Nov 2011 18:22:36 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 30 Nov 2011 18:22:36 +0200 From: Kostik Belousov To: Andreas Tobler Message-ID: <20111130162236.GA50300@deviant.kiev.zoral.com.ua> References: <4ED5BE19.70805@fgznet.ch> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="MaxviSYR+Gwa10R8" Content-Disposition: inline In-Reply-To: <4ED5BE19.70805@fgznet.ch> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: FreeBSD Arch Subject: Re: powerpc64 malloc limit? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Nov 2011 16:24:28 -0000 --MaxviSYR+Gwa10R8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Nov 30, 2011 at 06:24:41AM +0100, Andreas Tobler wrote: > All, >=20 > while working on gcc I found a very strange situation which renders my=20 > powerpc64 machine unusable. > The test case below tries to allocate that much memory as 'wanted'. The= =20 > same test case on amd64 returns w/o trying to allocate mem because the=20 > size is far to big. >=20 > I couldn't find the reason so far, that's why I'm here. >=20 > As Nathan pointed out the VM_MAXUSER_SIZE is the biggest on powerpc64: > #define VM_MAXUSER_ADDRESS (0x7ffffffffffff000UL) >=20 > So, I'd expect a system to return an allocation error when a user tries= =20 > to allocate too much memory and not really trying it and going to be=20 > unusable. Iow, I'd exepect the situation on powerpc64 as I see on amd64. >=20 > Can anybody explain me the situation, why do I not have a working limit= =20 > on powerpc64? >=20 > The machine itself has 7GB RAM and 12GB swap. The amd64 where I compared= =20 > has around 4GB/4GB RAM/swap. >=20 > TIA, > Andreas >=20 > include > #include >=20 > int main() > { > void *p; >=20 > p =3D (void*) malloc (1152921504606846968ULL); > if (p !=3D NULL) > printf("p =3D %p\n", p); >=20 > printf("p =3D %p\n", p); > return (0); > } First, you should provide details of what consistutes 'the unusable machine situation' on powerpc. That said, on amd64 the user map is between 0 and 0x7fffffffffff, which obviously less then the requested allocation size 0x100000000000000. If you look at the kdump output on amd64, you will see that malloc() tries to mmap() the area, fails and retries with obreak(). Default virtual memory limit is unlimited, so my best quess is that on amd64 vm_map_findspace() returns immediately. On powerpc64, I see no reason why vm_map_entry cannot be allocated, but please note that vm object and pages shall be only allocated on demand. So I am curious how does your machine breaks and where. --MaxviSYR+Gwa10R8 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk7WWEwACgkQC3+MBN1Mb4iCKwCfYPE7yETlYQz1KTZDJrcHQFYU eAkAnRLJQco/VXNjHPN3QMIYl1XyZo+C =Jwut -----END PGP SIGNATURE----- --MaxviSYR+Gwa10R8-- From owner-freebsd-arch@FreeBSD.ORG Wed Nov 30 16:53:09 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 27315106564A for ; Wed, 30 Nov 2011 16:53:09 +0000 (UTC) (envelope-from andreast-list@fgznet.ch) Received: from smtp.fgznet.ch (mail.fgznet.ch [81.92.96.47]) by mx1.freebsd.org (Postfix) with ESMTP id 88C938FC1F for ; Wed, 30 Nov 2011 16:53:08 +0000 (UTC) Received: from deuterium.andreas.nets (dhclient-91-190-14-19.flashcable.ch [91.190.14.19]) by smtp.fgznet.ch (8.13.8/8.13.8/Submit_SMTPAUTH) with ESMTP id pAUGpucZ040554; Wed, 30 Nov 2011 17:51:58 +0100 (CET) (envelope-from andreast-list@fgznet.ch) Message-ID: <4ED65F70.7050700@fgznet.ch> Date: Wed, 30 Nov 2011 17:53:04 +0100 From: Andreas Tobler User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: Kostik Belousov References: <4ED5BE19.70805@fgznet.ch> <20111130162236.GA50300@deviant.kiev.zoral.com.ua> In-Reply-To: <20111130162236.GA50300@deviant.kiev.zoral.com.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.64 on 81.92.96.47 Cc: FreeBSD Arch Subject: Re: powerpc64 malloc limit? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Nov 2011 16:53:09 -0000 On 30.11.11 17:22, Kostik Belousov wrote: > On Wed, Nov 30, 2011 at 06:24:41AM +0100, Andreas Tobler wrote: >> All, >> >> while working on gcc I found a very strange situation which renders my >> powerpc64 machine unusable. >> The test case below tries to allocate that much memory as 'wanted'. The >> same test case on amd64 returns w/o trying to allocate mem because the >> size is far to big. >> >> I couldn't find the reason so far, that's why I'm here. >> >> As Nathan pointed out the VM_MAXUSER_SIZE is the biggest on powerpc64: >> #define VM_MAXUSER_ADDRESS (0x7ffffffffffff000UL) >> >> So, I'd expect a system to return an allocation error when a user tries >> to allocate too much memory and not really trying it and going to be >> unusable. Iow, I'd exepect the situation on powerpc64 as I see on amd64. >> >> Can anybody explain me the situation, why do I not have a working limit >> on powerpc64? >> >> The machine itself has 7GB RAM and 12GB swap. The amd64 where I compared >> has around 4GB/4GB RAM/swap. >> >> TIA, >> Andreas >> >> include >> #include >> >> int main() >> { >> void *p; >> >> p = (void*) malloc (1152921504606846968ULL); >> if (p != NULL) >> printf("p = %p\n", p); >> >> printf("p = %p\n", p); >> return (0); >> } > > First, you should provide details of what consistutes 'the unusable > machine situation' on powerpc. I can not login anymore, everything is stuck except the core control mechanisms for example the fan controller. Top reports 'ugly' figures, below from a earlier try: last pid: 6790; load averages: 0.78, 0.84, 0.86 up 0+00:34:52 22:42:29 47 processes: 1 running, 46 sleeping CPU: 0.0% user, 0.0% nice, 15.4% system, 11.8% interrupt, 72.8% idle Mem: 5912M Active, 570M Inact, 280M Wired, 26M Cache, 104M Buf, 352K Free Swap: 12G Total, 9904M Used, 2383M Free, 80% Inuse, 178M Out PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 6768 andreast 1 52 01073741824G 6479M pfault 1 0:58 18.90% 31370. And after my mem and swap are full I see swap_pager_getswapspace(16) failed. In this state I can only power-cycle the machine. > That said, on amd64 the user map is between 0 and 0x7fffffffffff, which > obviously less then the requested allocation size 0x100000000000000. > If you look at the kdump output on amd64, you will see that malloc() > tries to mmap() the area, fails and retries with obreak(). Default > virtual memory limit is unlimited, so my best quess is that on amd64 > vm_map_findspace() returns immediately. > > On powerpc64, I see no reason why vm_map_entry cannot be allocated, but > please note that vm object and pages shall be only allocated on demand. > So I am curious how does your machine breaks and where. I would expect that the 'system' does not allow me to allocate that much of ram. Thanks! Andreas From owner-freebsd-arch@FreeBSD.ORG Wed Nov 30 17:10:03 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2D366106566B for ; Wed, 30 Nov 2011 17:10:03 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id B36728FC17 for ; Wed, 30 Nov 2011 17:10:02 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pAUH9bDa023632 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 30 Nov 2011 19:09:37 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pAUH9b5t079279; Wed, 30 Nov 2011 19:09:37 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pAUH9acd079278; Wed, 30 Nov 2011 19:09:36 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 30 Nov 2011 19:09:36 +0200 From: Kostik Belousov To: Andreas Tobler Message-ID: <20111130170936.GB50300@deviant.kiev.zoral.com.ua> References: <4ED5BE19.70805@fgznet.ch> <20111130162236.GA50300@deviant.kiev.zoral.com.ua> <4ED65F70.7050700@fgznet.ch> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="6VIkwkuKQ8ml0fyB" Content-Disposition: inline In-Reply-To: <4ED65F70.7050700@fgznet.ch> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: FreeBSD Arch Subject: Re: powerpc64 malloc limit? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Nov 2011 17:10:03 -0000 --6VIkwkuKQ8ml0fyB Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Nov 30, 2011 at 05:53:04PM +0100, Andreas Tobler wrote: > On 30.11.11 17:22, Kostik Belousov wrote: > >On Wed, Nov 30, 2011 at 06:24:41AM +0100, Andreas Tobler wrote: > >>All, > >> > >>while working on gcc I found a very strange situation which renders my > >>powerpc64 machine unusable. > >>The test case below tries to allocate that much memory as 'wanted'. The > >>same test case on amd64 returns w/o trying to allocate mem because the > >>size is far to big. > >> > >>I couldn't find the reason so far, that's why I'm here. > >> > >>As Nathan pointed out the VM_MAXUSER_SIZE is the biggest on powerpc64: > >>#define VM_MAXUSER_ADDRESS (0x7ffffffffffff000UL) > >> > >>So, I'd expect a system to return an allocation error when a user tries > >>to allocate too much memory and not really trying it and going to be > >>unusable. Iow, I'd exepect the situation on powerpc64 as I see on amd64. > >> > >>Can anybody explain me the situation, why do I not have a working limit > >>on powerpc64? > >> > >>The machine itself has 7GB RAM and 12GB swap. The amd64 where I compared > >>has around 4GB/4GB RAM/swap. > >> > >>TIA, > >>Andreas > >> > >>include > >>#include > >> > >>int main() > >>{ > >> void *p; > >> > >> p =3D (void*) malloc (1152921504606846968ULL); > >> if (p !=3D NULL) > >> printf("p =3D %p\n", p); > >> > >> printf("p =3D %p\n", p); > >> return (0); > >>} > > > >First, you should provide details of what consistutes 'the unusable > >machine situation' on powerpc. >=20 > I can not login anymore, everything is stuck except the core control=20 > mechanisms for example the fan controller. >=20 > Top reports 'ugly' figures, below from a earlier try: >=20 > last pid: 6790; load averages: 0.78, 0.84, 0.86 up 0+00:34:52 > 22:42:29 47 processes: 1 running, 46 sleeping > CPU: 0.0% user, 0.0% nice, 15.4% system, 11.8% interrupt, 72.8% idle > Mem: 5912M Active, 570M Inact, 280M Wired, 26M Cache, 104M Buf, 352K Free > Swap: 12G Total, 9904M Used, 2383M Free, 80% Inuse, 178M Out >=20 > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU > COMMAND > 6768 andreast 1 52 01073741824G 6479M pfault 1 0:58 > 18.90% 31370. >=20 > And after my mem and swap are full I see swap_pager_getswapspace(16) fail= ed. >=20 > In this state I can only power-cycle the machine. >=20 > >That said, on amd64 the user map is between 0 and 0x7fffffffffff, which > >obviously less then the requested allocation size 0x100000000000000. > >If you look at the kdump output on amd64, you will see that malloc() > >tries to mmap() the area, fails and retries with obreak(). Default > >virtual memory limit is unlimited, so my best quess is that on amd64 > >vm_map_findspace() returns immediately. > > > >On powerpc64, I see no reason why vm_map_entry cannot be allocated, but > >please note that vm object and pages shall be only allocated on demand. > >So I am curious how does your machine breaks and where. >=20 > I would expect that the 'system' does not allow me to allocate that much= =20 > of ram. Does the issue with machine going into limbo reproducable with the code you posted ? Or, do you need to actually touch the pages in the allocated region ? If the later (and I do expect that), then how many pages do you need to touch before machine breaks ? Is it single access that causes the havoc, or you need to touch the amount approximately equal to RAM+swap ? --6VIkwkuKQ8ml0fyB Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk7WY1AACgkQC3+MBN1Mb4j1awCg4DC4/lm/16bbiU0luriAlULM GjkAn3HkCD8WtD35Kys03IPHFtXlY6If =xbz2 -----END PGP SIGNATURE----- --6VIkwkuKQ8ml0fyB-- From owner-freebsd-arch@FreeBSD.ORG Wed Nov 30 17:44:26 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7657E1065672 for ; Wed, 30 Nov 2011 17:44:26 +0000 (UTC) (envelope-from andreast-list@fgznet.ch) Received: from smtp.fgznet.ch (mail.fgznet.ch [81.92.96.47]) by mx1.freebsd.org (Postfix) with ESMTP id 037838FC0A for ; Wed, 30 Nov 2011 17:44:25 +0000 (UTC) Received: from deuterium.andreas.nets (dhclient-91-190-14-19.flashcable.ch [91.190.14.19]) by smtp.fgznet.ch (8.13.8/8.13.8/Submit_SMTPAUTH) with ESMTP id pAUHhEr4024610; Wed, 30 Nov 2011 18:43:15 +0100 (CET) (envelope-from andreast-list@fgznet.ch) Message-ID: <4ED66B75.3060409@fgznet.ch> Date: Wed, 30 Nov 2011 18:44:21 +0100 From: Andreas Tobler User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: Kostik Belousov References: <4ED5BE19.70805@fgznet.ch> <20111130162236.GA50300@deviant.kiev.zoral.com.ua> <4ED65F70.7050700@fgznet.ch> <20111130170936.GB50300@deviant.kiev.zoral.com.ua> In-Reply-To: <20111130170936.GB50300@deviant.kiev.zoral.com.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.64 on 81.92.96.47 Cc: FreeBSD Arch Subject: Re: powerpc64 malloc limit? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Nov 2011 17:44:26 -0000 On 30.11.11 18:09, Kostik Belousov wrote: > On Wed, Nov 30, 2011 at 05:53:04PM +0100, Andreas Tobler wrote: >> On 30.11.11 17:22, Kostik Belousov wrote: >>> On Wed, Nov 30, 2011 at 06:24:41AM +0100, Andreas Tobler wrote: >>>> All, >>>> >>>> while working on gcc I found a very strange situation which renders my >>>> powerpc64 machine unusable. >>>> The test case below tries to allocate that much memory as 'wanted'. The >>>> same test case on amd64 returns w/o trying to allocate mem because the >>>> size is far to big. >>>> >>>> I couldn't find the reason so far, that's why I'm here. >>>> >>>> As Nathan pointed out the VM_MAXUSER_SIZE is the biggest on powerpc64: >>>> #define VM_MAXUSER_ADDRESS (0x7ffffffffffff000UL) >>>> >>>> So, I'd expect a system to return an allocation error when a user tries >>>> to allocate too much memory and not really trying it and going to be >>>> unusable. Iow, I'd exepect the situation on powerpc64 as I see on amd64. >>>> >>>> Can anybody explain me the situation, why do I not have a working limit >>>> on powerpc64? >>>> >>>> The machine itself has 7GB RAM and 12GB swap. The amd64 where I compared >>>> has around 4GB/4GB RAM/swap. >>>> >>>> TIA, >>>> Andreas >>>> >>>> include >>>> #include >>>> >>>> int main() >>>> { >>>> void *p; >>>> >>>> p = (void*) malloc (1152921504606846968ULL); >>>> if (p != NULL) >>>> printf("p = %p\n", p); >>>> >>>> printf("p = %p\n", p); >>>> return (0); >>>> } >>> >>> First, you should provide details of what consistutes 'the unusable >>> machine situation' on powerpc. >> >> I can not login anymore, everything is stuck except the core control >> mechanisms for example the fan controller. >> >> Top reports 'ugly' figures, below from a earlier try: >> >> last pid: 6790; load averages: 0.78, 0.84, 0.86 up 0+00:34:52 >> 22:42:29 47 processes: 1 running, 46 sleeping >> CPU: 0.0% user, 0.0% nice, 15.4% system, 11.8% interrupt, 72.8% idle >> Mem: 5912M Active, 570M Inact, 280M Wired, 26M Cache, 104M Buf, 352K Free >> Swap: 12G Total, 9904M Used, 2383M Free, 80% Inuse, 178M Out >> >> PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU >> COMMAND >> 6768 andreast 1 52 01073741824G 6479M pfault 1 0:58 >> 18.90% 31370. >> >> And after my mem and swap are full I see swap_pager_getswapspace(16) failed. >> >> In this state I can only power-cycle the machine. >> >>> That said, on amd64 the user map is between 0 and 0x7fffffffffff, which >>> obviously less then the requested allocation size 0x100000000000000. >>> If you look at the kdump output on amd64, you will see that malloc() >>> tries to mmap() the area, fails and retries with obreak(). Default >>> virtual memory limit is unlimited, so my best quess is that on amd64 >>> vm_map_findspace() returns immediately. >>> >>> On powerpc64, I see no reason why vm_map_entry cannot be allocated, but >>> please note that vm object and pages shall be only allocated on demand. >>> So I am curious how does your machine breaks and where. >> >> I would expect that the 'system' does not allow me to allocate that much >> of ram. > > Does the issue with machine going into limbo reproducable with the code > you posted ? If I understand you correctly, yes. I can launch the test case and the machine is immediately unusable. Means I can not kill the process nor can I log in. Also, top does not show anything useful. The original test case where I discovered this behavior behaves a bit different. http://gcc.gnu.org/viewcvs/trunk/libstdc%2B%2B-v3/testsuite/23_containers/vector/bool/modifiers/insert/31370.cc?revision=169421&view=markup Here I can follow how the ram and swap is eaten up. Top is reporting the figures. If everything is 'full', the swaper errors start to appear on the console. > Or, do you need to actually touch the pages in the allocated region ? If I have to, how would I do that? > If the later (and I do expect that), then how many pages do you need > to touch before machine breaks ? Is it single access that causes the > havoc, or you need to touch the amount approximately equal to RAM+swap ? Andreas From owner-freebsd-arch@FreeBSD.ORG Wed Nov 30 18:12:21 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5A6FB106564A for ; Wed, 30 Nov 2011 18:12:21 +0000 (UTC) (envelope-from peter@wemm.org) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 187038FC1C for ; Wed, 30 Nov 2011 18:12:20 +0000 (UTC) Received: by ywp17 with SMTP id 17so1441158ywp.13 for ; Wed, 30 Nov 2011 10:12:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wemm.org; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=ynOo6jJ4fRbqpa1HuLvos0C7Yy/B0d1o6oH7TJautMI=; b=j1CMYzmKzmBkqj5H1Z0SLUtnU513gtyx5kUMb0dqcR+FTKGvQxnfIbOeTJSUSIqUyF 2MXWk3+NYFk8bX+qfVq2Zd0z5HfLCcrEkVCCTDypZtFuWd5Oehvyj5tv7MorFP1AU4Qa tUrRxF4gFrXmotKkkCuUR7DJU8RGLNdAUAOhM= MIME-Version: 1.0 Received: by 10.50.213.6 with SMTP id no6mr4042563igc.51.1322676740094; Wed, 30 Nov 2011 10:12:20 -0800 (PST) Received: by 10.50.135.106 with HTTP; Wed, 30 Nov 2011 10:12:20 -0800 (PST) In-Reply-To: <4ED66B75.3060409@fgznet.ch> References: <4ED5BE19.70805@fgznet.ch> <20111130162236.GA50300@deviant.kiev.zoral.com.ua> <4ED65F70.7050700@fgznet.ch> <20111130170936.GB50300@deviant.kiev.zoral.com.ua> <4ED66B75.3060409@fgznet.ch> Date: Wed, 30 Nov 2011 10:12:20 -0800 Message-ID: From: Peter Wemm To: Andreas Tobler Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Kostik Belousov , FreeBSD Arch Subject: Re: powerpc64 malloc limit? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Nov 2011 18:12:21 -0000 On Wed, Nov 30, 2011 at 9:44 AM, Andreas Tobler w= rote: > On 30.11.11 18:09, Kostik Belousov wrote: >> >> On Wed, Nov 30, 2011 at 05:53:04PM +0100, Andreas Tobler wrote: >>> >>> On 30.11.11 17:22, Kostik Belousov wrote: >>>> >>>> On Wed, Nov 30, 2011 at 06:24:41AM +0100, Andreas Tobler wrote: >>>>> >>>>> All, >>>>> >>>>> while working on gcc I found a very strange situation which renders m= y >>>>> powerpc64 machine unusable. >>>>> The test case below tries to allocate that much memory as 'wanted'. T= he >>>>> same test case on amd64 returns w/o trying to allocate mem because th= e >>>>> size is far to big. >>>>> >>>>> I couldn't find the reason so far, that's why I'm here. >>>>> >>>>> As Nathan pointed out the VM_MAXUSER_SIZE is the biggest on powerpc64= : >>>>> #define VM_MAXUSER_ADDRESS =A0 =A0 =A0(0x7ffffffffffff000UL) >>>>> >>>>> So, I'd expect a system to return an allocation error when a user tri= es >>>>> to allocate too much memory and not really trying it and going to be >>>>> unusable. Iow, I'd exepect the situation on powerpc64 as I see on >>>>> amd64. >>>>> >>>>> Can anybody explain me the situation, why do I not have a working lim= it >>>>> on powerpc64? >>>>> >>>>> The machine itself has 7GB RAM and 12GB swap. The amd64 where I >>>>> compared >>>>> has around 4GB/4GB RAM/swap. >>>>> >>>>> TIA, >>>>> Andreas >>>>> >>>>> include >>>>> #include >>>>> >>>>> int main() >>>>> { >>>>> =A0 =A0 =A0 =A0 =A0void *p; >>>>> >>>>> =A0 =A0 =A0 =A0 =A0p =3D (void*) malloc (1152921504606846968ULL); >>>>> =A0 =A0 =A0 =A0 =A0if (p !=3D NULL) >>>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0printf("p =3D %p\n", p); >>>>> >>>>> =A0 =A0 =A0 =A0 =A0printf("p =3D %p\n", p); >>>>> =A0 =A0 =A0 =A0 =A0return (0); >>>>> } >>>> >>>> First, you should provide details of what consistutes 'the unusable >>>> machine situation' on powerpc. >>> >>> I can not login anymore, everything is stuck except the core control >>> mechanisms for example the fan controller. >>> >>> Top reports 'ugly' figures, below from a earlier try: >>> >>> last pid: =A06790; =A0load averages: =A00.78, =A00.84, =A00.86 =A0 =A0u= p 0+00:34:52 >>> 22:42:29 47 processes: =A01 running, 46 sleeping >>> CPU: =A00.0% user, =A00.0% nice, 15.4% system, 11.8% interrupt, 72.8% i= dle >>> Mem: 5912M Active, 570M Inact, 280M Wired, 26M Cache, 104M Buf, 352K Fr= ee >>> Swap: 12G Total, 9904M Used, 2383M Free, 80% Inuse, 178M Out >>> >>> =A0 =A0PID USERNAME =A0 =A0THR PRI NICE =A0 SIZE =A0 =A0RES STATE =A0 C= =A0 TIME =A0 WCPU >>> COMMAND >>> =A0 6768 andreast =A0 =A0 =A01 =A052 =A0 =A001073741824G =A06479M pfaul= t =A01 =A0 0:58 >>> 18.90% 31370. >>> >>> And after my mem and swap are full I see swap_pager_getswapspace(16) >>> failed. >>> >>> In this state I can only power-cycle the machine. >>> >>>> That said, on amd64 the user map is between 0 and 0x7fffffffffff, whic= h >>>> obviously less then the requested allocation size 0x100000000000000. >>>> If you look at the kdump output on amd64, you will see that malloc() >>>> tries to mmap() the area, fails and retries with obreak(). Default >>>> virtual memory limit is unlimited, so my best quess is that on amd64 >>>> vm_map_findspace() returns immediately. >>>> >>>> On powerpc64, I see no reason why vm_map_entry cannot be allocated, bu= t >>>> please note that vm object and pages shall be only allocated on demand= . >>>> So I am curious how does your machine breaks and where. >>> >>> I would expect that the 'system' does not allow me to allocate that muc= h >>> of ram. >> >> Does the issue with machine going into limbo reproducable with the code >> you posted ? > > If I understand you correctly, yes. I can launch the test case and the > machine is immediately unusable. Means I can not kill the process nor can= I > log in. Also, top does not show anything useful. > > The original test case where I discovered this behavior behaves a bit > different. > http://gcc.gnu.org/viewcvs/trunk/libstdc%2B%2B-v3/testsuite/23_containers= /vector/bool/modifiers/insert/31370.cc?revision=3D169421&view=3Dmarkup > > Here I can follow how the ram and swap is eaten up. Top is reporting the > figures. If everything is 'full', the swaper errors start to appear on th= e > console. > >> Or, do you need to actually touch the pages in the allocated region ? > > If I have to, how would I do that? > >> If the later (and I do expect that), then how many pages do you need >> to touch before machine breaks ? Is it single access that causes the >> havoc, or you need to touch the amount approximately equal to RAM+swap ? > > Andreas ia64 had some vaguely related excitement earlier in its life. If you created a 1TB sparse file and mmap'ed it over and over, tens, maybe hundreds of thousands of times, certain VM internal state got way out of hand. mmaping was fine, but unmapping took 36 hours of cpu runtime when I killed the process. It got so far out of hand because of the way ia64 handled just-in-time mappings on vhpt misses. It sounds a lot to me like ppc64 vm system consumes system resources to map something, which is unbounded, and those large address space operations are causing most/all of system ram to be wired for VM management and leading to a lockup. AMD64's user address space isn't actually all that big compared to other 64 bit systems. --=20 Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell From owner-freebsd-arch@FreeBSD.ORG Wed Nov 30 20:01:22 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E8A21106564A for ; Wed, 30 Nov 2011 20:01:22 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 8438B8FC0A for ; Wed, 30 Nov 2011 20:01:22 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pAUK14GB050607 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 30 Nov 2011 22:01:04 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pAUK139b079858; Wed, 30 Nov 2011 22:01:03 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pAUK131t079857; Wed, 30 Nov 2011 22:01:03 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 30 Nov 2011 22:01:03 +0200 From: Kostik Belousov To: Andreas Tobler Message-ID: <20111130200103.GE50300@deviant.kiev.zoral.com.ua> References: <4ED5BE19.70805@fgznet.ch> <20111130162236.GA50300@deviant.kiev.zoral.com.ua> <4ED65F70.7050700@fgznet.ch> <20111130170936.GB50300@deviant.kiev.zoral.com.ua> <4ED66B75.3060409@fgznet.ch> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="BJNipRl4fWPaRlLb" Content-Disposition: inline In-Reply-To: <4ED66B75.3060409@fgznet.ch> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: FreeBSD Arch Subject: Re: powerpc64 malloc limit? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Nov 2011 20:01:23 -0000 --BJNipRl4fWPaRlLb Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Nov 30, 2011 at 06:44:21PM +0100, Andreas Tobler wrote: > On 30.11.11 18:09, Kostik Belousov wrote: > >On Wed, Nov 30, 2011 at 05:53:04PM +0100, Andreas Tobler wrote: > >>On 30.11.11 17:22, Kostik Belousov wrote: > >>>On Wed, Nov 30, 2011 at 06:24:41AM +0100, Andreas Tobler wrote: > >>>>All, > >>>> > >>>>while working on gcc I found a very strange situation which renders my > >>>>powerpc64 machine unusable. > >>>>The test case below tries to allocate that much memory as 'wanted'. T= he > >>>>same test case on amd64 returns w/o trying to allocate mem because the > >>>>size is far to big. > >>>> > >>>>I couldn't find the reason so far, that's why I'm here. > >>>> > >>>>As Nathan pointed out the VM_MAXUSER_SIZE is the biggest on powerpc64: > >>>>#define VM_MAXUSER_ADDRESS (0x7ffffffffffff000UL) > >>>> > >>>>So, I'd expect a system to return an allocation error when a user tri= es > >>>>to allocate too much memory and not really trying it and going to be > >>>>unusable. Iow, I'd exepect the situation on powerpc64 as I see on amd= 64. > >>>> > >>>>Can anybody explain me the situation, why do I not have a working lim= it > >>>>on powerpc64? > >>>> > >>>>The machine itself has 7GB RAM and 12GB swap. The amd64 where I compa= red > >>>>has around 4GB/4GB RAM/swap. > >>>> > >>>>TIA, > >>>>Andreas > >>>> > >>>>include > >>>>#include > >>>> > >>>>int main() > >>>>{ > >>>> void *p; > >>>> > >>>> p =3D (void*) malloc (1152921504606846968ULL); > >>>> if (p !=3D NULL) > >>>> printf("p =3D %p\n", p); > >>>> > >>>> printf("p =3D %p\n", p); > >>>> return (0); > >>>>} > >>> > >>>First, you should provide details of what consistutes 'the unusable > >>>machine situation' on powerpc. > >> > >>I can not login anymore, everything is stuck except the core control > >>mechanisms for example the fan controller. > >> > >>Top reports 'ugly' figures, below from a earlier try: > >> > >>last pid: 6790; load averages: 0.78, 0.84, 0.86 up 0+00:34:52 > >>22:42:29 47 processes: 1 running, 46 sleeping > >>CPU: 0.0% user, 0.0% nice, 15.4% system, 11.8% interrupt, 72.8% idle > >>Mem: 5912M Active, 570M Inact, 280M Wired, 26M Cache, 104M Buf, 352K Fr= ee > >>Swap: 12G Total, 9904M Used, 2383M Free, 80% Inuse, 178M Out > >> > >> PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU > >>COMMAND > >> 6768 andreast 1 52 01073741824G 6479M pfault 1 0:58 > >>18.90% 31370. > >> > >>And after my mem and swap are full I see swap_pager_getswapspace(16)=20 > >>failed. > >> > >>In this state I can only power-cycle the machine. > >> > >>>That said, on amd64 the user map is between 0 and 0x7fffffffffff, which > >>>obviously less then the requested allocation size 0x100000000000000. > >>>If you look at the kdump output on amd64, you will see that malloc() > >>>tries to mmap() the area, fails and retries with obreak(). Default > >>>virtual memory limit is unlimited, so my best quess is that on amd64 > >>>vm_map_findspace() returns immediately. > >>> > >>>On powerpc64, I see no reason why vm_map_entry cannot be allocated, but > >>>please note that vm object and pages shall be only allocated on demand. > >>>So I am curious how does your machine breaks and where. > >> > >>I would expect that the 'system' does not allow me to allocate that much > >>of ram. > > > >Does the issue with machine going into limbo reproducable with the code > >you posted ? >=20 > If I understand you correctly, yes. I can launch the test case and the=20 > machine is immediately unusable. Means I can not kill the process nor=20 > can I log in. Also, top does not show anything useful. Again, let me restate my question: the single mmap() of the huge size is enough for powerpc64 machine to break apart ? What happen if you insert sleep(1000000); call before return ? Do not kill the process, I want to know is machine dead while the process sleeps. >=20 > The original test case where I discovered this behavior behaves a bit=20 > different. > http://gcc.gnu.org/viewcvs/trunk/libstdc%2B%2B-v3/testsuite/23_containers= /vector/bool/modifiers/insert/31370.cc?revision=3D169421&view=3Dmarkup >=20 > Here I can follow how the ram and swap is eaten up. Top is reporting the= =20 > figures. If everything is 'full', the swaper errors start to appear on=20 > the console. >=20 > >Or, do you need to actually touch the pages in the allocated region ? >=20 > If I have to, how would I do that? You read or write into each page in the region. >=20 > >If the later (and I do expect that), then how many pages do you need > >to touch before machine breaks ? Is it single access that causes the > >havoc, or you need to touch the amount approximately equal to RAM+swap ? >=20 > Andreas --BJNipRl4fWPaRlLb Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk7Wi38ACgkQC3+MBN1Mb4g7lQCcDFAgpunUqroZKNwgwUPuHIEn +EMAniqJfv8Uhm2rq3WrqjOJ/aXUsM3J =J9ox -----END PGP SIGNATURE----- --BJNipRl4fWPaRlLb-- From owner-freebsd-arch@FreeBSD.ORG Wed Nov 30 20:58:23 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E692C1065678 for ; Wed, 30 Nov 2011 20:58:23 +0000 (UTC) (envelope-from andreast-list@fgznet.ch) Received: from smtp.fgznet.ch (mail.fgznet.ch [81.92.96.47]) by mx1.freebsd.org (Postfix) with ESMTP id 8B6798FC08 for ; Wed, 30 Nov 2011 20:58:22 +0000 (UTC) Received: from deuterium.andreas.nets (dhclient-91-190-14-19.flashcable.ch [91.190.14.19]) by smtp.fgznet.ch (8.13.8/8.13.8/Submit_SMTPAUTH) with ESMTP id pAUKvA1E038483; Wed, 30 Nov 2011 21:57:12 +0100 (CET) (envelope-from andreast-list@fgznet.ch) Message-ID: <4ED698EB.8090904@fgznet.ch> Date: Wed, 30 Nov 2011 21:58:19 +0100 From: Andreas Tobler User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: Kostik Belousov References: <4ED5BE19.70805@fgznet.ch> <20111130162236.GA50300@deviant.kiev.zoral.com.ua> <4ED65F70.7050700@fgznet.ch> <20111130170936.GB50300@deviant.kiev.zoral.com.ua> <4ED66B75.3060409@fgznet.ch> <20111130200103.GE50300@deviant.kiev.zoral.com.ua> In-Reply-To: <20111130200103.GE50300@deviant.kiev.zoral.com.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.64 on 81.92.96.47 Cc: FreeBSD Arch Subject: Re: powerpc64 malloc limit? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Nov 2011 20:58:24 -0000 On 30.11.11 21:01, Kostik Belousov wrote: > On Wed, Nov 30, 2011 at 06:44:21PM +0100, Andreas Tobler wrote: >> On 30.11.11 18:09, Kostik Belousov wrote: >>> On Wed, Nov 30, 2011 at 05:53:04PM +0100, Andreas Tobler wrote: >>>> On 30.11.11 17:22, Kostik Belousov wrote: >>>>> On Wed, Nov 30, 2011 at 06:24:41AM +0100, Andreas Tobler wrote: >>>>>> All, >>>>>> >>>>>> while working on gcc I found a very strange situation which renders my >>>>>> powerpc64 machine unusable. >>>>>> The test case below tries to allocate that much memory as 'wanted'. The >>>>>> same test case on amd64 returns w/o trying to allocate mem because the >>>>>> size is far to big. >>>>>> >>>>>> I couldn't find the reason so far, that's why I'm here. >>>>>> >>>>>> As Nathan pointed out the VM_MAXUSER_SIZE is the biggest on powerpc64: >>>>>> #define VM_MAXUSER_ADDRESS (0x7ffffffffffff000UL) >>>>>> >>>>>> So, I'd expect a system to return an allocation error when a user tries >>>>>> to allocate too much memory and not really trying it and going to be >>>>>> unusable. Iow, I'd exepect the situation on powerpc64 as I see on amd64. >>>>>> >>>>>> Can anybody explain me the situation, why do I not have a working limit >>>>>> on powerpc64? >>>>>> >>>>>> The machine itself has 7GB RAM and 12GB swap. The amd64 where I compared >>>>>> has around 4GB/4GB RAM/swap. >>>>>> >>>>>> TIA, >>>>>> Andreas >>>>>> >>>>>> include >>>>>> #include >>>>>> >>>>>> int main() >>>>>> { >>>>>> void *p; >>>>>> >>>>>> p = (void*) malloc (1152921504606846968ULL); >>>>>> if (p != NULL) >>>>>> printf("p = %p\n", p); >>>>>> >>>>>> printf("p = %p\n", p); >>>>>> return (0); >>>>>> } >>>>> >>>>> First, you should provide details of what consistutes 'the unusable >>>>> machine situation' on powerpc. >>>> >>>> I can not login anymore, everything is stuck except the core control >>>> mechanisms for example the fan controller. >>>> >>>> Top reports 'ugly' figures, below from a earlier try: >>>> >>>> last pid: 6790; load averages: 0.78, 0.84, 0.86 up 0+00:34:52 >>>> 22:42:29 47 processes: 1 running, 46 sleeping >>>> CPU: 0.0% user, 0.0% nice, 15.4% system, 11.8% interrupt, 72.8% idle >>>> Mem: 5912M Active, 570M Inact, 280M Wired, 26M Cache, 104M Buf, 352K Free >>>> Swap: 12G Total, 9904M Used, 2383M Free, 80% Inuse, 178M Out >>>> >>>> PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU >>>> COMMAND >>>> 6768 andreast 1 52 01073741824G 6479M pfault 1 0:58 >>>> 18.90% 31370. >>>> >>>> And after my mem and swap are full I see swap_pager_getswapspace(16) >>>> failed. >>>> >>>> In this state I can only power-cycle the machine. >>>> >>>>> That said, on amd64 the user map is between 0 and 0x7fffffffffff, which >>>>> obviously less then the requested allocation size 0x100000000000000. >>>>> If you look at the kdump output on amd64, you will see that malloc() >>>>> tries to mmap() the area, fails and retries with obreak(). Default >>>>> virtual memory limit is unlimited, so my best quess is that on amd64 >>>>> vm_map_findspace() returns immediately. >>>>> >>>>> On powerpc64, I see no reason why vm_map_entry cannot be allocated, but >>>>> please note that vm object and pages shall be only allocated on demand. >>>>> So I am curious how does your machine breaks and where. >>>> >>>> I would expect that the 'system' does not allow me to allocate that much >>>> of ram. >>> >>> Does the issue with machine going into limbo reproducable with the code >>> you posted ? >> >> If I understand you correctly, yes. I can launch the test case and the >> machine is immediately unusable. Means I can not kill the process nor >> can I log in. Also, top does not show anything useful. > Again, let me restate my question: the single mmap() of the huge size is > enough for powerpc64 machine to break apart ? I can't answer. I don't know yet. > What happen if you insert sleep(1000000); call before return ? Do not kill > the process, I want to know is machine dead while the process sleeps. Ok, during the 'sleep' the machine is usable. top is reporting figures, I can log in and edit files. The process runs now for aboutt 30'. When I kill the process, I do not get back to the shell nor can I log in. Also top stops reporting. But as you said, I didn't kill in this run. Thanks, Andreas [bohrium:~] andreast% ./mega_malloc p = 0x50400000 p = 0x50400000 ... last pid: 1279; load averages: 0.05, 0.03, 0.04 up 0+00:30:34 21:56:40 41 processes: 1 running, 40 sleeping CPU: 0.0% user, 0.0% nice, 1.2% system, 0.2% interrupt, 98.6% idle Mem: 28M Active, 26M Inact, 89M Wired, 8K Cache, 725M Buf, 6642M Free Swap: 12G Total, 12G Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 1219 andreast 1 24 01073741824G 1608K nanslp 0 0:00 0.00% mega_m From owner-freebsd-arch@FreeBSD.ORG Wed Nov 30 21:24:58 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A957C106564A for ; Wed, 30 Nov 2011 21:24:58 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 7F7918FC14 for ; Wed, 30 Nov 2011 21:24:57 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pAULOd1j059043 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 30 Nov 2011 23:24:39 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pAULOdsB080128; Wed, 30 Nov 2011 23:24:39 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pAULOdQ1080127; Wed, 30 Nov 2011 23:24:39 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 30 Nov 2011 23:24:39 +0200 From: Kostik Belousov To: Andreas Tobler Message-ID: <20111130212439.GF50300@deviant.kiev.zoral.com.ua> References: <4ED5BE19.70805@fgznet.ch> <20111130162236.GA50300@deviant.kiev.zoral.com.ua> <4ED65F70.7050700@fgznet.ch> <20111130170936.GB50300@deviant.kiev.zoral.com.ua> <4ED66B75.3060409@fgznet.ch> <20111130200103.GE50300@deviant.kiev.zoral.com.ua> <4ED698EB.8090904@fgznet.ch> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="trzpwzIgNM352Iuf" Content-Disposition: inline In-Reply-To: <4ED698EB.8090904@fgznet.ch> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: FreeBSD Arch Subject: Re: powerpc64 malloc limit? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Nov 2011 21:24:58 -0000 --trzpwzIgNM352Iuf Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Nov 30, 2011 at 09:58:19PM +0100, Andreas Tobler wrote: > On 30.11.11 21:01, Kostik Belousov wrote: > >On Wed, Nov 30, 2011 at 06:44:21PM +0100, Andreas Tobler wrote: > >>On 30.11.11 18:09, Kostik Belousov wrote: > >>>On Wed, Nov 30, 2011 at 05:53:04PM +0100, Andreas Tobler wrote: > >>>>On 30.11.11 17:22, Kostik Belousov wrote: > >>>>>On Wed, Nov 30, 2011 at 06:24:41AM +0100, Andreas Tobler wrote: > >>>>>>All, > >>>>>> > >>>>>>while working on gcc I found a very strange situation which renders= my > >>>>>>powerpc64 machine unusable. > >>>>>>The test case below tries to allocate that much memory as 'wanted'.= =20 > >>>>>>The > >>>>>>same test case on amd64 returns w/o trying to allocate mem because = the > >>>>>>size is far to big. > >>>>>> > >>>>>>I couldn't find the reason so far, that's why I'm here. > >>>>>> > >>>>>>As Nathan pointed out the VM_MAXUSER_SIZE is the biggest on powerpc= 64: > >>>>>>#define VM_MAXUSER_ADDRESS (0x7ffffffffffff000UL) > >>>>>> > >>>>>>So, I'd expect a system to return an allocation error when a user= =20 > >>>>>>tries > >>>>>>to allocate too much memory and not really trying it and going to be > >>>>>>unusable. Iow, I'd exepect the situation on powerpc64 as I see on= =20 > >>>>>>amd64. > >>>>>> > >>>>>>Can anybody explain me the situation, why do I not have a working= =20 > >>>>>>limit > >>>>>>on powerpc64? > >>>>>> > >>>>>>The machine itself has 7GB RAM and 12GB swap. The amd64 where I=20 > >>>>>>compared > >>>>>>has around 4GB/4GB RAM/swap. > >>>>>> > >>>>>>TIA, > >>>>>>Andreas > >>>>>> > >>>>>>include > >>>>>>#include > >>>>>> > >>>>>>int main() > >>>>>>{ > >>>>>> void *p; > >>>>>> > >>>>>> p =3D (void*) malloc (1152921504606846968ULL); > >>>>>> if (p !=3D NULL) > >>>>>> printf("p =3D %p\n", p); > >>>>>> > >>>>>> printf("p =3D %p\n", p); > >>>>>> return (0); > >>>>>>} > >>>>> > >>>>>First, you should provide details of what consistutes 'the unusable > >>>>>machine situation' on powerpc. > >>>> > >>>>I can not login anymore, everything is stuck except the core control > >>>>mechanisms for example the fan controller. > >>>> > >>>>Top reports 'ugly' figures, below from a earlier try: > >>>> > >>>>last pid: 6790; load averages: 0.78, 0.84, 0.86 up 0+00:34:52 > >>>>22:42:29 47 processes: 1 running, 46 sleeping > >>>>CPU: 0.0% user, 0.0% nice, 15.4% system, 11.8% interrupt, 72.8% idle > >>>>Mem: 5912M Active, 570M Inact, 280M Wired, 26M Cache, 104M Buf, 352K= =20 > >>>>Free > >>>>Swap: 12G Total, 9904M Used, 2383M Free, 80% Inuse, 178M Out > >>>> > >>>> PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU > >>>>COMMAND > >>>> 6768 andreast 1 52 01073741824G 6479M pfault 1 0:58 > >>>>18.90% 31370. > >>>> > >>>>And after my mem and swap are full I see swap_pager_getswapspace(16) > >>>>failed. > >>>> > >>>>In this state I can only power-cycle the machine. > >>>> > >>>>>That said, on amd64 the user map is between 0 and 0x7fffffffffff, wh= ich > >>>>>obviously less then the requested allocation size 0x100000000000000. > >>>>>If you look at the kdump output on amd64, you will see that malloc() > >>>>>tries to mmap() the area, fails and retries with obreak(). Default > >>>>>virtual memory limit is unlimited, so my best quess is that on amd64 > >>>>>vm_map_findspace() returns immediately. > >>>>> > >>>>>On powerpc64, I see no reason why vm_map_entry cannot be allocated, = but > >>>>>please note that vm object and pages shall be only allocated on dema= nd. > >>>>>So I am curious how does your machine breaks and where. > >>>> > >>>>I would expect that the 'system' does not allow me to allocate that m= uch > >>>>of ram. > >>> > >>>Does the issue with machine going into limbo reproducable with the code > >>>you posted ? > >> > >>If I understand you correctly, yes. I can launch the test case and the > >>machine is immediately unusable. Means I can not kill the process nor > >>can I log in. Also, top does not show anything useful. > >Again, let me restate my question: the single mmap() of the huge size is > >enough for powerpc64 machine to break apart ? >=20 > I can't answer. I don't know yet. >=20 > >What happen if you insert sleep(1000000); call before return ? Do not ki= ll > >the process, I want to know is machine dead while the process sleeps. >=20 > Ok, during the 'sleep' the machine is usable. top is reporting figures,= =20 > I can log in and edit files. The process runs now for aboutt 30'. >=20 > When I kill the process, I do not get back to the shell nor can I log=20 > in. Also top stops reporting. > But as you said, I didn't kill in this run. Then, as Alan Cox pointed out, caused by the approach taken in powerpc64 pmap to handle pmap_remove(). It is definitely arch-specific. >=20 > Thanks, > Andreas >=20 > [bohrium:~] andreast% ./mega_malloc > p =3D 0x50400000 > p =3D 0x50400000 > ... >=20 > last pid: 1279; load averages: 0.05, 0.03, 0.04 up 0+00:30:34=20 > 21:56:40 > 41 processes: 1 running, 40 sleeping > CPU: 0.0% user, 0.0% nice, 1.2% system, 0.2% interrupt, 98.6% idle > Mem: 28M Active, 26M Inact, 89M Wired, 8K Cache, 725M Buf, 6642M Free > Swap: 12G Total, 12G Free >=20 > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU=20 > COMMAND > 1219 andreast 1 24 01073741824G 1608K nanslp 0 0:00=20 > 0.00% mega_m --trzpwzIgNM352Iuf Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk7WnxYACgkQC3+MBN1Mb4jj4ACgxe5J5gtPIAZmCYXpwwIID/Ri B+gAoNAQfNLTu7EOWf6U5gsCD52HLaol =ii8m -----END PGP SIGNATURE----- --trzpwzIgNM352Iuf-- From owner-freebsd-arch@FreeBSD.ORG Wed Nov 30 21:43:14 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A662F1065670 for ; Wed, 30 Nov 2011 21:43:14 +0000 (UTC) (envelope-from andreast-list@fgznet.ch) Received: from smtp.fgznet.ch (mail.fgznet.ch [81.92.96.47]) by mx1.freebsd.org (Postfix) with ESMTP id 4AB938FC08 for ; Wed, 30 Nov 2011 21:43:13 +0000 (UTC) Received: from deuterium.andreas.nets (dhclient-91-190-14-19.flashcable.ch [91.190.14.19]) by smtp.fgznet.ch (8.13.8/8.13.8/Submit_SMTPAUTH) with ESMTP id pAULg1Cg009303; Wed, 30 Nov 2011 22:42:03 +0100 (CET) (envelope-from andreast-list@fgznet.ch) Message-ID: <4ED6A36D.1050107@fgznet.ch> Date: Wed, 30 Nov 2011 22:43:09 +0100 From: Andreas Tobler User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: Kostik Belousov References: <4ED5BE19.70805@fgznet.ch> <20111130162236.GA50300@deviant.kiev.zoral.com.ua> <4ED65F70.7050700@fgznet.ch> <20111130170936.GB50300@deviant.kiev.zoral.com.ua> <4ED66B75.3060409@fgznet.ch> <20111130200103.GE50300@deviant.kiev.zoral.com.ua> <4ED698EB.8090904@fgznet.ch> <20111130212439.GF50300@deviant.kiev.zoral.com.ua> In-Reply-To: <20111130212439.GF50300@deviant.kiev.zoral.com.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.64 on 81.92.96.47 Cc: FreeBSD Arch Subject: Re: powerpc64 malloc limit? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Nov 2011 21:43:14 -0000 On 30.11.11 22:24, Kostik Belousov wrote: > On Wed, Nov 30, 2011 at 09:58:19PM +0100, Andreas Tobler wrote: >> On 30.11.11 21:01, Kostik Belousov wrote: >>> On Wed, Nov 30, 2011 at 06:44:21PM +0100, Andreas Tobler wrote: >>>> On 30.11.11 18:09, Kostik Belousov wrote: >>>>> On Wed, Nov 30, 2011 at 05:53:04PM +0100, Andreas Tobler wrote: >>>>>> On 30.11.11 17:22, Kostik Belousov wrote: >>>>>>> On Wed, Nov 30, 2011 at 06:24:41AM +0100, Andreas Tobler wrote: >>>>>>>> All, >>>>>>>> >>>>>>>> while working on gcc I found a very strange situation which renders my >>>>>>>> powerpc64 machine unusable. >>>>>>>> The test case below tries to allocate that much memory as 'wanted'. >>>>>>>> The >>>>>>>> same test case on amd64 returns w/o trying to allocate mem because the >>>>>>>> size is far to big. >>>>>>>> >>>>>>>> I couldn't find the reason so far, that's why I'm here. >>>>>>>> >>>>>>>> As Nathan pointed out the VM_MAXUSER_SIZE is the biggest on powerpc64: >>>>>>>> #define VM_MAXUSER_ADDRESS (0x7ffffffffffff000UL) >>>>>>>> >>>>>>>> So, I'd expect a system to return an allocation error when a user >>>>>>>> tries >>>>>>>> to allocate too much memory and not really trying it and going to be >>>>>>>> unusable. Iow, I'd exepect the situation on powerpc64 as I see on >>>>>>>> amd64. >>>>>>>> >>>>>>>> Can anybody explain me the situation, why do I not have a working >>>>>>>> limit >>>>>>>> on powerpc64? >>>>>>>> >>>>>>>> The machine itself has 7GB RAM and 12GB swap. The amd64 where I >>>>>>>> compared >>>>>>>> has around 4GB/4GB RAM/swap. >>>>>>>> >>>>>>>> TIA, >>>>>>>> Andreas >>>>>>>> >>>>>>>> include >>>>>>>> #include >>>>>>>> >>>>>>>> int main() >>>>>>>> { >>>>>>>> void *p; >>>>>>>> >>>>>>>> p = (void*) malloc (1152921504606846968ULL); >>>>>>>> if (p != NULL) >>>>>>>> printf("p = %p\n", p); >>>>>>>> >>>>>>>> printf("p = %p\n", p); >>>>>>>> return (0); >>>>>>>> } >>>>>>> >>>>>>> First, you should provide details of what consistutes 'the unusable >>>>>>> machine situation' on powerpc. >>>>>> >>>>>> I can not login anymore, everything is stuck except the core control >>>>>> mechanisms for example the fan controller. >>>>>> >>>>>> Top reports 'ugly' figures, below from a earlier try: >>>>>> >>>>>> last pid: 6790; load averages: 0.78, 0.84, 0.86 up 0+00:34:52 >>>>>> 22:42:29 47 processes: 1 running, 46 sleeping >>>>>> CPU: 0.0% user, 0.0% nice, 15.4% system, 11.8% interrupt, 72.8% idle >>>>>> Mem: 5912M Active, 570M Inact, 280M Wired, 26M Cache, 104M Buf, 352K >>>>>> Free >>>>>> Swap: 12G Total, 9904M Used, 2383M Free, 80% Inuse, 178M Out >>>>>> >>>>>> PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU >>>>>> COMMAND >>>>>> 6768 andreast 1 52 01073741824G 6479M pfault 1 0:58 >>>>>> 18.90% 31370. >>>>>> >>>>>> And after my mem and swap are full I see swap_pager_getswapspace(16) >>>>>> failed. >>>>>> >>>>>> In this state I can only power-cycle the machine. >>>>>> >>>>>>> That said, on amd64 the user map is between 0 and 0x7fffffffffff, which >>>>>>> obviously less then the requested allocation size 0x100000000000000. >>>>>>> If you look at the kdump output on amd64, you will see that malloc() >>>>>>> tries to mmap() the area, fails and retries with obreak(). Default >>>>>>> virtual memory limit is unlimited, so my best quess is that on amd64 >>>>>>> vm_map_findspace() returns immediately. >>>>>>> >>>>>>> On powerpc64, I see no reason why vm_map_entry cannot be allocated, but >>>>>>> please note that vm object and pages shall be only allocated on demand. >>>>>>> So I am curious how does your machine breaks and where. >>>>>> >>>>>> I would expect that the 'system' does not allow me to allocate that much >>>>>> of ram. >>>>> >>>>> Does the issue with machine going into limbo reproducable with the code >>>>> you posted ? >>>> >>>> If I understand you correctly, yes. I can launch the test case and the >>>> machine is immediately unusable. Means I can not kill the process nor >>>> can I log in. Also, top does not show anything useful. >>> Again, let me restate my question: the single mmap() of the huge size is >>> enough for powerpc64 machine to break apart ? >> >> I can't answer. I don't know yet. >> >>> What happen if you insert sleep(1000000); call before return ? Do not kill >>> the process, I want to know is machine dead while the process sleeps. >> >> Ok, during the 'sleep' the machine is usable. top is reporting figures, >> I can log in and edit files. The process runs now for aboutt 30'. >> >> When I kill the process, I do not get back to the shell nor can I log >> in. Also top stops reporting. >> But as you said, I didn't kill in this run. > Then, as Alan Cox pointed out, caused by the approach taken in powerpc64 > pmap to handle pmap_remove(). It is definitely arch-specific. Ok. I think you mean moea64_remove which is pmap_remove, right? Where did Alan pointed this out? Thank you for the analysis. Andreas From owner-freebsd-arch@FreeBSD.ORG Wed Nov 30 22:06:14 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6DA5D106566C for ; Wed, 30 Nov 2011 22:06:14 +0000 (UTC) (envelope-from alan.l.cox@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 25C118FC12 for ; Wed, 30 Nov 2011 22:06:14 +0000 (UTC) Received: by ghbg20 with SMTP id g20so1765591ghb.13 for ; Wed, 30 Nov 2011 14:06:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=nSPmn2NKjpE4lthXwghqtNjA/Q+lJW1tt2WEXLXea5U=; b=bPfxh8ffKdQQ+Z+kIvX5dz5QpWf3tLivpToRTHRtN7FL6sqIxcxhUx76Ljw9wsjHoc sVcJzGSFvbCas7LCtlV9uvrDn/tTW9ElSx6vcBjT4O273hYHows9hiFabQB6UlQQqi71 6UdgcEE0ZKUCMoMRjF2gQmBIXCiuz8iiP+JbQ= MIME-Version: 1.0 Received: by 10.68.41.70 with SMTP id d6mr821763pbl.49.1322690354757; Wed, 30 Nov 2011 13:59:14 -0800 (PST) Received: by 10.143.159.9 with HTTP; Wed, 30 Nov 2011 13:59:14 -0800 (PST) In-Reply-To: <4ED6A36D.1050107@fgznet.ch> References: <4ED5BE19.70805@fgznet.ch> <20111130162236.GA50300@deviant.kiev.zoral.com.ua> <4ED65F70.7050700@fgznet.ch> <20111130170936.GB50300@deviant.kiev.zoral.com.ua> <4ED66B75.3060409@fgznet.ch> <20111130200103.GE50300@deviant.kiev.zoral.com.ua> <4ED698EB.8090904@fgznet.ch> <20111130212439.GF50300@deviant.kiev.zoral.com.ua> <4ED6A36D.1050107@fgznet.ch> Date: Wed, 30 Nov 2011 15:59:14 -0600 Message-ID: From: Alan Cox To: Andreas Tobler Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Kostik Belousov , FreeBSD Arch Subject: Re: powerpc64 malloc limit? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: alc@freebsd.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Nov 2011 22:06:14 -0000 On Wed, Nov 30, 2011 at 3:43 PM, Andreas Tobler wrote: > On 30.11.11 22:24, Kostik Belousov wrote: > >> On Wed, Nov 30, 2011 at 09:58:19PM +0100, Andreas Tobler wrote: >> >>> On 30.11.11 21:01, Kostik Belousov wrote: >>> >>>> On Wed, Nov 30, 2011 at 06:44:21PM +0100, Andreas Tobler wrote: >>>> >>>>> On 30.11.11 18:09, Kostik Belousov wrote: >>>>> >>>>>> On Wed, Nov 30, 2011 at 05:53:04PM +0100, Andreas Tobler wrote: >>>>>> >>>>>>> On 30.11.11 17:22, Kostik Belousov wrote: >>>>>>> >>>>>>>> On Wed, Nov 30, 2011 at 06:24:41AM +0100, Andreas Tobler wrote: >>>>>>>> >>>>>>>>> All, >>>>>>>>> >>>>>>>>> while working on gcc I found a very strange situation which >>>>>>>>> renders my >>>>>>>>> powerpc64 machine unusable. >>>>>>>>> The test case below tries to allocate that much memory as 'wanted'. >>>>>>>>> The >>>>>>>>> same test case on amd64 returns w/o trying to allocate mem because >>>>>>>>> the >>>>>>>>> size is far to big. >>>>>>>>> >>>>>>>>> I couldn't find the reason so far, that's why I'm here. >>>>>>>>> >>>>>>>>> As Nathan pointed out the VM_MAXUSER_SIZE is the biggest on >>>>>>>>> powerpc64: >>>>>>>>> #define VM_MAXUSER_ADDRESS (0x7ffffffffffff000UL) >>>>>>>>> >>>>>>>>> So, I'd expect a system to return an allocation error when a user >>>>>>>>> tries >>>>>>>>> to allocate too much memory and not really trying it and going to >>>>>>>>> be >>>>>>>>> unusable. Iow, I'd exepect the situation on powerpc64 as I see on >>>>>>>>> amd64. >>>>>>>>> >>>>>>>>> Can anybody explain me the situation, why do I not have a working >>>>>>>>> limit >>>>>>>>> on powerpc64? >>>>>>>>> >>>>>>>>> The machine itself has 7GB RAM and 12GB swap. The amd64 where I >>>>>>>>> compared >>>>>>>>> has around 4GB/4GB RAM/swap. >>>>>>>>> >>>>>>>>> TIA, >>>>>>>>> Andreas >>>>>>>>> >>>>>>>>> include >>>>>>>>> #include >>>>>>>>> >>>>>>>>> int main() >>>>>>>>> { >>>>>>>>> void *p; >>>>>>>>> >>>>>>>>> p = (void*) malloc (1152921504606846968ULL); >>>>>>>>> if (p != NULL) >>>>>>>>> printf("p = %p\n", p); >>>>>>>>> >>>>>>>>> printf("p = %p\n", p); >>>>>>>>> return (0); >>>>>>>>> } >>>>>>>>> >>>>>>>> >>>>>>>> First, you should provide details of what consistutes 'the unusable >>>>>>>> machine situation' on powerpc. >>>>>>>> >>>>>>> >>>>>>> I can not login anymore, everything is stuck except the core control >>>>>>> mechanisms for example the fan controller. >>>>>>> >>>>>>> Top reports 'ugly' figures, below from a earlier try: >>>>>>> >>>>>>> last pid: 6790; load averages: 0.78, 0.84, 0.86 up 0+00:34:52 >>>>>>> 22:42:29 47 processes: 1 running, 46 sleeping >>>>>>> CPU: 0.0% user, 0.0% nice, 15.4% system, 11.8% interrupt, 72.8% >>>>>>> idle >>>>>>> Mem: 5912M Active, 570M Inact, 280M Wired, 26M Cache, 104M Buf, 352K >>>>>>> Free >>>>>>> Swap: 12G Total, 9904M Used, 2383M Free, 80% Inuse, 178M Out >>>>>>> >>>>>>> PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU >>>>>>> COMMAND >>>>>>> 6768 andreast 1 52 01073741824G 6479M pfault 1 0:58 >>>>>>> 18.90% 31370. >>>>>>> >>>>>>> And after my mem and swap are full I see swap_pager_getswapspace(16) >>>>>>> failed. >>>>>>> >>>>>>> In this state I can only power-cycle the machine. >>>>>>> >>>>>>> That said, on amd64 the user map is between 0 and 0x7fffffffffff, >>>>>>>> which >>>>>>>> obviously less then the requested allocation size 0x100000000000000. >>>>>>>> If you look at the kdump output on amd64, you will see that malloc() >>>>>>>> tries to mmap() the area, fails and retries with obreak(). Default >>>>>>>> virtual memory limit is unlimited, so my best quess is that on amd64 >>>>>>>> vm_map_findspace() returns immediately. >>>>>>>> >>>>>>>> On powerpc64, I see no reason why vm_map_entry cannot be allocated, >>>>>>>> but >>>>>>>> please note that vm object and pages shall be only allocated on >>>>>>>> demand. >>>>>>>> So I am curious how does your machine breaks and where. >>>>>>>> >>>>>>> >>>>>>> I would expect that the 'system' does not allow me to allocate that >>>>>>> much >>>>>>> of ram. >>>>>>> >>>>>> >>>>>> Does the issue with machine going into limbo reproducable with the >>>>>> code >>>>>> you posted ? >>>>>> >>>>> >>>>> If I understand you correctly, yes. I can launch the test case and the >>>>> machine is immediately unusable. Means I can not kill the process nor >>>>> can I log in. Also, top does not show anything useful. >>>>> >>>> Again, let me restate my question: the single mmap() of the huge size is >>>> enough for powerpc64 machine to break apart ? >>>> >>> >>> I can't answer. I don't know yet. >>> >>> What happen if you insert sleep(1000000); call before return ? Do not >>>> kill >>>> the process, I want to know is machine dead while the process sleeps. >>>> >>> >>> Ok, during the 'sleep' the machine is usable. top is reporting figures, >>> I can log in and edit files. The process runs now for aboutt 30'. >>> >>> When I kill the process, I do not get back to the shell nor can I log >>> in. Also top stops reporting. >>> But as you said, I didn't kill in this run. >>> >> Then, as Alan Cox pointed out, caused by the approach taken in powerpc64 >> pmap to handle pmap_remove(). It is definitely arch-specific. >> > > Ok. I think you mean moea64_remove which is pmap_remove, right? > > Where did Alan pointed this out? > > I was in a rush earlier, so I sent a short, cryptic note to Kostik privately. Alan From owner-freebsd-arch@FreeBSD.ORG Wed Nov 30 22:08:21 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CDB4C106566C for ; Wed, 30 Nov 2011 22:08:21 +0000 (UTC) (envelope-from andreast-list@fgznet.ch) Received: from smtp.fgznet.ch (mail.fgznet.ch [81.92.96.47]) by mx1.freebsd.org (Postfix) with ESMTP id 2DCBE8FC0A for ; Wed, 30 Nov 2011 22:08:20 +0000 (UTC) Received: from deuterium.andreas.nets (dhclient-91-190-14-19.flashcable.ch [91.190.14.19]) by smtp.fgznet.ch (8.13.8/8.13.8/Submit_SMTPAUTH) with ESMTP id pAUM78pC048334; Wed, 30 Nov 2011 23:07:09 +0100 (CET) (envelope-from andreast-list@fgznet.ch) Message-ID: <4ED6A951.4070202@fgznet.ch> Date: Wed, 30 Nov 2011 23:08:17 +0100 From: Andreas Tobler User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: alc@freebsd.org References: <4ED5BE19.70805@fgznet.ch> <20111130162236.GA50300@deviant.kiev.zoral.com.ua> <4ED65F70.7050700@fgznet.ch> <20111130170936.GB50300@deviant.kiev.zoral.com.ua> <4ED66B75.3060409@fgznet.ch> <20111130200103.GE50300@deviant.kiev.zoral.com.ua> <4ED698EB.8090904@fgznet.ch> <20111130212439.GF50300@deviant.kiev.zoral.com.ua> <4ED6A36D.1050107@fgznet.ch> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.64 on 81.92.96.47 Cc: Kostik Belousov , Alan Cox , FreeBSD Arch Subject: Re: powerpc64 malloc limit? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Nov 2011 22:08:21 -0000 On 30.11.11 22:59, Alan Cox wrote: > On Wed, Nov 30, 2011 at 3:43 PM, Andreas Tobler > wrote: > > On 30.11.11 22:24, Kostik Belousov wrote: > > On Wed, Nov 30, 2011 at 09:58:19PM +0100, Andreas Tobler wrote: > > On 30.11.11 21:01, Kostik Belousov wrote: > > On Wed, Nov 30, 2011 at 06:44:21PM +0100, Andreas Tobler > wrote: > > On 30.11.11 18:09, Kostik Belousov wrote: > > On Wed, Nov 30, 2011 at 05:53:04PM +0100, > Andreas Tobler wrote: > > On 30.11.11 17:22, Kostik Belousov wrote: > > On Wed, Nov 30, 2011 at 06:24:41AM > +0100, Andreas Tobler wrote: > > All, > > while working on gcc I found a very > strange situation which renders my > powerpc64 machine unusable. > The test case below tries to > allocate that much memory as 'wanted'. > The > same test case on amd64 returns w/o > trying to allocate mem because the > size is far to big. > > I couldn't find the reason so far, > that's why I'm here. > > As Nathan pointed out the > VM_MAXUSER_SIZE is the biggest on > powerpc64: > #define VM_MAXUSER_ADDRESS > (0x7ffffffffffff000UL) > > So, I'd expect a system to return an > allocation error when a user > tries > to allocate too much memory and not > really trying it and going to be > unusable. Iow, I'd exepect the > situation on powerpc64 as I see on > amd64. > > Can anybody explain me the > situation, why do I not have a working > limit > on powerpc64? > > The machine itself has 7GB RAM and > 12GB swap. The amd64 where I > compared > has around 4GB/4GB RAM/swap. > > TIA, > Andreas > > include > #include > > int main() > { > void *p; > > p = (void*) malloc > (1152921504606846968ULL); > if (p != NULL) > printf("p = %p\n", p); > > printf("p = %p\n", p); > return (0); > } > > > First, you should provide details of > what consistutes 'the unusable > machine situation' on powerpc. > > > I can not login anymore, everything is stuck > except the core control > mechanisms for example the fan controller. > > Top reports 'ugly' figures, below from a > earlier try: > > last pid: 6790; load averages: 0.78, > 0.84, 0.86 up 0+00:34:52 > 22:42:29 47 processes: 1 running, 46 sleeping > CPU: 0.0% user, 0.0% nice, 15.4% system, > 11.8% interrupt, 72.8% idle > Mem: 5912M Active, 570M Inact, 280M Wired, > 26M Cache, 104M Buf, 352K > Free > Swap: 12G Total, 9904M Used, 2383M Free, 80% > Inuse, 178M Out > > PID USERNAME THR PRI NICE SIZE > RES STATE C TIME WCPU > COMMAND > 6768 andreast 1 52 01073741824G > 6479M pfault 1 0:58 > 18.90% 31370. > > And after my mem and swap are full I see > swap_pager_getswapspace(16) > failed. > > In this state I can only power-cycle the > machine. > > That said, on amd64 the user map is > between 0 and 0x7fffffffffff, which > obviously less then the requested > allocation size 0x100000000000000. > If you look at the kdump output on > amd64, you will see that malloc() > tries to mmap() the area, fails and > retries with obreak(). Default > virtual memory limit is unlimited, so my > best quess is that on amd64 > vm_map_findspace() returns immediately. > > On powerpc64, I see no reason why > vm_map_entry cannot be allocated, but > please note that vm object and pages > shall be only allocated on demand. > So I am curious how does your machine > breaks and where. > > > I would expect that the 'system' does not > allow me to allocate that much > of ram. > > > Does the issue with machine going into limbo > reproducable with the code > you posted ? > > > If I understand you correctly, yes. I can launch the > test case and the > machine is immediately unusable. Means I can not > kill the process nor > can I log in. Also, top does not show anything useful. > > Again, let me restate my question: the single mmap() of > the huge size is > enough for powerpc64 machine to break apart ? > > > I can't answer. I don't know yet. > > What happen if you insert sleep(1000000); call before > return ? Do not kill > the process, I want to know is machine dead while the > process sleeps. > > > Ok, during the 'sleep' the machine is usable. top is > reporting figures, > I can log in and edit files. The process runs now for aboutt > 30'. > > When I kill the process, I do not get back to the shell nor > can I log > in. Also top stops reporting. > But as you said, I didn't kill in this run. > > Then, as Alan Cox pointed out, caused by the approach taken in > powerpc64 > pmap to handle pmap_remove(). It is definitely arch-specific. > > > Ok. I think you mean moea64_remove which is pmap_remove, right? > > Where did Alan pointed this out? > > > I was in a rush earlier, so I sent a short, cryptic note to Kostik > privately. Thank you. I have to talk with Nathan then. Well, I only wanted to make a test case work and I went through stages where I had to make gdb usable, assembler and linker adaptations, gcc improvments and now it seems that I have to dive into pmap & co :) A real exciting spare time business. Regards, Andreas From owner-freebsd-arch@FreeBSD.ORG Wed Nov 30 22:21:40 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9C641106566C for ; Wed, 30 Nov 2011 22:21:40 +0000 (UTC) (envelope-from alan.l.cox@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3BBEA8FC0A for ; Wed, 30 Nov 2011 22:21:39 +0000 (UTC) Received: by yenq9 with SMTP id q9so1628454yen.13 for ; Wed, 30 Nov 2011 14:21:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=wJR27wwqVuetIRET3qEKQ2E0+H6r3oB89gMAR1suBdg=; b=X4YMddEwi4Fu0zsJsax56dE2xCfCo0cUUVOp7eY0QRymSf5tikOTUuYu8cfCSx9gON L2jRCdsDPs1OOawwhS3qHbUWthxNo2PJ5og7LXQl9+IAJFkp0Sm+W/DeDVr5icybIVMI uw3T3+Z2vYtT/NwP0NAnRJ+JJgUJB3+fD6p/8= MIME-Version: 1.0 Received: by 10.68.28.230 with SMTP id e6mr464563pbh.117.1322689852816; Wed, 30 Nov 2011 13:50:52 -0800 (PST) Received: by 10.143.159.9 with HTTP; Wed, 30 Nov 2011 13:50:52 -0800 (PST) In-Reply-To: References: <4ED5BE19.70805@fgznet.ch> <20111130162236.GA50300@deviant.kiev.zoral.com.ua> <4ED65F70.7050700@fgznet.ch> <20111130170936.GB50300@deviant.kiev.zoral.com.ua> <4ED66B75.3060409@fgznet.ch> Date: Wed, 30 Nov 2011 15:50:52 -0600 Message-ID: From: Alan Cox To: Peter Wemm Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Kostik Belousov , Andreas Tobler , FreeBSD Arch Subject: Re: powerpc64 malloc limit? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: alc@freebsd.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Nov 2011 22:21:40 -0000 On Wed, Nov 30, 2011 at 12:12 PM, Peter Wemm wrote: > On Wed, Nov 30, 2011 at 9:44 AM, Andreas Tobler > wrote: > > On 30.11.11 18:09, Kostik Belousov wrote: > >> > >> On Wed, Nov 30, 2011 at 05:53:04PM +0100, Andreas Tobler wrote: > >>> > >>> On 30.11.11 17:22, Kostik Belousov wrote: > >>>> > >>>> On Wed, Nov 30, 2011 at 06:24:41AM +0100, Andreas Tobler wrote: > >>>>> > >>>>> All, > >>>>> > >>>>> while working on gcc I found a very strange situation which renders > my > >>>>> powerpc64 machine unusable. > >>>>> The test case below tries to allocate that much memory as 'wanted'. > The > >>>>> same test case on amd64 returns w/o trying to allocate mem because > the > >>>>> size is far to big. > >>>>> > >>>>> I couldn't find the reason so far, that's why I'm here. > >>>>> > >>>>> As Nathan pointed out the VM_MAXUSER_SIZE is the biggest on > powerpc64: > >>>>> #define VM_MAXUSER_ADDRESS (0x7ffffffffffff000UL) > >>>>> > >>>>> So, I'd expect a system to return an allocation error when a user > tries > >>>>> to allocate too much memory and not really trying it and going to be > >>>>> unusable. Iow, I'd exepect the situation on powerpc64 as I see on > >>>>> amd64. > >>>>> > >>>>> Can anybody explain me the situation, why do I not have a working > limit > >>>>> on powerpc64? > >>>>> > >>>>> The machine itself has 7GB RAM and 12GB swap. The amd64 where I > >>>>> compared > >>>>> has around 4GB/4GB RAM/swap. > >>>>> > >>>>> TIA, > >>>>> Andreas > >>>>> > >>>>> include > >>>>> #include > >>>>> > >>>>> int main() > >>>>> { > >>>>> void *p; > >>>>> > >>>>> p = (void*) malloc (1152921504606846968ULL); > >>>>> if (p != NULL) > >>>>> printf("p = %p\n", p); > >>>>> > >>>>> printf("p = %p\n", p); > >>>>> return (0); > >>>>> } > >>>> > >>>> First, you should provide details of what consistutes 'the unusable > >>>> machine situation' on powerpc. > >>> > >>> I can not login anymore, everything is stuck except the core control > >>> mechanisms for example the fan controller. > >>> > >>> Top reports 'ugly' figures, below from a earlier try: > >>> > >>> last pid: 6790; load averages: 0.78, 0.84, 0.86 up 0+00:34:52 > >>> 22:42:29 47 processes: 1 running, 46 sleeping > >>> CPU: 0.0% user, 0.0% nice, 15.4% system, 11.8% interrupt, 72.8% idle > >>> Mem: 5912M Active, 570M Inact, 280M Wired, 26M Cache, 104M Buf, 352K > Free > >>> Swap: 12G Total, 9904M Used, 2383M Free, 80% Inuse, 178M Out > >>> > >>> PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU > >>> COMMAND > >>> 6768 andreast 1 52 01073741824G 6479M pfault 1 0:58 > >>> 18.90% 31370. > >>> > >>> And after my mem and swap are full I see swap_pager_getswapspace(16) > >>> failed. > >>> > >>> In this state I can only power-cycle the machine. > >>> > >>>> That said, on amd64 the user map is between 0 and 0x7fffffffffff, > which > >>>> obviously less then the requested allocation size 0x100000000000000. > >>>> If you look at the kdump output on amd64, you will see that malloc() > >>>> tries to mmap() the area, fails and retries with obreak(). Default > >>>> virtual memory limit is unlimited, so my best quess is that on amd64 > >>>> vm_map_findspace() returns immediately. > >>>> > >>>> On powerpc64, I see no reason why vm_map_entry cannot be allocated, > but > >>>> please note that vm object and pages shall be only allocated on > demand. > >>>> So I am curious how does your machine breaks and where. > >>> > >>> I would expect that the 'system' does not allow me to allocate that > much > >>> of ram. > >> > >> Does the issue with machine going into limbo reproducable with the code > >> you posted ? > > > > If I understand you correctly, yes. I can launch the test case and the > > machine is immediately unusable. Means I can not kill the process nor > can I > > log in. Also, top does not show anything useful. > > > > The original test case where I discovered this behavior behaves a bit > > different. > > > http://gcc.gnu.org/viewcvs/trunk/libstdc%2B%2B-v3/testsuite/23_containers/vector/bool/modifiers/insert/31370.cc?revision=169421&view=markup > > > > Here I can follow how the ram and swap is eaten up. Top is reporting the > > figures. If everything is 'full', the swaper errors start to appear on > the > > console. > > > >> Or, do you need to actually touch the pages in the allocated region ? > > > > If I have to, how would I do that? > > > >> If the later (and I do expect that), then how many pages do you need > >> to touch before machine breaks ? Is it single access that causes the > >> havoc, or you need to touch the amount approximately equal to RAM+swap ? > > > > Andreas > > ia64 had some vaguely related excitement earlier in its life. If > you created a 1TB sparse file and mmap'ed it over and over, tens, > maybe hundreds of thousands of times, certain VM internal state got > way out of hand. mmaping was fine, but unmapping took 36 hours of cpu > runtime when I killed the process. It got so far out of hand because > of the way ia64 handled just-in-time mappings on vhpt misses. > > There is a fundamental scalability problem with the powerpc64/aim pmap. See revision 212360. In a nutshell, unlike amd64, ia64, and most other pmap implementations, the powerpc64/aim pmap implementation doesn't link together all of the pv entries belonging to a pmap into a list. So, the powerpc64/aim implementations of range operations, like pmap_remove(), don't handle large, sparsely populated ranges as efficiently as ia64 does. Moreover, powerpc64/aim can't effectively implement pmap_remove_pages(), and so it doesn't even try. Alan From owner-freebsd-arch@FreeBSD.ORG Thu Dec 1 02:08:46 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CC4E6106564A; Thu, 1 Dec 2011 02:08:46 +0000 (UTC) (envelope-from das@FreeBSD.ORG) Received: from zim.MIT.EDU (ZIM.MIT.EDU [18.95.3.101]) by mx1.freebsd.org (Postfix) with ESMTP id 8F7E08FC12; Thu, 1 Dec 2011 02:08:46 +0000 (UTC) Received: from zim.MIT.EDU (localhost [127.0.0.1]) by zim.MIT.EDU (8.14.5/8.14.2) with ESMTP id pB11niCM078204; Wed, 30 Nov 2011 20:49:44 -0500 (EST) (envelope-from das@FreeBSD.ORG) Received: (from das@localhost) by zim.MIT.EDU (8.14.5/8.14.2/Submit) id pB11nixZ078203; Wed, 30 Nov 2011 20:49:44 -0500 (EST) (envelope-from das@FreeBSD.ORG) Date: Wed, 30 Nov 2011 20:49:44 -0500 From: David Schultz To: John Baldwin Message-ID: <20111201014944.GA78010@zim.MIT.EDU> Mail-Followup-To: John Baldwin , freebsd-arch@freebsd.org, Zack Kirsch , mdf@freebsd.org References: <20111130154604.B949@besplex.bde.org> <201111301032.04102.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201111301032.04102.jhb@freebsd.org> Cc: Zack Kirsch , mdf@FreeBSD.ORG, freebsd-arch@FreeBSD.ORG Subject: Re: Use of bool / stdbool.h in kernel X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Dec 2011 02:08:46 -0000 On Wed, Nov 30, 2011, John Baldwin wrote: > On Wednesday, November 30, 2011 12:13:53 am Bruce Evans wrote: > > On Tue, 29 Nov 2011 mdf@freebsd.org wrote: > > > > > At $WORK we have a hack in one of the *.mk files to allow including > > > stdbool.h in the kernel and we use it extensively. This is not > > > allowed by style(9), as far as I can tell, because the file is in > > > include/stdbool.h and those files are not allowed to be included in > > > kernel sources. > > > > Including stdbool.h in the kernel is not a style bug, but unsupported. > > > > > What I want to check on is, would it be acceptable to move stdbool.h > > > from include/stdbool.h to sys/sys/stdbool.h (i.e. like errno.h) and > > > then include it in the kernel as ? That is, is the > > > > Would be a larger style bug, especially if it were actually used. > > Even its spellings of TRUE and FALSE are strange. Even in userland > > stdbool.h is considered so useful that it is never used in src/bin > > and is only used a few times on other src/*bin. src/bin never uses > > TRUE of FALSE either. > > I suspect there is some bias here though due to the fact that there wasn't > a standard bool type when most of this code was written. :) I don't think > that means we have to forgo use of the new type now that it is in fact > standardized in C99. I would be happy to have 'bool' available and the > lowercase 'true' and 'false' are fine with me. The lowercase 'true' and 'false' are intended to mimic C++, where they are keywords. Regardless of how you prefer to capitalize them, using them instead of 0 and 1 makes the intent much clearer. This is especially true in the kernel, where non-zero could mean true, or it could be an error code. Unfortunately, the "new type" is mostly useless, aside from improving readability. Unlike modern languages, C doesn't consider it a compile-time error to mix up bools and ints. From owner-freebsd-arch@FreeBSD.ORG Thu Dec 1 14:37:00 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C174B106564A; Thu, 1 Dec 2011 14:37:00 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 345838FC0A; Thu, 1 Dec 2011 14:36:59 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApYAAHCJ106DaFvO/2dsb2JhbABEhQOVU5EogXIBAQUjVhsOCgICDRkCWQYTrhiRe4EwgjyGHoEWBIgojC+SMA X-IronPort-AV: E=Sophos;i="4.71,278,1320642000"; d="scan'208";a="147808740" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 01 Dec 2011 09:07:18 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 57439B3F38; Thu, 1 Dec 2011 09:07:18 -0500 (EST) Date: Thu, 1 Dec 2011 09:07:18 -0500 (EST) From: Rick Macklem To: David Schultz Message-ID: <1697386241.723355.1322748438341.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <20111201014944.GA78010@zim.MIT.EDU> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: Zack Kirsch , mdf@FreeBSD.ORG, John Baldwin , freebsd-arch@FreeBSD.ORG Subject: Re: Use of bool / stdbool.h in kernel X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Dec 2011 14:37:00 -0000 David Schultz wrote: > On Wed, Nov 30, 2011, John Baldwin wrote: > > On Wednesday, November 30, 2011 12:13:53 am Bruce Evans wrote: > > > On Tue, 29 Nov 2011 mdf@freebsd.org wrote: > > > > > > > At $WORK we have a hack in one of the *.mk files to allow > > > > including > > > > stdbool.h in the kernel and we use it extensively. This is not > > > > allowed by style(9), as far as I can tell, because the file is > > > > in > > > > include/stdbool.h and those files are not allowed to be included > > > > in > > > > kernel sources. > > > > > > Including stdbool.h in the kernel is not a style bug, but > > > unsupported. > > > > > > > What I want to check on is, would it be acceptable to move > > > > stdbool.h > > > > from include/stdbool.h to sys/sys/stdbool.h (i.e. like errno.h) > > > > and > > > > then include it in the kernel as ? That is, is > > > > the > > > > > > Would be a larger style bug, especially if it were actually used. > > > Even its spellings of TRUE and FALSE are strange. Even in userland > > > stdbool.h is considered so useful that it is never used in src/bin > > > and is only used a few times on other src/*bin. src/bin never uses > > > TRUE of FALSE either. > > > > I suspect there is some bias here though due to the fact that there > > wasn't > > a standard bool type when most of this code was written. :) I don't > > think > > that means we have to forgo use of the new type now that it is in > > fact > > standardized in C99. I would be happy to have 'bool' available and > > the > > lowercase 'true' and 'false' are fine with me. > > The lowercase 'true' and 'false' are intended to mimic C++, where > they are keywords. Regardless of how you prefer to capitalize > them, using them instead of 0 and 1 makes the intent much clearer. > This is especially true in the kernel, where non-zero could mean > true, or it could be an error code. > > Unfortunately, the "new type" is mostly useless, aside from > improving readability. Unlike modern languages, C doesn't > consider it a compile-time error to mix up bools and ints. > If this is added, would the style gods approve of the following: (A) bool test_func(); if (test_func()) ... instead of: (B) int test_func(); if (test_func() != 0) ... Personally, I prefer the former, but understand that it isn't currently style(9) compliant. Being able to do (A) instead of (B) would be why I'd like stdbool.h to be added to the kernel, if it will be allowed after the change? rick From owner-freebsd-arch@FreeBSD.ORG Thu Dec 1 15:12:18 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 72719106566C; Thu, 1 Dec 2011 15:12:18 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 47C708FC0A; Thu, 1 Dec 2011 15:12:18 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id EF54B46B0C; Thu, 1 Dec 2011 10:12:17 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 710C0B914; Thu, 1 Dec 2011 10:12:17 -0500 (EST) From: John Baldwin To: David Schultz Date: Thu, 1 Dec 2011 10:07:23 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) References: <201111301032.04102.jhb@freebsd.org> <20111201014944.GA78010@zim.MIT.EDU> In-Reply-To: <20111201014944.GA78010@zim.MIT.EDU> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201112011007.23430.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 01 Dec 2011 10:12:17 -0500 (EST) Cc: Zack Kirsch , mdf@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Use of bool / stdbool.h in kernel X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Dec 2011 15:12:18 -0000 On Wednesday, November 30, 2011 8:49:44 pm David Schultz wrote: > On Wed, Nov 30, 2011, John Baldwin wrote: > > On Wednesday, November 30, 2011 12:13:53 am Bruce Evans wrote: > > > On Tue, 29 Nov 2011 mdf@freebsd.org wrote: > > > > > > > At $WORK we have a hack in one of the *.mk files to allow including > > > > stdbool.h in the kernel and we use it extensively. This is not > > > > allowed by style(9), as far as I can tell, because the file is in > > > > include/stdbool.h and those files are not allowed to be included in > > > > kernel sources. > > > > > > Including stdbool.h in the kernel is not a style bug, but unsupported. > > > > > > > What I want to check on is, would it be acceptable to move stdbool.h > > > > from include/stdbool.h to sys/sys/stdbool.h (i.e. like errno.h) and > > > > then include it in the kernel as ? That is, is the > > > > > > Would be a larger style bug, especially if it were actually used. > > > Even its spellings of TRUE and FALSE are strange. Even in userland > > > stdbool.h is considered so useful that it is never used in src/bin > > > and is only used a few times on other src/*bin. src/bin never uses > > > TRUE of FALSE either. > > > > I suspect there is some bias here though due to the fact that there wasn't > > a standard bool type when most of this code was written. :) I don't think > > that means we have to forgo use of the new type now that it is in fact > > standardized in C99. I would be happy to have 'bool' available and the > > lowercase 'true' and 'false' are fine with me. > > The lowercase 'true' and 'false' are intended to mimic C++, where > they are keywords. Regardless of how you prefer to capitalize > them, using them instead of 0 and 1 makes the intent much clearer. > This is especially true in the kernel, where non-zero could mean > true, or it could be an error code. > > Unfortunately, the "new type" is mostly useless, aside from > improving readability. Unlike modern languages, C doesn't > consider it a compile-time error to mix up bools and ints. Yes, I consider it a readability aide and think it is fine as such. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Dec 1 15:12:19 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3D8C31065675; Thu, 1 Dec 2011 15:12:19 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id EEE618FC0C; Thu, 1 Dec 2011 15:12:18 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 882D046B09; Thu, 1 Dec 2011 10:12:18 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id DDD6BB941; Thu, 1 Dec 2011 10:12:17 -0500 (EST) From: John Baldwin To: Rick Macklem Date: Thu, 1 Dec 2011 10:12:16 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) References: <1697386241.723355.1322748438341.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <1697386241.723355.1322748438341.JavaMail.root@erie.cs.uoguelph.ca> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201112011012.16891.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 01 Dec 2011 10:12:18 -0500 (EST) Cc: Zack Kirsch , mdf@freebsd.org, David Schultz , freebsd-arch@freebsd.org Subject: Re: Use of bool / stdbool.h in kernel X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Dec 2011 15:12:19 -0000 On Thursday, December 01, 2011 9:07:18 am Rick Macklem wrote: > David Schultz wrote: > > On Wed, Nov 30, 2011, John Baldwin wrote: > > > On Wednesday, November 30, 2011 12:13:53 am Bruce Evans wrote: > > > > On Tue, 29 Nov 2011 mdf@freebsd.org wrote: > > > > > > > > > At $WORK we have a hack in one of the *.mk files to allow > > > > > including > > > > > stdbool.h in the kernel and we use it extensively. This is not > > > > > allowed by style(9), as far as I can tell, because the file is > > > > > in > > > > > include/stdbool.h and those files are not allowed to be included > > > > > in > > > > > kernel sources. > > > > > > > > Including stdbool.h in the kernel is not a style bug, but > > > > unsupported. > > > > > > > > > What I want to check on is, would it be acceptable to move > > > > > stdbool.h > > > > > from include/stdbool.h to sys/sys/stdbool.h (i.e. like errno.h) > > > > > and > > > > > then include it in the kernel as ? That is, is > > > > > the > > > > > > > > Would be a larger style bug, especially if it were actually used. > > > > Even its spellings of TRUE and FALSE are strange. Even in userland > > > > stdbool.h is considered so useful that it is never used in src/bin > > > > and is only used a few times on other src/*bin. src/bin never uses > > > > TRUE of FALSE either. > > > > > > I suspect there is some bias here though due to the fact that there > > > wasn't > > > a standard bool type when most of this code was written. :) I don't > > > think > > > that means we have to forgo use of the new type now that it is in > > > fact > > > standardized in C99. I would be happy to have 'bool' available and > > > the > > > lowercase 'true' and 'false' are fine with me. > > > > The lowercase 'true' and 'false' are intended to mimic C++, where > > they are keywords. Regardless of how you prefer to capitalize > > them, using them instead of 0 and 1 makes the intent much clearer. > > This is especially true in the kernel, where non-zero could mean > > true, or it could be an error code. > > > > Unfortunately, the "new type" is mostly useless, aside from > > improving readability. Unlike modern languages, C doesn't > > consider it a compile-time error to mix up bools and ints. > > > If this is added, would the style gods approve of the following: > > (A) bool test_func(); > > if (test_func()) > ... > > instead of: > > (B) int test_func(); > > if (test_func() != 0) > ... > > Personally, I prefer the former, but understand that it isn't > currently style(9) compliant. Being able to do (A) instead of > (B) would be why I'd like stdbool.h to be added to the kernel, > if it will be allowed after the change? My understanding is that (A) is fine if test_func() is returning an actual boolean value (so the return value is only true and false). A case where this isn't really true is when the function returns an errno as that is returning an int with many values other than just 0 and 1. Granted, it is a common style violation (I am guilty) to do: if (error) return (error); I also consider it a boolean test to test a single-bit flag in a flags field: if (p->p_flag & P_PROFIL) vs. if ((p->p_flag & P_PROFIL) != 0) If you had a multi-bit field though then I think it is appropriate to be more explicit about which values of that field are being tested for. Thus: #define FLAGS_SIMPLE 0x00001 #define FLAGS_FIELD 0x000fe if (foo & FLAGS_SIMPLE) ... if ((foo & FLAGS_FIELD) != 0) ... -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Dec 1 15:43:20 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6B1D81065670; Thu, 1 Dec 2011 15:43:20 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id C5CAE8FC17; Thu, 1 Dec 2011 15:43:19 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApYAAKif106DaFvO/2dsb2JhbABEhQOVU5EogXIBAQQBIwRSBRYOCgICDRkCWQYTiAemFZF5gTCCPIYegRYEiCiML5Iw X-IronPort-AV: E=Sophos;i="4.71,278,1320642000"; d="scan'208";a="147824628" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 01 Dec 2011 10:43:18 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id E0E65B3F99; Thu, 1 Dec 2011 10:43:18 -0500 (EST) Date: Thu, 1 Dec 2011 10:43:18 -0500 (EST) From: Rick Macklem To: John Baldwin Message-ID: <685533129.733983.1322754198902.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <201112011012.16891.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: Zack Kirsch , mdf@freebsd.org, David Schultz , freebsd-arch@freebsd.org Subject: Re: Use of bool / stdbool.h in kernel X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Dec 2011 15:43:20 -0000 John Baldwin wrote: > On Thursday, December 01, 2011 9:07:18 am Rick Macklem wrote: > > David Schultz wrote: > > > On Wed, Nov 30, 2011, John Baldwin wrote: > > > > On Wednesday, November 30, 2011 12:13:53 am Bruce Evans wrote: > > > > > On Tue, 29 Nov 2011 mdf@freebsd.org wrote: > > > > > > > > > > > At $WORK we have a hack in one of the *.mk files to allow > > > > > > including > > > > > > stdbool.h in the kernel and we use it extensively. This is > > > > > > not > > > > > > allowed by style(9), as far as I can tell, because the file > > > > > > is > > > > > > in > > > > > > include/stdbool.h and those files are not allowed to be > > > > > > included > > > > > > in > > > > > > kernel sources. > > > > > > > > > > Including stdbool.h in the kernel is not a style bug, but > > > > > unsupported. > > > > > > > > > > > What I want to check on is, would it be acceptable to move > > > > > > stdbool.h > > > > > > from include/stdbool.h to sys/sys/stdbool.h (i.e. like > > > > > > errno.h) > > > > > > and > > > > > > then include it in the kernel as ? That is, > > > > > > is > > > > > > the > > > > > > > > > > Would be a larger style bug, especially if it were actually > > > > > used. > > > > > Even its spellings of TRUE and FALSE are strange. Even in > > > > > userland > > > > > stdbool.h is considered so useful that it is never used in > > > > > src/bin > > > > > and is only used a few times on other src/*bin. src/bin never > > > > > uses > > > > > TRUE of FALSE either. > > > > > > > > I suspect there is some bias here though due to the fact that > > > > there > > > > wasn't > > > > a standard bool type when most of this code was written. :) I > > > > don't > > > > think > > > > that means we have to forgo use of the new type now that it is > > > > in > > > > fact > > > > standardized in C99. I would be happy to have 'bool' available > > > > and > > > > the > > > > lowercase 'true' and 'false' are fine with me. > > > > > > The lowercase 'true' and 'false' are intended to mimic C++, where > > > they are keywords. Regardless of how you prefer to capitalize > > > them, using them instead of 0 and 1 makes the intent much clearer. > > > This is especially true in the kernel, where non-zero could mean > > > true, or it could be an error code. > > > > > > Unfortunately, the "new type" is mostly useless, aside from > > > improving readability. Unlike modern languages, C doesn't > > > consider it a compile-time error to mix up bools and ints. > > > > > If this is added, would the style gods approve of the following: > > > > (A) bool test_func(); > > > > if (test_func()) > > ... > > > > instead of: > > > > (B) int test_func(); > > > > if (test_func() != 0) > > ... > > > > Personally, I prefer the former, but understand that it isn't > > currently style(9) compliant. Being able to do (A) instead of > > (B) would be why I'd like stdbool.h to be added to the kernel, > > if it will be allowed after the change? > > My understanding is that (A) is fine if test_func() is returning an > actual boolean value (so the return value is only true and false). > A case where this isn't really true is when the function returns an > errno as that is returning an int with many values other than just > 0 and 1. Granted, it is a common style violation (I am guilty) to do: > > if (error) > return (error); > Yes, guilty as well. Once I became aware that it was a style(9) violation I've resisted doing it, but barely;-) Thanks for the commments, rick > I also consider it a boolean test to test a single-bit flag in a flags > field: > > if (p->p_flag & P_PROFIL) > > vs. > > if ((p->p_flag & P_PROFIL) != 0) > > If you had a multi-bit field though then I think it is appropriate to > be more > explicit about which values of that field are being tested for. Thus: > > #define FLAGS_SIMPLE 0x00001 > #define FLAGS_FIELD 0x000fe > > if (foo & FLAGS_SIMPLE) > ... > > if ((foo & FLAGS_FIELD) != 0) > ... > > -- > John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Dec 1 15:44:00 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EEF81106566C; Thu, 1 Dec 2011 15:44:00 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from argol.doit.wisc.edu (argol.doit.wisc.edu [144.92.197.212]) by mx1.freebsd.org (Postfix) with ESMTP id BC1A88FC13; Thu, 1 Dec 2011 15:44:00 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from avs-daemon.smtpauth3.wiscmail.wisc.edu by smtpauth3.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) id <0LVJ00H024XB9U00@smtpauth3.wiscmail.wisc.edu>; Thu, 01 Dec 2011 08:43:59 -0600 (CST) Received: from comporellon.tachypleus.net ([unknown] [76.210.61.201]) by smtpauth3.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) with ESMTPSA id <0LVJ006GK4X9F330@smtpauth3.wiscmail.wisc.edu>; Thu, 01 Dec 2011 08:43:57 -0600 (CST) Date: Thu, 01 Dec 2011 08:43:56 -0600 From: Nathan Whitehorn In-reply-to: To: alc@freebsd.org Message-id: <4ED792AC.4000501@freebsd.org> X-Spam-Report: AuthenticatedSender=yes, SenderIP=76.210.61.201 X-Spam-PmxInfo: Server=avs-14, Version=5.6.1.2065439, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2011.12.1.143315, SenderIP=76.210.61.201 References: <4ED5BE19.70805@fgznet.ch> <20111130162236.GA50300@deviant.kiev.zoral.com.ua> <4ED65F70.7050700@fgznet.ch> <20111130170936.GB50300@deviant.kiev.zoral.com.ua> <4ED66B75.3060409@fgznet.ch> User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111113 Thunderbird/8.0 Cc: Kostik Belousov , Andreas Tobler , Alan Cox , FreeBSD Arch Subject: Re: powerpc64 malloc limit? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Dec 2011 15:44:01 -0000 On 11/30/11 15:50, Alan Cox wrote: > On Wed, Nov 30, 2011 at 12:12 PM, Peter Wemm wrote: > >> On Wed, Nov 30, 2011 at 9:44 AM, Andreas Tobler >> wrote: >>> On 30.11.11 18:09, Kostik Belousov wrote: >>>> On Wed, Nov 30, 2011 at 05:53:04PM +0100, Andreas Tobler wrote: >>>>> On 30.11.11 17:22, Kostik Belousov wrote: >>>>>> On Wed, Nov 30, 2011 at 06:24:41AM +0100, Andreas Tobler wrote: >>>>>>> All, >>>>>>> >>>>>>> while working on gcc I found a very strange situation which renders >> my >>>>>>> powerpc64 machine unusable. >>>>>>> The test case below tries to allocate that much memory as 'wanted'. >> The >>>>>>> same test case on amd64 returns w/o trying to allocate mem because >> the >>>>>>> size is far to big. >>>>>>> >>>>>>> I couldn't find the reason so far, that's why I'm here. >>>>>>> >>>>>>> As Nathan pointed out the VM_MAXUSER_SIZE is the biggest on >> powerpc64: >>>>>>> #define VM_MAXUSER_ADDRESS (0x7ffffffffffff000UL) >>>>>>> >>>>>>> So, I'd expect a system to return an allocation error when a user >> tries >>>>>>> to allocate too much memory and not really trying it and going to be >>>>>>> unusable. Iow, I'd exepect the situation on powerpc64 as I see on >>>>>>> amd64. >>>>>>> >>>>>>> Can anybody explain me the situation, why do I not have a working >> limit >>>>>>> on powerpc64? >>>>>>> >>>>>>> The machine itself has 7GB RAM and 12GB swap. The amd64 where I >>>>>>> compared >>>>>>> has around 4GB/4GB RAM/swap. >>>>>>> >>>>>>> TIA, >>>>>>> Andreas >>>>>>> >>>>>>> include >>>>>>> #include >>>>>>> >>>>>>> int main() >>>>>>> { >>>>>>> void *p; >>>>>>> >>>>>>> p = (void*) malloc (1152921504606846968ULL); >>>>>>> if (p != NULL) >>>>>>> printf("p = %p\n", p); >>>>>>> >>>>>>> printf("p = %p\n", p); >>>>>>> return (0); >>>>>>> } >>>>>> First, you should provide details of what consistutes 'the unusable >>>>>> machine situation' on powerpc. >>>>> I can not login anymore, everything is stuck except the core control >>>>> mechanisms for example the fan controller. >>>>> >>>>> Top reports 'ugly' figures, below from a earlier try: >>>>> >>>>> last pid: 6790; load averages: 0.78, 0.84, 0.86 up 0+00:34:52 >>>>> 22:42:29 47 processes: 1 running, 46 sleeping >>>>> CPU: 0.0% user, 0.0% nice, 15.4% system, 11.8% interrupt, 72.8% idle >>>>> Mem: 5912M Active, 570M Inact, 280M Wired, 26M Cache, 104M Buf, 352K >> Free >>>>> Swap: 12G Total, 9904M Used, 2383M Free, 80% Inuse, 178M Out >>>>> >>>>> PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU >>>>> COMMAND >>>>> 6768 andreast 1 52 01073741824G 6479M pfault 1 0:58 >>>>> 18.90% 31370. >>>>> >>>>> And after my mem and swap are full I see swap_pager_getswapspace(16) >>>>> failed. >>>>> >>>>> In this state I can only power-cycle the machine. >>>>> >>>>>> That said, on amd64 the user map is between 0 and 0x7fffffffffff, >> which >>>>>> obviously less then the requested allocation size 0x100000000000000. >>>>>> If you look at the kdump output on amd64, you will see that malloc() >>>>>> tries to mmap() the area, fails and retries with obreak(). Default >>>>>> virtual memory limit is unlimited, so my best quess is that on amd64 >>>>>> vm_map_findspace() returns immediately. >>>>>> >>>>>> On powerpc64, I see no reason why vm_map_entry cannot be allocated, >> but >>>>>> please note that vm object and pages shall be only allocated on >> demand. >>>>>> So I am curious how does your machine breaks and where. >>>>> I would expect that the 'system' does not allow me to allocate that >> much >>>>> of ram. >>>> Does the issue with machine going into limbo reproducable with the code >>>> you posted ? >>> If I understand you correctly, yes. I can launch the test case and the >>> machine is immediately unusable. Means I can not kill the process nor >> can I >>> log in. Also, top does not show anything useful. >>> >>> The original test case where I discovered this behavior behaves a bit >>> different. >>> >> http://gcc.gnu.org/viewcvs/trunk/libstdc%2B%2B-v3/testsuite/23_containers/vector/bool/modifiers/insert/31370.cc?revision=169421&view=markup >>> Here I can follow how the ram and swap is eaten up. Top is reporting the >>> figures. If everything is 'full', the swaper errors start to appear on >> the >>> console. >>> >>>> Or, do you need to actually touch the pages in the allocated region ? >>> If I have to, how would I do that? >>> >>>> If the later (and I do expect that), then how many pages do you need >>>> to touch before machine breaks ? Is it single access that causes the >>>> havoc, or you need to touch the amount approximately equal to RAM+swap ? >>> Andreas >> ia64 had some vaguely related excitement earlier in its life. If >> you created a 1TB sparse file and mmap'ed it over and over, tens, >> maybe hundreds of thousands of times, certain VM internal state got >> way out of hand. mmaping was fine, but unmapping took 36 hours of cpu >> runtime when I killed the process. It got so far out of hand because >> of the way ia64 handled just-in-time mappings on vhpt misses. >> >> > There is a fundamental scalability problem with the powerpc64/aim pmap. > See revision 212360. In a nutshell, unlike amd64, ia64, and most other > pmap implementations, the powerpc64/aim pmap implementation doesn't link > together all of the pv entries belonging to a pmap into a list. So, the > powerpc64/aim implementations of range operations, like pmap_remove(), > don't handle large, sparsely populated ranges as efficiently as ia64 does. > Moreover, powerpc64/aim can't effectively implement pmap_remove_pages(), > and so it doesn't even try. > This is really irritating to fix. The PMAP layer is really designed around a tree-based page table layout, which PowerPC/AIM does not have. One fix for at least some issues might be to also link the PVO structures into the pmap -- some things would not be efficient, since it would be a flat list, but it's at least not that complicated. -Nathan From owner-freebsd-arch@FreeBSD.ORG Thu Dec 1 16:11:28 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 823EA1065675 for ; Thu, 1 Dec 2011 16:11:28 +0000 (UTC) (envelope-from peter@wemm.org) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 254428FC0C for ; Thu, 1 Dec 2011 16:11:27 +0000 (UTC) Received: by iakl21 with SMTP id l21so4186576iak.13 for ; Thu, 01 Dec 2011 08:11:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wemm.org; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=0p8mivGhTQcRgDo4pmGSbPhYj7zHLeDOR+6vobNtZyM=; b=QbRy8zuehZCAMk+YMV3SPKMsmZMibKiE0v7u+htsGPgLyKUzR/AgjSBKpd+3aV9XnP ZLcsuymrlm6UdN7/rfS7SS7+SyvTpCpb64uPjXkKx2O8z6yJidE86U0tvWbaiZUJft6c 9rWe/JAkhRLTNK7xujLwcHw5Shoarx6y7adSw= MIME-Version: 1.0 Received: by 10.43.44.199 with SMTP id uh7mr8896917icb.25.1322755887450; Thu, 01 Dec 2011 08:11:27 -0800 (PST) Received: by 10.50.135.106 with HTTP; Thu, 1 Dec 2011 08:11:27 -0800 (PST) In-Reply-To: <4ED792AC.4000501@freebsd.org> References: <4ED5BE19.70805@fgznet.ch> <20111130162236.GA50300@deviant.kiev.zoral.com.ua> <4ED65F70.7050700@fgznet.ch> <20111130170936.GB50300@deviant.kiev.zoral.com.ua> <4ED66B75.3060409@fgznet.ch> <4ED792AC.4000501@freebsd.org> Date: Thu, 1 Dec 2011 08:11:27 -0800 Message-ID: From: Peter Wemm To: Nathan Whitehorn Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: alc@freebsd.org, Kostik Belousov , Alan Cox , Andreas Tobler , FreeBSD Arch Subject: Re: powerpc64 malloc limit? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Dec 2011 16:11:28 -0000 On Thu, Dec 1, 2011 at 6:43 AM, Nathan Whitehorn w= rote: > On 11/30/11 15:50, Alan Cox wrote: >> >> On Wed, Nov 30, 2011 at 12:12 PM, Peter Wemm =A0wrote: >> >>> On Wed, Nov 30, 2011 at 9:44 AM, Andreas Tobler >>> wrote: >>>> >>>> On 30.11.11 18:09, Kostik Belousov wrote: >>>>> >>>>> On Wed, Nov 30, 2011 at 05:53:04PM +0100, Andreas Tobler wrote: >>>>>> >>>>>> On 30.11.11 17:22, Kostik Belousov wrote: >>>>>>> >>>>>>> On Wed, Nov 30, 2011 at 06:24:41AM +0100, Andreas Tobler wrote: >>>>>>>> >>>>>>>> All, >>>>>>>> >>>>>>>> while working on gcc I found a very strange situation which render= s >>> >>> my >>>>>>>> >>>>>>>> powerpc64 machine unusable. >>>>>>>> The test case below tries to allocate that much memory as 'wanted'= . >>> >>> The >>>>>>>> >>>>>>>> same test case on amd64 returns w/o trying to allocate mem because >>> >>> the >>>>>>>> >>>>>>>> size is far to big. >>>>>>>> >>>>>>>> I couldn't find the reason so far, that's why I'm here. >>>>>>>> >>>>>>>> As Nathan pointed out the VM_MAXUSER_SIZE is the biggest on >>> >>> powerpc64: >>>>>>>> >>>>>>>> #define VM_MAXUSER_ADDRESS =A0 =A0 =A0(0x7ffffffffffff000UL) >>>>>>>> >>>>>>>> So, I'd expect a system to return an allocation error when a user >>> >>> tries >>>>>>>> >>>>>>>> to allocate too much memory and not really trying it and going to = be >>>>>>>> unusable. Iow, I'd exepect the situation on powerpc64 as I see on >>>>>>>> amd64. >>>>>>>> >>>>>>>> Can anybody explain me the situation, why do I not have a working >>> >>> limit >>>>>>>> >>>>>>>> on powerpc64? >>>>>>>> >>>>>>>> The machine itself has 7GB RAM and 12GB swap. The amd64 where I >>>>>>>> compared >>>>>>>> has around 4GB/4GB RAM/swap. >>>>>>>> >>>>>>>> TIA, >>>>>>>> Andreas >>>>>>>> >>>>>>>> include >>>>>>>> #include >>>>>>>> >>>>>>>> int main() >>>>>>>> { >>>>>>>> =A0 =A0 =A0 =A0 =A0void *p; >>>>>>>> >>>>>>>> =A0 =A0 =A0 =A0 =A0p =3D (void*) malloc (1152921504606846968ULL); >>>>>>>> =A0 =A0 =A0 =A0 =A0if (p !=3D NULL) >>>>>>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0printf("p =3D %p\n", p); >>>>>>>> >>>>>>>> =A0 =A0 =A0 =A0 =A0printf("p =3D %p\n", p); >>>>>>>> =A0 =A0 =A0 =A0 =A0return (0); >>>>>>>> } >>>>>>> >>>>>>> First, you should provide details of what consistutes 'the unusable >>>>>>> machine situation' on powerpc. >>>>>> >>>>>> I can not login anymore, everything is stuck except the core control >>>>>> mechanisms for example the fan controller. >>>>>> >>>>>> Top reports 'ugly' figures, below from a earlier try: >>>>>> >>>>>> last pid: =A06790; =A0load averages: =A00.78, =A00.84, =A00.86 =A0 = =A0up 0+00:34:52 >>>>>> 22:42:29 47 processes: =A01 running, 46 sleeping >>>>>> CPU: =A00.0% user, =A00.0% nice, 15.4% system, 11.8% interrupt, 72.8= % idle >>>>>> Mem: 5912M Active, 570M Inact, 280M Wired, 26M Cache, 104M Buf, 352K >>> >>> Free >>>>>> >>>>>> Swap: 12G Total, 9904M Used, 2383M Free, 80% Inuse, 178M Out >>>>>> >>>>>> =A0 =A0PID USERNAME =A0 =A0THR PRI NICE =A0 SIZE =A0 =A0RES STATE = =A0 C =A0 TIME =A0 WCPU >>>>>> COMMAND >>>>>> =A0 6768 andreast =A0 =A0 =A01 =A052 =A0 =A001073741824G =A06479M pf= ault =A01 =A0 0:58 >>>>>> 18.90% 31370. >>>>>> >>>>>> And after my mem and swap are full I see swap_pager_getswapspace(16) >>>>>> failed. >>>>>> >>>>>> In this state I can only power-cycle the machine. >>>>>> >>>>>>> That said, on amd64 the user map is between 0 and 0x7fffffffffff, >>> >>> which >>>>>>> >>>>>>> obviously less then the requested allocation size 0x100000000000000= . >>>>>>> If you look at the kdump output on amd64, you will see that malloc(= ) >>>>>>> tries to mmap() the area, fails and retries with obreak(). Default >>>>>>> virtual memory limit is unlimited, so my best quess is that on amd6= 4 >>>>>>> vm_map_findspace() returns immediately. >>>>>>> >>>>>>> On powerpc64, I see no reason why vm_map_entry cannot be allocated, >>> >>> but >>>>>>> >>>>>>> please note that vm object and pages shall be only allocated on >>> >>> demand. >>>>>>> >>>>>>> So I am curious how does your machine breaks and where. >>>>>> >>>>>> I would expect that the 'system' does not allow me to allocate that >>> >>> much >>>>>> >>>>>> of ram. >>>>> >>>>> Does the issue with machine going into limbo reproducable with the co= de >>>>> you posted ? >>>> >>>> If I understand you correctly, yes. I can launch the test case and the >>>> machine is immediately unusable. Means I can not kill the process nor >>> >>> can I >>>> >>>> log in. Also, top does not show anything useful. >>>> >>>> The original test case where I discovered this behavior behaves a bit >>>> different. >>>> >>> >>> http://gcc.gnu.org/viewcvs/trunk/libstdc%2B%2B-v3/testsuite/23_containe= rs/vector/bool/modifiers/insert/31370.cc?revision=3D169421&view=3Dmarkup >>>> >>>> Here I can follow how the ram and swap is eaten up. Top is reporting t= he >>>> figures. If everything is 'full', the swaper errors start to appear on >>> >>> the >>>> >>>> console. >>>> >>>>> Or, do you need to actually touch the pages in the allocated region ? >>>> >>>> If I have to, how would I do that? >>>> >>>>> If the later (and I do expect that), then how many pages do you need >>>>> to touch before machine breaks ? Is it single access that causes the >>>>> havoc, or you need to touch the amount approximately equal to RAM+swa= p >>>>> ? >>>> >>>> Andreas >>> >>> ia64 had some vaguely related excitement earlier in its life. =A0 =A0If >>> you created a 1TB sparse file and mmap'ed it over and over, tens, >>> maybe hundreds of thousands of times, certain VM internal state got >>> way out of hand. =A0mmaping was fine, but unmapping took 36 hours of cp= u >>> runtime when I killed the process. =A0It got so far out of hand because >>> of the way ia64 handled just-in-time mappings on vhpt misses. >>> >>> >> There is a fundamental scalability problem with the powerpc64/aim pmap. >> See revision 212360. =A0In a nutshell, unlike amd64, ia64, and most othe= r >> pmap implementations, the powerpc64/aim pmap implementation doesn't link >> together all of the pv entries belonging to a pmap into a list. =A0So, t= he >> powerpc64/aim implementations of range operations, like pmap_remove(), >> don't handle large, sparsely populated ranges as efficiently as ia64 doe= s. >> Moreover, powerpc64/aim can't effectively implement pmap_remove_pages(), >> and so it doesn't even try. >> > > This is really irritating to fix. The PMAP layer is really designed aroun= d a > tree-based page table layout, which PowerPC/AIM does not have. One fix fo= r > at least some issues might be to also link the PVO structures into the pm= ap > -- some things would not be efficient, since it would be a flat list, but > it's at least not that complicated. > -Nathan It's not entirely that simple. The pmap layer exists so that the upper level vm layers don't have to learn anything about page trees. Its already been shown to work on tree-less systems like ia64. On MIPS, you could, if you wished, write a TLB miss handler that parsed the upper layer vm structures if you were crazy enough. So no, the pmap interface isn't designed around assuming trees. Its just that the most often used pmap implementations happen to use them. But not all. The upper layers do make some assumptions about what operations are cheap vs expensive though and on other pmaps we made implementation changes to support observed use cases. The scenarios that Alan mentioned remind me of what John Dyson was working on back in the late 90's when working with things like buildworld speed (fork/exec/exit) and ld.so mmap/munmap speed. In particular a second linked list was added to pv entries to speed up an operation that needed to be faster and API's were added to access the speedup. That was more about speeding up pv entry management than assuming trees. This particular address space rundown optimization doesn't seem to exist in some of the other BSD platform ports that people have used as models. There's another optimization that Apple uses that we don't. They added an extra has in there somewhere to deal with a certain scalability issue when pv entry chains get very long due to their prelinked shared library stuff. --=20 Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell