From owner-freebsd-arch@FreeBSD.ORG Wed Nov 30 22:08:21 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CDB4C106566C for ; Wed, 30 Nov 2011 22:08:21 +0000 (UTC) (envelope-from andreast-list@fgznet.ch) Received: from smtp.fgznet.ch (mail.fgznet.ch [81.92.96.47]) by mx1.freebsd.org (Postfix) with ESMTP id 2DCBE8FC0A for ; Wed, 30 Nov 2011 22:08:20 +0000 (UTC) Received: from deuterium.andreas.nets (dhclient-91-190-14-19.flashcable.ch [91.190.14.19]) by smtp.fgznet.ch (8.13.8/8.13.8/Submit_SMTPAUTH) with ESMTP id pAUM78pC048334; Wed, 30 Nov 2011 23:07:09 +0100 (CET) (envelope-from andreast-list@fgznet.ch) Message-ID: <4ED6A951.4070202@fgznet.ch> Date: Wed, 30 Nov 2011 23:08:17 +0100 From: Andreas Tobler User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: alc@freebsd.org References: <4ED5BE19.70805@fgznet.ch> <20111130162236.GA50300@deviant.kiev.zoral.com.ua> <4ED65F70.7050700@fgznet.ch> <20111130170936.GB50300@deviant.kiev.zoral.com.ua> <4ED66B75.3060409@fgznet.ch> <20111130200103.GE50300@deviant.kiev.zoral.com.ua> <4ED698EB.8090904@fgznet.ch> <20111130212439.GF50300@deviant.kiev.zoral.com.ua> <4ED6A36D.1050107@fgznet.ch> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.64 on 81.92.96.47 Cc: Kostik Belousov , Alan Cox , FreeBSD Arch Subject: Re: powerpc64 malloc limit? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Nov 2011 22:08:21 -0000 On 30.11.11 22:59, Alan Cox wrote: > On Wed, Nov 30, 2011 at 3:43 PM, Andreas Tobler > wrote: > > On 30.11.11 22:24, Kostik Belousov wrote: > > On Wed, Nov 30, 2011 at 09:58:19PM +0100, Andreas Tobler wrote: > > On 30.11.11 21:01, Kostik Belousov wrote: > > On Wed, Nov 30, 2011 at 06:44:21PM +0100, Andreas Tobler > wrote: > > On 30.11.11 18:09, Kostik Belousov wrote: > > On Wed, Nov 30, 2011 at 05:53:04PM +0100, > Andreas Tobler wrote: > > On 30.11.11 17:22, Kostik Belousov wrote: > > On Wed, Nov 30, 2011 at 06:24:41AM > +0100, Andreas Tobler wrote: > > All, > > while working on gcc I found a very > strange situation which renders my > powerpc64 machine unusable. > The test case below tries to > allocate that much memory as 'wanted'. > The > same test case on amd64 returns w/o > trying to allocate mem because the > size is far to big. > > I couldn't find the reason so far, > that's why I'm here. > > As Nathan pointed out the > VM_MAXUSER_SIZE is the biggest on > powerpc64: > #define VM_MAXUSER_ADDRESS > (0x7ffffffffffff000UL) > > So, I'd expect a system to return an > allocation error when a user > tries > to allocate too much memory and not > really trying it and going to be > unusable. Iow, I'd exepect the > situation on powerpc64 as I see on > amd64. > > Can anybody explain me the > situation, why do I not have a working > limit > on powerpc64? > > The machine itself has 7GB RAM and > 12GB swap. The amd64 where I > compared > has around 4GB/4GB RAM/swap. > > TIA, > Andreas > > include > #include > > int main() > { > void *p; > > p = (void*) malloc > (1152921504606846968ULL); > if (p != NULL) > printf("p = %p\n", p); > > printf("p = %p\n", p); > return (0); > } > > > First, you should provide details of > what consistutes 'the unusable > machine situation' on powerpc. > > > I can not login anymore, everything is stuck > except the core control > mechanisms for example the fan controller. > > Top reports 'ugly' figures, below from a > earlier try: > > last pid: 6790; load averages: 0.78, > 0.84, 0.86 up 0+00:34:52 > 22:42:29 47 processes: 1 running, 46 sleeping > CPU: 0.0% user, 0.0% nice, 15.4% system, > 11.8% interrupt, 72.8% idle > Mem: 5912M Active, 570M Inact, 280M Wired, > 26M Cache, 104M Buf, 352K > Free > Swap: 12G Total, 9904M Used, 2383M Free, 80% > Inuse, 178M Out > > PID USERNAME THR PRI NICE SIZE > RES STATE C TIME WCPU > COMMAND > 6768 andreast 1 52 01073741824G > 6479M pfault 1 0:58 > 18.90% 31370. > > And after my mem and swap are full I see > swap_pager_getswapspace(16) > failed. > > In this state I can only power-cycle the > machine. > > That said, on amd64 the user map is > between 0 and 0x7fffffffffff, which > obviously less then the requested > allocation size 0x100000000000000. > If you look at the kdump output on > amd64, you will see that malloc() > tries to mmap() the area, fails and > retries with obreak(). Default > virtual memory limit is unlimited, so my > best quess is that on amd64 > vm_map_findspace() returns immediately. > > On powerpc64, I see no reason why > vm_map_entry cannot be allocated, but > please note that vm object and pages > shall be only allocated on demand. > So I am curious how does your machine > breaks and where. > > > I would expect that the 'system' does not > allow me to allocate that much > of ram. > > > Does the issue with machine going into limbo > reproducable with the code > you posted ? > > > If I understand you correctly, yes. I can launch the > test case and the > machine is immediately unusable. Means I can not > kill the process nor > can I log in. Also, top does not show anything useful. > > Again, let me restate my question: the single mmap() of > the huge size is > enough for powerpc64 machine to break apart ? > > > I can't answer. I don't know yet. > > What happen if you insert sleep(1000000); call before > return ? Do not kill > the process, I want to know is machine dead while the > process sleeps. > > > Ok, during the 'sleep' the machine is usable. top is > reporting figures, > I can log in and edit files. The process runs now for aboutt > 30'. > > When I kill the process, I do not get back to the shell nor > can I log > in. Also top stops reporting. > But as you said, I didn't kill in this run. > > Then, as Alan Cox pointed out, caused by the approach taken in > powerpc64 > pmap to handle pmap_remove(). It is definitely arch-specific. > > > Ok. I think you mean moea64_remove which is pmap_remove, right? > > Where did Alan pointed this out? > > > I was in a rush earlier, so I sent a short, cryptic note to Kostik > privately. Thank you. I have to talk with Nathan then. Well, I only wanted to make a test case work and I went through stages where I had to make gdb usable, assembler and linker adaptations, gcc improvments and now it seems that I have to dive into pmap & co :) A real exciting spare time business. Regards, Andreas