From owner-freebsd-arch@FreeBSD.ORG Wed Nov 30 21:24:58 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A957C106564A for ; Wed, 30 Nov 2011 21:24:58 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 7F7918FC14 for ; Wed, 30 Nov 2011 21:24:57 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pAULOd1j059043 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 30 Nov 2011 23:24:39 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pAULOdsB080128; Wed, 30 Nov 2011 23:24:39 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pAULOdQ1080127; Wed, 30 Nov 2011 23:24:39 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 30 Nov 2011 23:24:39 +0200 From: Kostik Belousov To: Andreas Tobler Message-ID: <20111130212439.GF50300@deviant.kiev.zoral.com.ua> References: <4ED5BE19.70805@fgznet.ch> <20111130162236.GA50300@deviant.kiev.zoral.com.ua> <4ED65F70.7050700@fgznet.ch> <20111130170936.GB50300@deviant.kiev.zoral.com.ua> <4ED66B75.3060409@fgznet.ch> <20111130200103.GE50300@deviant.kiev.zoral.com.ua> <4ED698EB.8090904@fgznet.ch> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="trzpwzIgNM352Iuf" Content-Disposition: inline In-Reply-To: <4ED698EB.8090904@fgznet.ch> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: FreeBSD Arch Subject: Re: powerpc64 malloc limit? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Nov 2011 21:24:58 -0000 --trzpwzIgNM352Iuf Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Nov 30, 2011 at 09:58:19PM +0100, Andreas Tobler wrote: > On 30.11.11 21:01, Kostik Belousov wrote: > >On Wed, Nov 30, 2011 at 06:44:21PM +0100, Andreas Tobler wrote: > >>On 30.11.11 18:09, Kostik Belousov wrote: > >>>On Wed, Nov 30, 2011 at 05:53:04PM +0100, Andreas Tobler wrote: > >>>>On 30.11.11 17:22, Kostik Belousov wrote: > >>>>>On Wed, Nov 30, 2011 at 06:24:41AM +0100, Andreas Tobler wrote: > >>>>>>All, > >>>>>> > >>>>>>while working on gcc I found a very strange situation which renders= my > >>>>>>powerpc64 machine unusable. > >>>>>>The test case below tries to allocate that much memory as 'wanted'.= =20 > >>>>>>The > >>>>>>same test case on amd64 returns w/o trying to allocate mem because = the > >>>>>>size is far to big. > >>>>>> > >>>>>>I couldn't find the reason so far, that's why I'm here. > >>>>>> > >>>>>>As Nathan pointed out the VM_MAXUSER_SIZE is the biggest on powerpc= 64: > >>>>>>#define VM_MAXUSER_ADDRESS (0x7ffffffffffff000UL) > >>>>>> > >>>>>>So, I'd expect a system to return an allocation error when a user= =20 > >>>>>>tries > >>>>>>to allocate too much memory and not really trying it and going to be > >>>>>>unusable. Iow, I'd exepect the situation on powerpc64 as I see on= =20 > >>>>>>amd64. > >>>>>> > >>>>>>Can anybody explain me the situation, why do I not have a working= =20 > >>>>>>limit > >>>>>>on powerpc64? > >>>>>> > >>>>>>The machine itself has 7GB RAM and 12GB swap. The amd64 where I=20 > >>>>>>compared > >>>>>>has around 4GB/4GB RAM/swap. > >>>>>> > >>>>>>TIA, > >>>>>>Andreas > >>>>>> > >>>>>>include > >>>>>>#include > >>>>>> > >>>>>>int main() > >>>>>>{ > >>>>>> void *p; > >>>>>> > >>>>>> p =3D (void*) malloc (1152921504606846968ULL); > >>>>>> if (p !=3D NULL) > >>>>>> printf("p =3D %p\n", p); > >>>>>> > >>>>>> printf("p =3D %p\n", p); > >>>>>> return (0); > >>>>>>} > >>>>> > >>>>>First, you should provide details of what consistutes 'the unusable > >>>>>machine situation' on powerpc. > >>>> > >>>>I can not login anymore, everything is stuck except the core control > >>>>mechanisms for example the fan controller. > >>>> > >>>>Top reports 'ugly' figures, below from a earlier try: > >>>> > >>>>last pid: 6790; load averages: 0.78, 0.84, 0.86 up 0+00:34:52 > >>>>22:42:29 47 processes: 1 running, 46 sleeping > >>>>CPU: 0.0% user, 0.0% nice, 15.4% system, 11.8% interrupt, 72.8% idle > >>>>Mem: 5912M Active, 570M Inact, 280M Wired, 26M Cache, 104M Buf, 352K= =20 > >>>>Free > >>>>Swap: 12G Total, 9904M Used, 2383M Free, 80% Inuse, 178M Out > >>>> > >>>> PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU > >>>>COMMAND > >>>> 6768 andreast 1 52 01073741824G 6479M pfault 1 0:58 > >>>>18.90% 31370. > >>>> > >>>>And after my mem and swap are full I see swap_pager_getswapspace(16) > >>>>failed. > >>>> > >>>>In this state I can only power-cycle the machine. > >>>> > >>>>>That said, on amd64 the user map is between 0 and 0x7fffffffffff, wh= ich > >>>>>obviously less then the requested allocation size 0x100000000000000. > >>>>>If you look at the kdump output on amd64, you will see that malloc() > >>>>>tries to mmap() the area, fails and retries with obreak(). Default > >>>>>virtual memory limit is unlimited, so my best quess is that on amd64 > >>>>>vm_map_findspace() returns immediately. > >>>>> > >>>>>On powerpc64, I see no reason why vm_map_entry cannot be allocated, = but > >>>>>please note that vm object and pages shall be only allocated on dema= nd. > >>>>>So I am curious how does your machine breaks and where. > >>>> > >>>>I would expect that the 'system' does not allow me to allocate that m= uch > >>>>of ram. > >>> > >>>Does the issue with machine going into limbo reproducable with the code > >>>you posted ? > >> > >>If I understand you correctly, yes. I can launch the test case and the > >>machine is immediately unusable. Means I can not kill the process nor > >>can I log in. Also, top does not show anything useful. > >Again, let me restate my question: the single mmap() of the huge size is > >enough for powerpc64 machine to break apart ? >=20 > I can't answer. I don't know yet. >=20 > >What happen if you insert sleep(1000000); call before return ? Do not ki= ll > >the process, I want to know is machine dead while the process sleeps. >=20 > Ok, during the 'sleep' the machine is usable. top is reporting figures,= =20 > I can log in and edit files. The process runs now for aboutt 30'. >=20 > When I kill the process, I do not get back to the shell nor can I log=20 > in. Also top stops reporting. > But as you said, I didn't kill in this run. Then, as Alan Cox pointed out, caused by the approach taken in powerpc64 pmap to handle pmap_remove(). It is definitely arch-specific. >=20 > Thanks, > Andreas >=20 > [bohrium:~] andreast% ./mega_malloc > p =3D 0x50400000 > p =3D 0x50400000 > ... >=20 > last pid: 1279; load averages: 0.05, 0.03, 0.04 up 0+00:30:34=20 > 21:56:40 > 41 processes: 1 running, 40 sleeping > CPU: 0.0% user, 0.0% nice, 1.2% system, 0.2% interrupt, 98.6% idle > Mem: 28M Active, 26M Inact, 89M Wired, 8K Cache, 725M Buf, 6642M Free > Swap: 12G Total, 12G Free >=20 > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU=20 > COMMAND > 1219 andreast 1 24 01073741824G 1608K nanslp 0 0:00=20 > 0.00% mega_m --trzpwzIgNM352Iuf Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk7WnxYACgkQC3+MBN1Mb4jj4ACgxe5J5gtPIAZmCYXpwwIID/Ri B+gAoNAQfNLTu7EOWf6U5gsCD52HLaol =ii8m -----END PGP SIGNATURE----- --trzpwzIgNM352Iuf--