Date: Wed, 30 Nov 2011 23:24:39 +0200 From: Kostik Belousov <kostikbel@gmail.com> To: Andreas Tobler <andreast-list@fgznet.ch> Cc: FreeBSD Arch <freebsd-arch@freebsd.org> Subject: Re: powerpc64 malloc limit? Message-ID: <20111130212439.GF50300@deviant.kiev.zoral.com.ua> In-Reply-To: <4ED698EB.8090904@fgznet.ch> References: <4ED5BE19.70805@fgznet.ch> <20111130162236.GA50300@deviant.kiev.zoral.com.ua> <4ED65F70.7050700@fgznet.ch> <20111130170936.GB50300@deviant.kiev.zoral.com.ua> <4ED66B75.3060409@fgznet.ch> <20111130200103.GE50300@deviant.kiev.zoral.com.ua> <4ED698EB.8090904@fgznet.ch>
next in thread | previous in thread | raw e-mail | index | archive | help
--trzpwzIgNM352Iuf
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Wed, Nov 30, 2011 at 09:58:19PM +0100, Andreas Tobler wrote:
> On 30.11.11 21:01, Kostik Belousov wrote:
> >On Wed, Nov 30, 2011 at 06:44:21PM +0100, Andreas Tobler wrote:
> >>On 30.11.11 18:09, Kostik Belousov wrote:
> >>>On Wed, Nov 30, 2011 at 05:53:04PM +0100, Andreas Tobler wrote:
> >>>>On 30.11.11 17:22, Kostik Belousov wrote:
> >>>>>On Wed, Nov 30, 2011 at 06:24:41AM +0100, Andreas Tobler wrote:
> >>>>>>All,
> >>>>>>
> >>>>>>while working on gcc I found a very strange situation which renders=
my
> >>>>>>powerpc64 machine unusable.
> >>>>>>The test case below tries to allocate that much memory as 'wanted'.=
=20
> >>>>>>The
> >>>>>>same test case on amd64 returns w/o trying to allocate mem because =
the
> >>>>>>size is far to big.
> >>>>>>
> >>>>>>I couldn't find the reason so far, that's why I'm here.
> >>>>>>
> >>>>>>As Nathan pointed out the VM_MAXUSER_SIZE is the biggest on powerpc=
64:
> >>>>>>#define VM_MAXUSER_ADDRESS (0x7ffffffffffff000UL)
> >>>>>>
> >>>>>>So, I'd expect a system to return an allocation error when a user=
=20
> >>>>>>tries
> >>>>>>to allocate too much memory and not really trying it and going to be
> >>>>>>unusable. Iow, I'd exepect the situation on powerpc64 as I see on=
=20
> >>>>>>amd64.
> >>>>>>
> >>>>>>Can anybody explain me the situation, why do I not have a working=
=20
> >>>>>>limit
> >>>>>>on powerpc64?
> >>>>>>
> >>>>>>The machine itself has 7GB RAM and 12GB swap. The amd64 where I=20
> >>>>>>compared
> >>>>>>has around 4GB/4GB RAM/swap.
> >>>>>>
> >>>>>>TIA,
> >>>>>>Andreas
> >>>>>>
> >>>>>>include<stdlib.h>
> >>>>>>#include<stdio.h>
> >>>>>>
> >>>>>>int main()
> >>>>>>{
> >>>>>> void *p;
> >>>>>>
> >>>>>> p =3D (void*) malloc (1152921504606846968ULL);
> >>>>>> if (p !=3D NULL)
> >>>>>> printf("p =3D %p\n", p);
> >>>>>>
> >>>>>> printf("p =3D %p\n", p);
> >>>>>> return (0);
> >>>>>>}
> >>>>>
> >>>>>First, you should provide details of what consistutes 'the unusable
> >>>>>machine situation' on powerpc.
> >>>>
> >>>>I can not login anymore, everything is stuck except the core control
> >>>>mechanisms for example the fan controller.
> >>>>
> >>>>Top reports 'ugly' figures, below from a earlier try:
> >>>>
> >>>>last pid: 6790; load averages: 0.78, 0.84, 0.86 up 0+00:34:52
> >>>>22:42:29 47 processes: 1 running, 46 sleeping
> >>>>CPU: 0.0% user, 0.0% nice, 15.4% system, 11.8% interrupt, 72.8% idle
> >>>>Mem: 5912M Active, 570M Inact, 280M Wired, 26M Cache, 104M Buf, 352K=
=20
> >>>>Free
> >>>>Swap: 12G Total, 9904M Used, 2383M Free, 80% Inuse, 178M Out
> >>>>
> >>>> PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU
> >>>>COMMAND
> >>>> 6768 andreast 1 52 01073741824G 6479M pfault 1 0:58
> >>>>18.90% 31370.
> >>>>
> >>>>And after my mem and swap are full I see swap_pager_getswapspace(16)
> >>>>failed.
> >>>>
> >>>>In this state I can only power-cycle the machine.
> >>>>
> >>>>>That said, on amd64 the user map is between 0 and 0x7fffffffffff, wh=
ich
> >>>>>obviously less then the requested allocation size 0x100000000000000.
> >>>>>If you look at the kdump output on amd64, you will see that malloc()
> >>>>>tries to mmap() the area, fails and retries with obreak(). Default
> >>>>>virtual memory limit is unlimited, so my best quess is that on amd64
> >>>>>vm_map_findspace() returns immediately.
> >>>>>
> >>>>>On powerpc64, I see no reason why vm_map_entry cannot be allocated, =
but
> >>>>>please note that vm object and pages shall be only allocated on dema=
nd.
> >>>>>So I am curious how does your machine breaks and where.
> >>>>
> >>>>I would expect that the 'system' does not allow me to allocate that m=
uch
> >>>>of ram.
> >>>
> >>>Does the issue with machine going into limbo reproducable with the code
> >>>you posted ?
> >>
> >>If I understand you correctly, yes. I can launch the test case and the
> >>machine is immediately unusable. Means I can not kill the process nor
> >>can I log in. Also, top does not show anything useful.
> >Again, let me restate my question: the single mmap() of the huge size is
> >enough for powerpc64 machine to break apart ?
>=20
> I can't answer. I don't know yet.
>=20
> >What happen if you insert sleep(1000000); call before return ? Do not ki=
ll
> >the process, I want to know is machine dead while the process sleeps.
>=20
> Ok, during the 'sleep' the machine is usable. top is reporting figures,=
=20
> I can log in and edit files. The process runs now for aboutt 30'.
>=20
> When I kill the process, I do not get back to the shell nor can I log=20
> in. Also top stops reporting.
> But as you said, I didn't kill in this run.
Then, as Alan Cox pointed out, caused by the approach taken in powerpc64
pmap to handle pmap_remove(). It is definitely arch-specific.
>=20
> Thanks,
> Andreas
>=20
> [bohrium:~] andreast% ./mega_malloc
> p =3D 0x50400000
> p =3D 0x50400000
> ...
>=20
> last pid: 1279; load averages: 0.05, 0.03, 0.04 up 0+00:30:34=20
> 21:56:40
> 41 processes: 1 running, 40 sleeping
> CPU: 0.0% user, 0.0% nice, 1.2% system, 0.2% interrupt, 98.6% idle
> Mem: 28M Active, 26M Inact, 89M Wired, 8K Cache, 725M Buf, 6642M Free
> Swap: 12G Total, 12G Free
>=20
> PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU=20
> COMMAND
> 1219 andreast 1 24 01073741824G 1608K nanslp 0 0:00=20
> 0.00% mega_m
--trzpwzIgNM352Iuf
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (FreeBSD)
iEYEARECAAYFAk7WnxYACgkQC3+MBN1Mb4jj4ACgxe5J5gtPIAZmCYXpwwIID/Ri
B+gAoNAQfNLTu7EOWf6U5gsCD52HLaol
=ii8m
-----END PGP SIGNATURE-----
--trzpwzIgNM352Iuf--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20111130212439.GF50300>