Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 22 May 2015 20:38:15 +0200
From:      =?UTF-8?Q?Micha=C5=82_Stanek?= <mst@semihalf.com>
To:        Konstantin Belousov <kostikbel@gmail.com>
Cc:        freebsd-current@freebsd.org, freebsd-arm@freebsd.org
Subject:   Re: Fwd: UMA initialization failure with 48 core ARM64
Message-ID:  <CAMiGqYjPgkCk-X%2BQVso%2BNE9AzOKsZCpqh%2BQVDw9-ppcE4L_BCA@mail.gmail.com>

next in thread | raw e-mail | index | archive | help
2015-05-22 16:36 GMT+02:00 Konstantin Belousov <kostikbel@gmail.com>:

> On Fri, May 22, 2015 at 04:14:13PM +0200, Micha?? Stanek wrote:
> > Success! I am finally able to boot 48 cores. I have been trying out
> > different values given to uma_prealloc() and uma_zone_reserve(). It
> started
> > working when I increased the parameter in uma_prealloc() 32 times and
> left
> > uma_zone_reserve() as it was originally. UMA_BOOT_PAGES also needs to be
> > set to 512.
> >
> > Thank you very much for your help. Do you know how the value to
> > uma_prealloc() should scale with the number of CPUs? If it is not
> obvious,
> > then maybe for now we should make a #define with a value to multiply
> > BT_MAXALLOC by, with a comment that a higher number is required on
> > platforms with many CPUs. What do you think the final fix should look
> like?
>
> I suspect it is not only the number of CPUs which makes the play.  Note
> that the number of tags is already scaled with the number of CPUs.
>
> It is also the question of how much the given architecture needs to
> allocate
> before the normal uma/vmem mechanisms start working.  My quess is that
> arm64
> performes more kva_alloc()s on early stages than other architectures.
>

I am forwarding the result of my conversation with Konstantin Belousov.
With his help I was able to boot 48 cores on an arm64 platform. I needed to
set UMA_BOOT_PAGES=512 and increase the parameter given to uma_prealloc()
in vmem_startup() 32 times (giving 32 * BT_MAXALLOC). It looks like this
should be made configurable to avoid running out of space for initial
allocations on some platforms. In our case, the panic happened still in
SI_SUB_VM sysinit.

Best regards,
Michal Stanek



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAMiGqYjPgkCk-X%2BQVso%2BNE9AzOKsZCpqh%2BQVDw9-ppcE4L_BCA>