Date: Fri, 22 May 2015 20:38:15 +0200 From: =?UTF-8?Q?Micha=C5=82_Stanek?= <mst@semihalf.com> To: Konstantin Belousov <kostikbel@gmail.com> Cc: freebsd-current@freebsd.org, freebsd-arm@freebsd.org Subject: Re: Fwd: UMA initialization failure with 48 core ARM64 Message-ID: <CAMiGqYjPgkCk-X%2BQVso%2BNE9AzOKsZCpqh%2BQVDw9-ppcE4L_BCA@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
2015-05-22 16:36 GMT+02:00 Konstantin Belousov <kostikbel@gmail.com>: > On Fri, May 22, 2015 at 04:14:13PM +0200, Micha?? Stanek wrote: > > Success! I am finally able to boot 48 cores. I have been trying out > > different values given to uma_prealloc() and uma_zone_reserve(). It > started > > working when I increased the parameter in uma_prealloc() 32 times and > left > > uma_zone_reserve() as it was originally. UMA_BOOT_PAGES also needs to be > > set to 512. > > > > Thank you very much for your help. Do you know how the value to > > uma_prealloc() should scale with the number of CPUs? If it is not > obvious, > > then maybe for now we should make a #define with a value to multiply > > BT_MAXALLOC by, with a comment that a higher number is required on > > platforms with many CPUs. What do you think the final fix should look > like? > > I suspect it is not only the number of CPUs which makes the play. Note > that the number of tags is already scaled with the number of CPUs. > > It is also the question of how much the given architecture needs to > allocate > before the normal uma/vmem mechanisms start working. My quess is that > arm64 > performes more kva_alloc()s on early stages than other architectures. > I am forwarding the result of my conversation with Konstantin Belousov. With his help I was able to boot 48 cores on an arm64 platform. I needed to set UMA_BOOT_PAGES=512 and increase the parameter given to uma_prealloc() in vmem_startup() 32 times (giving 32 * BT_MAXALLOC). It looks like this should be made configurable to avoid running out of space for initial allocations on some platforms. In our case, the panic happened still in SI_SUB_VM sysinit. Best regards, Michal Stanek
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAMiGqYjPgkCk-X%2BQVso%2BNE9AzOKsZCpqh%2BQVDw9-ppcE4L_BCA>