Date: Tue, 13 Aug 2013 17:47:34 -1000 (HST) From: Jeff Roberson <jroberson@jroberson.net> To: Jim Harris <jim.harris@gmail.com> Cc: Alan Cox <alc@freebsd.org>, Attilio Rao <attilio@freebsd.org>, FreeBSD current <freebsd-current@freebsd.org>, Konstantin Belousov <kib@freebsd.org>, Colin Percival <cperciva@freebsd.org> Subject: Re: panic: UMA: Increase vm.boot_pages with 32 CPUs Message-ID: <alpine.BSF.2.00.1308131747050.4083@desktop> In-Reply-To: <CAJP=Hc9KZfD-uKkOM%2BO=oAHiGj3vNwRa-fMgtKWEacOKL110DQ@mail.gmail.com> References: <5208A488.2050603@freebsd.org> <alpine.BSF.2.00.1308131203350.4083@desktop> <CAJP=Hc9KZfD-uKkOM%2BO=oAHiGj3vNwRa-fMgtKWEacOKL110DQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --2547152148-481170591-1376452058=:4083 Content-Type: TEXT/PLAIN; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8BIT On Tue, 13 Aug 2013, Jim Harris wrote: > > > > On Tue, Aug 13, 2013 at 3:05 PM, Jeff Roberson <jroberson@jroberson.net> > wrote: > On Mon, 12 Aug 2013, Colin Percival wrote: > > Hi all, > > A HEAD@254238 kernel fails to boot in EC2 with > panic: UMA: Increase vm.boot_pages > > on 32-CPU instances. Instances with up to 16 CPUs > boot fine. > > I know there has been some mucking about with VM > recently -- anyone want > to claim this, or should I start doing a binary > search? > > > It's not any one commit really, just creeping demand for more pages > before the VM can get started. I would suggest making boot pages > scale with MAXCPU. Or just raising it as the panic suggests. We > could rewrite the way that the vm gets these early pages but it's a > lot of work and typically people just bump it and forget about it. > > > I ran into this problem today when enabling hyperthreading on my dual-socket > Xeon E5 system. > > It looks like r254025 is actually the culprit. Specifically, the new > mallocinit()/kmeminit() now invoke the new vmem_init() before > uma_startup2(), which allocates 16 zones out of the boot pages if I am > reading this correctly. This is all done before uma_startup2() is called, > triggering the panic. > I just disabled the quantum caches in vmem which allocate those 16 zones. This may alleviate the problem for now. Thanks, Jeff > Anything less than 28 CPUs, and the zone size (uma_zone + uma_cache * > (mp_maxid + 1)) is <= PAGE_SIZE and we can successfully boot. So at 32 > CPUs, we need two boot pages per zone which consumes more than the default > 64 boot pages. The size of these structures do not appear to have > materially changed any time recently. > > Scaling with MAXCPU seems to be an OK solution, but should it be based > directly on the size of (uma_zone + uma_cache * MAXCPU)? I am not very > familiar with uma startup, but it seems like these zones are the primary > consumers of the boot pages, so the UMA_BOOT_PAGES default should be based > directly on that size.. > > Regards, > > -Jim > > > --2547152148-481170591-1376452058=:4083--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.BSF.2.00.1308131747050.4083>