Date: Mon, 11 Sep 2017 20:35:23 -0700 From: Mark Johnston <markj@FreeBSD.org> To: "Andrey V. Elsukov" <bu7cher@yandex.ru> Cc: Raphael Kubo da Costa <rakuco@FreeBSD.org>, freebsd-current@FreeBSD.org, cem@FreeBSD.org Subject: Re: r323412: Panic on boot (slab->us_keg == keg) Message-ID: <20170912033523.GA89424@wkstn-mjohnston.west.isilon.com> In-Reply-To: <8ccad12b-3d31-6478-7884-c003bc314990@yandex.ru> References: <86vakp1vsq.fsf@orwell> <fded3e63-bb68-3fc5-e080-f7ed9aea00bc@yandex.ru> <8ccad12b-3d31-6478-7884-c003bc314990@yandex.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Sep 11, 2017 at 09:15:51PM +0300, Andrey V. Elsukov wrote:
> On 11.09.2017 15:23, Andrey V. Elsukov wrote:
> > --- trap 0xc, rip = 0xffffffff80d84870, rsp = 0xffffffff82193970, rbp =
> > 0xffffffff821939b0 ---
> > zone_import() at zone_import+0x110/frame 0xffffffff821939b0
> > zone_alloc_item() at zone_alloc_item+0x36/frame 0xffffffff821939f0
> > uma_startup() at uma_startup+0x1d0/frame 0xffffffff82193ae0
> > vm_page_startup() at vm_page_startup+0x34e/frame 0xffffffff82193b30
> > vm_mem_init() at vm_mem_init+0x1a/frame 0xffffffff82193b50
> > mi_startup() at mi_startup+0x9c/frame 0xffffffff82193b70
> > btext() at btext+0x2c
> > Uptime: 1s
>
> I bisected revisions, and the last working is r322988.
> This machine is E5-2660 v4@ based.
>
> [...]
> FreeBSD/SMP: 2 package(s) x 14 core(s) x 2 hardware threads
>
> Also I determined that it can successfully boot with disabled
> hyper-threading.
After the change to CACHE_LINE_SIZE, we have
sizeof(struct uma_zone) == 448 and sizeof(struct uma_cache) == 64. With
56 CPUs, we therefore need 4032 bytes per UMA zone, plus 80 bytes for
the slab header - "internal" zones always keep the slab header in the
slab itself. That's slightly larger than one page, but the UMA zone
zone's keg will have uk_ppera == 1. So, when allocating slabzone,
keg_alloc_slab() will call startup_alloc(uk_ppera * PAGE_SIZE), which
will allocate 4096 bytes for a structure that is 4032 + 80 = 4112 bytes
in size.
I think the bug is that keg_large_init() doesn't take
sizeof(struct uma_slab) into account when setting uk_ppera for the keg.
In particular, the bug isn't specific to the bootup process; it only
affects internal zones with an item size in the range [4016, 4096].
The patch below should fix this - could you give it a try?
diff --git a/sys/vm/uma_core.c b/sys/vm/uma_core.c
index 44c91e66769a..48daeb18f9c3 100644
--- a/sys/vm/uma_core.c
+++ b/sys/vm/uma_core.c
@@ -1306,10 +1306,6 @@ keg_large_init(uma_keg_t keg)
keg->uk_ipers = 1;
keg->uk_rsize = keg->uk_size;
- /* We can't do OFFPAGE if we're internal, bail out here. */
- if (keg->uk_flags & UMA_ZFLAG_INTERNAL)
- return;
-
/* Check whether we have enough space to not do OFFPAGE. */
if ((keg->uk_flags & UMA_ZONE_OFFPAGE) == 0) {
shsize = sizeof(struct uma_slab);
@@ -1317,8 +1313,17 @@ keg_large_init(uma_keg_t keg)
shsize = (shsize & ~UMA_ALIGN_PTR) +
(UMA_ALIGN_PTR + 1);
- if ((PAGE_SIZE * keg->uk_ppera) - keg->uk_rsize < shsize)
- keg->uk_flags |= UMA_ZONE_OFFPAGE;
+ if ((PAGE_SIZE * keg->uk_ppera) - keg->uk_rsize < shsize) {
+ /*
+ * We can't do offpage if we're internal, in which case
+ * we need an extra page per allocation to contain the
+ * slab header.
+ */
+ if ((keg->uk_flags & UMA_ZFLAG_INTERNAL) == 0)
+ keg->uk_flags |= UMA_ZONE_OFFPAGE;
+ else
+ keg->uk_ppera++;
+ }
}
if ((keg->uk_flags & UMA_ZONE_OFFPAGE) &&
diff --git a/sys/vm/vm_page.c b/sys/vm/vm_page.c
index ee7b93bbd719..477a816b0bd2 100644
--- a/sys/vm/vm_page.c
+++ b/sys/vm/vm_page.c
@@ -475,7 +475,8 @@ vm_page_startup(vm_offset_t vaddr)
* in proportion to the zone structure size.
*/
pages_per_zone = howmany(sizeof(struct uma_zone) +
- sizeof(struct uma_cache) * (mp_maxid + 1), UMA_SLAB_SIZE);
+ sizeof(struct uma_slab) + sizeof(struct uma_cache) * (mp_maxid + 1),
+ UMA_SLAB_SIZE);
if (pages_per_zone > 1) {
/* Reserve more pages so that we don't run out. */
boot_pages = UMA_BOOT_PAGES_ZONES * pages_per_zone;
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20170912033523.GA89424>