From owner-freebsd-current@FreeBSD.ORG Mon Aug 12 15:30:17 2013 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 2EE9C6C7 for ; Mon, 12 Aug 2013 15:30:17 +0000 (UTC) (envelope-from davide.italiano@gmail.com) Received: from mail-vb0-x231.google.com (mail-vb0-x231.google.com [IPv6:2607:f8b0:400c:c02::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E447D26F5 for ; Mon, 12 Aug 2013 15:30:16 +0000 (UTC) Received: by mail-vb0-f49.google.com with SMTP id w16so5764466vbb.22 for ; Mon, 12 Aug 2013 08:30:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Ix3dRRbk5RYbra5eJU9gtDB1Ehfoye5V/7McaRK6yJE=; b=Eo6669Mr2eLskr+Uf+uu+VnaaIID18zAq6MlKhQ9+lLl1Jw70Z1aEsscMvmLWhZ0Y5 Ka6T7FvqwwLC6UjhqDOHn/sBmzjps9W1319B1gTgeZThR94VxwdacZKR9FxJmdgsUzwP tuk6j/CGkBWJ3CO5hM2ZXuBDIoAVHivPOr2FqcF52MplvwSNuLQH3QYCQUNZ7A+SDcx+ 9MJWtSgyiY6H3m1FHcsCknOob6azS777pLSw/tGZdU1fSJQy35yegVZr0V9uyXF1Co5d F9Qr2y+t8OM2QK1ISi28o8q/4mSbS6r8MgiQlBz+uWCeWjzPszbki2QY/ViLuTB8Jn6g 68MA== MIME-Version: 1.0 X-Received: by 10.220.182.193 with SMTP id cd1mr6778976vcb.32.1376321415901; Mon, 12 Aug 2013 08:30:15 -0700 (PDT) Sender: davide.italiano@gmail.com Received: by 10.220.21.140 with HTTP; Mon, 12 Aug 2013 08:30:15 -0700 (PDT) In-Reply-To: <20130812151314.GE1570@albert.catwhisker.org> References: <20130812151314.GE1570@albert.catwhisker.org> Date: Mon, 12 Aug 2013 08:30:15 -0700 X-Google-Sender-Auth: tWsyRnZUNhFntl4edRPjLhwTLDU Message-ID: Subject: Re: Early drop to debugger with DEBUG_MEMGUARD From: Davide Italiano To: David Wolfskill Content-Type: text/plain; charset=ISO-8859-1 Cc: current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Aug 2013 15:30:17 -0000 On Mon, Aug 12, 2013 at 8:13 AM, David Wolfskill wrote: > I first noticed this on my laptop on 08 Aug, after having built & booted > > FreeBSD 10.0-CURRENT #975 r253985M/253985:1000041: Tue Aug 6 05:28:39 PDT 2013 root@localhost:/common/S4/obj/usr/src/sys/CANARY i386 > > OK. I'm away from home, and Internet access is a bit flaky, so > initially, I suspected that something may have gone wrong with my > source update; I later determined that disabling "options DEBUG_MEMGUARD" > would avoid the panic. > > That said, I had been running a kernel with DEBUG_MEMGUARD for quite > a while without issues; I suspect that this drop to debugger either > reflects a real problem that disabling DEBUG_MEMGUARD merely hides > or htat the assert in src/sys/kern/subr_vmem.c:1050 isn't actually > correct in all cases. > > So I finally(!) had a chance to try to reproduce the error on a > machine with a serial console; here's a cut/paste from that: > > ... > | 7. Boot [V]erbose: NO | `:` `:` > | | .-- `--. > | | .---.....----. > +-----------------------------------------+ > > > Booting... > GDB: no debug ports present > KDB: debugger backends: ddb > KDB: current backend: ddb > Copyright (c) 1992-2013 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 10.0-CURRENT #0 r254245M/254246:1000042: Mon Aug 12 07:20:47 PDT 2013 > root@freebeast.catwhisker.org:/common/S3/obj/usr/src/sys/MEMGUARD i386 > FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 > WARNING: WITNESS option enabled, expect reduced performance. > panic: Assertion strat == M_BESTFIT || strat == M_FIRSTFIT failed at /usr/src/sys/kern/subr_vmem.c:1050 > cpuid = 0 > KDB: stack backtrace: > db_trace_self_wrapper(c116fcdc,73752f20,72732f72,79732f63,656b2f73,...) at db_trace_self_wrapper+0x2d/frame 0xc1820ba0 > kdb_backtrace(c11c4b23,0,c0f8a835,c1820c74,c0f8a835,...) at kdb_backtrace+0x30/frame 0xc1820c08 > vpanic(c12eea08,100,c0f8a835,c1820c74,c1820c74,...) at vpanic+0x11f/frame 0xc1820c44 > kassert_panic(c0f8a835,c1172e98,c1172e39,41a,8,...) at kassert_panic+0xea/frame 0xc1820c68 > vmem_alloc(c130d680,6681000,2,c1820cc0,3b5,...) at vmem_alloc+0x53/frame 0xc1820ca0 > memguard_init(c130d680,c0a9fa50,c6800000,20281000,1000,10000,0) at memguard_init+0x29/frame 0xc1820cc4 > kmeminit(c14b9fd4,c10efc89,0,0,c1820d30,...) at kmeminit+0x171/frame 0xc1820cf0 > mallocinit(0,0,2,0,c11d3728,...) at mallocinit+0x32/frame 0xc1820d30 > mi_startup() at mi_startup+0xf7/frame 0xc1820d58 > begin() at begin+0x2c > KDB: enter: panic > [ thread pid 0 tid 0 ] > Stopped at kdb_enter+0x3d: movl $0,kdb_why > db> > > As you can see, this is well before any device probes or much of > anything else. Thus, I suspect that it's fairly possible that the > assertion may well be OK after a certain point in the boot sequence, > but decidedly *not* OK in this specific instance. Or perhaps the > assertion just doesn't play well with DEBUG_MEMGUARD. > > I'm not about to pretend that I have anywhere near enough familiarity > with what's going on to even suggest a fix, but it seems to me that > Something Is Wrong Here. > > The kernel config (in this case) is: > > include GENERIC > > ident MEMGUARD > > options DEBUG_MEMGUARD > > > The system was running a copy of: > > FreeBSD 10.0-CURRENT #1243 r254245M/254246:1000042: Mon Aug 12 05:39:42 PDT 2013 root@freebeast.catwhisker.org:/common/S4/obj/usr/src/sys/GENERIC i386 > > but with a newly-built MEMGUARD kernel (as above), built from the same > sources. > > I have some time to poke at it for the next few hours; subject to > my Internet access & available time, I'm happy to do that, try > patches, or whatever, but I could use a bit of guidance. > > Since it's been completely reproducible for me, I suspect that > anyone with sufficiently recenty sources running head can reproduce > it merely by enabling "options DEBUG_MEMGUARD", rebuilding the > kernel, and booting it. > > Peace, > david > -- > David H. Wolfskill david@catwhisker.org > Taliban: Evil men with guns afraid of truth from a 14-year old girl. > > See http://www.catwhisker.org/~david/publickey.gpg for my public key. vmem_alloc() KPI needs the consumer to specify exactly a strategy for allocation, which is one of two between: M_FIRSTFIT/M_BESTFIT (fast allocation vs low fragmentation), and that's the assertion that's not respected within the code. 1050 MPASS(strat == M_BESTFIT || strat == M_FIRSTFIT); It looks like memguard_init() doesn't specify none of these two strategies. 209 vmem_alloc(parent, memguard_mapsize, M_WAITOK, &base); My guess is that you need to OR one between M_BESTFIT/M_FIRSTFIT with M_WAITOK to have your kernel booting. What's better between the two probably will need some measurements but this should at least make your kernel booting. I cannot test this change myself as long as I'm out of town until tomorrow afternoon as well, but I will take a further look when I'll come back. If in the meanwhile you want to try this change, be my guest. Thanks, -- Davide "There are no solved problems; there are only problems that are more or less solved" -- Henri Poincare