From owner-freebsd-arch@FreeBSD.ORG Sat Jul 19 18:29:20 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 063B0677; Sat, 19 Jul 2014 18:29:20 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8158D2202; Sat, 19 Jul 2014 18:29:19 +0000 (UTC) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id s6JIT930088464 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 19 Jul 2014 21:29:09 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua s6JIT930088464 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id s6JIT9Ix088463; Sat, 19 Jul 2014 21:29:09 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 19 Jul 2014 21:29:09 +0300 From: Konstantin Belousov To: Marcel Moolenaar Subject: Re: KDB entry on NMI Message-ID: <20140719182909.GU93733@kib.kiev.ua> References: <20140718160708.GO93733@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Tx24CLeOShJjHgAY" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: amd64@freebsd.org, arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Jul 2014 18:29:20 -0000 --Tx24CLeOShJjHgAY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jul 19, 2014 at 10:58:18AM -0700, Marcel Moolenaar wrote: >=20 > On Jul 18, 2014, at 9:07 AM, Konstantin Belousov wr= ote: >=20 > > It was mentioned somewhere recently, that typical BIOS today configures > > NMI delivery on the hardware events as broadcast. When I developerd > > the dmar(4) busdma backend, I indeed met the problem, and wrote a > > prototype which avoided startup of ddb on all cores. Instead, the patch > > implements custom spinlock, which allows only one core to win, other > > cores ignore the NMI, by spinning on lock. > >=20 > > The issue which I see on at least two different machines with different > > Intel chipsets, is that NMI is somehow sticky, i.e. it is re-delivered > > after the handler executes iret. I am not sure what the problem is, > > whether it is due to hardware needing some ACK, or a bug in code. > >=20 > > Anyway, even on two-cores machine, having both cores simultaneously > > enter NMI makes the use of ddb impossible, so I believe the patch is > > improvement. I make measures to ensure that reboot from ddb prompt > > works. > >=20 > > Thought ? >=20 > One may call kdb_enter on different CPUs at the same time and it's > also possible to call panic on multiple CPUs at the same time (but > we serialize panic() right now). What if we let kdb_enter at al deal > with concurrency, instead of doing it specifically for NMIs? Then, on 80-threads machine I get the 80 ddb sessions on NMI broadcast, like now. With your proposal, it will be somewhat better, since sessions are serialized, so I can do the reboot from the first one. Still, I hope to understand what I am missing to stop NMI from delivering in loop. Then, having only one ddb entry would mean that I should return only once. >=20 > Also: we may want to do something else than going to the debugger > when we see an NMI. More complexity in the NMI handler and specific > to entering the debugger seems to move us away from doing other > things more easily. I agree there. >=20 > Aside: I've always wanted to have the ability to have the kernel > debugger switch to a different CPU so that you can create DDB > commands that dump hardware resources like TLBs, etc. To support > this, you want the KDB layer to have good CPU handling, which > possibly makes it also a good place to handle concurrent entry > into the debugger from different CPUs. Me too. I have another half-finished patch which does this, it allows to migrate the ddb from one cpu to another. It worked by signalling a destination cpu that it should activate, while source cpu starts spinning. I do not remember exact problems which were unresolved. I needed this because some state is CPU-local, cannot be accessed =66rom other cores, and is not saved in pcb. I definitely looked at EFER and MISC_FEATURES MSRs, and local apic state. --Tx24CLeOShJjHgAY Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJTyrj1AAoJEJDCuSvBvK1BodgQAIk2aC6e9so+6O0rLJAizzzz 1SZ3bYLSpNN/MbdPxQM/JudjI4qRuqMXl4vSOdOpN8oUlrKKpZBKBj7YfDyjwA3L aEeyG1iAhBjEMLhF2Rty2wDVXfOxBM/bZiTmPdl69VGbmvsqIchhJFNbUh3tTY1i a4JKQDg32LyUNvkbxvw++wco+Ts2ptIASHVkayI1zM428uaxA2r2DnFajCFFAbot dh/jvWJsXHEiQhHFOCoP1UfrQaLWbl+mTsvBjXW3lSldL4SkAQPSuW8NtUE1+kfY DBVEjMHXFvR2cT9TVJgkOhzosFKQ+Z/hrjd2tyIknaptk2kQPZJUX/qO9KH1Yt1+ UoeTVBKXUySYcfDZX/CzThLmsZwURnfq/7ZjV64P0x44FfWIBe+os5sUA0dKBmS2 62kkOFFkOwi4oSFM6ymm0JJhYLxHfGULIgqTeVa+XEO4RHb8qmU199559R/YSSDK f6nlV8hLaHkbhBSZ2EZAhRmXyuBB+P6l6rOoIEbujTAQGIy3xHvCEUFIHDl3BtN6 6aiFEKXPQJFoEvItYwDW3VSjbvCO9jkyA0+suCaL36isEM8w4Z4OfAQszgBRouqk /SXHoI6KIe/14z0JZohYYavPSC3q51/WeHYO1k7q50IIEiwhKUNwkyRAv0DJPIf2 WwSwITcNPpypeeamVfaB =qos5 -----END PGP SIGNATURE----- --Tx24CLeOShJjHgAY--