From owner-freebsd-hackers@freebsd.org Wed Feb 24 23:49:42 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 10EAB54D223 for ; Wed, 24 Feb 2021 23:49:42 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DmCM95XtQz4nZQ; Wed, 24 Feb 2021 23:49:41 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 11ONnQWg011804 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 25 Feb 2021 01:49:29 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 11ONnQWg011804 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 11ONnQqM011803; Thu, 25 Feb 2021 01:49:26 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 25 Feb 2021 01:49:26 +0200 From: Konstantin Belousov To: Ravi Pokala Cc: "freebsd-hackers@freebsd.org" Subject: Re: `sysctl vm.pmap.kernel_maps' spins on 12.2-RELEASE-p3 w/ nvdimm.ko Message-ID: References: <2C7B4C6A-0432-44A6-B512-D7114F2B092B@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2C7B4C6A-0432-44A6-B512-D7114F2B092B@freebsd.org> X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4DmCM95XtQz4nZQ X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2021 23:49:42 -0000 On Wed, Feb 24, 2021 at 03:37:12PM -0800, Ravi Pokala wrote: > Hi folks, > > A colleague and I both independently observed `sysctl -a' appear to hang on nodes running FreeBSD 12.2-RELEASE-p3; it didn't emit any output, and ^C didn't kill it. We could still establish a new terminal session to the node, via SSH or serial console, and we were able to see that it was actually spinning, not hung, and was consuming an entire CPU. > > We eventually determined that it was specifically `sysctl vm.pmap.kernel_maps' which was spinning, and subsequently that it only spinned if nvdimm.ko was loaded. It was not necessary to access the device node associated with the NVDIMM; merely having the module loaded was sufficient. > > I know nvdimm(4) isn't terribly widely used, but hopefully someone who uses it can at least confirm my findings on this. Help in debugging would be even more appreciated. > How large your nvdimms are? Their' SPAs are mapped into KVA fully and this could be quite large. It could be busy dumping page tables. Try to skip large map in pmap.c:sysctl_kmaps() (just increment i over it).