From owner-svn-src-all@freebsd.org Thu May 14 16:06:54 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C5E5D2F5C0E; Thu, 14 May 2020 16:06:54 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49NGdB4ssVz4VgW; Thu, 14 May 2020 16:06:54 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 88B52D9D5; Thu, 14 May 2020 16:06:54 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 04EG6s5P017231; Thu, 14 May 2020 16:06:54 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 04EG6sMi017229; Thu, 14 May 2020 16:06:54 GMT (envelope-from markj@FreeBSD.org) Message-Id: <202005141606.04EG6sMi017229@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Thu, 14 May 2020 16:06:54 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r361032 - head/sys/vm X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: head/sys/vm X-SVN-Commit-Revision: 361032 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 May 2020 16:06:54 -0000 Author: markj Date: Thu May 14 16:06:54 2020 New Revision: 361032 URL: https://svnweb.freebsd.org/changeset/base/361032 Log: Allocate UMA per-CPU counters earlier. Otherwise anything counted before SI_SUB_VM_CONF is discarded. However, it is useful to be able to see stats from allocations done early during boot. Reviewed by: kib Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D24756 Modified: head/sys/vm/uma_core.c Modified: head/sys/vm/uma_core.c ============================================================================== --- head/sys/vm/uma_core.c Thu May 14 15:49:37 2020 (r361031) +++ head/sys/vm/uma_core.c Thu May 14 16:06:54 2020 (r361032) @@ -192,6 +192,7 @@ SYSCTL_ULONG(_vm, OID_AUTO, uma_kmem_total, CTLFLAG_RD static enum { BOOT_COLD, BOOT_KVA, + BOOT_PCPU, BOOT_RUNNING, BOOT_SHUTDOWN, } booted = BOOT_COLD; @@ -304,7 +305,6 @@ static int hash_alloc(struct uma_hash *, u_int); static int hash_expand(struct uma_hash *, struct uma_hash *); static void hash_free(struct uma_hash *hash); static void uma_timeout(void *); -static void uma_startup3(void); static void uma_shutdown(void); static void *zone_alloc_item(uma_zone_t, void *, int, int); static void zone_free_item(uma_zone_t, void *, void *, enum zfreeskip); @@ -360,8 +360,6 @@ SYSCTL_COUNTER_U64(_vm_debug, OID_AUTO, skipped, CTLFL &uma_skip_cnt, "memory items skipped, not debugged"); #endif -SYSINIT(uma_startup3, SI_SUB_VM_CONF, SI_ORDER_SECOND, uma_startup3, NULL); - SYSCTL_NODE(_vm, OID_AUTO, uma, CTLFLAG_RW | CTLFLAG_MPSAFE, 0, "Universal Memory Allocator"); @@ -2658,9 +2656,10 @@ zone_ctor(void *mem, int size, void *udata, int flags) (UMA_ZONE_INHERIT | UMA_ZFLAG_INHERIT)); out: - if (__predict_true(booted >= BOOT_RUNNING)) { + if (booted >= BOOT_PCPU) { zone_alloc_counters(zone, NULL); - zone_alloc_sysctl(zone, NULL); + if (booted >= BOOT_RUNNING) + zone_alloc_sysctl(zone, NULL); } else { zone->uz_allocs = EARLY_COUNTER; zone->uz_frees = EARLY_COUNTER; @@ -2904,10 +2903,23 @@ uma_startup2(void) } /* + * Allocate counters as early as possible so that boot-time allocations are + * accounted more precisely. + */ +static void +uma_startup_pcpu(void *arg __unused) +{ + + zone_foreach_unlocked(zone_alloc_counters, NULL); + booted = BOOT_PCPU; +} +SYSINIT(uma_startup_pcpu, SI_SUB_COUNTER, SI_ORDER_ANY, uma_startup_pcpu, NULL); + +/* * Finish our initialization steps. */ static void -uma_startup3(void) +uma_startup3(void *arg __unused) { #ifdef INVARIANTS @@ -2915,7 +2927,6 @@ uma_startup3(void) uma_dbg_cnt = counter_u64_alloc(M_WAITOK); uma_skip_cnt = counter_u64_alloc(M_WAITOK); #endif - zone_foreach_unlocked(zone_alloc_counters, NULL); zone_foreach_unlocked(zone_alloc_sysctl, NULL); callout_init(&uma_callout, 1); callout_reset(&uma_callout, UMA_TIMEOUT * hz, uma_timeout, NULL); @@ -2924,6 +2935,7 @@ uma_startup3(void) EVENTHANDLER_REGISTER(shutdown_post_sync, uma_shutdown, NULL, EVENTHANDLER_PRI_FIRST); } +SYSINIT(uma_startup3, SI_SUB_VM_CONF, SI_ORDER_SECOND, uma_startup3, NULL); static void uma_shutdown(void)