From owner-svn-src-all@freebsd.org Tue Nov 10 14:23:47 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9DA9B432612; Tue, 10 Nov 2020 14:23:47 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CVqq7477jz4TCK; Tue, 10 Nov 2020 14:23:47 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8095823FCA; Tue, 10 Nov 2020 14:23:47 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0AAENl0j080608; Tue, 10 Nov 2020 14:23:47 GMT (envelope-from mjg@FreeBSD.org) Received: (from mjg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0AAENlZH080607; Tue, 10 Nov 2020 14:23:47 GMT (envelope-from mjg@FreeBSD.org) Message-Id: <202011101423.0AAENlZH080607@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mjg set sender to mjg@FreeBSD.org using -f From: Mateusz Guzik Date: Tue, 10 Nov 2020 14:23:47 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r367569 - head/sys/contrib/openzfs/module/zfs X-SVN-Group: head X-SVN-Commit-Author: mjg X-SVN-Commit-Paths: head/sys/contrib/openzfs/module/zfs X-SVN-Commit-Revision: 367569 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Nov 2020 14:23:47 -0000 Author: mjg Date: Tue Nov 10 14:23:46 2020 New Revision: 367569 URL: https://svnweb.freebsd.org/changeset/base/367569 Log: zfs: combine zio caches if possible This deduplicates 2 sets of caches using the same sizes. Memory savings fluctuate a lot, one sample result is buildworld on zfs saving ~180MB RAM in reduced page count associated with zio caches. Modified: head/sys/contrib/openzfs/module/zfs/zio.c Modified: head/sys/contrib/openzfs/module/zfs/zio.c ============================================================================== --- head/sys/contrib/openzfs/module/zfs/zio.c Tue Nov 10 14:21:23 2020 (r367568) +++ head/sys/contrib/openzfs/module/zfs/zio.c Tue Nov 10 14:23:46 2020 (r367569) @@ -204,6 +204,19 @@ zio_init(void) if (align != 0) { char name[36]; + if (cflags == data_cflags) { + /* + * Resulting kmem caches would be identical. + * Save memory by creating only one. + */ + (void) snprintf(name, sizeof (name), + "zio_buf_comb_%lu", (ulong_t)size); + zio_buf_cache[c] = kmem_cache_create(name, + size, align, NULL, NULL, NULL, NULL, NULL, + cflags); + zio_data_buf_cache[c] = zio_buf_cache[c]; + continue; + } (void) snprintf(name, sizeof (name), "zio_buf_%lu", (ulong_t)size); zio_buf_cache[c] = kmem_cache_create(name, size, @@ -234,37 +247,55 @@ zio_init(void) void zio_fini(void) { - size_t c; - kmem_cache_t *last_cache = NULL; - kmem_cache_t *last_data_cache = NULL; + size_t i, j, n; + kmem_cache_t *cache; - for (c = 0; c < SPA_MAXBLOCKSIZE >> SPA_MINBLOCKSHIFT; c++) { -#ifdef _ILP32 - /* - * Cache size limited to 1M on 32-bit platforms until ARC - * buffers no longer require virtual address space. - */ - if (((c + 1) << SPA_MINBLOCKSHIFT) > zfs_max_recordsize) - break; -#endif + n = SPA_MAXBLOCKSIZE >> SPA_MINBLOCKSHIFT; + #if defined(ZFS_DEBUG) && !defined(_KERNEL) - if (zio_buf_cache_allocs[c] != zio_buf_cache_frees[c]) + for (i = 0; i < n; i++) { + if (zio_buf_cache_allocs[i] != zio_buf_cache_frees[i]) (void) printf("zio_fini: [%d] %llu != %llu\n", - (int)((c + 1) << SPA_MINBLOCKSHIFT), - (long long unsigned)zio_buf_cache_allocs[c], - (long long unsigned)zio_buf_cache_frees[c]); + (int)((i + 1) << SPA_MINBLOCKSHIFT), + (long long unsigned)zio_buf_cache_allocs[i], + (long long unsigned)zio_buf_cache_frees[i]); + } #endif - if (zio_buf_cache[c] != last_cache) { - last_cache = zio_buf_cache[c]; - kmem_cache_destroy(zio_buf_cache[c]); + + /* + * The same kmem cache can show up multiple times in both zio_buf_cache + * and zio_data_buf_cache. Do a wasteful but trivially correct scan to + * sort it out. + */ + for (i = 0; i < n; i++) { + cache = zio_buf_cache[i]; + if (cache == NULL) + continue; + for (j = i; j < n; j++) { + if (cache == zio_buf_cache[j]) + zio_buf_cache[j] = NULL; + if (cache == zio_data_buf_cache[j]) + zio_data_buf_cache[j] = NULL; } - zio_buf_cache[c] = NULL; + kmem_cache_destroy(cache); + } - if (zio_data_buf_cache[c] != last_data_cache) { - last_data_cache = zio_data_buf_cache[c]; - kmem_cache_destroy(zio_data_buf_cache[c]); + for (i = 0; i < n; i++) { + cache = zio_data_buf_cache[i]; + if (cache == NULL) + continue; + for (j = i; j < n; j++) { + if (cache == zio_data_buf_cache[j]) + zio_data_buf_cache[j] = NULL; } - zio_data_buf_cache[c] = NULL; + kmem_cache_destroy(cache); + } + + for (i = 0; i < n; i++) { + if (zio_buf_cache[i] != NULL) + panic("zio_fini: zio_buf_cache[%d] != NULL", (int)i); + if (zio_data_buf_cache[i] != NULL) + panic("zio_fini: zio_data_buf_cache[%d] != NULL", (int)i); } kmem_cache_destroy(zio_link_cache);