From owner-svn-src-head@freebsd.org Tue Feb 4 22:40:12 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A4390237412; Tue, 4 Feb 2020 22:40:12 +0000 (UTC) (envelope-from rlibby@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48C0583q0fz3CjM; Tue, 4 Feb 2020 22:40:12 +0000 (UTC) (envelope-from rlibby@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 52A4420EDA; Tue, 4 Feb 2020 22:40:12 +0000 (UTC) (envelope-from rlibby@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 014MeCiH068136; Tue, 4 Feb 2020 22:40:12 GMT (envelope-from rlibby@FreeBSD.org) Received: (from rlibby@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 014MeBfO068132; Tue, 4 Feb 2020 22:40:11 GMT (envelope-from rlibby@FreeBSD.org) Message-Id: <202002042240.014MeBfO068132@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rlibby set sender to rlibby@FreeBSD.org using -f From: Ryan Libby Date: Tue, 4 Feb 2020 22:40:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r357547 - in head: share/man/man9 sys/vm X-SVN-Group: head X-SVN-Commit-Author: rlibby X-SVN-Commit-Paths: in head: share/man/man9 sys/vm X-SVN-Commit-Revision: 357547 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Feb 2020 22:40:12 -0000 Author: rlibby Date: Tue Feb 4 22:40:11 2020 New Revision: 357547 URL: https://svnweb.freebsd.org/changeset/base/357547 Log: uma: add UMA_ZONE_CONTIG, and a default contig_alloc For now, copy the mbuf allocator. Reviewed by: jeff, markj (previous version) Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D23237 Modified: head/share/man/man9/zone.9 head/sys/vm/uma.h head/sys/vm/uma_core.c head/sys/vm/uma_int.h Modified: head/share/man/man9/zone.9 ============================================================================== --- head/share/man/man9/zone.9 Tue Feb 4 22:39:58 2020 (r357546) +++ head/share/man/man9/zone.9 Tue Feb 4 22:40:11 2020 (r357547) @@ -25,7 +25,7 @@ .\" .\" $FreeBSD$ .\" -.Dd January 8, 2020 +.Dd February 4, 2020 .Dt UMA 9 .Os .Sh NAME @@ -340,6 +340,10 @@ was allocated will cause mixing in per-CPU caches. See .Xr numa 4 for more details. +.It Dv UMA_ZONE_CONTIG +Items in this zone must be contiguous in physical address space. +Items will follow normal alignment constraints and may span page boundaries +between pages with contiguous physical addresses. .El .Pp Zones can be destroyed using @@ -465,12 +469,8 @@ and .Fn uma_zone_set_freef functions allow a zone's default slab allocation and free functions to be overridden. -This is useful if the zone's items have special memory allocation constraints. -For example, if multi-page objects are required to be physically contiguous, -an -.Fa allocf -function which requests contiguous memory from the kernel's page allocator -may be used. +This is useful if memory with special constraints such as attributes, +alignment, or address ranges must be used. .Pp The .Fn uma_zone_set_max Modified: head/sys/vm/uma.h ============================================================================== --- head/sys/vm/uma.h Tue Feb 4 22:39:58 2020 (r357546) +++ head/sys/vm/uma.h Tue Feb 4 22:40:11 2020 (r357547) @@ -236,6 +236,10 @@ uma_zone_t uma_zcache_create(char *name, int size, uma * overlap when adding new features. */ #define UMA_ZONE_ZINIT 0x0002 /* Initialize with zeros */ +#define UMA_ZONE_CONTIG 0x0004 /* + * Physical memory underlying an object + * must be contiguous. + */ #define UMA_ZONE_NOTOUCH 0x0008 /* UMA may not access the memory */ #define UMA_ZONE_MALLOC 0x0010 /* For use by malloc(9) only! */ #define UMA_ZONE_NOFREE 0x0020 /* Do not free slabs of this type! */ @@ -639,8 +643,7 @@ smr_t uma_zone_get_smr(uma_zone_t zone); #define UMA_SLAB_BOOT 0x01 /* Slab alloced from boot pages */ #define UMA_SLAB_KERNEL 0x04 /* Slab alloced from kmem */ #define UMA_SLAB_PRIV 0x08 /* Slab alloced from priv allocator */ -#define UMA_SLAB_OFFP 0x10 /* Slab is managed separately */ -/* 0x02, 0x40, and 0x80 are available */ +/* 0x02, 0x10, 0x40, and 0x80 are available */ /* * Used to pre-fill a zone with some number of items Modified: head/sys/vm/uma_core.c ============================================================================== --- head/sys/vm/uma_core.c Tue Feb 4 22:39:58 2020 (r357546) +++ head/sys/vm/uma_core.c Tue Feb 4 22:40:11 2020 (r357547) @@ -271,6 +271,7 @@ static void *noobj_alloc(uma_zone_t, vm_size_t, int, u static void *page_alloc(uma_zone_t, vm_size_t, int, uint8_t *, int); static void *pcpu_page_alloc(uma_zone_t, vm_size_t, int, uint8_t *, int); static void *startup_alloc(uma_zone_t, vm_size_t, int, uint8_t *, int); +static void *contig_alloc(uma_zone_t, vm_size_t, int, uint8_t *, int); static void page_free(void *, vm_size_t, uint8_t); static void pcpu_page_free(void *, vm_size_t, uint8_t); static uma_slab_t keg_alloc_slab(uma_keg_t, uma_zone_t, int, int, int); @@ -1660,6 +1661,19 @@ noobj_alloc(uma_zone_t zone, vm_size_t bytes, int doma } /* + * Allocate physically contiguous pages. + */ +static void * +contig_alloc(uma_zone_t zone, vm_size_t bytes, int domain, uint8_t *pflag, + int wait) +{ + + *pflag = UMA_SLAB_KERNEL; + return ((void *)kmem_alloc_contig_domainset(DOMAINSET_FIXED(domain), + bytes, wait, 0, ~(vm_paddr_t)0, 1, 0, VM_MEMATTR_DEFAULT)); +} + +/* * Frees a number of pages to the system * * Arguments: @@ -1679,8 +1693,8 @@ page_free(void *mem, vm_size_t size, uint8_t flags) return; } - if ((flags & UMA_SLAB_KERNEL) == 0) - panic("UMA: page_free used with invalid flags %x", flags); + KASSERT((flags & UMA_SLAB_KERNEL) != 0, + ("UMA: page_free used with invalid flags %x", flags)); kmem_free((vm_offset_t)mem, size); } @@ -2044,6 +2058,8 @@ keg_ctor(void *mem, int size, void *udata, int flags) keg->uk_allocf = startup_alloc; else if (keg->uk_flags & UMA_ZONE_PCPU) keg->uk_allocf = pcpu_page_alloc; + else if ((keg->uk_flags & UMA_ZONE_CONTIG) != 0 && keg->uk_ppera > 1) + keg->uk_allocf = contig_alloc; else keg->uk_allocf = page_alloc; #ifdef UMA_MD_SMALL_ALLOC @@ -2105,10 +2121,17 @@ zone_kva_available(uma_zone_t zone, void *unused) if ((zone->uz_flags & UMA_ZFLAG_CACHE) != 0) return; KEG_GET(zone, keg); - if (keg->uk_flags & UMA_ZONE_PCPU) - keg->uk_allocf = pcpu_page_alloc; - else if (keg->uk_allocf == startup_alloc) - keg->uk_allocf = page_alloc; + + if (keg->uk_allocf == startup_alloc) { + /* Switch to the real allocator. */ + if (keg->uk_flags & UMA_ZONE_PCPU) + keg->uk_allocf = pcpu_page_alloc; + else if ((keg->uk_flags & UMA_ZONE_CONTIG) != 0 && + keg->uk_ppera > 1) + keg->uk_allocf = contig_alloc; + else + keg->uk_allocf = page_alloc; + } } static void Modified: head/sys/vm/uma_int.h ============================================================================== --- head/sys/vm/uma_int.h Tue Feb 4 22:39:58 2020 (r357546) +++ head/sys/vm/uma_int.h Tue Feb 4 22:40:11 2020 (r357547) @@ -200,6 +200,7 @@ "\6NOFREE" \ "\5MALLOC" \ "\4NOTOUCH" \ + "\3CONTIG" \ "\2ZINIT" /*