From owner-svn-src-stable@freebsd.org Sat May 27 07:47:54 2017 Return-Path: Delivered-To: svn-src-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C266AD8458C; Sat, 27 May 2017 07:47:54 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 841FD1834; Sat, 27 May 2017 07:47:54 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v4R7lri6076849; Sat, 27 May 2017 07:47:53 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v4R7lrMx076844; Sat, 27 May 2017 07:47:53 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201705270747.v4R7lrMx076844@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Sat, 27 May 2017 07:47:53 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r318976 - in stable/11/sys: arm/arm arm64/arm64 mips/mips x86/x86 X-SVN-Group: stable-11 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for all the -stable branches of the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 May 2017 07:47:54 -0000 Author: hselasky Date: Sat May 27 07:47:52 2017 New Revision: 318976 URL: https://svnweb.freebsd.org/changeset/base/318976 Log: MFC r318353: Avoid use of contiguous memory allocations in busdma when possible. This patch improves the boundary checks in busdma to allow more cases using the regular page based kernel memory allocator. Especially in the case of having a non-zero boundary in the parent DMA tag. For example AMD64 based platforms set the PCI DMA tag boundary to PCI_DMA_BOUNDARY, 4GB, which before this patch caused contiguous memory allocations to be preferred when allocating more than PAGE_SIZE bytes. Even if the required alignment was less than PAGE_SIZE bytes. This patch also fixes the nsegments check for using kmem_alloc_attr() when the maximum segment size is less than PAGE_SIZE bytes. Updated some comments describing the code in question. Differential Revision: https://reviews.freebsd.org/D10645 Reviewed by: kib, jhb, gallatin, scottl Sponsored by: Mellanox Technologies Modified: stable/11/sys/arm/arm/busdma_machdep-v4.c stable/11/sys/arm/arm/busdma_machdep-v6.c stable/11/sys/arm64/arm64/busdma_bounce.c stable/11/sys/mips/mips/busdma_machdep.c stable/11/sys/x86/x86/busdma_bounce.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/arm/arm/busdma_machdep-v4.c ============================================================================== --- stable/11/sys/arm/arm/busdma_machdep-v4.c Sat May 27 06:24:06 2017 (r318975) +++ stable/11/sys/arm/arm/busdma_machdep-v4.c Sat May 27 07:47:52 2017 (r318976) @@ -736,8 +736,10 @@ bus_dmamem_alloc(bus_dma_tag_t dmat, voi if (bufzone != NULL && dmat->alignment <= bufzone->size && !_bus_dma_can_bounce(dmat->lowaddr, dmat->highaddr)) { *vaddr = uma_zalloc(bufzone->umazone, mflags); - } else if (dmat->nsegments >= btoc(dmat->maxsize) && - dmat->alignment <= PAGE_SIZE && dmat->boundary == 0) { + } else if (dmat->nsegments >= + howmany(dmat->maxsize, MIN(dmat->maxsegsz, PAGE_SIZE)) && + dmat->alignment <= PAGE_SIZE && + (dmat->boundary % PAGE_SIZE) == 0) { *vaddr = (void *)kmem_alloc_attr(kernel_arena, dmat->maxsize, mflags, 0, dmat->lowaddr, memattr); } else { Modified: stable/11/sys/arm/arm/busdma_machdep-v6.c ============================================================================== --- stable/11/sys/arm/arm/busdma_machdep-v6.c Sat May 27 06:24:06 2017 (r318975) +++ stable/11/sys/arm/arm/busdma_machdep-v6.c Sat May 27 07:47:52 2017 (r318976) @@ -779,7 +779,9 @@ bus_dmamem_alloc(bus_dma_tag_t dmat, voi * (the allocator aligns buffers to their size boundaries). * - There's no need to handle lowaddr/highaddr exclusion zones. * else allocate non-contiguous pages if... - * - The page count that could get allocated doesn't exceed nsegments. + * - The page count that could get allocated doesn't exceed + * nsegments also when the maximum segment size is less + * than PAGE_SIZE. * - The alignment constraint isn't larger than a page boundary. * - There are no boundary-crossing constraints. * else allocate a block of contiguous pages because one or more of the @@ -788,8 +790,10 @@ bus_dmamem_alloc(bus_dma_tag_t dmat, voi if (bufzone != NULL && dmat->alignment <= bufzone->size && !exclusion_bounce(dmat)) { *vaddr = uma_zalloc(bufzone->umazone, mflags); - } else if (dmat->nsegments >= btoc(dmat->maxsize) && - dmat->alignment <= PAGE_SIZE && dmat->boundary == 0) { + } else if (dmat->nsegments >= + howmany(dmat->maxsize, MIN(dmat->maxsegsz, PAGE_SIZE)) && + dmat->alignment <= PAGE_SIZE && + (dmat->boundary % PAGE_SIZE) == 0) { *vaddr = (void *)kmem_alloc_attr(kernel_arena, dmat->maxsize, mflags, 0, dmat->lowaddr, memattr); } else { Modified: stable/11/sys/arm64/arm64/busdma_bounce.c ============================================================================== --- stable/11/sys/arm64/arm64/busdma_bounce.c Sat May 27 06:24:06 2017 (r318975) +++ stable/11/sys/arm64/arm64/busdma_bounce.c Sat May 27 07:47:52 2017 (r318976) @@ -461,22 +461,35 @@ bounce_bus_dmamem_alloc(bus_dma_tag_t dm (*mapp)->flags = DMAMAP_FROM_DMAMEM; /* - * XXX: - * (dmat->alignment <= dmat->maxsize) is just a quick hack; the exact - * alignment guarantees of malloc need to be nailed down, and the - * code below should be rewritten to take that into account. + * Allocate the buffer from the malloc(9) allocator if... + * - It's small enough to fit into a single power of two sized bucket. + * - The alignment is less than or equal to the maximum size + * - The low address requirement is fulfilled. + * else allocate non-contiguous pages if... + * - The page count that could get allocated doesn't exceed + * nsegments also when the maximum segment size is less + * than PAGE_SIZE. + * - The alignment constraint isn't larger than a page boundary. + * - There are no boundary-crossing constraints. + * else allocate a block of contiguous pages because one or more of the + * constraints is something that only the contig allocator can fulfill. * - * In the meantime, we'll warn the user if malloc gets it wrong. + * NOTE: The (dmat->common.alignment <= dmat->maxsize) check + * below is just a quick hack. The exact alignment guarantees + * of malloc(9) need to be nailed down, and the code below + * should be rewritten to take that into account. + * + * In the meantime warn the user if malloc gets it wrong. */ if ((dmat->common.maxsize <= PAGE_SIZE) && (dmat->common.alignment <= dmat->common.maxsize) && dmat->common.lowaddr >= ptoa((vm_paddr_t)Maxmem) && attr == VM_MEMATTR_DEFAULT) { *vaddr = malloc(dmat->common.maxsize, M_DEVBUF, mflags); - } else if (dmat->common.nsegments >= btoc(dmat->common.maxsize) && + } else if (dmat->common.nsegments >= + howmany(dmat->common.maxsize, MIN(dmat->common.maxsegsz, PAGE_SIZE)) && dmat->common.alignment <= PAGE_SIZE && - (dmat->common.boundary == 0 || - dmat->common.boundary >= dmat->common.lowaddr)) { + (dmat->common.boundary % PAGE_SIZE) == 0) { /* Page-based multi-segment allocations allowed */ *vaddr = (void *)kmem_alloc_attr(kernel_arena, dmat->common.maxsize, mflags, 0ul, dmat->common.lowaddr, Modified: stable/11/sys/mips/mips/busdma_machdep.c ============================================================================== --- stable/11/sys/mips/mips/busdma_machdep.c Sat May 27 06:24:06 2017 (r318975) +++ stable/11/sys/mips/mips/busdma_machdep.c Sat May 27 07:47:52 2017 (r318976) @@ -680,7 +680,9 @@ bus_dmamem_alloc(bus_dma_tag_t dmat, voi * (the allocator aligns buffers to their size boundaries). * - There's no need to handle lowaddr/highaddr exclusion zones. * else allocate non-contiguous pages if... - * - The page count that could get allocated doesn't exceed nsegments. + * - The page count that could get allocated doesn't exceed + * nsegments also when the maximum segment size is less + * than PAGE_SIZE. * - The alignment constraint isn't larger than a page boundary. * - There are no boundary-crossing constraints. * else allocate a block of contiguous pages because one or more of the @@ -689,8 +691,10 @@ bus_dmamem_alloc(bus_dma_tag_t dmat, voi if (bufzone != NULL && dmat->alignment <= bufzone->size && !_bus_dma_can_bounce(dmat->lowaddr, dmat->highaddr)) { vaddr = uma_zalloc(bufzone->umazone, mflags); - } else if (dmat->nsegments >= btoc(dmat->maxsize) && - dmat->alignment <= PAGE_SIZE && dmat->boundary == 0) { + } else if (dmat->nsegments >= + howmany(dmat->maxsize, MIN(dmat->maxsegsz, PAGE_SIZE)) && + dmat->alignment <= PAGE_SIZE && + (dmat->boundary % PAGE_SIZE) == 0) { vaddr = (void *)kmem_alloc_attr(kernel_arena, dmat->maxsize, mflags, 0, dmat->lowaddr, memattr); } else { Modified: stable/11/sys/x86/x86/busdma_bounce.c ============================================================================== --- stable/11/sys/x86/x86/busdma_bounce.c Sat May 27 06:24:06 2017 (r318975) +++ stable/11/sys/x86/x86/busdma_bounce.c Sat May 27 07:47:52 2017 (r318976) @@ -396,23 +396,36 @@ bounce_bus_dmamem_alloc(bus_dma_tag_t dm else attr = VM_MEMATTR_DEFAULT; - /* - * XXX: - * (dmat->alignment <= dmat->maxsize) is just a quick hack; the exact - * alignment guarantees of malloc need to be nailed down, and the - * code below should be rewritten to take that into account. + /* + * Allocate the buffer from the malloc(9) allocator if... + * - It's small enough to fit into a single power of two sized bucket. + * - The alignment is less than or equal to the maximum size + * - The low address requirement is fulfilled. + * else allocate non-contiguous pages if... + * - The page count that could get allocated doesn't exceed + * nsegments also when the maximum segment size is less + * than PAGE_SIZE. + * - The alignment constraint isn't larger than a page boundary. + * - There are no boundary-crossing constraints. + * else allocate a block of contiguous pages because one or more of the + * constraints is something that only the contig allocator can fulfill. + * + * NOTE: The (dmat->common.alignment <= dmat->maxsize) check + * below is just a quick hack. The exact alignment guarantees + * of malloc(9) need to be nailed down, and the code below + * should be rewritten to take that into account. * - * In the meantime, we'll warn the user if malloc gets it wrong. + * In the meantime warn the user if malloc gets it wrong. */ if ((dmat->common.maxsize <= PAGE_SIZE) && (dmat->common.alignment <= dmat->common.maxsize) && dmat->common.lowaddr >= ptoa((vm_paddr_t)Maxmem) && attr == VM_MEMATTR_DEFAULT) { *vaddr = malloc(dmat->common.maxsize, M_DEVBUF, mflags); - } else if (dmat->common.nsegments >= btoc(dmat->common.maxsize) && + } else if (dmat->common.nsegments >= + howmany(dmat->common.maxsize, MIN(dmat->common.maxsegsz, PAGE_SIZE)) && dmat->common.alignment <= PAGE_SIZE && - (dmat->common.boundary == 0 || - dmat->common.boundary >= dmat->common.lowaddr)) { + (dmat->common.boundary % PAGE_SIZE) == 0) { /* Page-based multi-segment allocations allowed */ *vaddr = (void *)kmem_alloc_attr(kernel_arena, dmat->common.maxsize, mflags, 0ul, dmat->common.lowaddr,