From owner-svn-src-head@freebsd.org Mon Jul 2 17:18:48 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E1C361030513; Mon, 2 Jul 2018 17:18:47 +0000 (UTC) (envelope-from alc@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 90F6971814; Mon, 2 Jul 2018 17:18:47 +0000 (UTC) (envelope-from alc@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6D15610668; Mon, 2 Jul 2018 17:18:47 +0000 (UTC) (envelope-from alc@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w62HIl9L041893; Mon, 2 Jul 2018 17:18:47 GMT (envelope-from alc@FreeBSD.org) Received: (from alc@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w62HIlOt041892; Mon, 2 Jul 2018 17:18:47 GMT (envelope-from alc@FreeBSD.org) Message-Id: <201807021718.w62HIlOt041892@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: alc set sender to alc@FreeBSD.org using -f From: Alan Cox Date: Mon, 2 Jul 2018 17:18:47 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r335865 - head/sys/vm X-SVN-Group: head X-SVN-Commit-Author: alc X-SVN-Commit-Paths: head/sys/vm X-SVN-Commit-Revision: 335865 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2018 17:18:48 -0000 Author: alc Date: Mon Jul 2 17:18:46 2018 New Revision: 335865 URL: https://svnweb.freebsd.org/changeset/base/335865 Log: Introduce vm_phys_enq_range(), and call it in vm_phys_alloc_npages() and vm_phys_alloc_seg_contig() instead of vm_phys_free_contig(). In short, vm_phys_enq_range() is simpler and faster than the more general vm_phys_free_contig(), and in the case of vm_phys_alloc_seg_contig(), vm_phys_free_contig() was placing the excess physical pages at the wrong end of the queues. In collaboration with: Doug Moore Modified: head/sys/vm/vm_phys.c Modified: head/sys/vm/vm_phys.c ============================================================================== --- head/sys/vm/vm_phys.c Mon Jul 2 14:15:30 2018 (r335864) +++ head/sys/vm/vm_phys.c Mon Jul 2 17:18:46 2018 (r335865) @@ -605,6 +605,43 @@ vm_phys_split_pages(vm_page_t m, int oind, struct vm_f } /* + * Add the physical pages [m, m + npages) at the end of a power-of-two aligned + * and sized set to the specified free list. + * + * When this function is called by a page allocation function, the caller + * should request insertion at the head unless the lower-order queues are + * known to be empty. The objective being to reduce the likelihood of long- + * term fragmentation by promoting contemporaneous allocation and (hopefully) + * deallocation. + * + * The physical page m's buddy must not be free. + */ +static void +vm_phys_enq_range(vm_page_t m, u_int npages, struct vm_freelist *fl, int tail) +{ + u_int n; + int order; + + KASSERT(npages > 0, ("vm_phys_enq_range: npages is 0")); + KASSERT(((VM_PAGE_TO_PHYS(m) + npages * PAGE_SIZE) & + ((PAGE_SIZE << (fls(npages) - 1)) - 1)) == 0, + ("vm_phys_enq_range: page %p and npages %u are misaligned", + m, npages)); + do { + KASSERT(m->order == VM_NFREEORDER, + ("vm_phys_enq_range: page %p has unexpected order %d", + m, m->order)); + order = ffs(npages) - 1; + KASSERT(order < VM_NFREEORDER, + ("vm_phys_enq_range: order %d is out of range", order)); + vm_freelist_add(fl, m, order, tail); + n = 1 << order; + m += n; + npages -= n; + } while (npages > 0); +} + +/* * Tries to allocate the specified number of pages from the specified pool * within the specified domain. Returns the actual number of allocated pages * and a pointer to each page through the array ma[]. @@ -644,7 +681,12 @@ vm_phys_alloc_npages(int domain, int pool, int npages, for (end = i + need; i < end;) ma[i++] = m++; if (need < avail) { - vm_phys_free_contig(m, avail - need); + /* + * Return excess pages to fl. Its + * order [0, oind) queues are empty. + */ + vm_phys_enq_range(m, avail - need, fl, + 1); return (npages); } else if (i == npages) return (npages); @@ -662,8 +704,13 @@ vm_phys_alloc_npages(int domain, int pool, int npages, for (end = i + need; i < end;) ma[i++] = m++; if (need < avail) { - vm_phys_free_contig(m, avail - - need); + /* + * Return excess pages to fl. + * Its order [0, oind) queues + * are empty. + */ + vm_phys_enq_range(m, avail - + need, fl, 1); return (npages); } else if (i == npages) return (npages); @@ -1303,8 +1350,10 @@ done: } /* Return excess pages to the free lists. */ npages_end = roundup2(npages, 1 << oind); - if (npages < npages_end) - vm_phys_free_contig(&m_ret[npages], npages_end - npages); + if (npages < npages_end) { + fl = (*seg->free_queues)[VM_FREEPOOL_DEFAULT]; + vm_phys_enq_range(&m_ret[npages], npages_end - npages, fl, 0); + } return (m_ret); }