From owner-p4-projects@FreeBSD.ORG Sun Feb 5 21:48:48 2006 Return-Path: X-Original-To: p4-projects@freebsd.org Delivered-To: p4-projects@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 32767) id E359A16A420; Sun, 5 Feb 2006 21:48:47 +0000 (GMT) X-Original-To: perforce@freebsd.org Delivered-To: perforce@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8CA5616A420 for ; Sun, 5 Feb 2006 21:48:47 +0000 (GMT) (envelope-from alc@freebsd.org) Received: from repoman.freebsd.org (repoman.freebsd.org [216.136.204.115]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1CE7543D46 for ; Sun, 5 Feb 2006 21:48:47 +0000 (GMT) (envelope-from alc@freebsd.org) Received: from repoman.freebsd.org (localhost [127.0.0.1]) by repoman.freebsd.org (8.13.1/8.13.1) with ESMTP id k15LmknN077452 for ; Sun, 5 Feb 2006 21:48:47 GMT (envelope-from alc@freebsd.org) Received: (from perforce@localhost) by repoman.freebsd.org (8.13.1/8.13.1/Submit) id k15Lmkem077449 for perforce@freebsd.org; Sun, 5 Feb 2006 21:48:46 GMT (envelope-from alc@freebsd.org) Date: Sun, 5 Feb 2006 21:48:46 GMT Message-Id: <200602052148.k15Lmkem077449@repoman.freebsd.org> X-Authentication-Warning: repoman.freebsd.org: perforce set sender to alc@freebsd.org using -f From: Alan Cox To: Perforce Change Reviews Cc: Subject: PERFORCE change 91189 for review X-BeenThere: p4-projects@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: p4 projects tree changes List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Feb 2006 21:48:49 -0000 http://perforce.freebsd.org/chv.cgi?CH=91189 Change 91189 by alc@alc_home on 2006/02/05 21:48:15 A contiguity-aware reimplementation of uma_small_alloc(). This improves performance by reducing the number of 2MB page TLB misses for heap allocated data accessed through the direct map. Affected files ... .. //depot/projects/superpages/src/sys/amd64/amd64/uma_machdep.c#2 edit Differences ... ==== //depot/projects/superpages/src/sys/amd64/amd64/uma_machdep.c#2 (text+ko) ==== @@ -28,10 +28,12 @@ __FBSDID("$FreeBSD: src/sys/amd64/amd64/uma_machdep.c,v 1.1 2003/10/14 05:51:31 alc Exp $"); #include +#include #include #include #include #include +#include #include #include #include @@ -39,11 +41,29 @@ #include #include +static struct vm_object uma_small_object_store; + +static void uma_small_init(void *notused); +SYSINIT(uma_small, SI_SUB_VM, SI_ORDER_SECOND, uma_small_init, NULL) + +static void +uma_small_init(void *notused) +{ + vm_object_t object; + + object = &uma_small_object_store; + VM_OBJECT_LOCK_INIT(object, "uma small object"); + _vm_object_allocate(OBJT_DEFAULT, cnt.v_page_count, object); + object->pg_color = 0; + vm_object_set_flag(object, OBJ_SUPERPAGES); +} + void * uma_small_alloc(uma_zone_t zone, int bytes, u_int8_t *flags, int wait) { - static vm_pindex_t colour; + vm_object_t object; vm_page_t m; + vm_pindex_t pindex; void *va; int pflags; @@ -54,8 +74,12 @@ pflags = VM_ALLOC_SYSTEM; if (wait & M_ZERO) pflags |= VM_ALLOC_ZERO; - for (;;) { - m = vm_page_alloc(NULL, colour++, pflags | VM_ALLOC_NOOBJ); + for (object = &uma_small_object_store;;) { + VM_OBJECT_LOCK(object); + m = TAILQ_LAST(&object->memq, pglist); + pindex = m != NULL ? m->pindex + 1 : 0; + m = vm_page_alloc(object, pindex, pflags); + VM_OBJECT_UNLOCK(object); if (m == NULL) { if (wait & M_NOWAIT) return (NULL); @@ -73,10 +97,14 @@ void uma_small_free(void *mem, int size, u_int8_t flags) { + vm_object_t object; vm_page_t m; + object = &uma_small_object_store; m = PHYS_TO_VM_PAGE(DMAP_TO_PHYS((vm_offset_t)mem)); + VM_OBJECT_LOCK(object); vm_page_lock_queues(); vm_page_free(m); vm_page_unlock_queues(); + VM_OBJECT_UNLOCK(object); }