From owner-svn-src-all@FreeBSD.ORG  Fri Apr  6 16:41:20 2012
Return-Path: <owner-svn-src-all@FreeBSD.ORG>
Delivered-To: svn-src-all@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id 32A4E1065670;
	Fri,  6 Apr 2012 16:41:20 +0000 (UTC) (envelope-from alc@FreeBSD.org)
Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c])
	by mx1.freebsd.org (Postfix) with ESMTP id 1D7418FC14;
	Fri,  6 Apr 2012 16:41:20 +0000 (UTC)
Received: from svn.freebsd.org (localhost [127.0.0.1])
	by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q36GfJMe017533;
	Fri, 6 Apr 2012 16:41:19 GMT (envelope-from alc@svn.freebsd.org)
Received: (from alc@localhost)
	by svn.freebsd.org (8.14.4/8.14.4/Submit) id q36GfJkI017531;
	Fri, 6 Apr 2012 16:41:19 GMT (envelope-from alc@svn.freebsd.org)
Message-Id: <201204061641.q36GfJkI017531@svn.freebsd.org>
From: Alan Cox <alc@FreeBSD.org>
Date: Fri, 6 Apr 2012 16:41:19 +0000 (UTC)
To: src-committers@freebsd.org, svn-src-all@freebsd.org,
	svn-src-head@freebsd.org
X-SVN-Group: head
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Cc: 
Subject: svn commit: r233954 - head/sys/amd64/amd64
X-BeenThere: svn-src-all@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: "SVN commit messages for the entire src tree \(except for &quot;
	user&quot; and &quot; projects&quot; \)" <svn-src-all.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/svn-src-all>,
	<mailto:svn-src-all-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-all>
List-Post: <mailto:svn-src-all@freebsd.org>
List-Help: <mailto:svn-src-all-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/svn-src-all>,
	<mailto:svn-src-all-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 06 Apr 2012 16:41:20 -0000

Author: alc
Date: Fri Apr  6 16:41:19 2012
New Revision: 233954
URL: http://svn.freebsd.org/changeset/base/233954

Log:
  Micro-optimize free_pv_entry() for the expected case.

Modified:
  head/sys/amd64/amd64/pmap.c

Modified: head/sys/amd64/amd64/pmap.c
==============================================================================
--- head/sys/amd64/amd64/pmap.c	Fri Apr  6 16:32:29 2012	(r233953)
+++ head/sys/amd64/amd64/pmap.c	Fri Apr  6 16:41:19 2012	(r233954)
@@ -2095,7 +2095,6 @@ pmap_collect(pmap_t locked_pmap, struct 
 	}
 }
 
-
 /*
  * free the pv_entry back to the free list
  */
@@ -2116,13 +2115,16 @@ free_pv_entry(pmap_t pmap, pv_entry_t pv
 	field = idx / 64;
 	bit = idx % 64;
 	pc->pc_map[field] |= 1ul << bit;
-	/* move to head of list */
-	TAILQ_REMOVE(&pmap->pm_pvchunk, pc, pc_list);
 	if (pc->pc_map[0] != PC_FREE0 || pc->pc_map[1] != PC_FREE1 ||
 	    pc->pc_map[2] != PC_FREE2) {
-		TAILQ_INSERT_HEAD(&pmap->pm_pvchunk, pc, pc_list);
+		/* 98% of the time, pc is already at the head of the list. */
+		if (__predict_false(pc != TAILQ_FIRST(&pmap->pm_pvchunk))) {
+			TAILQ_REMOVE(&pmap->pm_pvchunk, pc, pc_list);
+			TAILQ_INSERT_HEAD(&pmap->pm_pvchunk, pc, pc_list);
+		}
 		return;
 	}
+	TAILQ_REMOVE(&pmap->pm_pvchunk, pc, pc_list);
 	PV_STAT(pv_entry_spare -= _NPCPV);
 	PV_STAT(pc_chunk_count--);
 	PV_STAT(pc_chunk_frees++);