From owner-svn-src-all@freebsd.org Thu Jun 23 08:28:14 2016 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5403FAC6715; Thu, 23 Jun 2016 08:28:14 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0AE04209F; Thu, 23 Jun 2016 08:28:13 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u5N8SDAJ050388; Thu, 23 Jun 2016 08:28:13 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u5N8SDaT050387; Thu, 23 Jun 2016 08:28:13 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201606230828.u5N8SDaT050387@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Thu, 23 Jun 2016 08:28:13 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r302131 - head/sys/vm X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jun 2016 08:28:14 -0000 Author: kib Date: Thu Jun 23 08:28:13 2016 New Revision: 302131 URL: https://svnweb.freebsd.org/changeset/base/302131 Log: In vm_page_xunbusy_maybelocked(), add fast path for unbusy when no waiters exist, same as for vm_page_xunbusy(). If previous value of busy_lock was VPB_SINGLE_EXCLUSIVER, no waiters existed and wakeup is not needed. Move common code from vm_page_xunbusy_maybelocked() and vm_page_xunbusy_hard() to vm_page_xunbusy_locked(). Reviewed by: alc Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Approved by: re (gjb) Modified: head/sys/vm/vm_page.c Modified: head/sys/vm/vm_page.c ============================================================================== --- head/sys/vm/vm_page.c Thu Jun 23 08:27:38 2016 (r302130) +++ head/sys/vm/vm_page.c Thu Jun 23 08:28:13 2016 (r302131) @@ -760,17 +760,36 @@ vm_page_trysbusy(vm_page_t m) } static void +vm_page_xunbusy_locked(vm_page_t m) +{ + + vm_page_assert_xbusied(m); + vm_page_assert_locked(m); + + atomic_store_rel_int(&m->busy_lock, VPB_UNBUSIED); + /* There is a waiter, do wakeup() instead of vm_page_flash(). */ + wakeup(m); +} + +static void vm_page_xunbusy_maybelocked(vm_page_t m) { bool lockacq; vm_page_assert_xbusied(m); + /* + * Fast path for unbusy. If it succeeds, we know that there + * are no waiters, so we do not need a wakeup. + */ + if (atomic_cmpset_rel_int(&m->busy_lock, VPB_SINGLE_EXCLUSIVER, + VPB_UNBUSIED)) + return; + lockacq = !mtx_owned(vm_page_lockptr(m)); if (lockacq) vm_page_lock(m); - vm_page_flash(m); - atomic_store_rel_int(&m->busy_lock, VPB_UNBUSIED); + vm_page_xunbusy_locked(m); if (lockacq) vm_page_unlock(m); } @@ -788,8 +807,7 @@ vm_page_xunbusy_hard(vm_page_t m) vm_page_assert_xbusied(m); vm_page_lock(m); - atomic_store_rel_int(&m->busy_lock, VPB_UNBUSIED); - wakeup(m); + vm_page_xunbusy_locked(m); vm_page_unlock(m); }