From owner-svn-src-all@FreeBSD.ORG Mon Jul 5 21:13:33 2010 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7859A106566B; Mon, 5 Jul 2010 21:13:33 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 66D918FC13; Mon, 5 Jul 2010 21:13:33 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o65LDXdF066162; Mon, 5 Jul 2010 21:13:33 GMT (envelope-from kib@svn.freebsd.org) Received: (from kib@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o65LDXFG066161; Mon, 5 Jul 2010 21:13:33 GMT (envelope-from kib@svn.freebsd.org) Message-Id: <201007052113.o65LDXFG066161@svn.freebsd.org> From: Konstantin Belousov Date: Mon, 5 Jul 2010 21:13:33 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r209713 - in head/sys: kern vm X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2010 21:13:33 -0000 Author: kib Date: Mon Jul 5 21:13:32 2010 New Revision: 209713 URL: http://svn.freebsd.org/changeset/base/209713 Log: Add the ability for the allocflag argument of the vm_page_grab() to specify the increment of vm_pageout_deficit when sleeping due to page shortage. Then, in allocbuf(), the code to allocate pages when extending vmio buffer can be replaced by a call to vm_page_grab(). Suggested and reviewed by: alc MFC after: 2 weeks Modified: head/sys/kern/vfs_bio.c head/sys/vm/vm_page.c head/sys/vm/vm_page.h Modified: head/sys/kern/vfs_bio.c ============================================================================== --- head/sys/kern/vfs_bio.c Mon Jul 5 19:01:10 2010 (r209712) +++ head/sys/kern/vfs_bio.c Mon Jul 5 21:13:32 2010 (r209713) @@ -3013,63 +3013,24 @@ allocbuf(struct buf *bp, int size) VM_OBJECT_LOCK(obj); while (bp->b_npages < desiredpages) { vm_page_t m; - vm_pindex_t pi; - - pi = OFF_TO_IDX(bp->b_offset) + bp->b_npages; - if ((m = vm_page_lookup(obj, pi)) == NULL) { - /* - * note: must allocate system pages - * since blocking here could intefere - * with paging I/O, no matter which - * process we are. - */ - m = vm_page_alloc(obj, pi, - VM_ALLOC_NOBUSY | VM_ALLOC_SYSTEM | - VM_ALLOC_WIRED); - if (m == NULL) { - atomic_add_int(&vm_pageout_deficit, - desiredpages - bp->b_npages); - VM_OBJECT_UNLOCK(obj); - VM_WAIT; - VM_OBJECT_LOCK(obj); - } else { - if (m->valid == 0) - bp->b_flags &= ~B_CACHE; - bp->b_pages[bp->b_npages] = m; - ++bp->b_npages; - } - continue; - } /* - * We found a page. If we have to sleep on it, - * retry because it might have gotten freed out - * from under us. + * We must allocate system pages since blocking + * here could intefere with paging I/O, no + * matter which process we are. * * We can only test VPO_BUSY here. Blocking on * m->busy might lead to a deadlock: - * * vm_fault->getpages->cluster_read->allocbuf - * - */ - if ((m->oflags & VPO_BUSY) != 0) { - /* - * Reference the page before unlocking - * and sleeping so that the page daemon - * is less likely to reclaim it. - */ - vm_page_lock_queues(); - vm_page_flag_set(m, PG_REFERENCED); - vm_page_sleep(m, "pgtblk"); - continue; - } - - /* - * We have a good page. + * Thus, we specify VM_ALLOC_IGN_SBUSY. */ - vm_page_lock(m); - vm_page_wire(m); - vm_page_unlock(m); + m = vm_page_grab(obj, OFF_TO_IDX(bp->b_offset) + + bp->b_npages, VM_ALLOC_NOBUSY | + VM_ALLOC_SYSTEM | VM_ALLOC_WIRED | + VM_ALLOC_RETRY | VM_ALLOC_IGN_SBUSY | + VM_ALLOC_COUNT(desiredpages - bp->b_npages)); + if (m->valid == 0) + bp->b_flags &= ~B_CACHE; bp->b_pages[bp->b_npages] = m; ++bp->b_npages; } Modified: head/sys/vm/vm_page.c ============================================================================== --- head/sys/vm/vm_page.c Mon Jul 5 19:01:10 2010 (r209712) +++ head/sys/vm/vm_page.c Mon Jul 5 21:13:32 2010 (r209713) @@ -2038,11 +2038,13 @@ vm_page_t vm_page_grab(vm_object_t object, vm_pindex_t pindex, int allocflags) { vm_page_t m; + u_int count; VM_OBJECT_LOCK_ASSERT(object, MA_OWNED); retrylookup: if ((m = vm_page_lookup(object, pindex)) != NULL) { - if ((m->oflags & VPO_BUSY) != 0 || m->busy != 0) { + if ((m->oflags & VPO_BUSY) != 0 || + ((allocflags & VM_ALLOC_IGN_SBUSY) == 0 && m->busy != 0)) { if ((allocflags & VM_ALLOC_RETRY) != 0) { /* * Reference the page before unlocking and @@ -2067,9 +2069,13 @@ retrylookup: return (m); } } - m = vm_page_alloc(object, pindex, allocflags & ~VM_ALLOC_RETRY); + m = vm_page_alloc(object, pindex, allocflags & ~(VM_ALLOC_RETRY | + VM_ALLOC_IGN_SBUSY | VM_ALLOC_COUNT_MASK)); if (m == NULL) { VM_OBJECT_UNLOCK(object); + count = (u_int)allocflags >> VM_ALLOC_COUNT_SHIFT; + if (count > 0) + atomic_add_int(&vm_pageout_deficit, count); VM_WAIT; VM_OBJECT_LOCK(object); if ((allocflags & VM_ALLOC_RETRY) == 0) Modified: head/sys/vm/vm_page.h ============================================================================== --- head/sys/vm/vm_page.h Mon Jul 5 19:01:10 2010 (r209712) +++ head/sys/vm/vm_page.h Mon Jul 5 21:13:32 2010 (r209713) @@ -317,6 +317,11 @@ extern struct vpglocks vm_page_queue_loc #define VM_ALLOC_NOBUSY 0x0200 /* Do not busy the page */ #define VM_ALLOC_IFCACHED 0x0400 /* Fail if the page is not cached */ #define VM_ALLOC_IFNOTCACHED 0x0800 /* Fail if the page is cached */ +#define VM_ALLOC_IGN_SBUSY 0x1000 /* vm_page_grab() only */ + +#define VM_ALLOC_COUNT_SHIFT 16 +#define VM_ALLOC_COUNT(count) ((count) << VM_ALLOC_COUNT_SHIFT) +#define VM_ALLOC_COUNT_MASK VM_ALLOC_COUNT(0xffff) void vm_page_flag_set(vm_page_t m, unsigned short bits); void vm_page_flag_clear(vm_page_t m, unsigned short bits);