From owner-p4-projects@FreeBSD.ORG Sun Nov 18 07:30:48 2007 Return-Path: Delivered-To: p4-projects@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 32767) id 8F3F616A41B; Sun, 18 Nov 2007 07:30:48 +0000 (UTC) Delivered-To: perforce@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F315916A419 for ; Sun, 18 Nov 2007 07:30:47 +0000 (UTC) (envelope-from kmacy@freebsd.org) Received: from repoman.freebsd.org (repoman.freebsd.org [IPv6:2001:4f8:fff6::29]) by mx1.freebsd.org (Postfix) with ESMTP id E001313C44B for ; Sun, 18 Nov 2007 07:30:47 +0000 (UTC) (envelope-from kmacy@freebsd.org) Received: from repoman.freebsd.org (localhost [127.0.0.1]) by repoman.freebsd.org (8.14.1/8.14.1) with ESMTP id lAI7UlUp093832 for ; Sun, 18 Nov 2007 07:30:47 GMT (envelope-from kmacy@freebsd.org) Received: (from perforce@localhost) by repoman.freebsd.org (8.14.1/8.14.1/Submit) id lAI7UlQl093829 for perforce@freebsd.org; Sun, 18 Nov 2007 07:30:47 GMT (envelope-from kmacy@freebsd.org) Date: Sun, 18 Nov 2007 07:30:47 GMT Message-Id: <200711180730.lAI7UlQl093829@repoman.freebsd.org> X-Authentication-Warning: repoman.freebsd.org: perforce set sender to kmacy@freebsd.org using -f From: Kip Macy To: Perforce Change Reviews Cc: Subject: PERFORCE change 129199 for review X-BeenThere: p4-projects@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: p4 projects tree changes List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Nov 2007 07:30:48 -0000 http://perforce.freebsd.org/chv.cgi?CH=129199 Change 129199 by kmacy@kmacy:storage:toestack on 2007/11/18 07:29:55 - Add efficient routine for holding user pages for reed/write in order to allow devices to do DMA to user pages - this needs an additional per process accounting check to prevent a user application from holding too many pages at once by sending large amounts of data over slow connections Affected files ... .. //depot/projects/toestack/sys/vm/vm_extern.h#4 edit .. //depot/projects/toestack/sys/vm/vm_fault.c#4 edit Differences ... ==== //depot/projects/toestack/sys/vm/vm_extern.h#4 (text+ko) ==== @@ -98,5 +98,8 @@ void vm_thread_swapout(struct thread *td); void *contigmalloc2(vm_page_t m, vm_pindex_t npages, int flags); + +#define VM_HOLD_WRITEABLE 0x1 +int vm_fault_hold_user_pages(vm_offset_t addr, int len, vm_page_t *mp, int *count, int flags); #endif /* _KERNEL */ #endif /* !_VM_EXTERN_H_ */ ==== //depot/projects/toestack/sys/vm/vm_fault.c#4 (text+ko) ==== @@ -1308,3 +1308,111 @@ /* return number of pages */ return i; } + +/* + * This routine takes a user address range and does the following: + * - validate that the user has access to those pages (flags indicates read or write) - if not fail + * - validate that count is enough to hold range number of pages - if not fail + * - fault in any non-resident pages + * - if the user is doing a read force a write fault for any COWed pages + * - if the user is doing a read mark all pages as dirty + * - hold all pages + * - return number of pages in count + */ + +int +vm_fault_hold_user_pages(vm_offset_t addr, int len, vm_page_t *mp, int *count, int flags) +{ + + vm_offset_t start, va; + vm_paddr_t pa; + int pageslen, faults, rv; + + struct thread *td; + vm_map_t map; + pmap_t pmap; + vm_page_t m, *pages; + vm_prot_t prot; + + start = addr & ~PAGE_MASK; + pageslen = roundup2(addr + len, PAGE_SIZE); + if (*count < (pageslen >> PAGE_SHIFT)) + return (EFBIG); + + *count = pageslen >> PAGE_SHIFT; + /* + * Check that virtual address range is legal + * This check is somewhat bogus as on some architectures kernel + * and user do not share VA - however, it appears that all FreeBSD + * architectures define it + */ + if (addr + len > VM_MAXUSER_ADDRESS) + return (EFAULT); + + td = curthread; + map = &td->td_proc->p_vmspace->vm_map; + pmap = &td->td_proc->p_vmspace->vm_pmap; + pages = mp; + + prot = (flags & VM_HOLD_WRITEABLE) ? VM_PROT_WRITE : VM_PROT_READ; + bzero(pages, sizeof(vm_page_t *) * (*count)); +retry: + + /* + * First optimistically assume that all pages are resident (and R/W if for write) + * if so just mark pages as held (and dirty if for write) and return + */ + vm_page_lock_queues(); + for (pages = mp, faults = 0, va = start; va < pageslen; va += PAGE_SIZE, pages++) { + /* + * Assure that we only hold the page once + */ + if (*pages == NULL) { + /* + * page queue mutex is recursable so this is OK + * it would be really nice if we had an unlocked version of this so + * we were only acquiring the pmap lock 1 time as opposed to potentially + * many dozens of times + */ + m = pmap_extract_and_hold(pmap, va, prot); + if (m == NULL) { + faults++; + continue; + } + *pages = m; + if (flags & VM_HOLD_WRITEABLE) + vm_page_dirty(m); + } + } + vm_page_unlock_queues(); + + if (faults == 0) + return (0); + /* + * Pages either have insufficient permissions or are not present + * trigger a fault where neccessary + * + */ + for (va = start; va < pageslen; va += PAGE_SIZE) { + m = NULL; + pa = pmap_extract(pmap, va); + rv = 0; + if (pa) + m = PHYS_TO_VM_PAGE(pa); + if (flags & VM_HOLD_WRITEABLE) { + if (m == NULL || (m->flags & PG_WRITEABLE) == 0) + rv = vm_fault(map, va, VM_PROT_WRITE, VM_FAULT_DIRTY); + } else if (m == NULL) + rv = vm_fault(map, va, VM_PROT_READ, VM_FAULT_NORMAL); + if (rv) + goto error; + } + goto retry; + + for (pages = mp, va = start; va < pageslen; va += PAGE_SIZE, pages++) + if (*pages) + vm_page_unhold(*pages); + + return (EFAULT); +} +