From owner-freebsd-arch@FreeBSD.ORG Sun Feb 3 10:04:48 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id AEF696FD; Sun, 3 Feb 2013 10:04:48 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id BDE6EFD9; Sun, 3 Feb 2013 10:04:47 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA06990; Sun, 03 Feb 2013 12:04:44 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1U1wRI-0006Aj-15; Sun, 03 Feb 2013 12:04:44 +0200 Message-ID: <510E363B.9050207@FreeBSD.org> Date: Sun, 03 Feb 2013 12:04:43 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130121 Thunderbird/17.0.2 MIME-Version: 1.0 To: Alan Cox Subject: Re: kva size on amd64 References: <507E7E59.8060201@FreeBSD.org> <51098743.2050603@FreeBSD.org> <510A2C09.6030709@FreeBSD.org> <510AB848.3010806@rice.edu> <510B8F2B.5070609@FreeBSD.org> <510D5C37.6000507@rice.edu> In-Reply-To: <510D5C37.6000507@rice.edu> X-Enigmail-Version: 1.4.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: alc@FreeBSD.org, Alan Cox , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Feb 2013 10:04:48 -0000 on 02/02/2013 20:34 Alan Cox said the following: > On 02/01/2013 03:47, Andriy Gapon wrote: >> Actually, I have been obsessed quite for some time with an idea of confining ZFS >> to its own submap. But ZFS does its allocations through malloc(9) and uma(9) >> (depending on configuration). It seemed like a bit of work to provide support >> for per-zone or per-tag submaps in uma and malloc. >> What is your opinion of this approach? > > I'm skeptical that it would accomplish anything. Specifically, I don't > think that it would have any impact on the fragmentation problem that we > have with ZFS. On amd64, with its direct map, all small allocations are > implemented by uma_small_alloc() and accessed through the direct map, > rather than coming from the kmem map. Outside of ZFS, large, multipage > allocations from the kmem map aren't that common. So, for all practical > purposes, ZFS has the kmem map to itself. I agree. My line of thinking was this. Allocate smaller kmem_map bu default (e.g. 1/3 of memory as it was before). If ZFS is not used, then that's it. If ZFS is used then it allocated an additional submap out of kernel_map. The submap would be sized based on configured or autoconfigured maximum size of ARC, e.g. 1.5 * arc_max_size. But that's probably more wasteful than needed for those who use ZFS. > While I'm here, I'll offer some other food for thought. In HEAD, we > have a new-ish function, vm_page_alloc_contig(), that can allocate > contiguous, unmapped physical pages either to an arbitrary vm object or > VM_ALLOC_NOOBJ, just like vm_page_alloc(). Moreover, just like > vm_page_alloc(), it honors the VM_ALLOC_{NORMAL,SYSTEM,INTERRUPT} > thresholds and wakes the page daemon when appropriate. > > Using this function, you could rewrite the multipage allocation code to > first attempt allocation through vm_page_alloc_contig() and then fall > back to the kmem map only if vm_page_alloc_contig() fails. This is a very interesting idea for systems with direct map! P.S. I hope I don't get an indigestion with all the food :-) -- Andriy Gapon