From owner-freebsd-arch@FreeBSD.ORG Mon Jul 26 19:29:07 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7824A1065676 for ; Mon, 26 Jul 2010 19:29:07 +0000 (UTC) (envelope-from peter@wemm.org) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 5272B8FC13 for ; Mon, 26 Jul 2010 19:29:07 +0000 (UTC) Received: by pzk7 with SMTP id 7so1306491pzk.13 for ; Mon, 26 Jul 2010 12:29:06 -0700 (PDT) MIME-Version: 1.0 Received: by 10.142.200.21 with SMTP id x21mr3698302wff.207.1280172546682; Mon, 26 Jul 2010 12:29:06 -0700 (PDT) Received: by 10.229.237.73 with HTTP; Mon, 26 Jul 2010 12:29:05 -0700 (PDT) In-Reply-To: <4C4DD1AA.3050906@freebsd.org> References: <4C4DB2B8.9080404@freebsd.org> <4C4DD1AA.3050906@freebsd.org> Date: Mon, 26 Jul 2010 12:29:05 -0700 Message-ID: From: Peter Wemm To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 Cc: Matthew Fleming , freebsd-arch@freebsd.org Subject: Re: amd64: change VM_KMEM_SIZE_SCALE to 1? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jul 2010 19:29:07 -0000 On Mon, Jul 26, 2010 at 11:19 AM, Andriy Gapon wrote: > on 26/07/2010 20:04 Matthew Fleming said the following: >> On Mon, Jul 26, 2010 at 9:07 AM, Andriy Gapon wrote: >>> Anyone knows any reason why VM_KMEM_SIZE_SCALE on amd64 should not be set to 1? >>> I mean things potentially breaking, or some unpleasant surprise for an >>> administrator/user... The amd64 kernel has a fixed size limit of kva, of which the kmem_map must fit inside. Most consumers of malloc() use the free direct map region, but there are some notable abusers of malloc (zfs being the prime offender) that prevent the use of the free direct map region for its allocations. I'm not familiar with how VM_KMEM_SIZE_SCALE's calculations work but I think it would be a crying shame to waste a huge chunk of finite kva space on systems that aren't handicapped by ZFS's abuse of malloc(). We've run out of kva space on amd64 in the past. To recap.. the amd64 kernel has a place to do temporary mappings. This space is finite. 6G on newer systems, 2G on older ones. This is most often used to remap discontiguous pages into virtually contiguous address space. The kernel also sets up a 1:1 virtual<->physical map region so it can get to any page on the system without requiring a kva mapping. If its clear that changing VM_KMEM_SIZE_SCALE makes sense for the common case then that's different. Of course, with machines with 128G / 256G of physical ram either already here or just around the corner, its time to start thinking hard about physical ram based scaling calculations again. That hard limit of 512G of physical ram doesn't seem so distant anymore.. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell