From owner-freebsd-stable@FreeBSD.ORG Tue Aug 5 18:29:25 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6C50B106567E for ; Tue, 5 Aug 2008 18:29:25 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id 244038FC2B for ; Tue, 5 Aug 2008 18:29:24 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.14.1/8.14.1) with ESMTP id m75ITM21029178; Tue, 5 Aug 2008 11:29:22 -0700 (PDT) Received: (from dillon@localhost) by apollo.backplane.com (8.14.1/8.13.4/Submit) id m75ITLmS029177; Tue, 5 Aug 2008 11:29:21 -0700 (PDT) Date: Tue, 5 Aug 2008 11:29:21 -0700 (PDT) From: Matthew Dillon Message-Id: <200808051829.m75ITLmS029177@apollo.backplane.com> To: Max Laier References: <47713ee10808050839k5b258831x66bc52f70b2c355b@mail.gmail.com> <200808051919.34577.max@love2party.net> Cc: freebsd-stable@freebsd.org, Lin Jui-Nan Eric Subject: Re: Max size of one swap slice X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2008 18:29:25 -0000 :> Recently we found that we can only allocate 32GB for one swap slice. :> Does there is any sysctl oid or any kernel option to increase it? Why :> we have this restriction? : :this is a consequence of the data structure used to manage swap space. See :sys/blist.h for details. It *seems* that you *might* be able to increase the :coverage by decreasing BLIST_META_RADIX, but that's from a quick glance and :most certainly not a good idea. : :However, the blist is a abstract enough API so that you can likely replace it :with something that supports 64bit addresses (and thus 512*2^64 bytes of swap :space per device) ... but I don't see why you'd want to do something like :this. Remember that you need memory to manage your swap space as well! : :-- :/"\ Best regards, | mlaier@freebsd.org :\ / Max Laier | ICQ #67774661 The core structures can handle 2 billion swap pages == 2TB of swap, but the blist code hits arithmatic overflows if a single blist has more then (0x40000000 / BLIST_META_RADIX) = 1G/16 = 64M swap blocks, or 256GB. I think the VM/BIO system had additional overflow issues due to conversions back and forth between PAGE_SIZE and DEV_BSIZE which further restricted the limit to 32GB. Those restrictions may be gone now that FreeBSD is using 64 bit block numbers, so you may be able to pop it up to 256GB with virtually no effort (but you need to test it significantly!). With some work on the blist code only (not its structures) the arithmatic overflow issues could also be resolved, increasing the swap capability to 2TB. I do not recommend changing any of the core blist structure, particularly not BLIST_META_RADIX. Just don't try :-). You do NOT want to bump the swap block number fields to 64 bits. Also note that significant memory is used to manage that much swap. It's a factor of 1:16384 or so for the blist structures and probably about the same amount for the vm_object tracking structures. 32G of swap needs around 2-4MB of wired ram. -Matt Matthew Dillon