From owner-freebsd-current@FreeBSD.ORG Fri Oct 10 20:42:55 2014 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 149A3B7C for ; Fri, 10 Oct 2014 20:42:55 +0000 (UTC) Received: from mx2.shrew.net (mx2.shrew.net [38.97.5.132]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C1DD7FBA for ; Fri, 10 Oct 2014 20:42:53 +0000 (UTC) Received: from mail.shrew.net (mail.shrew.prv [10.24.10.20]) by mx2.shrew.net (8.14.7/8.14.7) with ESMTP id s9AKSsDI025791 for ; Fri, 10 Oct 2014 15:28:54 -0500 (CDT) (envelope-from mgrooms@shrew.net) Received: from [10.16.32.30] (rrcs-50-84-127-134.sw.biz.rr.com [50.84.127.134]) by mail.shrew.net (Postfix) with ESMTPSA id 760251884F7 for ; Fri, 10 Oct 2014 15:28:43 -0500 (CDT) Message-ID: <543841B8.4070007@shrew.net> Date: Fri, 10 Oct 2014 15:29:44 -0500 From: Matthew Grooms User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-current@freebsd.org Subject: Resizing a zpool as a VMware ESXi guest ... Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mx2.shrew.net [10.24.10.11]); Fri, 10 Oct 2014 15:28:54 -0500 (CDT) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Oct 2014 20:42:55 -0000 All, I am a long time user and advocate of FreeBSD and manage a several deployments of FreeBSD in a few data centers. Now that these environments are almost always virtual, it would make sense that FreeBSD support for basic features such as dynamic disk resizing. It looks like most of the parts are intended to work. Kudos to the FreeBSD foundation for seeing the need and sponsoring dynamic increase of online UFS filesystems via growfs. Unfortunately, it would appear that there are still problems in this area, such as ... a) cam/geom recognizing when a drive's size has increased b) zpool recognizing when a gpt partition size has increased For example, if I do an install of FreeBSD 10 on VMware using ZFS, I see the following ... root@zpool-test:~ # gpart show => 34 16777149 da0 GPT (8.0G) 34 1024 1 freebsd-boot (512K) 1058 4194304 2 freebsd-swap (2.0G) 4195362 12581821 3 freebsd-zfs (6.0G) If I increase the VM disk size using VMware to 16G and rescan using camcontrol, this is what I see ... root@zpool-test:~ # camcontrol rescan all Re-scan of bus 0 was successful Re-scan of bus 1 was successful Re-scan of bus 2 was successful root@zpool-test:~ # gpart show => 34 16777149 da0 GPT (8.0G) 34 1024 1 freebsd-boot (512K) 1058 4194304 2 freebsd-swap (2.0G) 4195362 12581821 3 freebsd-zfs (6.0G) The GPT label still appears to be 8G. If I reboot the VM, it picks up the correct size ... root@zpool-test:~ # gpart show => 34 16777149 da0 GPT (16G) [CORRUPT] 34 1024 1 freebsd-boot (512K) 1058 4194304 2 freebsd-swap (2.0G) 4195362 12581821 3 freebsd-zfs (6.0G) Now I have 16G to play with. I'll expand the freebsd-zfs partition to claim the additional space ... root@zpool-test:~ # gpart recover da0 da0 recovered root@zpool-test:~ # gpart show => 34 33554365 da0 GPT (16G) 34 1024 1 freebsd-boot (512K) 1058 4194304 2 freebsd-swap (2.0G) 4195362 12581821 3 freebsd-zfs (6.0G) 16777183 16777216 - free - (8.0G) root@zpool-test:~ # gpart resize -i 3 da0 root@zpool-test:~ # gpart show => 34 33554365 da0 GPT (16G) 34 1024 1 freebsd-boot (512K) 1058 4194304 2 freebsd-swap (2.0G) 4195362 29359037 3 freebsd-zfs (14G) Now I want the claim the additional 14 gigs of space for my zpool ... root@zpool-test:~ # zpool status pool: zroot state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 gptid/352086bd-50b5-11e4-95b8-0050569b2a04 ONLINE 0 0 0 root@zpool-test:~ # zpool set autoexpand=on zroot root@zpool-test:~ # zpool online -e zroot gptid/352086bd-50b5-11e4-95b8-0050569b2a04 root@zpool-test:~ # zpool list NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT zroot 5.97G 876M 5.11G 14% 1.00x ONLINE - The zpool appears to still only have 5.11G free. Lets reboot and try again ... root@zpool-test:~ # zpool set autoexpand=on zroot root@zpool-test:~ # zpool online -e zroot gptid/352086bd-50b5-11e4-95b8-0050569b2a04 root@zpool-test:~ # zpool list NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT zroot 14.0G 876M 13.1G 6% 1.00x ONLINE - Now I have 13.1G free. I can add this space to any of my zfs volumes and it picks the change up immediately. So the question remains, why do I need to reboot the OS twice to allocate new disk space to a volume? FreeBSD is first and foremost a server operating system. Servers are commonly deployed in data centers. Virtual environments are now commonplace in data centers, not the exception to the rule. VMware still has the vast majority of the private virutal environment market. I assume that most would expect things like this to work out of the box. Did I miss a required step or is this fixed in CURRENT? Thanks, -Matthew