From owner-freebsd-fs@FreeBSD.ORG Sun Dec 30 17:20:19 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8FD9216A419 for ; Sun, 30 Dec 2007 17:20:19 +0000 (UTC) (envelope-from johan@stromnet.se) Received: from core.stromnet.se (core.stromnet.se [83.218.84.131]) by mx1.freebsd.org (Postfix) with ESMTP id 2AC0B13C455 for ; Sun, 30 Dec 2007 17:20:18 +0000 (UTC) (envelope-from johan@stromnet.se) Received: from localhost (unknown [83.218.84.135]) by core.stromnet.se (Postfix) with ESMTP id EF34BD467B2 for ; Sun, 30 Dec 2007 18:20:17 +0100 (CET) X-Virus-Scanned: amavisd-new at stromnet.se Received: from core.stromnet.se ([83.218.84.131]) by localhost (core.stromnet.se [83.218.84.135]) (amavisd-new, port 10024) with ESMTP id 9qOqHg6Og0GB for ; Sun, 30 Dec 2007 18:20:15 +0100 (CET) Received: from [172.28.1.102] (90-224-172-102-no129.tbcn.telia.com [90.224.172.102]) by core.stromnet.se (Postfix) with ESMTP id B1870D467B0 for ; Sun, 30 Dec 2007 18:20:15 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v753) In-Reply-To: <5A6CFB06-4175-452F-BFC9-323C2023D2F6@stromnet.se> References: <5A6CFB06-4175-452F-BFC9-323C2023D2F6@stromnet.se> Content-Type: text/plain; charset=ISO-8859-1; delsp=yes; format=flowed Message-Id: <66640AA3-31BC-4E00-9E59-AF8FEA25EFD1@stromnet.se> Content-Transfer-Encoding: quoted-printable From: =?ISO-8859-1?Q?Johan_Str=F6m?= Date: Sun, 30 Dec 2007 18:19:27 +0100 To: freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.753) Subject: Re: ZFS replace/expand problem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Dec 2007 17:20:19 -0000 On Dec 27, 2007, at 20:25 , Johan Str=F6m wrote: > Hello list > > First of all, I want to thank everybody involved in writing and =20 > porting ZFS to FreeBSD, its working (except for this problem) great =20= > for me! > > Now to my problem. To sumarize it, I want to replace two mirrored =20 > disk with bigger ones. Replace works well but the vdev doesnt =20 > expand until i do export/import. Details follows: > > I currently have the following setup: > > back-1 /$ zpool status > pool: tank > state: ONLINE > scrub: none requested > config: > NAME STATE READ WRITE CKSUM > tank ONLINE 0 0 0 > mirror ONLINE 0 0 0 > ad14s1d ONLINE 0 0 0 > ad16s1d ONLINE 0 0 0 > mirror ONLINE 0 0 0 > ad8 ONLINE 0 0 0 > ad10s2 ONLINE 0 0 0 > mirror ONLINE 0 0 0 > ad12 ONLINE 0 0 0 > ad10s1 ONLINE 0 0 0 > > The ad8/ad10/ad12 setup is kindof stupid, I know.. ad8 is a 80Gb =20 > and ad10 is a 120Gb, and a10 200Gb.. But now I want to replace =20 > those two mirrors with 4x 300GB (or rather 2x300 and 2x320). So my =20 > plan was to do something like: > > zpool replace tank ad8 ad18 > zpool replace tank ad10s2 ad20 > > where ad18 and ad20 are the two 300Gbs.. Then the same thing for =20 > ad12 and ad10s1.. But before I did that i wanted to make sure that =20 > it would actually expand as I'ev read, so i tried this first.. > On ad18/ad20 I had ad*s1a, a 500MB partition, and ad*s1g a ~280Gb =20 > partition. So i created a testtank with first ad*s1a: > > back-1 /$ zpool create testtank mirror /dev/ad18s1a /dev/ad20s1a > back-1 /$ zpool list > NAME SIZE USED AVAIL CAP HEALTH =20 > ALTROOT > tank 878G 812G 65.1G 92% ONLINE - > testtank 492M 111K 492M 0% ONLINE - > > back-1 /$ zpool status > .. > pool: testtank > state: ONLINE > scrub: none requested > config: > > NAME STATE READ WRITE CKSUM > testtank ONLINE 0 0 0 > mirror ONLINE 0 0 0 > ad18s1a ONLINE 0 0 0 > ad20s1a ONLINE 0 0 0 > > errors: No known data errors > back-1 /storage$ zpool replace testtank ad18s1a ad18s1g > status now shows > mirror ONLINE 0 0 0 > replacing ONLINE 0 0 0 > ad18s1a ONLINE 0 0 0 > ad18s1g ONLINE 0 0 0 > ad20s1a ONLINE 0 0 0 > > when that was done (and only ad18s1g was showing) i did > > back-1 /storage$ zpool replace testtank ad20s1a ad20s1g > > and then same replacing output as above (but for ad20) > Okey, so now when this is done.. it should have expanded one would =20 > think, right? > > back-1 /storage$ zpool list > NAME SIZE USED AVAIL CAP HEALTH =20 > ALTROOT > .. > testtank 492M 218K 492M 0% ONLINE - > > > Nope.. Waited a while, nothing happened.. Some googling gave me =20 > that export/import could be done: > > back-1 /storage$ zpool export testtank > back-1 /storage$ zpool import testtank > back-1 /storage$ zpool list > NAME SIZE USED AVAIL CAP HEALTH =20 > ALTROOT > .. > testtank 289G 132K 289G 0% ONLINE - > > Yey! Okey so it expands, but only after export/import.. Havent =20 > realy found much docs about this but according to ppl in =20 > #opensolaris this should not be necessary. > Not a big deal in this test case, but doing it for my real tank =20 > will require me to take the system down on an external boot medium =20 > (CD or something) I guess, and then do zfs export/import there, and =20= > then boot back up.. > Any guidelines how to do this? Will doing import/export from a CD =20 > (rescue shell I guess) work as I expect? Or what would be the =20 > smartest way (the actual downtime isnt such a big deal as long as =20 > it is quick and works). > For the record, I found a somewhat easeier solution.. Just reboot and =20= it was updated! Tested with my testtank first, reboot, worked. Then =20 did the same with my real tank (but with ad20 and ad18, not using =20 slices), and the extra space showed up fine after reboot. Thanks again for ZFS!=20= From owner-freebsd-fs@FreeBSD.ORG Sun Dec 30 17:22:45 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BF82316A417 for ; Sun, 30 Dec 2007 17:22:45 +0000 (UTC) (envelope-from gergely.czuczy@harmless.hu) Received: from marvin.harmless.hu (marvin.harmless.hu [195.56.55.204]) by mx1.freebsd.org (Postfix) with ESMTP id 76F6B13C45B for ; Sun, 30 Dec 2007 17:22:45 +0000 (UTC) (envelope-from gergely.czuczy@harmless.hu) Received: from localhost (marvin-mail [192.168.0.2]) by marvin.harmless.hu (Postfix) with ESMTP id 829567C1ADC; Sun, 30 Dec 2007 18:22:43 +0100 (CET) X-Virus-Scanned: by amavisd-new-2.4.2 (20060627) (Debian) at harmless.hu Received: from marvin.harmless.hu ([192.168.0.2]) by localhost (marvin.harmless.hu [192.168.0.2]) (amavisd-new, port 10024) with ESMTP id n-twNHDnD3KO; Sun, 30 Dec 2007 18:22:43 +0100 (CET) Received: from marvin.harmless.hu (localhost [127.0.0.1]) by marvin.harmless.hu (Postfix) with ESMTP id 1AF627C1ADD; Sun, 30 Dec 2007 18:22:41 +0100 (CET) Date: Sun, 30 Dec 2007 18:22:41 +0100 From: Gergely CZUCZY To: Johan =?utf-8?B?U3Ryw7Zt?= Message-ID: <20071230172241.GA89549@harmless.hu> References: <5A6CFB06-4175-452F-BFC9-323C2023D2F6@stromnet.se> <66640AA3-31BC-4E00-9E59-AF8FEA25EFD1@stromnet.se> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=x-unknown; protocol="application/pgp-signature"; boundary="+HP7ph2BbKc20aGI" Content-Disposition: inline In-Reply-To: <66640AA3-31BC-4E00-9E59-AF8FEA25EFD1@stromnet.se> User-Agent: mutt-ng/devel-r804 (FreeBSD) Cc: freebsd-fs@freebsd.org Subject: Re: ZFS replace/expand problem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Dec 2007 17:22:45 -0000 --+HP7ph2BbKc20aGI Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Dec 30, 2007 at 06:19:27PM +0100, Johan Str=C3=B6m wrote: > On Dec 27, 2007, at 20:25 , Johan Str=C3=B6m wrote: >=20 > >Hello list > > > >First of all, I want to thank everybody involved in writing and porting = ZFS to FreeBSD, its working (except for this problem) great=20 > >for me! > > > >Now to my problem. To sumarize it, I want to replace two mirrored disk w= ith bigger ones. Replace works well but the vdev doesnt expand=20 > >until i do export/import. Details follows: same experience here. > For the record, I found a somewhat easeier solution.. Just reboot and it = was updated! Tested with my testtank first, reboot, worked.=20 > Then did the same with my real tank (but with ad20 and ad18, not using sl= ices), and the extra space showed up fine after reboot. Sherlock, rebooting includes an export and an import. Sincerely, Gergely Czuczy mailto: gergely.czuczy@harmless.hu --=20 Weenies test. Geniuses solve problems that arise. --+HP7ph2BbKc20aGI Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) owF9VD2PFEcQXb6SkQngFxSRQTczzM6xt3djLWDuuAMssGEXWTKB1TtTu9O6nu5x f+zcXoSIECJApCBhO0ciIMCx80tICEH8DAJcPbtrOXIwmv6oevXqve5+dvpE5/iZ ozdvH6w9ff7i2OuTv47XKmetnEYV0zMuo26SdKPNrd6lrWgj2sh7SbKJvbTfHa9v 9bp7zfuP20palDYazWvMwOKBvVgLxuV3kJdMG7QDZyfRZrCK2+GmVoZbrmQGXAou 8d+9kWbSTFBH12WuCi6nGfzmlMUiqjWXlo0FBsGPEoZOhrCDOawnIaRJ0gdmIdnI ultZ2v/pNqwlRDuEW6pkFGz1YHt9cG2jgkYTWBZcBsLw6Wl/kR76/DTJ0h78T9Yg TSj18g0UQoHgxvqZ/3a5NhbUBJgQIdyEhkkLVoEloH3AGer5WBVz6namxAwLGhAo SSCnwGQBtdLteBD8sjv0ibsa8dpwJwRuDTRK7/vd83iQY21hojQhcwO1ViRIdQGm GpldsvO7FZ5bUbujGg9YzVfRMYwUGEfu8kMk/P/y1UjO5Qi2oQyutdLEteBmH5pB wG0JYz6dogYl0cRwbxnt+RFLUgXGjnBKhFmBMygUGsLFg5p6XLJz0nIBnPb8OrV9 kVf+F5MdlnFhqDtStzFZYFiFPgg1R0llStQYBwSy2/aPxDZXuvD8J8qRigyMqrAp yUpkBjkRNUo4f9DiGG45skjjWCnbas4tyd0wA64uGJ2wczBCQ39ofKOkl6Wp9f5N vLvhMjds28UiXjQ0KlGSQkVLqGW8SidLBLT5570q7TIr0qQtzoruZgiSqDjjrTWC 9M3RXAjbbQ9G90hTR7VX2JSqIWauJi4SgU0s9bbgEwdDEkaofH/F0ONxmQtXoCG0 pc6LshKWcgfBkGJIUTEPg2AP9ZRGsH3o8sN5UJERVmUwXSzHebt8lW5zJdCYuHRB EEVegJ8RJacyXqsY9mjiDE2NP+WrA2f8NaD6mhvy7/GVE6c6/tlYvTlnjn+IOy+/ +dRJiktPHz5590P/j8/fPrrx14e082py9/6X2+7rg/Wj+38enX39/O/vP/7+Dw== =+xpF -----END PGP SIGNATURE----- --+HP7ph2BbKc20aGI-- From owner-freebsd-fs@FreeBSD.ORG Sun Dec 30 17:49:31 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A313416A418 for ; Sun, 30 Dec 2007 17:49:31 +0000 (UTC) (envelope-from johan@stromnet.se) Received: from core.stromnet.se (core.stromnet.se [83.218.84.131]) by mx1.freebsd.org (Postfix) with ESMTP id 58A5013C4D1 for ; Sun, 30 Dec 2007 17:49:31 +0000 (UTC) (envelope-from johan@stromnet.se) Received: from localhost (unknown [83.218.84.135]) by core.stromnet.se (Postfix) with ESMTP id D9361D467B2; Sun, 30 Dec 2007 18:49:29 +0100 (CET) X-Virus-Scanned: amavisd-new at stromnet.se Received: from core.stromnet.se ([83.218.84.131]) by localhost (core.stromnet.se [83.218.84.135]) (amavisd-new, port 10024) with ESMTP id SzCy2Dfly5rK; Sun, 30 Dec 2007 18:49:27 +0100 (CET) Received: from [172.28.1.102] (90-224-172-102-no129.tbcn.telia.com [90.224.172.102]) by core.stromnet.se (Postfix) with ESMTP id 98E6BD467B0; Sun, 30 Dec 2007 18:49:27 +0100 (CET) In-Reply-To: <20071230172241.GA89549@harmless.hu> References: <5A6CFB06-4175-452F-BFC9-323C2023D2F6@stromnet.se> <66640AA3-31BC-4E00-9E59-AF8FEA25EFD1@stromnet.se> <20071230172241.GA89549@harmless.hu> Mime-Version: 1.0 (Apple Message framework v753) Content-Type: text/plain; charset=ISO-8859-1; delsp=yes; format=flowed Message-Id: <01C889E4-1A89-4675-872B-D4192B7855FD@stromnet.se> Content-Transfer-Encoding: quoted-printable From: =?ISO-8859-1?Q?Johan_Str=F6m?= Date: Sun, 30 Dec 2007 18:48:39 +0100 To: Gergely CZUCZY X-Mailer: Apple Mail (2.753) Cc: freebsd-fs@freebsd.org Subject: Re: ZFS replace/expand problem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Dec 2007 17:49:31 -0000 On Dec 30, 2007, at 18:22 , Gergely CZUCZY wrote: > On Sun, Dec 30, 2007 at 06:19:27PM +0100, Johan Str=F6m wrote: >> On Dec 27, 2007, at 20:25 , Johan Str=F6m wrote: >> >>> Hello list >>> >>> First of all, I want to thank everybody involved in writing and =20 >>> porting ZFS to FreeBSD, its working (except for this problem) great >>> for me! >>> >>> Now to my problem. To sumarize it, I want to replace two mirrored =20= >>> disk with bigger ones. Replace works well but the vdev doesnt expand >>> until i do export/import. Details follows: > same experience here. > >> For the record, I found a somewhat easeier solution.. Just reboot =20 >> and it was updated! Tested with my testtank first, reboot, worked. >> Then did the same with my real tank (but with ad20 and ad18, not =20 >> using slices), and the extra space showed up fine after reboot. > Sherlock, rebooting includes an export and an import. Oh.. haha.. well, one learns new things everyday :) From owner-freebsd-fs@FreeBSD.ORG Mon Dec 31 11:06:56 2007 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F147D16A47E for ; Mon, 31 Dec 2007 11:06:56 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id DF9AA13C461 for ; Mon, 31 Dec 2007 11:06:56 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id lBVB6uRS037325 for ; Mon, 31 Dec 2007 11:06:56 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id lBVB6uYk037321 for freebsd-fs@FreeBSD.org; Mon, 31 Dec 2007 11:06:56 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 31 Dec 2007 11:06:56 GMT Message-Id: <200712311106.lBVB6uYk037321@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Dec 2007 11:06:57 -0000 Current FreeBSD problem reports Critical problems Serious problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o kern/114856 fs [ntfs] [patch] Bug in NTFS allows bogus file modes. o kern/116170 fs Kernel panic when mounting /tmp 4 problems total. Non-critical problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/114847 fs [ntfs] [patch] dirmask support for NTFS ala MSDOSFS o bin/118249 fs mv(1): moving a directory changes its mtime 2 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon Dec 31 23:48:53 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5CE6A16A41B for ; Mon, 31 Dec 2007 23:48:53 +0000 (UTC) (envelope-from davidt@yadt.co.uk) Received: from outcold.yadt.co.uk (outcold.yadt.co.uk [81.187.204.178]) by mx1.freebsd.org (Postfix) with ESMTP id 0FD2F13C455 for ; Mon, 31 Dec 2007 23:48:52 +0000 (UTC) (envelope-from davidt@yadt.co.uk) Received: from localhost (localhost [127.0.0.1]) by outcold.yadt.co.uk (Postfix) with ESMTP id 7078B23BD; Mon, 31 Dec 2007 23:30:15 +0000 (GMT) X-Virus-Scanned: amavisd-new at yadt.co.uk Received: from outcold.yadt.co.uk ([127.0.0.1]) by localhost (outcold.yadt.co.uk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5ZopmFTzvQm2; Mon, 31 Dec 2007 23:30:13 +0000 (GMT) Received: by outcold.yadt.co.uk (Postfix, from userid 1001) id CED166CF5; Mon, 31 Dec 2007 23:23:19 +0000 (GMT) Date: Mon, 31 Dec 2007 23:23:19 +0000 From: David Taylor To: freebsd-fs@freebsd.org Message-ID: <20071231232319.GA90972@outcold.yadt.co.uk> Mail-Followup-To: David Taylor , freebsd-fs@freebsd.org, pjd@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline User-Agent: Mutt/1.5.17 (2007-11-01) Cc: pjd@freebsd.org Subject: [PATCH] ZFS not caching on i386 with kmem_size >1GB X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Dec 2007 23:48:53 -0000 Hi, About 2 months ago I reported that I found ZFS extremely slow for some tasks (specifically upgrading ports). This was because ZFS was only using the absolute minimum cache size at all times. The problem is here in /sys/contrib/opensolaris/uts/common/fs/zfs/arc.c: static int arc_reclaim_needed(void) { ... if (kmem_used() > (kmem_size() * 4) / 5) return (1); } I'm running on i386 with kmem_size set to 1GB. As a result, the multiplication overflows and the test becomes (kmem_used() > 0). ZFS then always tries to shrink the cache, and never grows it above the absolute minimum size (about 30MB for each of c and p) The patch I have attached fixes the problem for me, although there is probably a better way to avoid the overflow (without calling kmem_size() twice). Best of all, portupgrade is now an order of magnitude faster! Of course, I'm now worried that my previously rock-solid settings will actually trigger the kmem_map too small panics when the cache actually fills up. -- David Taylor