From owner-freebsd-fs@FreeBSD.ORG Sun Aug 22 09:25:49 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6AF2610656A9 for ; Sun, 22 Aug 2010 09:25:49 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 1489E8FC28 for ; Sun, 22 Aug 2010 09:25:39 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 7D92E45E91; Wed, 18 Aug 2010 13:07:01 +0200 (CEST) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 9B77345E86; Wed, 18 Aug 2010 13:06:56 +0200 (CEST) Date: Wed, 18 Aug 2010 13:06:55 +0200 From: Pawel Jakub Dawidek To: Thomas Steen Rasmussen Message-ID: <20100818110655.GA2177@garage.freebsd.pl> References: <4C57E20E.2030908@gibfest.dk> <20100806135001.GF1710@garage.freebsd.pl> <4C5ECA78.6010803@gibfest.dk> <20100810075528.GA1754@garage.freebsd.pl> <4C61CF4D.4060009@gibfest.dk> <4C651B7E.5000805@gibfest.dk> <20100816221059.GE2611@garage.freebsd.pl> <4C6B08BD.9080206@gibfest.dk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="dDRMvlgZJXvWKvBx" Content-Disposition: inline In-Reply-To: <4C6B08BD.9080206@gibfest.dk> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: HAST initial sync speed X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Aug 2010 09:25:49 -0000 --dDRMvlgZJXvWKvBx Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 18, 2010 at 12:10:05AM +0200, Thomas Steen Rasmussen wrote: > I performed the tests with md devices like you asked. It seems like using > memory disks doesn't make any difference. It is still running at 2-300kBps > according to gstat. The network is plenty fast, like I mentioned in an > earlier mail, once the initial sync is done, I can reach almost wire spee= d, > over 100 megabytes per second. Could you edit sbin/hastd/proto_common.c and change MAX_SEND_SIZE at the begining of the file from 131072 to 32768? Then do the following: # cd /usr/src # make buildenv # cd sbin/hastd # make && make install ^D And please rerun the test with md(4) devices. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --dDRMvlgZJXvWKvBx Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxrvs8ACgkQForvXbEpPzQpzQCgviN7W2k5NGkEK6BZ2SeKCdwm 6+UAoKlYS50hpu65dRMiRV6zIwd1XnSq =ElmX -----END PGP SIGNATURE----- --dDRMvlgZJXvWKvBx-- From owner-freebsd-fs@FreeBSD.ORG Sun Aug 22 09:42:20 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A283710656A3 for ; Sun, 22 Aug 2010 09:42:20 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 3E6488FC0A for ; Sun, 22 Aug 2010 09:42:20 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 2AC5F45E86; Wed, 18 Aug 2010 14:15:56 +0200 (CEST) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 1AC1945CAC; Wed, 18 Aug 2010 14:15:51 +0200 (CEST) Date: Wed, 18 Aug 2010 14:15:50 +0200 From: Pawel Jakub Dawidek To: Ed Schouten Message-ID: <20100818121550.GD2177@garage.freebsd.pl> References: <4C6B9F51.1060009@freebsd.org> <20100818104853.GB2978@hoeg.nl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Km1U/tdNT/EmXiR1" Content-Disposition: inline In-Reply-To: <20100818104853.GB2978@hoeg.nl> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org, Daichi GOTO , freebsd-current@freebsd.org Subject: Re: Mounting cd9660 multiple times gives EBUSY [Was: unionfs a little improvement] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Aug 2010 09:42:20 -0000 --Km1U/tdNT/EmXiR1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 18, 2010 at 12:48:53PM +0200, Ed Schouten wrote: > Hi Daichi, >=20 > I think Keith Packard of Xorg once wrote a commit message along the > lines of "5000 lines of code removed, feature added" This seems to be > similar, albeit on a smaller scale. ;-) >=20 > Apart from this issue with unionfs, I am also experiencing another > issue, where for some reason I cannot perform a second mount of the CD > right after booting the system. Basically, my WIP FreeBSD boot CD does > the following (but written in C): >=20 > mount -t cd9660 /dev/iso9660/freebsd /mnt > mount -t tmpfs none /tmp > mount -t unionfs /tmp /mnt > mount -t devfs none /mnt/dev > chroot /mnt /sbin/init >=20 > The first step fails with EBUSY. I use the following hack to get it > working, but I don't think it's the proper way to solve it: What you are trying to do here is to mount /dev/iso9660/freebsd for the second time? This is not supported. The check is there to prevent doing this, as it will panic on you when you try to unmount first mount (not really a problem in your case, as the first mount is /, so you probably don't want to unmount it, but it is a problem in general). You should be able to reproduce the panic with your patch applied by doing the following: # mount -t cd9660 /dev/iso9660/freebsd /mnt0 # mount -t cd9660 /dev/iso9660/freebsd /mnt1 # umount /mnt0 --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --Km1U/tdNT/EmXiR1 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxrzvUACgkQForvXbEpPzRytgCgo8Cm2ShBhf8i+rFg9nvOiNn8 bMoAn2J+eG3iColgrWmofQQlfPP4OIvW =GD6x -----END PGP SIGNATURE----- --Km1U/tdNT/EmXiR1-- From owner-freebsd-fs@FreeBSD.ORG Sun Aug 22 10:15:40 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 44CA810656A7 for ; Sun, 22 Aug 2010 10:15:40 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id D947A8FC19 for ; Sun, 22 Aug 2010 10:15:39 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 59CB345F38; Wed, 18 Aug 2010 14:59:02 +0200 (CEST) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 97EC145E93; Wed, 18 Aug 2010 14:58:56 +0200 (CEST) Date: Wed, 18 Aug 2010 14:58:56 +0200 From: Pawel Jakub Dawidek To: Thomas Steen Rasmussen Message-ID: <20100818125856.GE2177@garage.freebsd.pl> References: <20100810075528.GA1754@garage.freebsd.pl> <4C61CF4D.4060009@gibfest.dk> <4C651B7E.5000805@gibfest.dk> <20100816221059.GE2611@garage.freebsd.pl> <4C6B08BD.9080206@gibfest.dk> <20100818110655.GA2177@garage.freebsd.pl> <4C6BC0BA.9030303@gibfest.dk> <4C6BC35B.9040000@gibfest.dk> <20100818121133.GC2177@garage.freebsd.pl> <4C6BD521.1060807@gibfest.dk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="5xSkJheCpeK0RUEJ" Content-Disposition: inline In-Reply-To: <4C6BD521.1060807@gibfest.dk> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: HAST initial sync speed X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Aug 2010 10:15:40 -0000 --5xSkJheCpeK0RUEJ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 18, 2010 at 02:42:09PM +0200, Thomas Steen Rasmussen wrote: > On 18-08-2010 14:11, Pawel Jakub Dawidek wrote: > >I'm very glad to hear that. I changed the default MAX_SEND_SIZE to 32kB > >as it seems to be the safest setting. I had similar problem in ggate. > > > > =20 >=20 > OK. So it will be (or has been) committed to the tree ? What about MFC ?= =20 > I should > probably know this, but: > What does this mean, like, when will this fix be in -STABLE or even in a= =20 > -RELEASE ? I just committed it to FreeBSD-CURRENT, it will be merged to FreeBSD-STABLE (8-STABLE) in three weeks and will be available in 8.2-RELEASE when it is released. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --5xSkJheCpeK0RUEJ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxr2Q8ACgkQForvXbEpPzR6oQCdGglutOcpNiQsD/+De86QFDKs 3pUAn3CpbjmWXjl7H8yFwP4kBZx2qZpf =6S5Y -----END PGP SIGNATURE----- --5xSkJheCpeK0RUEJ-- From owner-freebsd-fs@FreeBSD.ORG Sun Aug 22 10:49:00 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 25D77106566B for ; Sun, 22 Aug 2010 10:49:00 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id C301E8FC17 for ; Sun, 22 Aug 2010 10:48:59 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id C1F9245C9B; Wed, 18 Aug 2010 14:11:38 +0200 (CEST) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id D782B45683; Wed, 18 Aug 2010 14:11:34 +0200 (CEST) Date: Wed, 18 Aug 2010 14:11:33 +0200 From: Pawel Jakub Dawidek To: Thomas Steen Rasmussen Message-ID: <20100818121133.GC2177@garage.freebsd.pl> References: <20100806135001.GF1710@garage.freebsd.pl> <4C5ECA78.6010803@gibfest.dk> <20100810075528.GA1754@garage.freebsd.pl> <4C61CF4D.4060009@gibfest.dk> <4C651B7E.5000805@gibfest.dk> <20100816221059.GE2611@garage.freebsd.pl> <4C6B08BD.9080206@gibfest.dk> <20100818110655.GA2177@garage.freebsd.pl> <4C6BC0BA.9030303@gibfest.dk> <4C6BC35B.9040000@gibfest.dk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="qjNfmADvan18RZcF" Content-Disposition: inline In-Reply-To: <4C6BC35B.9040000@gibfest.dk> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: HAST initial sync speed X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Aug 2010 10:49:00 -0000 --qjNfmADvan18RZcF Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 18, 2010 at 01:26:19PM +0200, Thomas Steen Rasmussen wrote: > Sorry for responding to my own post. I recreated the HAST setup with the > four harddrives and it is now saturating the gigabit link, it is=20 > synching at a steady > rate of 112 megabytes per second, meaning that each of the disks are > reading/writing at just under 30 megabytes per second. It would probably = be > even faster if the network wasn't limiting it. >=20 > I built ZFS on top of it and everything seems to be working as expected. >=20 > Thank you again for looking at this. I'm very glad to hear that. I changed the default MAX_SEND_SIZE to 32kB as it seems to be the safest setting. I had similar problem in ggate. BTW. What network cards do you use there? --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --qjNfmADvan18RZcF Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxrzfUACgkQForvXbEpPzTh0gCeKwWQ2RwbF8ijXb3p7d00tJWw ZvEAnRyFW/DUrTC7q3bhN5V/k+MSmudC =JPPz -----END PGP SIGNATURE----- --qjNfmADvan18RZcF-- From owner-freebsd-fs@FreeBSD.ORG Sun Aug 22 11:19:11 2010 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E81510656A8; Sun, 22 Aug 2010 11:19:11 +0000 (UTC) (envelope-from gnehzuil@gmail.com) Received: from mail-pv0-f182.google.com (mail-pv0-f182.google.com [74.125.83.182]) by mx1.freebsd.org (Postfix) with ESMTP id D43BF8FC16; Sun, 22 Aug 2010 11:19:10 +0000 (UTC) Received: by pvg4 with SMTP id 4so2123869pvg.13 for ; Sun, 22 Aug 2010 04:19:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from :user-agent:mime-version:to:cc:subject:content-type; bh=PcZ0bgBC3xwzMc/uLV7p/u7OtvcwukReVxmozijQ/BU=; b=R3GQe82vfoJvaPkqct25x8VVV8nUFbtZtY0WE2EYvmaLcGk+R7t/9+Q9PuPrYr4OP7 Gl6ff5TnS7dQuDKv/D1zfBWil0QPv7FDoiuWJPLIHpn4ivRL/A9DAhcpq+R62Go0rVuF KyJPgWQmyWu3TZf2MmvLmsPfltoOw5HtpGScs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:cc:subject :content-type; b=QrGHyK4s97x58ugFo8zPzG3fmYQ1DmnCLEylfTmDvb4EMrUT6HyRtyL0y7HhCiE79w JQzWs+h7n4Uw/yLo9EW/mQoc6k6B884p0l9v0hDWkb7ZTrsQo5xHvah5TXgHwEuhdrDe FnBz8zfPvzRD5Nl+fGvZNnVb+gCKo5bpFWEAc= Received: by 10.114.190.20 with SMTP id n20mr4340525waf.10.1282475950385; Sun, 22 Aug 2010 04:19:10 -0700 (PDT) Received: from [192.168.1.37] ([166.111.68.197]) by mx.google.com with ESMTPS id q6sm9810271waj.22.2010.08.22.04.19.06 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sun, 22 Aug 2010 04:19:08 -0700 (PDT) Message-ID: <4C71075C.9010802@gmail.com> Date: Sun, 22 Aug 2010 19:17:48 +0800 From: gnehzuil User-Agent: Thunderbird 2.0.0.24 (X11/20100317) MIME-Version: 1.0 To: fs@freebsd.org Content-Type: multipart/mixed; boundary="------------000709010802090406030102" Cc: jhb@FreeBSD.org Subject: [patch] ext4fs read only mode X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Aug 2010 11:19:11 -0000 This is a multi-part message in MIME format. --------------000709010802090406030102 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi all, This patch makes ext2fs can read ext4 filesystem in read-only mode. There are two files in attachments. 'ext4fs_ro_makefile.patch' is for Makefile in modules/ext2fs/. 'ext4fs_ro_src.patch' is for source code in fs/ext2fs/. Please use the following command to mount disk: 'mount -t ext2fs -r /dev/XXX /YYY'. Now you can use it to read data from ext4 filesystem in the following features: + HAS_JOURNAL(*) + FILE_TYPE + SPARSE_SUPER + HUGE_FILE + EXTENTS + DIR_NLINK + UNINIT_BG + FLEX_BG(*) + EXTRA_ISIZE(*) + DIR_INDEX(**) * I don't implement this feature. However you don't need to worry about it because it doesn't be used in read-only mode. ** I have implemented a hash directory index in ext2_lookup() function. But there are two functions that I think they seem to be contaminated kernel source code. So this patch doesn't include hash directory index. Please test it. Best regards, lz --------------000709010802090406030102 Content-Type: text/x-patch; name="ext4fs_ro_makefile.patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="ext4fs_ro_makefile.patch" --- /usr/src/sys/modules/ext2fs/Makefile 2010-01-14 22:30:54.000000000 +0800 +++ Makefile 2010-08-22 22:53:28.000000000 +0800 @@ -3,8 +3,8 @@ .PATH: ${.CURDIR}/../../fs/ext2fs KMOD= ext2fs SRCS= opt_ddb.h opt_quota.h opt_suiddir.h vnode_if.h \ - ext2_alloc.c ext2_balloc.c ext2_bmap.c ext2_inode.c \ - ext2_inode_cnv.c ext2_lookup.c ext2_subr.c ext2_vfsops.c \ - ext2_vnops.c + ext2_alloc.c ext2_balloc.c ext2_bmap.c ext2_extents.c \ + ext2_inode.c ext2_inode_cnv.c ext2_lookup.c ext2_subr.c \ + ext2_vfsops.c ext2_vnops.c .include --------------000709010802090406030102 Content-Type: text/x-patch; name="ext4fs_ro_src.patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="ext4fs_ro_src.patch" diff -urN /usr/src/sys/fs/ext2fs/ext2_alloc.c src/ext2_alloc.c --- /usr/src/sys/fs/ext2fs/ext2_alloc.c 2010-01-14 22:30:54.000000000 +0800 +++ src/ext2_alloc.c 2010-08-22 22:41:52.000000000 +0800 @@ -116,12 +116,12 @@ if (cred == NOCRED) panic("ext2_alloc: missing credential"); #endif /* DIAGNOSTIC */ - if (size == fs->e2fs_bsize && fs->e2fs->e2fs_fbcount == 0) + if (size == fs->e2fs_bsize && fs->e2fs->e2fs_fbcount_lo == 0) goto nospace; if (cred->cr_uid != 0 && - fs->e2fs->e2fs_fbcount < fs->e2fs->e2fs_rbcount) + fs->e2fs->e2fs_fbcount_lo < fs->e2fs->e2fs_rbcount_lo) goto nospace; - if (bpref >= fs->e2fs->e2fs_bcount) + if (bpref >= fs->e2fs->e2fs_bcount_lo) bpref = 0; if (bpref == 0) cg = ino_to_cg(fs, ip->i_number); @@ -443,7 +443,7 @@ fs = pip->i_e2fs; avgifree = fs->e2fs->e2fs_ficount / fs->e2fs_gcount; - avgbfree = fs->e2fs->e2fs_fbcount / fs->e2fs_gcount; + avgbfree = fs->e2fs->e2fs_fbcount_lo / fs->e2fs_gcount; avgndir = fs->e2fs_total_dir / fs->e2fs_gcount; /* @@ -455,18 +455,18 @@ mincg = prefcg; minndir = fs->e2fs_ipg; for (cg = prefcg; cg < fs->e2fs_gcount; cg++) - if (fs->e2fs_gd[cg].ext2bgd_ndirs < minndir && - fs->e2fs_gd[cg].ext2bgd_nifree >= avgifree && - fs->e2fs_gd[cg].ext2bgd_nbfree >= avgbfree) { + if (fs->e2fs_gd[cg].ext2bgd_ndirs_lo < minndir && + fs->e2fs_gd[cg].ext2bgd_nifree_lo >= avgifree && + fs->e2fs_gd[cg].ext2bgd_nbfree_lo >= avgbfree) { mincg = cg; - minndir = fs->e2fs_gd[cg].ext2bgd_ndirs; + minndir = fs->e2fs_gd[cg].ext2bgd_ndirs_lo; } for (cg = 0; cg < prefcg; cg++) - if (fs->e2fs_gd[cg].ext2bgd_ndirs < minndir && - fs->e2fs_gd[cg].ext2bgd_nifree >= avgifree && - fs->e2fs_gd[cg].ext2bgd_nbfree >= avgbfree) { + if (fs->e2fs_gd[cg].ext2bgd_ndirs_lo < minndir && + fs->e2fs_gd[cg].ext2bgd_nifree_lo >= avgifree && + fs->e2fs_gd[cg].ext2bgd_nbfree_lo >= avgbfree) { mincg = cg; - minndir = fs->e2fs_gd[cg].ext2bgd_ndirs; + minndir = fs->e2fs_gd[cg].ext2bgd_ndirs_lo; } return (mincg); @@ -503,16 +503,16 @@ */ prefcg = ino_to_cg(fs, pip->i_number); for (cg = prefcg; cg < fs->e2fs_gcount; cg++) - if (fs->e2fs_gd[cg].ext2bgd_ndirs < maxndir && - fs->e2fs_gd[cg].ext2bgd_nifree >= minifree && - fs->e2fs_gd[cg].ext2bgd_nbfree >= minbfree) { + if (fs->e2fs_gd[cg].ext2bgd_ndirs_lo < maxndir && + fs->e2fs_gd[cg].ext2bgd_nifree_lo >= minifree && + fs->e2fs_gd[cg].ext2bgd_nbfree_lo >= minbfree) { if (fs->e2fs_contigdirs[cg] < maxcontigdirs) return (cg); } for (cg = 0; cg < prefcg; cg++) - if (fs->e2fs_gd[cg].ext2bgd_ndirs < maxndir && - fs->e2fs_gd[cg].ext2bgd_nifree >= minifree && - fs->e2fs_gd[cg].ext2bgd_nbfree >= minbfree) { + if (fs->e2fs_gd[cg].ext2bgd_ndirs_lo < maxndir && + fs->e2fs_gd[cg].ext2bgd_nifree_lo >= minifree && + fs->e2fs_gd[cg].ext2bgd_nbfree_lo >= minbfree) { if (fs->e2fs_contigdirs[cg] < maxcontigdirs) return (cg); } @@ -520,10 +520,10 @@ * This is a backstop when we have deficit in space. */ for (cg = prefcg; cg < fs->e2fs_gcount; cg++) - if (fs->e2fs_gd[cg].ext2bgd_nifree >= avgifree) + if (fs->e2fs_gd[cg].ext2bgd_nifree_lo >= avgifree) return (cg); for (cg = 0; cg < prefcg; cg++) - if (fs->e2fs_gd[cg].ext2bgd_nifree >= avgifree) + if (fs->e2fs_gd[cg].ext2bgd_nifree_lo >= avgifree) break; return (cg); } @@ -644,11 +644,11 @@ /* XXX ondisk32 */ fs = ip->i_e2fs; ump = ip->i_ump; - if (fs->e2fs_gd[cg].ext2bgd_nbfree == 0) + if (fs->e2fs_gd[cg].ext2bgd_nbfree_lo == 0) return (0); EXT2_UNLOCK(ump); error = bread(ip->i_devvp, fsbtodb(fs, - fs->e2fs_gd[cg].ext2bgd_b_bitmap), + fs->e2fs_gd[cg].ext2bgd_b_bitmap_lo), (int)fs->e2fs_bsize, NOCRED, &bp); if (error) { brelse(bp); @@ -709,8 +709,8 @@ #endif setbit(bbp, (daddr_t)bno); EXT2_LOCK(ump); - fs->e2fs->e2fs_fbcount--; - fs->e2fs_gd[cg].ext2bgd_nbfree--; + fs->e2fs->e2fs_fbcount_lo--; + fs->e2fs_gd[cg].ext2bgd_nbfree_lo--; fs->e2fs_fmod = 1; EXT2_UNLOCK(ump); bdwrite(bp); @@ -736,11 +736,11 @@ ipref = 0; fs = ip->i_e2fs; ump = ip->i_ump; - if (fs->e2fs_gd[cg].ext2bgd_nifree == 0) + if (fs->e2fs_gd[cg].ext2bgd_nifree_lo == 0) return (0); EXT2_UNLOCK(ump); error = bread(ip->i_devvp, fsbtodb(fs, - fs->e2fs_gd[cg].ext2bgd_i_bitmap), + fs->e2fs_gd[cg].ext2bgd_i_bitmap_lo), (int)fs->e2fs_bsize, NOCRED, &bp); if (error) { brelse(bp); @@ -781,11 +781,11 @@ gotit: setbit(ibp, ipref); EXT2_LOCK(ump); - fs->e2fs_gd[cg].ext2bgd_nifree--; + fs->e2fs_gd[cg].ext2bgd_nifree_lo--; fs->e2fs->e2fs_ficount--; fs->e2fs_fmod = 1; if ((mode & IFMT) == IFDIR) { - fs->e2fs_gd[cg].ext2bgd_ndirs++; + fs->e2fs_gd[cg].ext2bgd_ndirs_lo++; fs->e2fs_total_dir++; } EXT2_UNLOCK(ump); @@ -812,14 +812,14 @@ fs = ip->i_e2fs; ump = ip->i_ump; cg = dtog(fs, bno); - if ((u_int)bno >= fs->e2fs->e2fs_bcount) { + if ((u_int)bno >= fs->e2fs->e2fs_bcount_lo) { printf("bad block %lld, ino %llu\n", (long long)bno, (unsigned long long)ip->i_number); ext2_fserr(fs, ip->i_uid, "bad block"); return; } error = bread(ip->i_devvp, - fsbtodb(fs, fs->e2fs_gd[cg].ext2bgd_b_bitmap), + fsbtodb(fs, fs->e2fs_gd[cg].ext2bgd_b_bitmap_lo), (int)fs->e2fs_bsize, NOCRED, &bp); if (error) { brelse(bp); @@ -834,8 +834,8 @@ } clrbit(bbp, bno); EXT2_LOCK(ump); - fs->e2fs->e2fs_fbcount++; - fs->e2fs_gd[cg].ext2bgd_nbfree++; + fs->e2fs->e2fs_fbcount_lo++; + fs->e2fs_gd[cg].ext2bgd_nbfree_lo++; fs->e2fs_fmod = 1; EXT2_UNLOCK(ump); bdwrite(bp); @@ -868,7 +868,7 @@ cg = ino_to_cg(fs, ino); error = bread(pip->i_devvp, - fsbtodb(fs, fs->e2fs_gd[cg].ext2bgd_i_bitmap), + fsbtodb(fs, fs->e2fs_gd[cg].ext2bgd_i_bitmap_lo), (int)fs->e2fs_bsize, NOCRED, &bp); if (error) { brelse(bp); @@ -885,9 +885,9 @@ clrbit(ibp, ino); EXT2_LOCK(ump); fs->e2fs->e2fs_ficount++; - fs->e2fs_gd[cg].ext2bgd_nifree++; + fs->e2fs_gd[cg].ext2bgd_nifree_lo++; if ((mode & IFMT) == IFDIR) { - fs->e2fs_gd[cg].ext2bgd_ndirs--; + fs->e2fs_gd[cg].ext2bgd_ndirs_lo--; fs->e2fs_total_dir--; } fs->e2fs_fmod = 1; diff -urN /usr/src/sys/fs/ext2fs/ext2_bmap.c src/ext2_bmap.c --- /usr/src/sys/fs/ext2fs/ext2_bmap.c 2010-01-14 22:30:54.000000000 +0800 +++ src/ext2_bmap.c 2010-08-22 22:41:52.000000000 +0800 @@ -46,9 +46,62 @@ #include #include +#include #include #include #include +#include + +static int ext4_bmapext(struct vnode *, int32_t, int64_t *, int *, int *); + +/* + * This function converts the logical block number of a file to + * its physical block number on the disk within ext4 extents. + */ +static int +ext4_bmapext(struct vnode *vp, int32_t bn, int64_t *bnp, int *runp, int *runb) +{ + struct inode *ip; + struct m_ext2fs *fs; + struct ext4_extent *ep; + struct ext4_extent_header *ehp; + struct ext4_extent_path path; + daddr_t lbn; + int bsize; + int depth; + + ip = VTOI(vp); + fs = ip->i_e2fs; + lbn = bn; + bsize = blksize(fs, ip, lbn); + + /* + * TODO: need to implement read ahead to improve the performance. + */ + if (runp != NULL) + *runp = 0; + + if (runb != NULL) + *runb = 0; + + ext4_ext_find_extent(fs, ip, lbn, &path); + depth = ((struct ext4_extent_header *)(ip->i_db))->eh_depth; + if (path.ep_ext == NULL && depth != 0) + return (EIO); + + ehp = path.ep_header; + ep = path.ep_ext; + if (ep == NULL) + return (EIO); + + *bnp = fsbtodb(fs, (lbn - ep->e_blk + + (ep->e_start_lo | ((daddr_t)(ep->e_start_hi) << 31) << 1))); + + if (*bnp == 0) + *bnp = -1; + + return (0); +} /* * Bmap converts the logical block number of a file to its physical block @@ -66,7 +119,7 @@ int *a_runb; } */ *ap; { - int32_t blkno; + int64_t blkno; int error; /* @@ -78,8 +131,12 @@ if (ap->a_bnp == NULL) return (0); - error = ext2_bmaparray(ap->a_vp, ap->a_bn, &blkno, - ap->a_runp, ap->a_runb); + if (VTOI(ap->a_vp)->i_flags & EXT4_EXTENTS) + error = ext4_bmapext(ap->a_vp, ap->a_bn, &blkno, + ap->a_runp, ap->a_runb); + else + error = ext2_bmaparray(ap->a_vp, ap->a_bn, &blkno, + ap->a_runp, ap->a_runb); *ap->a_bnp = blkno; return (error); } @@ -102,7 +159,7 @@ ext2_bmaparray(vp, bn, bnp, runp, runb) struct vnode *vp; int32_t bn; - int32_t *bnp; + int64_t *bnp; int *runp; int *runb; { diff -urN /usr/src/sys/fs/ext2fs/ext2_dinode.h src/ext2_dinode.h --- /usr/src/sys/fs/ext2fs/ext2_dinode.h 2010-01-14 22:30:54.000000000 +0800 +++ src/ext2_dinode.h 2010-08-22 22:41:52.000000000 +0800 @@ -29,8 +29,6 @@ #ifndef _FS_EXT2FS_EXT2_DINODE_H_ #define _FS_EXT2FS_EXT2_DINODE_H_ -#define e2di_size_high e2di_dacl - /* * Inode flags * The current implementation uses only EXT2_IMMUTABLE and EXT2_APPEND flags @@ -43,7 +41,21 @@ #define EXT2_APPEND 0x00000020 /* writes to file may only append */ #define EXT2_NODUMP 0x00000040 /* do not dump file */ #define EXT2_NOATIME 0x00000080 /* do not update atime */ - +/* NOT implementation. Reserved for compression usage. */ +#define EXT4_DIRTY 0x00000100 +#define EXT4_COMPRBLK 0x00000200 /* One or more compressed clusters */ +#define EXT4_NOCOMPR 0x00000400 /* Don't compress */ +#define EXT4_ECOMPR 0x00000800 /* Compression error */ +/* End compression flags */ +#define EXT4_INDEX 0x00001000 /* Hash-indexed directory */ +#define EXT4_IMAGIC 0x00002000 /* AFS directory */ +#define EXT4_JOURNAL_DATA 0x00004000 /* File data should be journaled */ +#define EXT4_NOTAIL 0x00008000 /* File tail should not be merged */ +#define EXT4_DIRSYNC 0x00010000 /* dirsync behavior */ +#define EXT4_TOPDIR 0x00020000 /* top of directory hierarchies */ +#define EXT4_HUGE_FILE 0x00040000 /* Set to each huge file */ +#define EXT4_EXTENTS 0x00080000 /* Inode uses extents */ +#define EXT4_RESERVED 0x80000000 /* Reserved for ext4 lib */ /* * Structure of an inode on the disk @@ -51,27 +63,62 @@ struct ext2fs_dinode { u_int16_t e2di_mode; /* 0: IFMT, permissions; see below. */ u_int16_t e2di_uid; /* 2: Owner UID */ - u_int32_t e2di_size; /* 4: Size (in bytes) */ + u_int32_t e2di_size_lo; /* 4: Size (in bytes) */ u_int32_t e2di_atime; /* 8: Access time */ u_int32_t e2di_ctime; /* 12: Create time */ u_int32_t e2di_mtime; /* 16: Modification time */ u_int32_t e2di_dtime; /* 20: Deletion time */ u_int16_t e2di_gid; /* 24: Owner GID */ u_int16_t e2di_nlink; /* 26: File link count */ - u_int32_t e2di_nblock; /* 28: Blocks count */ + u_int32_t e2di_nblock_lo; /* 28: Blocks count */ u_int32_t e2di_flags; /* 32: Status flags (chflags) */ - u_int32_t e2di_linux_reserved1; /* 36 */ + union { + struct { + u_int32_t e2di_version; + } linux1; + struct { + u_int32_t e2di_translator; + } hurd1; + struct { + u_int32_t e2di_reserved1; + } masix1; + } osd1; /* 36: */ u_int32_t e2di_blocks[EXT2_N_BLOCKS]; /* 40: disk blocks */ u_int32_t e2di_gen; /* 100: generation number */ u_int32_t e2di_facl; /* 104: file ACL (not implemented) */ u_int32_t e2di_dacl; /* 108: dir ACL (not implemented) */ +#define e2di_size_high e2di_dacl u_int32_t e2di_faddr; /* 112: fragment address */ - u_int8_t e2di_nfrag; /* 116: fragment number */ - u_int8_t e2di_fsize; /* 117: fragment size */ - u_int16_t e2di_linux_reserved2; /* 118 */ - u_int16_t e2di_uid_high; /* 120: Owner UID top 16 bits */ - u_int16_t e2di_gid_high; /* 122: Owner GID top 16 bits */ - u_int32_t e2di_linux_reserved3; /* 124 */ + union { + struct { + u_int16_t e2di_l_blk_high; +#define e2di_nblock_high osd2.linux2.e2di_l_blk_high + u_int16_t e2di_l_facl_high; + u_int16_t e2di_l_uid_high; + u_int16_t e2di_l_gid_high; + u_int32_t e2di_l_reserved2; + } linux2; + struct { + u_int16_t e2di_h_reserved1; + u_int16_t e2di_h_mode_high; + u_int16_t e2di_h_uid_high; + u_int16_t e2di_h_gid_high; + u_int32_t e2di_h_author; + } hurd2; + struct { + u_int16_t e2di_m_reserved1; + u_int16_t e2di_m_facl_high; + u_int32_t e2di_reserved2[2]; + } masix2; + } osd2; + u_int16_t e2di_extra_isize; + u_int16_t e2di_pad; + u_int32_t e2di_ctime_extra; + u_int32_t e2di_mtime_extra; + u_int32_t e2di_atime_extra; + u_int32_t e2di_crtime; + u_int32_t e2di_crtime_extra; + u_int32_t e2di_version_hi; }; #endif /* _FS_EXT2FS_EXT2_DINODE_H_ */ diff -urN /usr/src/sys/fs/ext2fs/ext2_extents.c src/ext2_extents.c --- /usr/src/sys/fs/ext2fs/ext2_extents.c 1970-01-01 08:00:00.000000000 +0800 +++ src/ext2_extents.c 2010-08-22 22:41:52.000000000 +0800 @@ -0,0 +1,191 @@ +/*- + * Copyright (c) 2010, 2010 Zheng Liu + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD: src/sys/fs/ext2fs/ext2_extents.c,v 0.1 2010/07/02 17:22:00 lz Exp $ + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include + +static void ext4_ext_binsearch_index(struct inode *, struct ext4_extent_path *, daddr_t); +static void ext4_ext_binsearch(struct inode *, struct ext4_extent_path *, daddr_t); + +static void +ext4_ext_binsearch_index(struct inode *ip, struct ext4_extent_path *path, daddr_t lbn) +{ + struct ext4_extent_header *ehp = path->ep_header; + struct ext4_extent_index *l, *r, *m; + + l = (struct ext4_extent_index *)(((char *)(ehp) + + sizeof(struct ext4_extent_header))); + r = (struct ext4_extent_index *)(((char *)(ehp) + + sizeof(struct ext4_extent_header))) + ehp->eh_ecount - 1; + while (l <= r) { + m = l + (r - l) / 2; + if (lbn < m->ei_blk) + r = m - 1; + else + l = m + 1; + } + + path->ep_index = l - 1; +} + +static void +ext4_ext_binsearch(struct inode *ip, struct ext4_extent_path *path, daddr_t lbn) +{ + struct ext4_extent_header *ehp = path->ep_header; + struct ext4_extent *l, *r, *m; + + if (ehp->eh_ecount == 0) + return; + + l = (struct ext4_extent *)(((char *)(ehp) + + sizeof(struct ext4_extent_header))); + r = (struct ext4_extent *)(((char *)(ehp) + + sizeof(struct ext4_extent_header))) + ehp->eh_ecount - 1; + while (l <= r) { + m = l + (r - l) / 2; + if (lbn < m->e_blk) + r = m - 1; + else + l = m + 1; + } + + path->ep_ext = l - 1; +} + +/* + * find a block in ext4 extent cache. + */ +int +ext4_ext_in_cache(struct inode *ip, daddr_t lbn, struct ext4_extent *ep) +{ + struct ext4_extent_cache *ecp; + int ret = EXT4_EXT_CACHE_NO; + + ecp = &ip->i_ext_cache; + + /* cache is invalid */ + if (ecp->ec_type == EXT4_EXT_CACHE_NO) + return (ret); + + if (lbn >= ecp->ec_blk && lbn < ecp->ec_blk + ecp->ec_len) { + ep->e_blk = ecp->ec_blk; + ep->e_start_lo = (ecp->ec_start & 0xffffffff); + ep->e_start_hi = (((ecp->ec_start >> 31) >> 1) & 0xffff); + ep->e_len = ecp->ec_len; + ret = ecp->ec_type; + } + + return (ret); +} + +/* + * put a ext4_extent structure in ext4 cache. + */ +void +ext4_ext_put_cache(struct inode *ip, struct ext4_extent *ep, int type) +{ + struct ext4_extent_cache *ecp; + + ecp = &ip->i_ext_cache; + ecp->ec_type = type; + ecp->ec_blk = ep->e_blk; + ecp->ec_len = ep->e_len; + ecp->ec_start = (((daddr_t)(ep->e_start_hi) << 31) << 1) | ep->e_start_lo; +} + +/* + * find a extent. + */ +struct ext4_extent_path * +ext4_ext_find_extent(struct m_ext2fs *fs, struct inode *ip, + daddr_t lbn, struct ext4_extent_path *path) +{ + struct vnode *vp; + struct ext4_extent_header *ehp; + int depth, i, error, size; + daddr_t nblk; + + vp = ITOV(ip); + ehp = (struct ext4_extent_header *)((char *)ip->i_db); + depth = ehp->eh_depth; + + if (ehp->eh_magic != EXT4_EXT_MAGIC) + return (NULL); + + path->ep_header = ehp; + + i = depth; + while (i) { + ext4_ext_binsearch_index(ip, path, lbn); + path->ep_blk = (((daddr_t)(path->ep_index->ei_leaf_hi) << 31) << 1) | + path->ep_index->ei_leaf_lo; + path->ep_depth = i; + path->ep_ext = NULL; + + size = blksize(fs, ip, path->ep_blk); + nblk = path->ep_blk; + if (path->ep_bp != NULL) { + brelse(path->ep_bp); + path->ep_bp = NULL; + } + error = bread(ip->i_devvp, fsbtodb(fs, nblk), size, NOCRED, &path->ep_bp); + if (error) { + brelse(path->ep_bp); + path->ep_bp = NULL; + return (NULL); + } + ehp = (struct ext4_extent_header *)path->ep_bp->b_data; + path->ep_header = ehp; + i--; + } + + path->ep_depth = i; + path->ep_ext = NULL; + path->ep_index = NULL; + + ext4_ext_binsearch(ip, path, lbn); + if (path->ep_ext != NULL) + path->ep_blk = (((daddr_t)(path->ep_ext->e_start_hi) << 31) << 1) | + path->ep_ext->e_start_lo; + + return (path); +} diff -urN /usr/src/sys/fs/ext2fs/ext2_extents.h src/ext2_extents.h --- /usr/src/sys/fs/ext2fs/ext2_extents.h 1970-01-01 08:00:00.000000000 +0800 +++ src/ext2_extents.h 2010-08-22 22:41:52.000000000 +0800 @@ -0,0 +1,103 @@ +/*- + * Copyright (c) 2010, 2010 Zheng Liu + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD: src/sys/fs/ext2fs/ext2_extents.h,v 0.1 2010/06/22 18:01:51 lz Exp $ + */ +#ifndef _FS_EXT2FS_EXT2_EXTENTS_H_ +#define _FS_EXT2FS_EXT2_EXTENTS_H_ + +#include + +#define EXT4_EXT_MAGIC 0xf30a + +/* lock/unlock ext lock */ +#define EXT4_EXT_LOCK(ip) mtx_lock(&(ip)->i_ext_lock) +#define EXT4_EXT_UNLOCK(ip) mtx_unlock(&(ip)->i_ext_lock) + +#define EXT4_EXT_CACHE_NO 0 +#define EXT4_EXT_CACHE_GAP 1 +#define EXT4_EXT_CACHE_IN 2 + +/* + * ext4 file system extent on disk + */ +struct ext4_extent { + u_int32_t e_blk; /* first logical block */ + u_int16_t e_len; /* number of blocks */ + u_int16_t e_start_hi; /* high 16 bits of physical block */ + u_int32_t e_start_lo; /* low 32 bits of physical block */ +}; + +/* + * extent index on disk + */ +struct ext4_extent_index { + u_int32_t ei_blk; /* indexes logical blocks */ + u_int32_t ei_leaf_lo; /* pointes to physical block of the next level */ + u_int16_t ei_leaf_hi; /* high 16 bits of physical block */ + u_int16_t ei_unused; +}; + +/* + * extent tree header + */ +struct ext4_extent_header { + u_int16_t eh_magic; /* magic number: 0xf30a */ + u_int16_t eh_ecount; /* number of valid entries */ + u_int16_t eh_max; /* capacity of store in entries */ + u_int16_t eh_depth; /* the depth of extent tree */ + u_int32_t eh_gen; /* generation of extent tree */ +}; + +/* + * save cached extent + */ +struct ext4_extent_cache { + daddr_t ec_start; /* extent start */ + u_int32_t ec_blk; /* logical block */ + u_int32_t ec_len; + u_int32_t ec_type; +}; + +/* + * save path to some extent. + */ +struct ext4_extent_path { + daddr_t ep_blk; + u_int16_t ep_depth; + struct buf *ep_bp; + struct ext4_extent *ep_ext; + struct ext4_extent_index *ep_index; + struct ext4_extent_header *ep_header; +}; + +struct inode; +struct m_ext2fs; +int ext4_ext_in_cache(struct inode *, daddr_t, struct ext4_extent *); +void ext4_ext_put_cache(struct inode *, struct ext4_extent *, int); +struct ext4_extent_path *ext4_ext_find_extent(struct m_ext2fs *fs, struct inode *, + daddr_t, struct ext4_extent_path *); + +#endif /* !_FS_EXT2FS_EXT2_EXTENTS_H_ */ diff -urN /usr/src/sys/fs/ext2fs/ext2_extern.h src/ext2_extern.h --- /usr/src/sys/fs/ext2fs/ext2_extern.h 2010-01-14 22:30:54.000000000 +0800 +++ src/ext2_extern.h 2010-08-22 22:41:52.000000000 +0800 @@ -54,7 +54,7 @@ void ext2_blkfree(struct inode *, int32_t, long); int32_t ext2_blkpref(struct inode *, int32_t, int, int32_t *, int32_t); int ext2_bmap(struct vop_bmap_args *); -int ext2_bmaparray(struct vnode *, int32_t, int32_t *, int *, int *); +int ext2_bmaparray(struct vnode *, int32_t, int64_t *, int *, int *); void ext2_dirbad(struct inode *ip, doff_t offset, char *how); void ext2_ei2i(struct ext2fs_dinode *, struct inode *); int ext2_getlbns(struct vnode *, int32_t, struct indir *, int *); diff -urN /usr/src/sys/fs/ext2fs/ext2_inode.c src/ext2_inode.c --- /usr/src/sys/fs/ext2fs/ext2_inode.c 2010-01-14 22:30:54.000000000 +0800 +++ src/ext2_inode.c 2010-08-22 22:41:52.000000000 +0800 @@ -153,6 +153,7 @@ } fs = oip->i_e2fs; osize = oip->i_size; + /* * Lengthen the size of the file. We must ensure that the * last byte of the file is allocated. Since the smallest @@ -525,11 +526,15 @@ if (prtactive && vrefcnt(vp) != 0) vprint("ufs_reclaim: pushing active", vp); ip = VTOI(vp); + if (ip->i_flag & IN_LAZYMOD) { ip->i_flag |= IN_MODIFIED; ext2_update(vp, 0); } vfs_hash_remove(vp); + + mtx_destroy(&ip->i_ext_lock); + free(vp->v_data, M_EXT2NODE); vp->v_data = 0; vnode_destroy_vobject(vp); diff -urN /usr/src/sys/fs/ext2fs/ext2_inode_cnv.c src/ext2_inode_cnv.c --- /usr/src/sys/fs/ext2fs/ext2_inode_cnv.c 2010-01-14 22:30:54.000000000 +0800 +++ src/ext2_inode_cnv.c 2010-08-22 22:50:00.000000000 +0800 @@ -35,12 +35,15 @@ #include #include #include +#include void ext2_print_inode( in ) struct inode *in; { int i; + struct ext4_extent_header *ehp; + struct ext4_extent *ep; printf( "Inode: %5d", in->i_number); printf( /* "Inode: %5d" */ @@ -49,7 +52,7 @@ printf( "User: %5lu Group: %5lu Size: %lu\n", (unsigned long)in->i_uid, (unsigned long)in->i_gid, (unsigned long)in->i_size); - printf( "Links: %3d Blockcount: %d\n", + printf( "Links: %3d Blockcount: %lld\n", in->i_nlink, in->i_blocks); printf( "ctime: 0x%x", in->i_ctime); printf( "atime: 0x%x", in->i_atime); @@ -57,6 +60,15 @@ printf( "BLOCKS: "); for(i=0; i < (in->i_blocks <= 24 ? ((in->i_blocks+1)/2): 12); i++) printf("%d ", in->i_db[i]); + printf( "\n"); + + printf( "Extents:\n"); + ehp = (struct ext4_extent_header *)in->i_db; + printf( "Header (magic 0x%x entries %d max %d depth %d gen %d)\n", + ehp->eh_magic, ehp->eh_ecount, ehp->eh_max, ehp->eh_depth, ehp->eh_gen); + ep = (struct ext4_extent *)((char *)(in->i_db) + sizeof(struct ext4_extent_header)); + printf( "Index (blk %d len %d start_lo %d start_hi %d)\n", + ep->e_blk, ep->e_len, ep->e_start_lo, ep->e_start_hi); printf("\n"); } @@ -77,17 +89,18 @@ I can see that this might lead to problems in an undelete. */ ip->i_mode = ei->e2di_nlink ? ei->e2di_mode : 0; - ip->i_size = ei->e2di_size; + ip->i_size = ei->e2di_size_lo; if (S_ISREG(ip->i_mode)) ip->i_size |= ((u_int64_t)ei->e2di_size_high) << 32; ip->i_atime = ei->e2di_atime; ip->i_mtime = ei->e2di_mtime; ip->i_ctime = ei->e2di_ctime; - ip->i_flags = 0; - ip->i_flags |= (ei->e2di_flags & EXT2_APPEND) ? SF_APPEND : 0; - ip->i_flags |= (ei->e2di_flags & EXT2_IMMUTABLE) ? SF_IMMUTABLE : 0; - ip->i_flags |= (ei->e2di_flags & EXT2_NODUMP) ? UF_NODUMP : 0; - ip->i_blocks = ei->e2di_nblock; + ip->i_flags = ei->e2di_flags; /* we need to entire flags to check new features */ + ip->i_gen = ei->e2di_gen; + if (ip->i_e2fs->e2fs->e2fs_features_incompat & EXT4F_ROCOMPAT_HUGE_FILE) + ip->i_blocks = ((int64_t)(ei->e2di_nblock_high)) << 32 | ei->e2di_nblock_lo; + else + ip->i_blocks = ei->e2di_nblock_lo; ip->i_gen = ei->e2di_gen; ip->i_uid = ei->e2di_uid; ip->i_gid = ei->e2di_gid; @@ -115,7 +128,7 @@ has been deleted, this would correspond to a zero link count */ ei->e2di_dtime = ei->e2di_nlink ? 0 : ip->i_mtime; - ei->e2di_size = ip->i_size; + ei->e2di_size_lo = ip->i_size; if (S_ISREG(ip->i_mode)) ei->e2di_size_high = ip->i_size >> 32; ei->e2di_atime = ip->i_atime; @@ -126,7 +139,7 @@ ei->e2di_flags |= (ip->i_flags & SF_APPEND) ? EXT2_APPEND: 0; ei->e2di_flags |= (ip->i_flags & SF_IMMUTABLE) ? EXT2_IMMUTABLE: 0; ei->e2di_flags |= (ip->i_flags & UF_NODUMP) ? EXT2_NODUMP: 0; - ei->e2di_nblock = ip->i_blocks; + ei->e2di_nblock_lo = ip->i_blocks; ei->e2di_gen = ip->i_gen; ei->e2di_uid = ip->i_uid; ei->e2di_gid = ip->i_gid; diff -urN /usr/src/sys/fs/ext2fs/ext2_readwrite.c src/ext2_readwrite.c --- /usr/src/sys/fs/ext2fs/ext2_readwrite.c 2010-01-14 22:30:54.000000000 +0800 +++ src/ext2_readwrite.c 2010-08-22 22:41:52.000000000 +0800 @@ -36,6 +36,9 @@ * $FreeBSD: src/sys/fs/ext2fs/ext2_readwrite.c,v 1.1 2010/01/14 14:30:54 lulf Exp $ */ +#include +#include + /* XXX TODO: remove these obfuscations (as in ffs_vnops.c). */ #define BLKSIZE(a, b, c) blksize(a, b, c) #define FS struct m_ext2fs @@ -45,17 +48,124 @@ #define WRITE ext2_write #define WRITE_S "ext2_write" +static int ext4_ext_read(struct vop_read_args *); +static int ext2_ind_read(struct vop_read_args *); + /* - * Vnode op for reading. + * this function handles ext4 extents block mapping */ static int -READ(ap) - struct vop_read_args /* { - struct vnode *a_vp; - struct uio *a_uio; - int a_ioflag; - struct ucred *a_cred; - } */ *ap; +ext4_ext_read(struct vop_read_args *ap) +{ + struct vnode *vp; + struct inode *ip; + struct uio *uio; + struct m_ext2fs *fs; + struct buf *bp; + struct ext4_extent nex, *ep; + struct ext4_extent_header *ehp; + struct ext4_extent_path path; + daddr_t lbn, nextlbn, newblk = 0; + off_t bytesinfile; + u_short mode; + int cache_type; + int orig_resid; + int error = 0; + int depth = 0; + long size, xfersize, blkoffset; + + vp = ap->a_vp; + ip = VTOI(vp); + mode = ip->i_mode; + uio = ap->a_uio; + memset(&path, 0, sizeof(path)); + + orig_resid = uio->uio_resid; + KASSERT(orig_resid >= 0, ("ext2_read: uio->uio_resid < 0")); + if (orig_resid == 0) + return (0); + KASSERT(uio->uio_offset >= 0, ("ext2_read: uio->uio_offset < 0")); + fs = ip->I_FS; + if (uio->uio_offset < ip->i_size && uio->uio_offset >= fs->e2fs_maxfilesize) + return (EOVERFLOW); + + for (error = 0, bp = NULL; uio->uio_resid > 0; bp = NULL) { + if ((bytesinfile = ip->i_size - uio->uio_offset) <= 0) + break; + lbn = lblkno(fs, uio->uio_offset); + nextlbn = lbn + 1; + size = BLKSIZE(fs, ip, lbn); + blkoffset = blkoff(fs, uio->uio_offset); + + xfersize = fs->e2fs_fsize - blkoffset; + if (uio->uio_resid < xfersize) + xfersize = uio->uio_resid; + if (bytesinfile < xfersize) + xfersize = bytesinfile; + + /* get block from ext4 extent cache */ + cache_type = ext4_ext_in_cache(ip, lbn, &nex); + if (cache_type != 0) { + /* block does not be allocated yet */ + if (cache_type == EXT4_EXT_CACHE_GAP) + return (error); + else if (cache_type == EXT4_EXT_CACHE_IN) + newblk = lbn - nex.e_blk + + (nex.e_start_lo | ((daddr_t)(nex.e_start_hi) << 31) << 1); + } else { + ext4_ext_find_extent(fs, ip, lbn, &path); + depth = ((struct ext4_extent_header *)(ip->i_db))->eh_depth; + if (path.ep_ext == NULL && depth != 0) + return (EIO); + + ehp = path.ep_header; + ep = path.ep_ext; + if (ep == NULL) + return (EIO); + + ext4_ext_put_cache(ip, ep, EXT4_EXT_CACHE_IN); + + newblk = lbn - ep->e_blk + + (ep->e_start_lo | ((daddr_t)(ep->e_start_hi) << 31) << 1); + + if (path.ep_bp != NULL) { + brelse(path.ep_bp); + path.ep_bp = NULL; + } + } + + error = bread(ip->i_devvp, fsbtodb(fs, newblk), size, NOCRED, &bp); + if (error) { + brelse(bp); + bp = NULL; + break; + } + + size -= bp->b_resid; + if (size < xfersize) { + if (size == 0) + break; + xfersize = size; + } + error = uiomove((char *)bp->b_data + blkoffset, + (int)xfersize, uio); + if (error) + break; + + bqrelse(bp); + } + + if (bp != NULL) + bqrelse(bp); + + return (error); +} + +/* + * this function handles traditional block mapping + */ +static int +ext2_ind_read(struct vop_read_args *ap) { struct vnode *vp; struct inode *ip; @@ -152,6 +262,35 @@ } /* + * Vnode op for reading. + */ +static int +READ(ap) + struct vop_read_args /* { + struct vnode *a_vp; + struct uio *a_uio; + int a_ioflag; + struct ucred *a_cred; + } */ *ap; +{ + struct vnode *vp; + struct inode *ip; + int error; + + vp = ap->a_vp; + ip = VTOI(vp); + + /*EXT4_EXT_LOCK(ip);*/ + if (ip->i_flags & EXT4_EXTENTS) + error = ext4_ext_read(ap); + else + error = ext2_ind_read(ap); + /*EXT4_EXT_UNLOCK(ip);*/ + + return (error); +} + +/* * Vnode op for writing. */ static int diff -urN /usr/src/sys/fs/ext2fs/ext2_subr.c src/ext2_subr.c --- /usr/src/sys/fs/ext2fs/ext2_subr.c 2010-01-14 22:30:54.000000000 +0800 +++ src/ext2_subr.c 2010-08-22 22:41:52.000000000 +0800 @@ -50,6 +50,7 @@ #include #include #include +#include #ifdef KDB void ext2_checkoverlap(struct buf *, struct inode *); @@ -70,22 +71,57 @@ struct inode *ip; struct m_ext2fs *fs; struct buf *bp; + struct ext4_extent *ep; + struct ext4_extent_header *ehp; + struct ext4_extent_path path; int32_t lbn; int bsize, error; + int depth; + daddr_t newblk; ip = VTOI(vp); fs = ip->i_e2fs; lbn = lblkno(fs, offset); bsize = blksize(fs, ip, lbn); + memset(&path, 0, sizeof(path)); *bpp = NULL; - if ((error = bread(vp, lbn, bsize, NOCRED, &bp)) != 0) { - brelse(bp); - return (error); - } - if (res) - *res = (char *)bp->b_data + blkoff(fs, offset); - *bpp = bp; + if (ext4_ext_find_extent(fs, ip, lbn, &path) == NULL) + goto normal; + depth = ((struct ext4_extent_header *)(ip->i_db))->eh_depth; + if (path.ep_ext == NULL && depth != 0) + goto normal; + ehp = path.ep_header; + ep = path.ep_ext; + if (ep == NULL) + goto normal; + + newblk = lbn - ep->e_blk + + (ep->e_start_lo | ((daddr_t)(ep->e_start_hi) << 31) << 1); + + if (path.ep_bp != NULL) { + brelse(path.ep_bp); + path.ep_bp = NULL; + } + if ((error = bread(ip->i_devvp, fsbtodb(fs, newblk), bsize, NOCRED, &bp)) != 0) { + brelse(bp); + return (error); + } + if (res) + *res = (char *)bp->b_data + blkoff(fs, offset); + *bpp = bp; + return (0); + +normal: + if (*bpp == NULL) { + if ((error = bread(vp, lbn, bsize, NOCRED, &bp)) != 0) { + brelse(bp); + return (error); + } + if (res) + *res = (char *)bp->b_data + blkoff(fs, offset); + *bpp = bp; + } return (0); } diff -urN /usr/src/sys/fs/ext2fs/ext2_vfsops.c src/ext2_vfsops.c --- /usr/src/sys/fs/ext2fs/ext2_vfsops.c 2010-01-14 22:30:54.000000000 +0800 +++ src/ext2_vfsops.c 2010-08-22 22:41:52.000000000 +0800 @@ -51,6 +51,9 @@ #include #include #include +#include + +#include #include #include @@ -61,6 +64,7 @@ #include #include #include +#include static int ext2_flushfiles(struct mount *mp, int flags, struct thread *td); static int ext2_mountfs(struct vnode *, struct mount *); @@ -288,7 +292,8 @@ return (1); } if (es->e2fs_rev > E2FS_REV0) { - if (es->e2fs_features_incompat & ~EXT2F_INCOMPAT_SUPP) { + /*if (es->e2fs_features_incompat & ~EXT2F_INCOMPAT_SUPP) {*/ + if (es->e2fs_features_incompat & ~EXT4F_INCOMPAT_SUPP) { printf( "WARNING: mount of %s denied due to unsupported optional features\n", devtoname(dev)); @@ -334,7 +339,6 @@ } else { fs->e2fs_first_inode = es->e2fs_first_ino; fs->e2fs_isize = es->e2fs_inode_size; - /* * Simple sanity check for superblock inode size value. */ @@ -350,8 +354,9 @@ fs->e2fs_itpg = fs->e2fs_ipg /fs->e2fs_ipb; fs->e2fs_descpb = fs->e2fs_bsize / sizeof (struct ext2_gd); /* s_resuid / s_resgid ? */ - fs->e2fs_gcount = (es->e2fs_bcount - es->e2fs_first_dblock + - EXT2_BLOCKS_PER_GROUP(fs) - 1) / EXT2_BLOCKS_PER_GROUP(fs); + fs->e2fs_gcount = (((int64_t)(es->e2fs_bcount_hi) << 32 | es->e2fs_bcount_lo) + - es->e2fs_first_dblock + EXT2_BLOCKS_PER_GROUP(fs) - 1) / + EXT2_BLOCKS_PER_GROUP(fs); db_count = (fs->e2fs_gcount + EXT2_DESC_PER_BLOCK(fs) - 1) / EXT2_DESC_PER_BLOCK(fs); fs->e2fs_gdbcount = db_count; @@ -383,9 +388,10 @@ brelse(bp); bp = NULL; } + fs->e2fs_total_dir = 0; for (i=0; i < fs->e2fs_gcount; i++){ - fs->e2fs_total_dir += fs->e2fs_gd[i].ext2bgd_ndirs; + fs->e2fs_total_dir += (fs->e2fs_gd[i].ext2bgd_ndirs_lo); fs->e2fs_contigdirs[i] = 0; } if (es->e2fs_rev == E2FS_REV0 || @@ -393,6 +399,25 @@ fs->e2fs_maxfilesize = 0x7fffffff; else fs->e2fs_maxfilesize = 0x7fffffffffffffff; + + /* check inode size */ + if (fs->e2fs_isize > E2FS_REV0_INODE_SIZE) { + fs->e2fs_want_extra_isize = sizeof(struct ext2fs_dinode) - + E2FS_REV0_INODE_SIZE; + + if (es->e2fs_features_rocompat & EXT4F_ROCOMPAT_EXTRA_ISIZE) { + if (fs->e2fs_want_extra_isize < es->e2fs_want_extra_isize) + fs->e2fs_want_extra_isize = es->e2fs_want_extra_isize; + if (fs->e2fs_want_extra_isize < es->e2fs_min_extra_isize) + fs->e2fs_want_extra_isize = es->e2fs_min_extra_isize; + } + } + + if (E2FS_REV0_INODE_SIZE + fs->e2fs_want_extra_isize > + fs->e2fs_isize) + printf("EXT2-fs: no space for extra inode.\n"); + + return (0); } @@ -745,9 +770,12 @@ sbp->f_bsize = EXT2_FRAG_SIZE(fs); sbp->f_iosize = EXT2_BLOCK_SIZE(fs); - sbp->f_blocks = fs->e2fs->e2fs_bcount - overhead; - sbp->f_bfree = fs->e2fs->e2fs_fbcount; - sbp->f_bavail = sbp->f_bfree - fs->e2fs->e2fs_rbcount; + sbp->f_blocks = ((int64_t)(fs->e2fs->e2fs_bcount_hi) << 32 | + fs->e2fs->e2fs_bcount_lo) - overhead; + sbp->f_bfree = ((int64_t)(fs->e2fs->e2fs_fbcount_hi) << 32 | + fs->e2fs->e2fs_fbcount_lo); + sbp->f_bavail = sbp->f_bfree - ((int64_t)(fs->e2fs->e2fs_rbcount_hi) << 32 | + fs->e2fs->e2fs_rbcount_lo); sbp->f_files = fs->e2fs->e2fs_icount; sbp->f_ffree = fs->e2fs->e2fs_ficount; return (0); @@ -853,8 +881,8 @@ struct vnode *vp; struct cdev *dev; struct thread *td; - int i, error; - int used_blocks; + int error; + int i, used_blocks; td = curthread; error = vfs_hash_get(mp, ino, flags, td, vpp, NULL, NULL); @@ -910,6 +938,7 @@ *vpp = NULL; return (error); } + /* convert ext2 inode to dinode */ ext2_ei2i((struct ext2fs_dinode *) ((char *)bp->b_data + EXT2_INODE_SIZE(fs) * ino_to_fsbo(fs, ino)), ip); @@ -919,19 +948,31 @@ ip->i_prealloc_count = 0; ip->i_prealloc_block = 0; + /* initialize ext lock */ + bzero(&ip->i_ext_lock, sizeof(struct mtx)); + mtx_init(&ip->i_ext_lock, "inode ext lock", NULL, MTX_DEF); + /* * Now we want to make sure that block pointers for unused * blocks are zeroed out - ext2_balloc depends on this * although for regular files and directories only */ - if(S_ISDIR(ip->i_mode) || S_ISREG(ip->i_mode)) { - used_blocks = (ip->i_size+fs->e2fs_bsize-1) / fs->e2fs_bsize; - for(i = used_blocks; i < EXT2_NDIR_BLOCKS; i++) - ip->i_db[i] = 0; - } -/* - ext2_print_inode(ip); -*/ + + /* + * NOTE: When ext4 file system use extents, we don't zero + * block pointers. + */ + if (!(fs->e2fs->e2fs_features_incompat & EXT4F_INCOMPAT_EXTENTS)) { + if(S_ISDIR(ip->i_mode) || S_ISREG(ip->i_mode)) { + used_blocks = (ip->i_size+fs->e2fs_bsize-1) / fs->e2fs_bsize; + for(i = used_blocks; i < EXT2_NDIR_BLOCKS; i++) + ip->i_db[i] = 0; + } + } + + + /*ext2_print_inode(ip);*/ + bqrelse(bp); /* diff -urN /usr/src/sys/fs/ext2fs/ext2_vnops.c src/ext2_vnops.c --- /usr/src/sys/fs/ext2fs/ext2_vnops.c 2010-02-20 18:19:19.000000000 +0800 +++ src/ext2_vnops.c 2010-08-22 22:41:52.000000000 +0800 @@ -1419,7 +1419,7 @@ struct vnode *vp = ap->a_vp; struct inode *ip; struct bufobj *bo; - int32_t blkno; + int64_t blkno; int error; ip = VTOI(vp); diff -urN /usr/src/sys/fs/ext2fs/ext2fs.h src/ext2fs.h --- /usr/src/sys/fs/ext2fs/ext2fs.h 2010-01-14 22:30:54.000000000 +0800 +++ src/ext2fs.h 2010-08-22 22:50:30.000000000 +0800 @@ -38,6 +38,7 @@ #define _FS_EXT2FS_EXT2_FS_H #include +#include /* * Special inode numbers @@ -71,7 +72,7 @@ /* * Maximal count of links to a file */ -#define EXT2_LINK_MAX 32000 +#define EXT2_LINK_MAX 65000 /* * Constants relative to the data blocks @@ -95,9 +96,9 @@ */ struct ext2fs { u_int32_t e2fs_icount; /* Inode count */ - u_int32_t e2fs_bcount; /* blocks count */ - u_int32_t e2fs_rbcount; /* reserved blocks count */ - u_int32_t e2fs_fbcount; /* free blocks count */ + u_int32_t e2fs_bcount_lo; /* blocks count */ + u_int32_t e2fs_rbcount_lo; /* reserved blocks count */ + u_int32_t e2fs_fbcount_lo; /* free blocks count */ u_int32_t e2fs_ficount; /* free inodes count */ u_int32_t e2fs_first_dblock; /* first data block */ u_int32_t e2fs_log_bsize; /* block size = 1024*(2^e2fs_log_bsize) */ @@ -130,8 +131,36 @@ char e2fs_vname[16]; /* volume name */ char e2fs_fsmnt[64]; /* name mounted on */ u_int32_t e2fs_algo; /* For comcate for dir */ - u_int16_t e2fs_reserved_ngdb; /* # of reserved gd blocks for resize */ - u_int32_t reserved2[204]; + u_int8_t e2fs_prealloc_blk; /* number of blocks to try to preallocate */ + u_int8_t e2fs_prealloc_dblk; /* number of dirs to preallocate */ + u_int16_t e2fs_reserved_ngdb; /* # of reserved gd blocks for resize */ + u_int8_t e2fs_journal_uuid[16]; /* uuid of journal superblock */ + u_int32_t e2fs_journal_inum; /* inode number of journal file */ + u_int32_t e2fs_journal_dev; /* device number of journal file */ + u_int32_t e2fs_last_orphan; /* start of list of inodes to delete */ + u_int32_t e2fs_hash_seed[4]; /* HTREE hash seed */ + u_int8_t e2fs_def_hash_ver; /* default hash version to use */ + u_int8_t e2fs_char_pad; + u_int16_t e2fs_desc_size; /* size of group descriptor */ + u_int32_t e2fs_def_mnt_opts; + u_int32_t e2fs_first_meta_bg; /* first metablock block group */ + u_int32_t e2fs_mkfs_time; /* when the fs was created */ + u_int32_t e2fs_jnl_blks[17]; /* backup of the journal inode */ + u_int32_t e2fs_bcount_hi; /* block count */ + u_int32_t e2fs_rbcount_hi; /* reserved blocks count */ + u_int32_t e2fs_fbcount_hi; /* free blocks count */ + u_int16_t e2fs_min_extra_isize;/* all inodes have at least some bytes */ + u_int16_t e2fs_want_extra_isize; /* new inodes should reserve some bytes */ + u_int32_t e2fs_flags; /* miscellaneous flags */ + u_int16_t e2fs_raid_stride; /* RAID stride */ + u_int16_t e2fs_mmpintv; /* number of seconds to wait in MMP checking */ + u_int64_t e2fs_mmpblk; /* block for multi-mount protection */ + u_int32_t e2fs_raid_stripe_wid;/* blocks on all data disks (N * stride) */ + u_int8_t e2fs_log_gpf; /* FLEX_BG group size */ + u_int8_t e2fs_char_pad2; + u_int16_t e2fs_pad; + u_int64_t e2fs_kbytes_written; /* number of lifetime kilobytes written */ + u_int32_t reserved2[160]; }; @@ -173,7 +202,10 @@ uint8_t *e2fs_contigdirs; char e2fs_wasvalid; /* valid at mount time */ off_t e2fs_maxfilesize; - struct ext2_gd *e2fs_gd; /* Group Descriptors */ + struct ext2_gd *e2fs_gd; /* Group Descriptors */ + + u_int16_t e2fs_min_extra_isize; /* all inodes have at least some bytes */ + u_int16_t e2fs_want_extra_isize; /* new inodes should reserve some bytes */ }; /* @@ -198,13 +230,28 @@ */ #define EXT2F_COMPAT_PREALLOC 0x0001 #define EXT2F_COMPAT_RESIZE 0x0010 +#define EXT4F_COMPAT_IMAGIC_INODES 0x0002 +#define EXT4F_COMPAT_HAS_JOURNAL 0x0004 +#define EXT4F_COMPAT_EXT_ATTR 0x0008 +#define EXT4F_COMPAT_DIR_INDEX 0x0020 #define EXT2F_ROCOMPAT_SPARSESUPER 0x0001 #define EXT2F_ROCOMPAT_LARGEFILE 0x0002 #define EXT2F_ROCOMPAT_BTREE_DIR 0x0004 +#define EXT4F_ROCOMPAT_HUGE_FILE 0x0008 +#define EXT4F_ROCOMPAT_GDT_CSUM 0x0010 +#define EXT4F_ROCOMPAT_DIR_NLINK 0x0020 +#define EXT4F_ROCOMPAT_EXTRA_ISIZE 0x0040 #define EXT2F_INCOMPAT_COMP 0x0001 #define EXT2F_INCOMPAT_FTYPE 0x0002 +#define EXT4F_INCOMPAT_RECOVER 0x0004 +#define EXT4F_INCOMPAT_JOURNAL_DEV 0x0008 +#define EXT4F_INCOMPAT_META_BG 0x0010 +#define EXT4F_INCOMPAT_EXTENTS 0x0040 +#define EXT4F_INCOMPAT_64BIT 0x0080 +#define EXT4F_INCOMPAT_MMP 0x0100 +#define EXT4F_INCOMPAT_FLEX_BG 0x0200 /* * Features supported in this implementation @@ -220,6 +267,20 @@ #define EXT2F_INCOMPAT_SUPP EXT2F_INCOMPAT_FTYPE /* + * Features supported in ext4 read-only mode + */ +#define EXT4F_INCOMPAT_SUPP (EXT2F_INCOMPAT_FTYPE \ + | EXT4F_INCOMPAT_EXTENTS \ + | EXT4F_INCOMPAT_FLEX_BG) +#define EXT4F_ROCOMPAT_SUPP (EXT2F_ROCOMPAT_SPARSESUPER \ + | EXT2F_ROCOMPAT_LARGEFILE \ + | EXT2F_ROCOMPAT_BTREE_DIR \ + | EXT4F_ROCOMPAT_GDT_CSUM \ + | EXT4F_ROCOMPAT_DIR_NLINK \ + | EXT4F_ROCOMPAT_EXTRA_ISIZE \ + | EXT4F_ROCOMPAT_HUGE_FILE) + +/* * Feature set definitions */ #define EXT2_HAS_COMPAT_FEATURE(sb,mask) \ @@ -255,14 +316,26 @@ /* ext2 file system block group descriptor */ struct ext2_gd { - u_int32_t ext2bgd_b_bitmap; /* blocks bitmap block */ - u_int32_t ext2bgd_i_bitmap; /* inodes bitmap block */ - u_int32_t ext2bgd_i_tables; /* inodes table block */ - u_int16_t ext2bgd_nbfree; /* number of free blocks */ - u_int16_t ext2bgd_nifree; /* number of free inodes */ - u_int16_t ext2bgd_ndirs; /* number of directories */ - u_int16_t reserved; - u_int32_t reserved2[3]; + u_int32_t ext2bgd_b_bitmap_lo; /* blocks bitmap block */ + u_int32_t ext2bgd_i_bitmap_lo; /* inodes bitmap block */ + u_int32_t ext2bgd_i_tables_lo; /* inodes table block */ + u_int16_t ext2bgd_nbfree_lo; /* number of free blocks */ + u_int16_t ext2bgd_nifree_lo; /* number of free inodes */ + u_int16_t ext2bgd_ndirs_lo; /* number of directories */ + u_int16_t ext2bgd_flags; /* EXT4_BG_flags */ +#if 0 + u_int32_t reserved[2]; + u_int16_t ext2bgd_i_tables_unused_lo; /* number of unused inodes */ + u_int16_t ext2bgd_chksum; /* crc16 checksum */ + u_int32_t ext2bgd_b_bitmap_hi; /* blocks bitmap block MSB */ + u_int32_t ext2bgd_i_bitmap_hi; /* inodes bitmap block MSB */ + u_int32_t ext2bgd_i_tables_hi; /* inodes table block MSB */ + u_int16_t ext2bgd_nbfree_hi; /* number of free blocks MSB */ + u_int16_t ext2bgd_nifree_hi; /* number of free inodes MSB */ + u_int16_t ext2bgd_ndirs_hi; /* number of directories MSB */ + u_int16_t ext2bgd_i_tables_unused_hi; /* number of unused inodes MSB */ +#endif + u_int32_t reserved2[3]; }; /* EXT2FS metadatas are stored in little-endian byte order. These macros @@ -275,7 +348,7 @@ * Macro-instructions used to manage several block sizes */ #define EXT2_MIN_BLOCK_SIZE 1024 -#define EXT2_MAX_BLOCK_SIZE 4096 +#define EXT2_MAX_BLOCK_SIZE 65536 #define EXT2_MIN_BLOCK_LOG_SIZE 10 #if defined(_KERNEL) # define EXT2_BLOCK_SIZE(s) ((s)->e2fs_bsize) diff -urN /usr/src/sys/fs/ext2fs/fs.h src/fs.h --- /usr/src/sys/fs/ext2fs/fs.h 2010-01-14 22:30:54.000000000 +0800 +++ src/fs.h 2010-08-22 22:41:52.000000000 +0800 @@ -93,7 +93,7 @@ /* get block containing inode from its number x */ #define ino_to_fsba(fs, x) \ - ((fs)->e2fs_gd[ino_to_cg((fs), (x))].ext2bgd_i_tables + \ + ((fs)->e2fs_gd[ino_to_cg((fs), (x))].ext2bgd_i_tables_lo + \ (((x) - 1) % (fs)->e2fs->e2fs_ipg) / (fs)->e2fs_ipb) /* get offset for inode in block */ diff -urN /usr/src/sys/fs/ext2fs/inode.h src/inode.h --- /usr/src/sys/fs/ext2fs/inode.h 2010-01-14 22:30:54.000000000 +0800 +++ src/inode.h 2010-08-22 22:41:52.000000000 +0800 @@ -38,9 +38,13 @@ #ifndef _FS_EXT2FS_INODE_H_ #define _FS_EXT2FS_INODE_H_ +#include #include +#include #include +#include + #define ROOTINO ((ino_t)2) #define NDADDR 12 /* Direct addresses in inode. */ @@ -85,7 +89,7 @@ /* Fields from struct dinode in UFS. */ u_int16_t i_mode; /* IFMT, permissions; see below. */ - int16_t i_nlink; /* File link count. */ + u_int16_t i_nlink; /* File link count. */ u_int64_t i_size; /* File byte count. */ int32_t i_atime; /* Last access time. */ int32_t i_atimensec; /* Last access time. */ @@ -96,10 +100,15 @@ int32_t i_db[NDADDR]; /* Direct disk blocks. */ int32_t i_ib[NIADDR]; /* Indirect disk blocks. */ u_int32_t i_flags; /* Status flags (chflags). */ - int32_t i_blocks; /* Blocks actually held. */ + int64_t i_blocks; /* Blocks actually held. */ int32_t i_gen; /* Generation number. */ u_int32_t i_uid; /* File owner. */ u_int32_t i_gid; /* File group. */ + + /* ext4 extents support */ + struct mtx i_ext_lock; /* this lock only is required in read/write mode + but we still use it in read-only mode. */ + struct ext4_extent_cache i_ext_cache; /* cache for ext4 extent */ }; /* --------------000709010802090406030102-- From owner-freebsd-fs@FreeBSD.ORG Sun Aug 22 11:39:10 2010 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1F65B106566B for ; Sun, 22 Aug 2010 11:39:10 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id BE4588FC12 for ; Sun, 22 Aug 2010 11:38:59 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 3751D45F20; Thu, 19 Aug 2010 21:43:32 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 1532E45E93; Thu, 19 Aug 2010 21:43:27 +0200 (CEST) Date: Thu, 19 Aug 2010 21:43:25 +0200 From: Pawel Jakub Dawidek To: "Michael W. Lucas" Message-ID: <20100819194325.GA2719@garage.freebsd.pl> References: <20100818183300.GA98970@bewilderbeast.blackhelicopters.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="a8Wt8u1KmwUX3Y2C" Content-Disposition: inline In-Reply-To: <20100818183300.GA98970@bewilderbeast.blackhelicopters.org> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: fs@freebsd.org Subject: Re: HAST secondary identifying when it's dirty? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Aug 2010 11:39:10 -0000 --a8Wt8u1KmwUX3Y2C Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 18, 2010 at 02:33:00PM -0400, Michael W. Lucas wrote: > Hi, >=20 > I have a HAST device set up between two systems. It appears that the > secondary doesn't know when it's dirty. Is there any way for the > secondary to know that its copy is incomplete? >=20 > For example, I had to resync my HAST partition due to a firewall > issue. While the master knows the secondary is dirty, the secondary > thinks it's complete. For example: >=20 > master# hastctl status > mirror: > role: primary > provname: mirror > localpath: /dev/da0s2 > extentsize: 2097152 > keepdirty: 64 > remoteaddr: 192.168.0.1 > replication: memsync > status: complete > dirty: 24385683456 bytes >=20 > secondary# hastctl status > mirror: > role: secondary > provname: mirror > localpath: /dev/da0s2 > extentsize: 2097152 > keepdirty: 0 > remoteaddr: 192.168.0.2 > replication: memsync > status: complete > dirty: 0 bytes >=20 > The secondary doesn't realize it's 24GB out of sync. >=20 > Is there any way for the secondary to get this information? Nope. Secondary doesn't even know that synchronization is progress. The only thing you can do is to: # ssh primary hastctl status :) --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --a8Wt8u1KmwUX3Y2C Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxtiVEACgkQForvXbEpPzQSqACfRsg0I5A6/ATjYVgppUmK30nX IvQAnjugdF9BBNF41hbX72QBdnKz8vg8 =GRCr -----END PGP SIGNATURE----- --a8Wt8u1KmwUX3Y2C-- From owner-freebsd-fs@FreeBSD.ORG Sun Aug 22 20:35:53 2010 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 65E0110656A4; Sun, 22 Aug 2010 20:35:53 +0000 (UTC) (envelope-from sarawgi.aditya@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 230578FC18; Sun, 22 Aug 2010 20:35:52 +0000 (UTC) Received: by iwn36 with SMTP id 36so5774238iwn.13 for ; Sun, 22 Aug 2010 13:35:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=VtSDU9wnjkTAlgdhN7mn0upSWuvM34+QRsZF5kCpGJw=; b=D/yj+K73BLEwFFTt2E4JBg2/r+Ch+nZXtBNUrk9qQLZ73YjKVZKVhvkFhkYvG2Oz06 sSaZgcGijrYjgSwc7vcXk7wDtnoNQHbcMTwETrMsv2zKjoITv7k6cqf2gl6/z5Q58JNz AZ/lp9NJJh56Yjs3HEogzrt0X4ZsZ6znqMSyI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=GGWBf/nPUkERqHsvggxphCcI8yPLlVglJaJS5rF+S2fpgZfEXSk7vFrwjrJX7tZveS ssCBeHlqCPi/bmnmODoM9q7tU4y8N25UpnjViM8eGtLO2buCSV2G/ld/fwP6e527Gx+O 6qL1VsDMMtMmXUjuWFTA44Va/QxCS7z5UnyAc= MIME-Version: 1.0 Received: by 10.231.190.74 with SMTP id dh10mr5399281ibb.43.1282508030680; Sun, 22 Aug 2010 13:13:50 -0700 (PDT) Received: by 10.231.169.139 with HTTP; Sun, 22 Aug 2010 13:13:50 -0700 (PDT) In-Reply-To: <4C71075C.9010802@gmail.com> References: <4C71075C.9010802@gmail.com> Date: Mon, 23 Aug 2010 01:43:50 +0530 Message-ID: From: aditya sarawgi To: gnehzuil Content-Type: text/plain; charset=ISO-8859-1 Cc: fs@freebsd.org Subject: Re: [patch] ext4fs read only mode X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Aug 2010 20:35:53 -0000 On Sun, Aug 22, 2010 at 4:47 PM, gnehzuil wrote: > Hi all, > > This patch makes ext2fs can read ext4 filesystem in read-only mode. There > are two files in attachments. 'ext4fs_ro_makefile.patch' is for Makefile in > modules/ext2fs/. 'ext4fs_ro_src.patch' is for source code in fs/ext2fs/. > Please use the following command to mount disk: 'mount -t ext2fs -r /dev/XXX > /YYY'. > Hi, Since ext2fs and ext4fs are different, are we planning to have a different module for ext4fs and hence different source directories > Now you can use it to read data from ext4 filesystem in the following > features: > > + HAS_JOURNAL(*) > + FILE_TYPE > + SPARSE_SUPER > + HUGE_FILE > + EXTENTS > + DIR_NLINK > + UNINIT_BG > + FLEX_BG(*) > + EXTRA_ISIZE(*) > + DIR_INDEX(**) > > > * I don't implement this feature. However you don't need to worry about it > because it doesn't be used in read-only mode. > > ** I have implemented a hash directory index in ext2_lookup() function. But > there are two functions that I think they seem to be contaminated kernel > source code. So this patch doesn't include hash directory index. > > Please test it. > > > Best regards, > > lz > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > -- Cheers, Aditya Sarawgi From owner-freebsd-fs@FreeBSD.ORG Mon Aug 23 02:54:10 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D6BC91065672; Mon, 23 Aug 2010 02:54:10 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id AD0B68FC0A; Mon, 23 Aug 2010 02:54:10 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o7N2sAjM071090; Mon, 23 Aug 2010 02:54:10 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o7N2sADD071086; Mon, 23 Aug 2010 02:54:10 GMT (envelope-from linimon) Date: Mon, 23 Aug 2010 02:54:10 GMT Message-Id: <201008230254.o7N2sADD071086@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/149855: [gvinum] growfs causes fsck to report errors in Filesystem grown with gvinum X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 02:54:10 -0000 Old Synopsis: growfs causes fsck to report errors in Filesystem grown with gvinum New Synopsis: [gvinum] growfs causes fsck to report errors in Filesystem grown with gvinum Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Mon Aug 23 02:53:46 UTC 2010 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=149855 From owner-freebsd-fs@FreeBSD.ORG Mon Aug 23 11:06:58 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B72D71065757 for ; Mon, 23 Aug 2010 11:06:58 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 8AD4F8FC16 for ; Mon, 23 Aug 2010 11:06:58 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o7NB6wSd089068 for ; Mon, 23 Aug 2010 11:06:58 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o7NB6v2k089066 for freebsd-fs@FreeBSD.org; Mon, 23 Aug 2010 11:06:57 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 23 Aug 2010 11:06:57 GMT Message-Id: <201008231106.o7NB6v2k089066@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 11:06:58 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/149855 fs [gvinum] growfs causes fsck to report errors in Filesy o kern/149495 fs [zfs] chflags sappend on zfs not working right o kern/149173 fs [patch] [zfs] make OpenSolaris installa o kern/149022 fs [hang] File system operations hangs with suspfs state o kern/149015 fs [zfs] [patch] misc fixes for ZFS code to build on Glib o kern/149014 fs [zfs] [patch] declarations in ZFS libraries/utilities o kern/149013 fs [zfs] [patch] make ZFS makefiles use the libraries fro o kern/148504 fs [zfs] ZFS' zpool does not allow replacing drives to be o kern/148490 fs [zfs]: zpool attach - resilver bidirectionally, and re o kern/148368 fs [zfs] ZFS hanging forever on 8.1-PRERELEASE o bin/148296 fs [zfs] [loader] [patch] Very slow probe in /usr/src/sys o kern/148204 fs [nfs] UDP NFS causes overload o kern/148138 fs [zfs] zfs raidz pool commands freeze o kern/147903 fs [zfs] [panic] Kernel panics on faulty zfs device o kern/147881 fs [zfs] [patch] ZFS "sharenfs" doesn't allow different " o kern/147790 fs [zfs] zfs set acl(mode|inherit) fails on existing zfs o kern/147420 fs [ufs] [panic] ufs_dirbad, nullfs, jail panic (corrupt o kern/147292 fs [nfs] [patch] readahead missing in nfs client options o kern/146941 fs [zfs] [panic] Kernel Double Fault - Happens constantly o kern/146786 fs [zfs] zpool import hangs with checksum errors o kern/146708 fs [ufs] [panic] Kernel panic in softdep_disk_write_compl o kern/146528 fs [zfs] Severe memory leak in ZFS on i386 o kern/146502 fs [nfs] FreeBSD 8 NFS Client Connection to Server o kern/146375 fs [nfs] [patch] Typos in macro variables names in sys/fs s kern/145712 fs [zfs] cannot offline two drives in a raidz2 configurat o kern/145411 fs [xfs] [panic] Kernel panics shortly after mounting an o kern/145309 fs [disklabel] Editing disk label invalidates the whole d o kern/145272 fs [zfs] [panic] Panic during boot when accessing zfs on o kern/145246 fs [ufs] dirhash in 7.3 gratuitously frees hashes when it o kern/145238 fs [zfs] [panic] kernel panic on zpool clear tank o kern/145229 fs [zfs] Vast differences in ZFS ARC behavior between 8.0 o kern/145189 fs [nfs] nfsd performs abysmally under load o kern/144929 fs [ufs] [lor] vfs_bio.c + ufs_dirhash.c o kern/144458 fs [nfs] [patch] nfsd fails as a kld p kern/144447 fs [zfs] sharenfs fsunshare() & fsshare_main() non functi o kern/144416 fs [panic] Kernel panic on online filesystem optimization s kern/144415 fs [zfs] [panic] kernel panics on boot after zfs crash o kern/144234 fs [zfs] Cannot boot machine with recent gptzfsboot code o kern/143825 fs [nfs] [panic] Kernel panic on NFS client o kern/143345 fs [ext2fs] [patch] extfs minor header cleanups to better o kern/143212 fs [nfs] NFSv4 client strange work ... o kern/143184 fs [zfs] [lor] zfs/bufwait LOR o kern/142924 fs [ext2fs] [patch] Small cleanup for the inode struct in o kern/142914 fs [zfs] ZFS performance degradation over time o kern/142878 fs [zfs] [vfs] lock order reversal o kern/142597 fs [ext2fs] ext2fs does not work on filesystems with real o kern/142489 fs [zfs] [lor] allproc/zfs LOR o kern/142466 fs Update 7.2 -> 8.0 on Raid 1 ends with screwed raid [re o kern/142401 fs [ntfs] [patch] Minor updates to NTFS from NetBSD o kern/142306 fs [zfs] [panic] ZFS drive (from OSX Leopard) causes two o kern/142068 fs [ufs] BSD labels are got deleted spontaneously o kern/141897 fs [msdosfs] [panic] Kernel panic. msdofs: file name leng o kern/141463 fs [nfs] [panic] Frequent kernel panics after upgrade fro o kern/141305 fs [zfs] FreeBSD ZFS+sendfile severe performance issues ( o kern/141091 fs [patch] [nullfs] fix panics with DIAGNOSTIC enabled o kern/141086 fs [nfs] [panic] panic("nfs: bioread, not dir") on FreeBS o kern/141010 fs [zfs] "zfs scrub" fails when backed by files in UFS2 o kern/140888 fs [zfs] boot fail from zfs root while the pool resilveri o kern/140661 fs [zfs] [patch] /boot/loader fails to work on a GPT/ZFS- o kern/140640 fs [zfs] snapshot crash o kern/140134 fs [msdosfs] write and fsck destroy filesystem integrity o kern/140068 fs [smbfs] [patch] smbfs does not allow semicolon in file o kern/139725 fs [zfs] zdb(1) dumps core on i386 when examining zpool c o kern/139715 fs [zfs] vfs.numvnodes leak on busy zfs o bin/139651 fs [nfs] mount(8): read-only remount of NFS volume does n o kern/139597 fs [patch] [tmpfs] tmpfs initializes va_gen but doesn't u o kern/139564 fs [zfs] [panic] 8.0-RC1 - Fatal trap 12 at end of shutdo o kern/139407 fs [smbfs] [panic] smb mount causes system crash if remot o kern/139363 fs [nfs] diskless root nfs mount from non FreeBSD server o kern/138790 fs [zfs] ZFS ceases caching when mem demand is high o kern/138421 fs [ufs] [patch] remove UFS label limitations o kern/138202 fs mount_msdosfs(1) see only 2Gb f kern/137037 fs [zfs] [hang] zfs rollback on root causes FreeBSD to fr o kern/136968 fs [ufs] [lor] ufs/bufwait/ufs (open) o kern/136945 fs [ufs] [lor] filedesc structure/ufs (poll) o kern/136944 fs [ffs] [lor] bufwait/snaplk (fsync) o kern/136873 fs [ntfs] Missing directories/files on NTFS volume o kern/136865 fs [nfs] [patch] NFS exports atomic and on-the-fly atomic o kern/136470 fs [nfs] Cannot mount / in read-only, over NFS o kern/135546 fs [zfs] zfs.ko module doesn't ignore zpool.cache filenam o kern/135469 fs [ufs] [panic] kernel crash on md operation in ufs_dirb o kern/135050 fs [zfs] ZFS clears/hides disk errors on reboot o kern/134491 fs [zfs] Hot spares are rather cold... o kern/133676 fs [smbfs] [panic] umount -f'ing a vnode-based memory dis o kern/133614 fs [panic] panic: ffs_truncate: read-only filesystem o kern/133174 fs [msdosfs] [patch] msdosfs must support utf-encoded int f kern/133150 fs [zfs] Page fault with ZFS on 7.1-RELEASE/amd64 while w o kern/132960 fs [ufs] [panic] panic:ffs_blkfree: freeing free frag o kern/132397 fs reboot causes filesystem corruption (failure to sync b o kern/132331 fs [ufs] [lor] LOR ufs and syncer o kern/132237 fs [msdosfs] msdosfs has problems to read MSDOS Floppy o kern/132145 fs [panic] File System Hard Crashes o kern/131441 fs [unionfs] [nullfs] unionfs and/or nullfs not combineab o kern/131360 fs [nfs] poor scaling behavior of the NFS server under lo o kern/131342 fs [nfs] mounting/unmounting of disks causes NFS to fail o bin/131341 fs makefs: error "Bad file descriptor" on the mount poin o kern/130920 fs [msdosfs] cp(1) takes 100% CPU time while copying file o kern/130210 fs [nullfs] Error by check nullfs o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129488 fs [smbfs] Kernel "bug" when using smbfs in smbfs_smb.c: o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/129059 fs [zfs] [patch] ZFS bootloader whitelistable via WITHOUT f kern/128829 fs smbd(8) causes periodic panic on 7-RELEASE o kern/127420 fs [gjournal] [panic] Journal overflow on gmirrored gjour o kern/127029 fs [panic] mount(8): trying to mount a write protected zi o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file o kern/125895 fs [ffs] [panic] kernel: panic: ffs_blkfree: freeing free s kern/125738 fs [zfs] [request] SHA256 acceleration in ZFS p kern/124621 fs [ext3] [patch] Cannot mount ext2fs partition f bin/124424 fs [zfs] zfs(8): zfs list -r shows strange snapshots' siz o kern/123939 fs [msdosfs] corrupts new files o kern/122380 fs [ffs] ffs_valloc:dup alloc (Soekris 4801/7.0/USB Flash o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o bin/121898 fs [nullfs] pwd(1)/getcwd(2) fails with Permission denied o bin/121779 fs [ufs] snapinfo(8) (and related tools?) only work for t o bin/121366 fs [zfs] [patch] Automatic disk scrubbing from periodic(8 o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha f kern/120991 fs [panic] [fs] [snapshot] System crashes when manipulati o kern/120483 fs [ntfs] [patch] NTFS filesystem locking changes o kern/120482 fs [ntfs] [patch] Sync style changes between NetBSD and F f kern/119735 fs [zfs] geli + ZFS + samba starting on boot panics 7.0-B o kern/118912 fs [2tb] disk sizing/geometry problem with large array o kern/118713 fs [minidump] [patch] Display media size required for a k o bin/118249 fs mv(1): moving a directory changes its mtime o kern/118107 fs [ntfs] [panic] Kernel panic when accessing a file at N o bin/117315 fs [smbfs] mount_smbfs(8) and related options can't mount o kern/117314 fs [ntfs] Long-filename only NTFS fs'es cause kernel pani o kern/117158 fs [zfs] zpool scrub causes panic if geli vdevs detach on o bin/116980 fs [msdosfs] [patch] mount_msdosfs(8) resets some flags f o conf/116931 fs lack of fsck_cd9660 prevents mounting iso images with p kern/116608 fs [msdosfs] [patch] msdosfs fails to check mount options o kern/116583 fs [ffs] [hang] System freezes for short time when using o kern/116170 fs [panic] Kernel panic when mounting /tmp o kern/115645 fs [snapshots] [panic] lockmgr: thread 0xc4c00d80, not ex o bin/115361 fs [zfs] mount(8) gets into a state where it won't set/un o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o kern/113852 fs [smbfs] smbfs does not properly implement DFS referral o bin/113838 fs [patch] [request] mount(8): add support for relative p o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/111843 fs [msdosfs] Long Names of files are incorrectly created o kern/111782 fs [ufs] dump(8) fails horribly for large filesystems s bin/111146 fs [2tb] fsck(8) fails on 6T filesystem o kern/109024 fs [msdosfs] [iconv] mount_msdosfs: msdosfs_iconv: Operat o kern/109010 fs [msdosfs] can't mv directory within fat32 file system o bin/107829 fs [2TB] fdisk(8): invalid boundary checking in fdisk / w o kern/106107 fs [ufs] left-over fsck_snapshot after unfinished backgro o kern/106030 fs [ufs] [panic] panic in ufs from geom when a dead disk o kern/104406 fs [ufs] Processes get stuck in "ufs" state under persist o kern/104133 fs [ext2fs] EXT2FS module corrupts EXT2/3 filesystems o kern/103035 fs [ntfs] Directories in NTFS mounted disc images appear o kern/101324 fs [smbfs] smbfs sometimes not case sensitive when it's s o kern/99290 fs [ntfs] mount_ntfs ignorant of cluster sizes o kern/97377 fs [ntfs] [patch] syntax cleanup for ntfs_ihash.c o kern/95222 fs [iso9660] File sections on ISO9660 level 3 CDs ignored o kern/94849 fs [ufs] rename on UFS filesystem is not atomic o kern/94769 fs [ufs] Multiple file deletions on multi-snapshotted fil o kern/94733 fs [smbfs] smbfs may cause double unlock o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D o kern/92272 fs [ffs] [hang] Filling a filesystem while creating a sna f kern/91568 fs [ufs] [panic] writing to UFS/softupdates DVD media in o kern/91134 fs [smbfs] [patch] Preserve access and modification time a kern/90815 fs [smbfs] [patch] SMBFS with character conversions somet o kern/88657 fs [smbfs] windows client hang when browsing a samba shar o kern/88266 fs [smbfs] smbfs does not implement UIO_NOCOPY and sendfi o kern/87859 fs [smbfs] System reboot while umount smbfs. o kern/86587 fs [msdosfs] rm -r /PATH fails with lots of small files o kern/85326 fs [smbfs] [panic] saving a file via samba to an overquot o kern/84589 fs [2TB] 5.4-STABLE unresponsive during background fsck 2 o kern/80088 fs [smbfs] Incorrect file time setting on NTFS mounted vi o kern/73484 fs [ntfs] Kernel panic when doing `ls` from the client si o bin/73019 fs [ufs] fsck_ufs(8) cannot alloc 607016868 bytes for ino o kern/71774 fs [ntfs] NTFS cannot "see" files on a WinXP filesystem o kern/68978 fs [panic] [ufs] crashes with failing hard disk, loose po o kern/65920 fs [nwfs] Mounted Netware filesystem behaves strange o kern/65901 fs [smbfs] [patch] smbfs fails fsx write/truncate-down/tr a docs/61716 fs [patch] newfs(8) code and manpage are out of sync o kern/61503 fs [smbfs] mount_smbfs does not work as non-root o kern/55617 fs [smbfs] Accessing an nsmb-mounted drive via a smb expo o kern/51685 fs [hang] Unbounded inode allocation causes kernel to loc o kern/51583 fs [nullfs] [patch] allow to work with devices and socket o kern/36566 fs [smbfs] System reboot with dead smb mount and umount o kern/33464 fs [ufs] soft update inconsistencies after system crash o kern/18874 fs [2TB] 32bit NFS servers export wrong negative values t 188 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon Aug 23 11:19:59 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 84DB91065693 for ; Mon, 23 Aug 2010 11:19:59 +0000 (UTC) (envelope-from bugReporter@Haakh.de) Received: from mo-p00-ob.rzone.de (mo-p00-ob.rzone.de [81.169.146.162]) by mx1.freebsd.org (Postfix) with ESMTP id 04D9A8FC1D for ; Mon, 23 Aug 2010 11:19:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; t=1282562397; l=237; s=domk; d=haakh.de; h=Content-Transfer-Encoding:Content-Type:Subject:To:MIME-Version:From: Date:X-RZG-CLASS-ID:X-RZG-AUTH; bh=R0neLhjI4xWM7/PTBkRNkTO0SPo=; b=cqAe+rzCpkfb79XiJgQ+UiWT3JpBdOZ1lPRyRArxVJqpyis3f0ajMDMQJE7t/9jOG+1 UOJOCyU13ElEbfMNHL6QSPZO4rI7HEfCtF67/BoGK3r9fVmaBhDk08AP5lqqoGTd/Y4xG EXVbZMK6i7y6INPjNu7/z08K4BxIzsGLrSQ= X-RZG-AUTH: :LWQcbViwW/e6OTbW0dHzwKkCepY3+zAQY9KdRPw9VcHc3bN9H/z/W9i13I4= X-RZG-CLASS-ID: mo00 Received: from abaton.Haakh.de (p57A756BE.dip.t-dialin.net [87.167.86.190]) by post.strato.de (fruni mo8) (RZmta 23.5) with ESMTP id f04504m7NAh8NF for ; Mon, 23 Aug 2010 13:08:13 +0200 (MEST) Received: from Crabberio.Haakh.de (crabberio.haakh.de [IPv6:2001:5c0:1508:1500:213:d4ff:fed0:bb7f]) by abaton.Haakh.de (8.14.4/8.14.4) with ESMTP id o7NB8Bot082431 for ; Mon, 23 Aug 2010 13:08:12 +0200 (CEST) (envelope-from bugReporter@Haakh.de) Message-ID: <4C72569B.9050807@Haakh.de> Date: Mon, 23 Aug 2010 13:08:11 +0200 From: "Dr. A. Haakh" User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; de-AT; rv:1.8.1.24) Gecko/20100801 SeaMonkey/1.1.19 MIME-Version: 1.0 To: freebsd-fs@FreeBSD.org Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_HK_NAME_DR, T_RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on abaton.Haakh.de X-Mailman-Approved-At: Mon, 23 Aug 2010 11:41:15 +0000 Cc: Subject: devfs: proposal to create links for bootdevice X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 11:19:59 -0000 Hi, it would be nice if device-nodes like bda, bdb, bdc ... could be created at boot time referring to the partitions of the boot-device. With these nodes in your fstab you don't have to edit fstab if devices change. Andreas From owner-freebsd-fs@FreeBSD.ORG Mon Aug 23 12:03:50 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7EC9C1065694 for ; Mon, 23 Aug 2010 12:03:50 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 0D25E8FC18 for ; Mon, 23 Aug 2010 12:03:49 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 84A223F61E; Mon, 23 Aug 2010 11:46:11 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.4/8.14.4) with ESMTP id o7NBkBUa086012; Mon, 23 Aug 2010 11:46:11 GMT (envelope-from phk@critter.freebsd.dk) To: "Dr. A. Haakh" From: "Poul-Henning Kamp" In-Reply-To: Your message of "Mon, 23 Aug 2010 13:08:11 +0200." <4C72569B.9050807@Haakh.de> Date: Mon, 23 Aug 2010 11:46:11 +0000 Message-ID: <86011.1282563971@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: freebsd-fs@FreeBSD.org Subject: Re: devfs: proposal to create links for bootdevice X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 12:03:50 -0000 In message <4C72569B.9050807@Haakh.de>, "Dr. A. Haakh" writes: >Hi, > >it would be nice if device-nodes like bda, bdb, bdc ... could be created >at boot time referring to the partitions of the boot-device. > >With these nodes in your fstab you don't have to edit fstab if devices >change. Use g_label ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-fs@FreeBSD.ORG Mon Aug 23 12:58:15 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B8F981065693 for ; Mon, 23 Aug 2010 12:58:15 +0000 (UTC) (envelope-from bugReporter@Haakh.de) Received: from mo-p00-ob.rzone.de (mo-p00-ob.rzone.de [81.169.146.162]) by mx1.freebsd.org (Postfix) with ESMTP id 2EC058FC17 for ; Mon, 23 Aug 2010 12:58:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; t=1282568293; l=637; s=domk; d=haakh.de; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:References: Subject:CC:To:MIME-Version:From:Date:X-RZG-CLASS-ID:X-RZG-AUTH; bh=ymPZfu7FMpBwh1FCc0JBkLkx2p0=; b=uTmY8pdeIvW1gon9ACg1fGFqcE+FY0eH0VRROKxTP6w3Lixt2ubGTZ4w112bV7yfEbE uHdvY5vBNPaI7RQBEbsOykC2/PqenOI51yhQBV/3W/HztwkQd1Avh4R4bzHerHZ3aTPiu SzG5hyXaRYg4/ee6ZkmeocII2WVqSvS6Tmg= X-RZG-AUTH: :LWQcbViwW/e6OTbW0dHzwKkCepY3+zAQY9KdRPw9VcHc3bN9H/H+W1EMYwA= X-RZG-CLASS-ID: mo00 Received: from abaton.Haakh.de (p57A739B9.dip.t-dialin.net [87.167.57.185]) by post.strato.de (jimi mo18) (RZmta 23.5) with ESMTP id 505646m7NCJLfM ; Mon, 23 Aug 2010 14:58:13 +0200 (MEST) Received: from Crabberio.Haakh.de (crabberio.haakh.de [IPv6:2001:5c0:1508:1500:213:d4ff:fed0:bb7f]) by abaton.Haakh.de (8.14.4/8.14.4) with ESMTP id o7NCw9m8084035; Mon, 23 Aug 2010 14:58:09 +0200 (CEST) (envelope-from bugReporter@Haakh.de) Message-ID: <4C727061.10904@Haakh.de> Date: Mon, 23 Aug 2010 14:58:09 +0200 From: "Dr. A. Haakh" User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; de-AT; rv:1.8.1.24) Gecko/20100801 SeaMonkey/1.1.19 MIME-Version: 1.0 To: Poul-Henning Kamp References: <86011.1282563971@critter.freebsd.dk> In-Reply-To: <86011.1282563971@critter.freebsd.dk> Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_HK_NAME_DR, T_RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on abaton.Haakh.de X-Mailman-Approved-At: Mon, 23 Aug 2010 13:27:15 +0000 Cc: freebsd-fs@FreeBSD.org Subject: Re: devfs: proposal to create links for bootdevice X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 12:58:15 -0000 Poul-Henning Kamp schrieb: > In message <4C72569B.9050807@Haakh.de>, "Dr. A. Haakh" writes: > >> Hi, >> >> it would be nice if device-nodes like bda, bdb, bdc ... could be created >> at boot time referring to the partitions of the boot-device. >> >> With these nodes in your fstab you don't have to edit fstab if devices >> change. >> > > Use g_label ? > glabel is an option if you have just one slice to boot from. If you have several OSes using ufs (e.g 7.3 and 8.1) then you would have to use different labels and again different entries in fstab. Generic entries like the ones described above would be helpful. From owner-freebsd-fs@FreeBSD.ORG Mon Aug 23 15:13:51 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B22D410656AA for ; Mon, 23 Aug 2010 15:13:51 +0000 (UTC) (envelope-from ticso@cicely7.cicely.de) Received: from raven.bwct.de (raven.bwct.de [85.159.14.73]) by mx1.freebsd.org (Postfix) with ESMTP id 2599E8FC13 for ; Mon, 23 Aug 2010 15:13:50 +0000 (UTC) Received: from mail.cicely.de ([10.1.1.37]) by raven.bwct.de (8.13.4/8.13.4) with ESMTP id o7NFDmIp053445 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Mon, 23 Aug 2010 17:13:49 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (cicely7.cicely.de [10.1.1.9]) by mail.cicely.de (8.14.4/8.14.4) with ESMTP id o7NFDgAb050532 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 23 Aug 2010 17:13:42 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (localhost [127.0.0.1]) by cicely7.cicely.de (8.14.2/8.14.2) with ESMTP id o7NFDgke069074; Mon, 23 Aug 2010 17:13:42 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: (from ticso@localhost) by cicely7.cicely.de (8.14.2/8.14.2/Submit) id o7NFDeg9069073; Mon, 23 Aug 2010 17:13:40 +0200 (CEST) (envelope-from ticso) Date: Mon, 23 Aug 2010 17:13:40 +0200 From: Bernd Walter To: "Dr. A. Haakh" Message-ID: <20100823151340.GP61454@cicely7.cicely.de> References: <86011.1282563971@critter.freebsd.dk> <4C727061.10904@Haakh.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4C727061.10904@Haakh.de> X-Operating-System: FreeBSD cicely7.cicely.de 7.0-STABLE i386 User-Agent: Mutt/1.5.11 X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED=-1, BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01 autolearn=ham version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on spamd.cicely.de Cc: freebsd-fs@freebsd.org, Poul-Henning Kamp Subject: Re: devfs: proposal to create links for bootdevice X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: ticso@cicely.de List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 15:13:51 -0000 On Mon, Aug 23, 2010 at 02:58:09PM +0200, Dr. A. Haakh wrote: > > > Poul-Henning Kamp schrieb: > >In message <4C72569B.9050807@Haakh.de>, "Dr. A. Haakh" writes: > > > >>Hi, > >> > >>it would be nice if device-nodes like bda, bdb, bdc ... could be created > >>at boot time referring to the partitions of the boot-device. > >> > >>With these nodes in your fstab you don't have to edit fstab if devices > >>change. > >> > > > >Use g_label ? > > > glabel is an option if you have just one slice to boot from. If you have > several OSes using ufs (e.g 7.3 and 8.1) then you would have to > use different labels and again different entries in fstab. Generic > entries like the ones described above would be helpful. It won't work with /dev/bd* either. The loader knows the BIOS disk and the kernel gets the fstab entry via loader. If you put /dev/bda into fstab then the kernel doesn't know anything /dev/ad0a anymore. You need a bootstraping devicename anywhere. -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. From owner-freebsd-fs@FreeBSD.ORG Tue Aug 24 07:08:01 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C387F1065693 for ; Tue, 24 Aug 2010 07:08:01 +0000 (UTC) (envelope-from jrisom@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 88BBF8FC19 for ; Tue, 24 Aug 2010 07:08:01 +0000 (UTC) Received: by iwn36 with SMTP id 36so7306584iwn.13 for ; Tue, 24 Aug 2010 00:08:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from :user-agent:mime-version:to:subject:content-type :content-transfer-encoding; bh=fnUBXr5YHfmpxdmXRLHsirhTKscMl7GSEUAl/fMAxMc=; b=R+JpqCQuCXLj0I7JuBxj3gvPLQJtXOGZ/h94JVM45/d7UPQseIhxoa3tif+TpyJx8a d1kCoJxvWitzQbpIPTM2vWUTV18n37PvLJYr2Sk/AxyKt8SQvQGypSgT1piWmYtrA2B/ Bve3fXPytdldymQFABKGtoZFWS6hl4YBUFWyY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; b=pw6UYVmNDPPJSeZ6bc6skOt8XMdyoi+nqVBNYw6xbZ+CM3wIBf3rqEFhh5UNh775E1 sKFFWxTBAcia0y1+M7ghmKJfuA0HeLhSEyvrst0rTlRUbybo9O7YCtxgiDUrjszUb9/8 vf6gZ+c26k0E9+1eO4tgkbgTHg1P5ZhH9tC+c= Received: by 10.231.40.9 with SMTP id i9mr7954754ibe.5.1282632159280; Mon, 23 Aug 2010 23:42:39 -0700 (PDT) Received: from [192.168.1.3] (c-24-14-170-47.hsd1.il.comcast.net [24.14.170.47]) by mx.google.com with ESMTPS id g31sm7048693ibh.16.2010.08.23.23.42.37 (version=SSLv3 cipher=RC4-MD5); Mon, 23 Aug 2010 23:42:37 -0700 (PDT) Message-ID: <4C7369DF.9020600@gmail.com> Date: Tue, 24 Aug 2010 01:42:39 -0500 From: Joshua Isom User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Tue, 24 Aug 2010 10:54:58 +0000 Subject: Fwd: zpool bootfs property X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Aug 2010 07:08:01 -0000 Note: Redirected here from -questions due to lack of replies and increased relevancy here. I was just able to change my zfs boot pool to raid0 by resetting the bootfs property, adding the slice, and setting it back. I was able to find a webpage talking about importing FreeBSD zfs pools into opensolaris and having problems with disk management because of the bootfs property. With the property set, `zpool add ...` prints out "root pool can not have multiple vdevs or separate logs" which confused me because of FreeBSD supporting booting from raidz drives. So now I wonder, is the bootfs property needed in FreeBSD? If it needs set for at least booting, then will having it set cause headaches when working with a raidz pool and a failed drive? And should the wiki be updated? From owner-freebsd-fs@FreeBSD.ORG Tue Aug 24 15:00:40 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EA334106566C for ; Tue, 24 Aug 2010 15:00:40 +0000 (UTC) (envelope-from roberto@keltia.freenix.fr) Received: from keltia.net (unknown [IPv6:2a01:240:fe5c::41]) by mx1.freebsd.org (Postfix) with ESMTP id 8508E8FC12 for ; Tue, 24 Aug 2010 15:00:40 +0000 (UTC) Received: from roberto-al.eurocontrol.fr (aran.keltia.net [88.191.250.24]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: roberto) by keltia.net (Postfix/TLS) with ESMTPSA id 630279ADC; Tue, 24 Aug 2010 17:00:38 +0200 (CEST) Date: Tue, 24 Aug 2010 17:00:36 +0200 From: Ollivier Robert To: Peter Jeremy Message-ID: <20100824150035.GB99477@roberto-al.eurocontrol.fr> References: <20100810214418.GA28288@tolstoy.tols.org> <20100811014919.GA52992@icarus.home.lan> <20100811192537.GA44635@tolstoy.tols.org> <20100811214302.GB44635@tolstoy.tols.org> <20100812205625.GA79515@server.vk2pj.dyndns.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100812205625.GA79515@server.vk2pj.dyndns.org> X-Operating-System: MacOS X / Macbook Pro - FreeBSD 7.2 / Dell D820 SMP User-Agent: Mutt/1.5.20 (2009-06-14) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.3 (keltia.net); Tue, 24 Aug 2010 17:00:38 +0200 (CEST) Cc: freebsd-fs@freebsd.org Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Aug 2010 15:00:41 -0000 According to Peter Jeremy: > I suspect Artem is referring to his patch at http://pastebin.com/ZCkzkWcs > which I have tweaked somewhat (see the last patch in > http://www.freebsd.org/cgi/query-pr.cgi?pr=146410 ). Thanks, cou you please send it in a non-QP-encoded form please? > Whilst these patches _are_ hacks, they seem to do a good job of > making ZFS and UFS play together. Is the patch only useful in these mixed situations or could it be also interestng for those of use running full-zfs (cf. http://www.keltia.net/howtos/zfsboot)? -- Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- roberto@keltia.net In memoriam to Ondine : http://ondine.keltia.net/ From owner-freebsd-fs@FreeBSD.ORG Tue Aug 24 15:20:48 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E31FC1065697 for ; Tue, 24 Aug 2010 15:20:47 +0000 (UTC) (envelope-from darksoul@darkbsd.org) Received: from quasar.darkbsd.org (shinigami.darkbsd.org [82.227.96.182]) by mx1.freebsd.org (Postfix) with ESMTP id 149988FC16 for ; Tue, 24 Aug 2010 15:20:46 +0000 (UTC) Received: from quasar.darkbsd.org (localhost [127.0.0.1]) by quasar.darkbsd.org (Postfix) with ESMTP id E2AC657A3 for ; Tue, 24 Aug 2010 17:01:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=darkbsd.org; h=message-id :date:from:mime-version:to:subject:content-type; s=selector1; bh=Nv+9b68rrUR8TQQWXw3FYcQ6l9A=; b=c7SziiLYy88y/NEd4wuB47nprtIb bHMZDGy0rrXmZR/Aqj2+j8mxf+gxAypjfX8NxHwGp1s1WSTE3c9nSJx1xxqPWcBM dOQO2aUAJSMb4lQXdtWYWm4QkQfu/bbOY3r+mIDFtVsJtGuc/2Z5TxrvEqCqLlbP m6hT7gioF1ICl6Q= DomainKey-Signature: a=rsa-sha1; c=nofws; d=darkbsd.org; h=message-id :date:from:mime-version:to:subject:content-type; q=dns; s= selector1; b=0Alow0yup6We1jgPdQvwkuJPEqhrzQFxU4dj9+iw0aKaTj+q+U9 8vERm9BWOkQsVPrsAiE5ekL3ED7P55TUXnzHkj5Ut0E19zPmRD44s+UlU5XuVZEA ZdJT9Hy2iwLobS+htVdhj3lP5rlnTkl1nMqcavUSvTlcWfIRK2DlYjjo= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=darkbsd.org; h= content-type:content-type:subject:subject:mime-version :user-agent:from:from:date:date:message-id:received:received; s= selector1; t=1282662095; bh=wDMXpnnuadncwAHAAlsr/5WDwt3ZenyWorUh FJTr7zg=; b=WATCkf7zWcpt4HUGPBRKYNkHR14Mq+BXYwTUa0kwIebKZHeX9bp6 E5avGnGxvEx9TyfiKWwF+OYsK+2DdbThrAMSAu93m/uPQsXUIEiefhWPThCvxAHK sX38e9XD9glj12D+UMIwbvkBH1H1TcWiH5ZfIhwDKNk5HgdQAhuBek4= Received: from quasar.darkbsd.org ([127.0.0.1]) by quasar.darkbsd.org (quasar.darkbsd.org [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id yf7G+4pxkdgM for ; Tue, 24 Aug 2010 17:01:35 +0200 (CEST) Received: from [192.168.3.42] (archer.yomi.darkbsd.org [192.168.3.42]) (Authenticated sender: darksoul) by quasar.darkbsd.org (Postfix) with ESMTPSA id 3AF4A579C for ; Tue, 24 Aug 2010 17:01:33 +0200 (CEST) Message-ID: <4C73DED0.7070308@darkbsd.org> Date: Wed, 25 Aug 2010 00:01:36 +0900 From: DarkSoul User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.12pre) Gecko/20100809 Shredder/3.0.7pre MIME-Version: 1.0 To: freebsd-fs@freebsd.org X-Enigmail-Version: 1.0.1 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigDA26B68AD8F6B040659463B4" Subject: A way to alter ZFS metadata stored on a drive ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Aug 2010 15:20:48 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigDA26B68AD8F6B040659463B4 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, I currently am bumping against this bug : http://bugs.opensolaris.org/view_bug.do?bug_id=3D6782540 Long story short, I had a dead drive on my pool, I tried to replace it only to have the planned spare drive die on me. Which led to this : pool: prana state: DEGRADED scrub: none requested config: NAME STATE READ WRITE CKSUM prana DEGRADED 0 0 0 raidz1 ONLINE 0 0 0 ad16 ONLINE 0 0 0 ad30 ONLINE 0 0 0 ad28 ONLINE 0 0 0 ad18 ONLINE 0 0 0 ad14 ONLINE 0 0 0 raidz1 DEGRADED 0 0 0 ada2 ONLINE 0 0 0 ad4 ONLINE 0 0 0 ad6 ONLINE 0 0 0 ad8 ONLINE 0 0 0 replacing UNAVAIL 0 12.2K 0 insufficient replicas 16391273719868046777 UNAVAIL 0 14.5K 0 was /dev/ada3/ol= d 17286942469715238412 UNAVAIL 0 14.5K 0 was /dev/ada3 raidz1 ONLINE 0 0 0 ada3 ONLINE 0 0 0 ada1 ONLINE 0 0 0 ada0 ONLINE 0 0 0 ad32 ONLINE 0 0 0 ad34 ONLINE 0 0 0 cache ad12 ONLINE 0 0 0 ad37 ONLINE 0 0 0 Here is the zdb dump : prana version=3D14 name=3D'prana' state=3D0 txg=3D3789517 pool_guid=3D4093564297936254749 hostname=3D'' vdev_tree type=3D'root' id=3D0 guid=3D4093564297936254749 children[0] type=3D'raidz' id=3D0 guid=3D2955456027802876323 nparity=3D1 metaslab_array=3D14 metaslab_shift=3D36 ashift=3D9 asize=3D7501485178880 is_log=3D0 children[0] type=3D'disk' id=3D0 guid=3D14068763453858628066 path=3D'/dev/ad16' whole_disk=3D0 DTL=3D706 children[1] type=3D'disk' id=3D1 guid=3D3644994118268583474 path=3D'/dev/ad30' whole_disk=3D0 DTL=3D4152 children[2] type=3D'disk' id=3D2 guid=3D14192589917373989656 path=3D'/dev/ad28' whole_disk=3D0 DTL=3D93 children[3] type=3D'disk' id=3D3 guid=3D12279545963786751752 path=3D'/dev/ad18' whole_disk=3D0 DTL=3D380 children[4] type=3D'disk' id=3D4 guid=3D9980201723692421139 path=3D'/dev/ad14' whole_disk=3D0 DTL=3D647 children[1] type=3D'raidz' id=3D1 guid=3D1078755695237588414 nparity=3D1 metaslab_array=3D175 metaslab_shift=3D36 ashift=3D9 asize=3D7501485178880 is_log=3D0 children[0] type=3D'disk' id=3D0 guid=3D12900041001921590764 path=3D'/dev/ada2' whole_disk=3D0 DTL=3D4127 children[1] type=3D'disk' id=3D1 guid=3D7067019390683734240 path=3D'/dev/ad4' whole_disk=3D0 DTL=3D198 children[2] type=3D'disk' id=3D2 guid=3D783285185901703545 path=3D'/dev/ad6' whole_disk=3D0 DTL=3D197 children[3] type=3D'disk' id=3D3 guid=3D9688838378677189049 path=3D'/dev/ad8' whole_disk=3D0 DTL=3D196 children[4] type=3D'replacing' id=3D4 guid=3D500478404305589015 whole_disk=3D0 children[0] type=3D'disk' id=3D0 guid=3D16391273719868046777 path=3D'/dev/ada3/old' whole_disk=3D0 not_present=3D1 DTL=3D195 children[1] type=3D'disk' id=3D1 guid=3D17286942469715238412 path=3D'/dev/ada3' whole_disk=3D0 not_present=3D1 DTL=3D4146 children[2] type=3D'raidz' id=3D2 guid=3D4090878701271390479 nparity=3D1 metaslab_array=3D164 metaslab_shift=3D36 ashift=3D9 asize=3D7501485178880 is_log=3D0 children[0] type=3D'disk' id=3D0 guid=3D2581994749932752632 path=3D'/dev/ada3' whole_disk=3D0 DTL=3D394 children[1] type=3D'disk' id=3D1 guid=3D3261576559815570039 path=3D'/dev/ada1' whole_disk=3D0 DTL=3D393 children[2] type=3D'disk' id=3D2 guid=3D2403801851152895606 path=3D'/dev/ada0' whole_disk=3D0 DTL=3D4105 children[3] type=3D'disk' id=3D3 guid=3D5795813053837736090 path=3D'/dev/ad32' whole_disk=3D0 DTL=3D391 children[4] type=3D'disk' id=3D4 guid=3D4547118883658878725 path=3D'/dev/ad34' whole_disk=3D0 DTL=3D390 I now can't detach/remove/replace the following drive because "no valid replicas" or because you "can't replace a replacing drive". I gather the problem would be solved if I had a drive with the proper ZDB information and GUID to fool the pool, but I am clueless as to how to generate this information, and it doesn't seem from what the manpage says, that zdb allows you to modify this information. Either that, or find a way to remove these bogus devices, or have the pool "forget" that it must replace a device that does not exist anymore, with a device that does not exist anymore. I would gladly welcome any clue as to how the ZDB metadata work and are stored on the disk, as I can't find written specifications short of probing the ZFS code itself. I can provide any information required, such as the first 512K for each drive (containing the ZDB copies). Thanks in advance for any help. --=20 Stephane LAPIE, EPITA SRS, Promo 2005 "Even when they have digital readouts, I can't understand them." --MegaTokyo --------------enigDA26B68AD8F6B040659463B4 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkxz3tYACgkQ24Ql8u6TF2OimACg9g8vHDCE1ndEp7SzB4y3v89O 2bwAoL1zVbjaeH44PLW1mfdVKkHROHDZ =PUe+ -----END PGP SIGNATURE----- --------------enigDA26B68AD8F6B040659463B4-- From owner-freebsd-fs@FreeBSD.ORG Tue Aug 24 20:05:38 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8860F1065697 for ; Tue, 24 Aug 2010 20:05:38 +0000 (UTC) (envelope-from peterjeremy@acm.org) Received: from mail16.syd.optusnet.com.au (mail16.syd.optusnet.com.au [211.29.132.197]) by mx1.freebsd.org (Postfix) with ESMTP id 0BF6E8FC0A for ; Tue, 24 Aug 2010 20:05:37 +0000 (UTC) Received: from server.vk2pj.dyndns.org (c220-239-116-103.belrs4.nsw.optusnet.com.au [220.239.116.103]) by mail16.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id o7OK5YLP018531 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 25 Aug 2010 06:05:35 +1000 X-Bogosity: Ham, spamicity=0.000000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.4/8.14.4) with ESMTP id o7OK5SoE032819; Wed, 25 Aug 2010 06:05:29 +1000 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.4/8.14.4/Submit) id o7OK5S8h032818; Wed, 25 Aug 2010 06:05:28 +1000 (EST) (envelope-from peter) Date: Wed, 25 Aug 2010 06:05:27 +1000 From: Peter Jeremy To: Ollivier Robert Message-ID: <20100824200527.GC11990@server.vk2pj.dyndns.org> References: <20100810214418.GA28288@tolstoy.tols.org> <20100811014919.GA52992@icarus.home.lan> <20100811192537.GA44635@tolstoy.tols.org> <20100811214302.GB44635@tolstoy.tols.org> <20100812205625.GA79515@server.vk2pj.dyndns.org> <20100824150035.GB99477@roberto-al.eurocontrol.fr> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="MW5yreqqjyrRcusr" Content-Disposition: inline In-Reply-To: <20100824150035.GB99477@roberto-al.eurocontrol.fr> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-fs@freebsd.org Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Aug 2010 20:05:38 -0000 --MW5yreqqjyrRcusr Content-Type: multipart/mixed; boundary="3V7upXqbjpZ4EhLz" Content-Disposition: inline --3V7upXqbjpZ4EhLz Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2010-Aug-24 17:00:36 +0200, Ollivier Robert = wrote: >According to Peter Jeremy: >> I suspect Artem is referring to his patch at http://pastebin.com/ZCkzkWcs >> which I have tweaked somewhat (see the last patch in=20 >> http://www.freebsd.org/cgi/query-pr.cgi?pr=3D146410 ). > >Thanks, cou you please send it in a non-QP-encoded form please? See attached. >> Whilst these patches _are_ hacks, they seem to do a good job of >> making ZFS and UFS play together. > >Is the patch only useful in these mixed situations or could it be also int= erestng for those of use running full-zfs (cf. http://www.keltia.net/howtos= /zfsboot)? I think it will be useful. As well as the trivial fix to count "cache" as "free" space (which is now in -stable), the intent of the patch is to improve the ability of ZFS to apply pressure to the VM subsystem. In theory, this should improve overall system performance even in a ZFS-only environment where there is memory pressure due to large, long-running processes --=20 Peter Jeremy --3V7upXqbjpZ4EhLz Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="arc.patch2" Content-Transfer-Encoding: quoted-printable Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /usr/ncvs/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.= c,v retrieving revision 1.22.2.6 diff -u -r1.22.2.6 arc.c --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c 24 May 2010 20:09:= 40 -0000 1.22.2.6 +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c 12 Jul 2010 09:21:= 31 -0000 @@ -183,10 +183,15 @@ int zfs_arc_shrink_shift =3D 0; int zfs_arc_p_min_shift =3D 0; =20 +uint64_t zfs_arc_bp_active; +uint64_t zfs_arc_bp_inactive; + TUNABLE_QUAD("vfs.zfs.arc_max", &zfs_arc_max); TUNABLE_QUAD("vfs.zfs.arc_min", &zfs_arc_min); TUNABLE_QUAD("vfs.zfs.arc_meta_limit", &zfs_arc_meta_limit); TUNABLE_INT("vfs.zfs.mdcomp_disable", &zfs_mdcomp_disable); +TUNABLE_QUAD("vfs.zfs.arc_bp_active", &zfs_arc_bp_active); +TUNABLE_QUAD("vfs.zfs.arc_bp_inactive", &zfs_arc_bp_inactive); SYSCTL_DECL(_vfs_zfs); SYSCTL_QUAD(_vfs_zfs, OID_AUTO, arc_max, CTLFLAG_RDTUN, &zfs_arc_max, 0, "Maximum ARC size"); @@ -195,6 +200,11 @@ SYSCTL_INT(_vfs_zfs, OID_AUTO, mdcomp_disable, CTLFLAG_RDTUN, &zfs_mdcomp_disable, 0, "Disable metadata compression"); =20 +SYSCTL_QUAD(_vfs_zfs, OID_AUTO, arc_bp_active, CTLFLAG_RW|CTLFLAG_TUN, &zf= s_arc_bp_active, 0, + "Start ARC backpressure if active memory is below this limit"); +SYSCTL_QUAD(_vfs_zfs, OID_AUTO, arc_bp_inactive, CTLFLAG_RW|CTLFLAG_TUN, &= zfs_arc_bp_inactive, 0, + "Start ARC backpressure if inactive memory is below this limit"); + /* * Note that buffers can be in one of 6 states: * ARC_anon - anonymous (discussed below) @@ -2103,7 +2113,6 @@ } =20 static int needfree =3D 0; - static int arc_reclaim_needed(void) { @@ -2112,20 +2121,58 @@ #endif =20 #ifdef _KERNEL - if (needfree) - return (1); + /* We've grown too much, */ if (arc_size > arc_c_max) return (1); + + /* Pagedaemon is stuck, let's free something right away */ + if (vm_pageout_pages_needed) + return 1; + + /* Check if inactive list have grown too much */ + if ( zfs_arc_bp_inactive + && (ptoa((uintmax_t)cnt.v_inactive_count) > zfs_arc_bp_inactive)) { + /* tell pager to reap 1/2th of inactive queue*/ + atomic_add_int(&vm_pageout_deficit, cnt.v_inactive_count/2); + pagedaemon_wakeup(); + return needfree; + } + + /* Same for active list... */ + if ( zfs_arc_bp_active + && (ptoa((uintmax_t)cnt.v_active_count) > zfs_arc_bp_active)) { + atomic_add_int(&vm_pageout_deficit, cnt.v_active_count/2); + pagedaemon_wakeup(); + return needfree; + } + +=09 + /* Old style behavior -- ARC gives up memory whenever page daemon asks.. = */ + if (needfree) + return 1; + + /* + We got here either because active/inactive lists are + getting short or because we've been called during voluntary + ARC size checks. Kind of gray area... + */ + + /* If we didn't reach our minimum yet, don't rush to give memory up..*/ if (arc_size <=3D arc_c_min) return (0); =20 + /* If we're really short on memory now, give it up. */ + if (vm_page_count_min()) { + return (1); + } +=09 /* - * If pages are needed or we're within 2048 pages - * of needing to page need to reclaim + * If we're within 2048 pages of pagedaemon start, reclaim... */ - if (vm_pages_needed || (vm_paging_target() > -2048)) + if (vm_pages_needed && (vm_paging_target() > -2048)) return (1); =20 + #if 0 /* * take 'desfree' extra pages, so we reclaim sooner, rather than later @@ -2169,8 +2216,6 @@ return (1); #endif #else - if (kmem_used() > (kmem_size() * 3) / 4) - return (1); #endif =20 #else @@ -2279,7 +2324,7 @@ if (arc_eviction_list !=3D NULL) arc_do_user_evicts(); =20 - if (arc_reclaim_needed()) { + if (needfree) { needfree =3D 0; #ifdef _KERNEL wakeup(&needfree); @@ -3611,10 +3656,17 @@ { #ifdef _KERNEL uint64_t inflight_data =3D arc_anon->arcs_size; - uint64_t available_memory =3D ptoa((uintmax_t)cnt.v_free_count); + uint64_t available_memory; static uint64_t page_load =3D 0; static uint64_t last_txg =3D 0; =20 + /* How much memory is potentially available */ + available_memory =3D (uint64_t)cnt.v_free_count + cnt.v_cache_count; + if (available_memory > cnt.v_free_min) + available_memory =3D ptoa(available_memory - cnt.v_free_min); + else + available_memory =3D 0; + #if 0 #if defined(__i386) available_memory =3D --3V7upXqbjpZ4EhLz-- --MW5yreqqjyrRcusr Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.15 (FreeBSD) iEYEARECAAYFAkx0JgcACgkQ/opHv/APuIeoIgCdH59RLQGI4ozPMQ1fhLJVQWnK RawAoLgB04nVkzDhwfY0iPiBMoJxTzI0 =vOyZ -----END PGP SIGNATURE----- --MW5yreqqjyrRcusr-- From owner-freebsd-fs@FreeBSD.ORG Tue Aug 24 20:33:34 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9D006106564A for ; Tue, 24 Aug 2010 20:33:34 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id DE7CC8FC08 for ; Tue, 24 Aug 2010 20:33:33 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id XAA13771; Tue, 24 Aug 2010 23:33:23 +0300 (EEST) (envelope-from avg@icyb.net.ua) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Oo0BP-000HwX-F7; Tue, 24 Aug 2010 23:33:23 +0300 Message-ID: <4C742C8F.2030401@icyb.net.ua> Date: Tue, 24 Aug 2010 23:33:19 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.8) Gecko/20100822 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Peter Jeremy References: <20100810214418.GA28288@tolstoy.tols.org> <20100811014919.GA52992@icarus.home.lan> <20100811192537.GA44635@tolstoy.tols.org> <20100811214302.GB44635@tolstoy.tols.org> <20100812205625.GA79515@server.vk2pj.dyndns.org> <20100824150035.GB99477@roberto-al.eurocontrol.fr> <20100824200527.GC11990@server.vk2pj.dyndns.org> In-Reply-To: <20100824200527.GC11990@server.vk2pj.dyndns.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=KOI8-U Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Aug 2010 20:33:34 -0000 on 24/08/2010 23:05 Peter Jeremy said the following: > On 2010-Aug-24 17:00:36 +0200, Ollivier Robert wrote: >> According to Peter Jeremy: >>> I suspect Artem is referring to his patch at http://pastebin.com/ZCkzkWcs >>> which I have tweaked somewhat (see the last patch in >>> http://www.freebsd.org/cgi/query-pr.cgi?pr=146410 ). >> >> Thanks, cou you please send it in a non-QP-encoded form please? > > See attached. >>> Whilst these patches _are_ hacks, they seem to do a good job of >>> making ZFS and UFS play together. >> >> Is the patch only useful in these mixed situations or could it be also interestng for those of use running full-zfs (cf. http://www.keltia.net/howtos/zfsboot)? > > I think it will be useful. As well as the trivial fix to count > "cache" as "free" space (which is now in -stable), the intent of the > patch is to improve the ability of ZFS to apply pressure to the VM > subsystem. In theory, this should improve overall system performance > even in a ZFS-only environment where there is memory pressure due to > large, long-running processes Peter, Ollivier, would it be possible for you to test patch informally specified in the following post? http://thread.gmane.org/gmane.os.freebsd.devel.hackers/40788 -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Tue Aug 24 21:00:51 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 239E41065674 for ; Tue, 24 Aug 2010 21:00:51 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 3D4BE8FC1D for ; Tue, 24 Aug 2010 21:00:48 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Oo0bt-0004gs-Bk for freebsd-fs@freebsd.org; Tue, 24 Aug 2010 23:00:45 +0200 Received: from 193.33.173.33 ([193.33.173.33]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 24 Aug 2010 23:00:45 +0200 Received: from c.kworr by 193.33.173.33 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 24 Aug 2010 23:00:45 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Volodymyr Kostyrko Date: Wed, 25 Aug 2010 00:00:33 +0300 Lines: 36 Message-ID: References: <4C73DED0.7070308@darkbsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 193.33.173.33 User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; uk-UA; rv:1.9.2.8) Gecko/20100807 Thunderbird/3.1.2 In-Reply-To: <4C73DED0.7070308@darkbsd.org> Subject: Re: A way to alter ZFS metadata stored on a drive ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Aug 2010 21:00:51 -0000 24.08.2010 18:01, DarkSoul wrote: > I now can't detach/remove/replace the following drive because "no valid > replicas" or because you "can't replace a replacing drive". > > I gather the problem would be solved if I had a drive with the proper > ZDB information and GUID to fool the pool, but I am clueless as to how > to generate this information, and it doesn't seem from what the manpage > says, that zdb allows you to modify this information. Maybe. > Either that, or find a way to remove these bogus devices, or have the > pool "forget" that it must replace a device that does not exist anymore, > with a device that does not exist anymore. > > I would gladly welcome any clue as to how the ZDB metadata work and are > stored on the disk, as I can't find written specifications short of > probing the ZFS code itself. > > I can provide any information required, such as the first 512K for each > drive (containing the ZDB copies). > > Thanks in advance for any help. http://hub.opensolaris.org/bin/download/Community+Group+zfs/docs/ondiskformat0822.pdf If you are familiar with python you can use this one: http://code.google.com/p/zfs-recover It's a sample python script that's trying to read ZFS from disk. I going to commit nvlists unwinding in two days or so. -- Sphinx of black quartz judge my vow. From owner-freebsd-fs@FreeBSD.ORG Tue Aug 24 21:06:21 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C5C1410656AA for ; Tue, 24 Aug 2010 21:06:21 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 55D9E8FC1A for ; Tue, 24 Aug 2010 21:06:20 +0000 (UTC) Received: by gxk24 with SMTP id 24so3215200gxk.13 for ; Tue, 24 Aug 2010 14:06:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=k/br++JWrP0uyPg9cOh6Gl2SeaMf0l2Zw81S6Xrk5rg=; b=D47il3tW4h6lsheFRKWMuHDyR4nU0zwTEECXk/6Lv4p3gWKVL2kjNVN8sk49V/0G9v /sJ5lDgYl0JHFkHqrHQM9EhJLN8Pxtsmjcz0MXV1W/Xd7OXLMqTQhnpxn437O5bGYPYv 9jtA4VPdX28rC77gYETBdLJW/f8ZlaxxDaDrY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=rF+wQ7HH4CfIpZs/VRo/Bcl3kPmm4Q+hiFoR2gh7ByR71MBdf/4r6a+0zttBEaQ6y/ /tPQehSWsSeDy4rrsZuW0XwC+XTwCGXgayonlh/U+jVi8G8gHI+QQXAFQ2fiDUZ/bwH7 ZkNVjy2fUXWLl0vzyIXSLr50+wFHx2AxjWnHM= MIME-Version: 1.0 Received: by 10.151.109.12 with SMTP id l12mr3686598ybm.388.1282683980366; Tue, 24 Aug 2010 14:06:20 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.220.49.70 with HTTP; Tue, 24 Aug 2010 14:06:20 -0700 (PDT) In-Reply-To: <4C742C8F.2030401@icyb.net.ua> References: <20100810214418.GA28288@tolstoy.tols.org> <20100811014919.GA52992@icarus.home.lan> <20100811192537.GA44635@tolstoy.tols.org> <20100811214302.GB44635@tolstoy.tols.org> <20100812205625.GA79515@server.vk2pj.dyndns.org> <20100824150035.GB99477@roberto-al.eurocontrol.fr> <20100824200527.GC11990@server.vk2pj.dyndns.org> <4C742C8F.2030401@icyb.net.ua> Date: Tue, 24 Aug 2010 14:06:20 -0700 X-Google-Sender-Auth: PYQdY2nc_4YUeO4nCrXKCnSIpKU Message-ID: From: Artem Belevich To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Aug 2010 21:06:21 -0000 I've tried the change on my ZFS-only 8-stable box which has v15+metaslab patches applied and so far it seems to work OK under memory shortage. As intended ZFS does not start giving up memory until pagedaemon wakes up. I'll bang on the box some more through the week. --Artem 2010/8/24 Andriy Gapon : > on 24/08/2010 23:05 Peter Jeremy said the following: >> On 2010-Aug-24 17:00:36 +0200, Ollivier Robert wrote: >>> According to Peter Jeremy: >>>> I suspect Artem is referring to his patch at http://pastebin.com/ZCkzk= Wcs >>>> which I have tweaked somewhat (see the last patch in >>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=3D146410 ). >>> >>> Thanks, cou you please send it in a non-QP-encoded form please? >> >> See attached. >>>> Whilst these patches _are_ hacks, they seem to do a good job of >>>> making ZFS and UFS play together. >>> >>> Is the patch only useful in these mixed situations or could it be also = interestng for those of use running full-zfs (cf. http://www.keltia.net/how= tos/zfsboot)? >> >> I think it will be useful. =A0As well as the trivial fix to count >> "cache" as "free" space (which is now in -stable), the intent of the >> patch is to improve the ability of ZFS to apply pressure to the VM >> subsystem. =A0In theory, this should improve overall system performance >> even in a ZFS-only environment where there is memory pressure due to >> large, long-running processes > > Peter, Ollivier, > > would it be possible for you to test patch informally specified in the fo= llowing > post? > http://thread.gmane.org/gmane.os.freebsd.devel.hackers/40788 > > -- > Andriy Gapon > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Tue Aug 24 21:07:53 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E9E010656A7 for ; Tue, 24 Aug 2010 21:07:53 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id EC63E8FC0A for ; Tue, 24 Aug 2010 21:07:52 +0000 (UTC) Received: by qwg5 with SMTP id 5so7307556qwg.13 for ; Tue, 24 Aug 2010 14:07:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=k/br++JWrP0uyPg9cOh6Gl2SeaMf0l2Zw81S6Xrk5rg=; b=ScXtju+7B00wTdGhS+u3Wc8lnN2lJgNtJyVM77jWx/BYbItD9hgLtvGclYr4MUDtJJ vlfgK4FOC3gLSnzv9Vy2puIlQliBLuMZsWgGg3vsUK/LLujuoxdFd7LJjKUIvOmfi/lh qeE3TN3Ydm4PQtOZUjAiN9yh968YaPLRD+P2U= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=W7Tk8Nbwlh5Q/eatOUPdzlTsxapxfyFrvLEgu+FVWInMGzfkMpisMhRnBmjCjLf7KQ y8GTqTHXBYA1UR8xGB8QcYZVZX2S/I75JeTOBRjx1jSEXlmMZsr9fr2SgiNerqyah1UI z2nICSux3geTBq8kvLfQcat3TOiU6+UYd1GOA= MIME-Version: 1.0 Received: by 10.229.235.146 with SMTP id kg18mr3113027qcb.205.1282684070692; Tue, 24 Aug 2010 14:07:50 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.220.49.70 with HTTP; Tue, 24 Aug 2010 14:07:50 -0700 (PDT) In-Reply-To: <4C742C8F.2030401@icyb.net.ua> References: <20100810214418.GA28288@tolstoy.tols.org> <20100811014919.GA52992@icarus.home.lan> <20100811192537.GA44635@tolstoy.tols.org> <20100811214302.GB44635@tolstoy.tols.org> <20100812205625.GA79515@server.vk2pj.dyndns.org> <20100824150035.GB99477@roberto-al.eurocontrol.fr> <20100824200527.GC11990@server.vk2pj.dyndns.org> <4C742C8F.2030401@icyb.net.ua> Date: Tue, 24 Aug 2010 14:07:50 -0700 X-Google-Sender-Auth: mV_SLbVnr237UvTZAtMN67xSra4 Message-ID: From: Artem Belevich To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Aug 2010 21:07:53 -0000 I've tried the change on my ZFS-only 8-stable box which has v15+metaslab patches applied and so far it seems to work OK under memory shortage. As intended ZFS does not start giving up memory until pagedaemon wakes up. I'll bang on the box some more through the week. --Artem 2010/8/24 Andriy Gapon : > on 24/08/2010 23:05 Peter Jeremy said the following: >> On 2010-Aug-24 17:00:36 +0200, Ollivier Robert wrote: >>> According to Peter Jeremy: >>>> I suspect Artem is referring to his patch at http://pastebin.com/ZCkzk= Wcs >>>> which I have tweaked somewhat (see the last patch in >>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=3D146410 ). >>> >>> Thanks, cou you please send it in a non-QP-encoded form please? >> >> See attached. >>>> Whilst these patches _are_ hacks, they seem to do a good job of >>>> making ZFS and UFS play together. >>> >>> Is the patch only useful in these mixed situations or could it be also = interestng for those of use running full-zfs (cf. http://www.keltia.net/how= tos/zfsboot)? >> >> I think it will be useful. =A0As well as the trivial fix to count >> "cache" as "free" space (which is now in -stable), the intent of the >> patch is to improve the ability of ZFS to apply pressure to the VM >> subsystem. =A0In theory, this should improve overall system performance >> even in a ZFS-only environment where there is memory pressure due to >> large, long-running processes > > Peter, Ollivier, > > would it be possible for you to test patch informally specified in the fo= llowing > post? > http://thread.gmane.org/gmane.os.freebsd.devel.hackers/40788 > > -- > Andriy Gapon > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Tue Aug 24 21:11:41 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8E7141065679 for ; Tue, 24 Aug 2010 21:11:41 +0000 (UTC) (envelope-from marco@tolstoy.tols.org) Received: from tolstoy.tols.org (tolstoy.tols.org [IPv6:2a02:898:0:20::57:1]) by mx1.freebsd.org (Postfix) with ESMTP id 1CC328FC1E for ; Tue, 24 Aug 2010 21:11:40 +0000 (UTC) Received: from tolstoy.tols.org (localhost [127.0.0.1]) by tolstoy.tols.org (8.14.4/8.14.4) with ESMTP id o7OLBbOg018241 for ; Tue, 24 Aug 2010 21:11:37 GMT (envelope-from marco@tolstoy.tols.org) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.96.1 at tolstoy.tols.org Received: (from marco@localhost) by tolstoy.tols.org (8.14.4/8.14.4/Submit) id o7OLBbNM018240 for freebsd-fs@freebsd.org; Tue, 24 Aug 2010 23:11:37 +0200 (CEST) (envelope-from marco) Date: Tue, 24 Aug 2010 23:11:37 +0200 From: Marco van Tol To: freebsd-fs@freebsd.org Message-ID: <20100824211136.GA18104@tolstoy.tols.org> Mail-Followup-To: freebsd-fs@freebsd.org References: <20100810214418.GA28288@tolstoy.tols.org> <20100811014919.GA52992@icarus.home.lan> <20100811192537.GA44635@tolstoy.tols.org> <20100811214302.GB44635@tolstoy.tols.org> <20100812205625.GA79515@server.vk2pj.dyndns.org> <20100824150035.GB99477@roberto-al.eurocontrol.fr> <20100824200527.GC11990@server.vk2pj.dyndns.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="jRHKVT23PllUwdXP" Content-Disposition: inline In-Reply-To: <20100824200527.GC11990@server.vk2pj.dyndns.org> User-Agent: Mutt/1.4.2.3i X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on tolstoy.tols.org Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Aug 2010 21:11:41 -0000 --jRHKVT23PllUwdXP Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 25, 2010 at 06:05:27AM +1000, Peter Jeremy wrote: > On 2010-Aug-24 17:00:36 +0200, Ollivier Robert wrote: > >According to Peter Jeremy: > >> I suspect Artem is referring to his patch at http://pastebin.com/ZCkzk= Wcs > >> which I have tweaked somewhat (see the last patch in=20 > >> http://www.freebsd.org/cgi/query-pr.cgi?pr=3D146410 ). > > > >Thanks, cou you please send it in a non-QP-encoded form please? >=20 > See attached. >=20 > >> Whilst these patches _are_ hacks, they seem to do a good job of > >> making ZFS and UFS play together. > > > >Is the patch only useful in these mixed situations or could it be also i= nterestng for those of use running full-zfs (cf. http://www.keltia.net/howt= os/zfsboot)? >=20 > I think it will be useful. As well as the trivial fix to count > "cache" as "free" space (which is now in -stable), the intent of the > patch is to improve the ability of ZFS to apply pressure to the VM > subsystem. In theory, this should improve overall system performance > even in a ZFS-only environment where there is memory pressure due to > large, long-running processes Would the inactive pages of the long running processes be swapped out in favour of the arc? That would be great I think as long as that is made clear at an obvious place so people don't get worried about swap usage on a system that "shouldn't need to swap out anything". /usr/src/UPDATING springs to mind, together with quick hints on how to see that the swapped out pages are harmless. :) Marco van Tol P.S. Forgive me if what I say is bs, within what I know it makes sense. :) --=20 A male gynecologist is like an auto mechanic who never owned a car. - Carrie Snow --jRHKVT23PllUwdXP Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkx0NYcACgkQbdVEUcCKvoLYHACgvutHMn7h5i/Z0w4yRTlvPA6a Gj0An2f/1OkN8d9adcxHN95rB5b50tUR =NyDj -----END PGP SIGNATURE----- --jRHKVT23PllUwdXP-- From owner-freebsd-fs@FreeBSD.ORG Tue Aug 24 21:55:15 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0B3BE1065672 for ; Tue, 24 Aug 2010 21:55:15 +0000 (UTC) (envelope-from roberto@keltia.net) Received: from keltia.net (unknown [IPv6:2a01:240:fe5c::41]) by mx1.freebsd.org (Postfix) with ESMTP id B0CB08FC14 for ; Tue, 24 Aug 2010 21:55:14 +0000 (UTC) Received: from lonrach.keltia.net (unknown [IPv6:2a01:240:fe5c:0:d69a:20ff:fed0:3a83]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: roberto) by keltia.net (Postfix/TLS) with ESMTPSA id 56BA99FA1; Tue, 24 Aug 2010 23:55:13 +0200 (CEST) Date: Tue, 24 Aug 2010 23:55:09 +0200 From: Ollivier Robert To: Andriy Gapon Message-ID: <20100824215508.GA15597@lonrach.keltia.net> References: <20100810214418.GA28288@tolstoy.tols.org> <20100811014919.GA52992@icarus.home.lan> <20100811192537.GA44635@tolstoy.tols.org> <20100811214302.GB44635@tolstoy.tols.org> <20100812205625.GA79515@server.vk2pj.dyndns.org> <20100824150035.GB99477@roberto-al.eurocontrol.fr> <20100824200527.GC11990@server.vk2pj.dyndns.org> <4C742C8F.2030401@icyb.net.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4C742C8F.2030401@icyb.net.ua> X-Operating-System: MacOS X / MBP 4,1 - FreeBSD 8.0 / T3500-E5520 Nehalem User-Agent: Mutt/1.5.20 (2009-06-14) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.3 (keltia.net); Tue, 24 Aug 2010 23:55:13 +0200 (CEST) Cc: freebsd-fs@freebsd.org Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Aug 2010 21:55:15 -0000 According to Andriy Gapon: > would it be possible for you to test patch informally specified in the following > post? > http://thread.gmane.org/gmane.os.freebsd.devel.hackers/40788 I've been running with that patch for months now, I think you told me about it on IRC. Works fine. -- Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- roberto@keltia.freenix.fr In memoriam to Ondine : http://ondine.keltia.net/ From owner-freebsd-fs@FreeBSD.ORG Tue Aug 24 22:13:50 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2E53810656A6 for ; Tue, 24 Aug 2010 22:13:50 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 710AB8FC13 for ; Tue, 24 Aug 2010 22:13:48 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id BAA15816; Wed, 25 Aug 2010 01:13:44 +0300 (EEST) (envelope-from avg@icyb.net.ua) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Oo1kV-000I2N-QP; Wed, 25 Aug 2010 01:13:43 +0300 Message-ID: <4C744417.8080106@icyb.net.ua> Date: Wed, 25 Aug 2010 01:13:43 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.8) Gecko/20100822 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Ollivier Robert References: <20100810214418.GA28288@tolstoy.tols.org> <20100811014919.GA52992@icarus.home.lan> <20100811192537.GA44635@tolstoy.tols.org> <20100811214302.GB44635@tolstoy.tols.org> <20100812205625.GA79515@server.vk2pj.dyndns.org> <20100824150035.GB99477@roberto-al.eurocontrol.fr> <20100824200527.GC11990@server.vk2pj.dyndns.org> <4C742C8F.2030401@icyb.net.ua> <20100824215508.GA15597@lonrach.keltia.net> In-Reply-To: <20100824215508.GA15597@lonrach.keltia.net> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Aug 2010 22:13:50 -0000 on 25/08/2010 00:55 Ollivier Robert said the following: > According to Andriy Gapon: >> would it be possible for you to test patch informally specified in the >> following post? >> http://thread.gmane.org/gmane.os.freebsd.devel.hackers/40788 > > I've been running with that patch for months now, I think you told me about > it on IRC. Works fine. > Could you please double-check? Is the check for vm_paging_needed() indeed? That could have been a slightly different patch. -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Tue Aug 24 23:50:09 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2A5D21065674 for ; Tue, 24 Aug 2010 23:50:09 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id F3D898FC08 for ; Tue, 24 Aug 2010 23:50:08 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o7ONo8xi089152 for ; Tue, 24 Aug 2010 23:50:08 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o7ONo826089151; Tue, 24 Aug 2010 23:50:08 GMT (envelope-from gnats) Date: Tue, 24 Aug 2010 23:50:08 GMT Message-Id: <201008242350.o7ONo826089151@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Alexander Best Cc: Subject: Re: docs/61716: [patch] newfs(8) code and manpage are out of sync X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Alexander Best List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Aug 2010 23:50:09 -0000 The following reply was made to PR docs/61716; it has been noted by GNATS. From: Alexander Best To: bug-followup@freebsd.org Cc: Subject: Re: docs/61716: [patch] newfs(8) code and manpage are out of sync Date: Tue, 24 Aug 2010 23:40:24 +0000 to the person committing the patch to the newfs(8) manual: please bump the date to "August 20, 2010". thanks. alex -- a13x From owner-freebsd-fs@FreeBSD.ORG Wed Aug 25 00:25:04 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1E74610656AA for ; Wed, 25 Aug 2010 00:25:04 +0000 (UTC) (envelope-from alangrow@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 424FB8FC08 for ; Wed, 25 Aug 2010 00:25:02 +0000 (UTC) Received: by ywt2 with SMTP id 2so21288ywt.13 for ; Tue, 24 Aug 2010 17:25:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:date:from:to:subject :message-id:mime-version:content-type:content-disposition:user-agent; bh=6D/WXgQ9bmTH9Ki8yd4BUNQpLzzNtDH//Yj7gbiuWY8=; b=c27J1lPUnb9IDThgV7L9uAJMRpun7ydllhNlFCB8HpvNjk7UQCunug+HBVy5vg337e 49xtcE4dgOjVgDxG3l55wUFg1jp80Fouaz5OQJWc/2E6J98sBlzWraPEaRgFvseigOhq M6L+fWVZD6DtukhuyCP7bOIVtL8V6jiex4ey4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:bcc:subject:message-id:mime-version:content-type :content-disposition:user-agent; b=tTFpZy5lVUxNEUa+uUUxIe2zGGPdt28bmyiI86oL5A7JxoBbvMtpzOwDTj+gqP6A2s PMiT1s2II6IR3zWviE2fyeRlc+rhC7jHg1922CGWBXihGdDf2vvf0C/N65i4xGuMu3VM AkLHE+84cvIipoADO1jwK38rzrou4+mhE22kc= Received: by 10.101.185.6 with SMTP id m6mr8185984anp.22.1282694387367; Tue, 24 Aug 2010 16:59:47 -0700 (PDT) Received: from scalar (pool-71-248-132-20.dllstx.dsl-w.verizon.net [71.248.132.20]) by mx.google.com with ESMTPS id p16sm879387anh.35.2010.08.24.16.59.45 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 24 Aug 2010 16:59:46 -0700 (PDT) Date: Tue, 24 Aug 2010 19:59:43 -0400 From: Alan Grow To: freebsd-fs@freebsd.org Message-ID: <20100824235943.GD3287@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.20 (2009-06-14) Subject: processes stuck in "zfs" state when opening a directory X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Aug 2010 00:25:04 -0000 Hello, After a recent bout of intense filesystem activity, processes on an 8.0-RELEASE box are getting stuck in the "zfs" state when they try to open a certain directory. A zfs scrub shows no errors. Neither does smartctl. Nothing in the system logs. Is there anything I can do to help narrow down the problem? Unfortunately I don't have console access. Thanks, -Alan From owner-freebsd-fs@FreeBSD.ORG Wed Aug 25 13:21:38 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D6C9C106566C for ; Wed, 25 Aug 2010 13:21:38 +0000 (UTC) (envelope-from roberto@keltia.freenix.fr) Received: from keltia.net (unknown [IPv6:2a01:240:fe5c::41]) by mx1.freebsd.org (Postfix) with ESMTP id 874FA8FC1A for ; Wed, 25 Aug 2010 13:21:38 +0000 (UTC) Received: from roberto-al.eurocontrol.fr (aran.keltia.net [88.191.250.24]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: roberto) by keltia.net (Postfix/TLS) with ESMTPSA id D37189931 for ; Wed, 25 Aug 2010 15:21:36 +0200 (CEST) Date: Wed, 25 Aug 2010 15:21:31 +0200 From: Ollivier Robert To: freebsd-fs@freebsd.org Message-ID: <20100825132131.GA56434@roberto-al.eurocontrol.fr> References: <20100811014919.GA52992@icarus.home.lan> <20100811192537.GA44635@tolstoy.tols.org> <20100811214302.GB44635@tolstoy.tols.org> <20100812205625.GA79515@server.vk2pj.dyndns.org> <20100824150035.GB99477@roberto-al.eurocontrol.fr> <20100824200527.GC11990@server.vk2pj.dyndns.org> <4C742C8F.2030401@icyb.net.ua> <20100824215508.GA15597@lonrach.keltia.net> <4C744417.8080106@icyb.net.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4C744417.8080106@icyb.net.ua> X-Operating-System: MacOS X / Macbook Pro - FreeBSD 7.2 / Dell D820 SMP User-Agent: Mutt/1.5.20 (2009-06-14) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.3 (keltia.net); Wed, 25 Aug 2010 15:21:37 +0200 (CEST) Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Aug 2010 13:21:38 -0000 According to Andriy Gapon: > Could you please double-check? Is the check for vm_paging_needed() indeed? > That could have been a slightly different patch. 695 [15:19] root@centre:src/sys# svn diff cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Index: cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c =================================================================== --- cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (revision 210515) +++ cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (working copy) @@ -2123,8 +2123,9 @@ /* * If pages are needed or we're within 2048 pages * of needing to page need to reclaim + * XXX Try to workaround zfs overzealousness from gav/avg */ - if (vm_pages_needed || (vm_paging_target() > -2048)) + if (vm_pages_needed) return (1); #if 0 -- Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- roberto@keltia.freenix.fr In memoriam to Ondine : http://ondine.keltia.net/ From owner-freebsd-fs@FreeBSD.ORG Wed Aug 25 14:43:42 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 063DB1065693 for ; Wed, 25 Aug 2010 14:43:42 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 4B9F88FC1A for ; Wed, 25 Aug 2010 14:43:40 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA04986; Wed, 25 Aug 2010 17:43:32 +0300 (EEST) (envelope-from avg@icyb.net.ua) Message-ID: <4C752C13.3080508@icyb.net.ua> Date: Wed, 25 Aug 2010 17:43:31 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.8) Gecko/20100823 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Ollivier Robert References: <20100811014919.GA52992@icarus.home.lan> <20100811192537.GA44635@tolstoy.tols.org> <20100811214302.GB44635@tolstoy.tols.org> <20100812205625.GA79515@server.vk2pj.dyndns.org> <20100824150035.GB99477@roberto-al.eurocontrol.fr> <20100824200527.GC11990@server.vk2pj.dyndns.org> <4C742C8F.2030401@icyb.net.ua> <20100824215508.GA15597@lonrach.keltia.net> <4C744417.8080106@icyb.net.ua> <20100825132131.GA56434@roberto-al.eurocontrol.fr> In-Reply-To: <20100825132131.GA56434@roberto-al.eurocontrol.fr> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Aug 2010 14:43:42 -0000 on 25/08/2010 16:21 Ollivier Robert said the following: > According to Andriy Gapon: >> Could you please double-check? Is the check for vm_paging_needed() indeed? >> That could have been a slightly different patch. > > 695 [15:19] root@centre:src/sys# svn diff cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c > Index: cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c > =================================================================== > --- cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (revision 210515) > +++ cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (working copy) > @@ -2123,8 +2123,9 @@ > /* > * If pages are needed or we're within 2048 pages > * of needing to page need to reclaim > + * XXX Try to workaround zfs overzealousness from gav/avg > */ > - if (vm_pages_needed || (vm_paging_target() > -2048)) > + if (vm_pages_needed) > return (1); Note that vm_pages_needed != vm_paging_needed(). Talk about confusing :) -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Wed Aug 25 14:49:19 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 97B68106567A for ; Wed, 25 Aug 2010 14:49:19 +0000 (UTC) (envelope-from roberto@keltia.freenix.fr) Received: from keltia.net (unknown [IPv6:2a01:240:fe5c::41]) by mx1.freebsd.org (Postfix) with ESMTP id 5113B8FC12 for ; Wed, 25 Aug 2010 14:49:19 +0000 (UTC) Received: from roberto-al.eurocontrol.fr (aran.keltia.net [88.191.250.24]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: roberto) by keltia.net (Postfix/TLS) with ESMTPSA id 64FE49A92 for ; Wed, 25 Aug 2010 16:49:18 +0200 (CEST) Date: Wed, 25 Aug 2010 16:49:16 +0200 From: Ollivier Robert To: freebsd-fs@freebsd.org Message-ID: <20100825144916.GB56434@roberto-al.eurocontrol.fr> References: <20100824235943.GD3287@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100824235943.GD3287@gmail.com> X-Operating-System: MacOS X / Macbook Pro - FreeBSD 7.2 / Dell D820 SMP User-Agent: Mutt/1.5.20 (2009-06-14) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.3 (keltia.net); Wed, 25 Aug 2010 16:49:18 +0200 (CEST) Subject: Re: processes stuck in "zfs" state when opening a directory X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Aug 2010 14:49:19 -0000 According to Alan Grow: > Is there anything I can do to help narrow down the problem? > Unfortunately I don't have console access. Increase kern.maxvnodes /etc/sysctl.conf: ##-- tuning kern.maxvnodes=260000 -- Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- roberto@keltia.freenix.fr In memoriam to Ondine : http://ondine.keltia.net/ From owner-freebsd-fs@FreeBSD.ORG Wed Aug 25 14:50:51 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DB7A21065693 for ; Wed, 25 Aug 2010 14:50:51 +0000 (UTC) (envelope-from roberto@keltia.freenix.fr) Received: from keltia.net (unknown [IPv6:2a01:240:fe5c::41]) by mx1.freebsd.org (Postfix) with ESMTP id 8D4D48FC18 for ; Wed, 25 Aug 2010 14:50:51 +0000 (UTC) Received: from roberto-al.eurocontrol.fr (aran.keltia.net [88.191.250.24]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: roberto) by keltia.net (Postfix/TLS) with ESMTPSA id 80B779AA2 for ; Wed, 25 Aug 2010 16:50:50 +0200 (CEST) Date: Wed, 25 Aug 2010 16:50:48 +0200 From: Ollivier Robert To: freebsd-fs@freebsd.org Message-ID: <20100825145047.GC56434@roberto-al.eurocontrol.fr> References: <20100811214302.GB44635@tolstoy.tols.org> <20100812205625.GA79515@server.vk2pj.dyndns.org> <20100824150035.GB99477@roberto-al.eurocontrol.fr> <20100824200527.GC11990@server.vk2pj.dyndns.org> <4C742C8F.2030401@icyb.net.ua> <20100824215508.GA15597@lonrach.keltia.net> <4C744417.8080106@icyb.net.ua> <20100825132131.GA56434@roberto-al.eurocontrol.fr> <4C752C13.3080508@icyb.net.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4C752C13.3080508@icyb.net.ua> X-Operating-System: MacOS X / Macbook Pro - FreeBSD 7.2 / Dell D820 SMP User-Agent: Mutt/1.5.20 (2009-06-14) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.3 (keltia.net); Wed, 25 Aug 2010 16:50:50 +0200 (CEST) Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Aug 2010 14:50:51 -0000 According to Andriy Gapon: > > - if (vm_pages_needed || (vm_paging_target() > -2048)) > > + if (vm_pages_needed) > > return (1); > > Note that vm_pages_needed != vm_paging_needed(). > Talk about confusing :) Right, my bad. Is there anyone tracking somewhere (wiki?) the various patches floating around? -- Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- roberto@keltia.freenix.fr In memoriam to Ondine : http://ondine.keltia.net/ From owner-freebsd-fs@FreeBSD.ORG Wed Aug 25 14:53:28 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4B31210656A3 for ; Wed, 25 Aug 2010 14:53:28 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 927008FC1F for ; Wed, 25 Aug 2010 14:53:27 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA05148 for ; Wed, 25 Aug 2010 17:53:25 +0300 (EEST) (envelope-from avg@icyb.net.ua) Message-ID: <4C752E65.3070701@icyb.net.ua> Date: Wed, 25 Aug 2010 17:53:25 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.8) Gecko/20100823 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <20100810214418.GA28288@tolstoy.tols.org> <20100811014919.GA52992@icarus.home.lan> <20100811192537.GA44635@tolstoy.tols.org> <20100811214302.GB44635@tolstoy.tols.org> <20100812205625.GA79515@server.vk2pj.dyndns.org> <20100824150035.GB99477@roberto-al.eurocontrol.fr> <20100824200527.GC11990@server.vk2pj.dyndns.org> <20100824211136.GA18104@tolstoy.tols.org> In-Reply-To: <20100824211136.GA18104@tolstoy.tols.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=KOI8-U Content-Transfer-Encoding: 7bit Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Aug 2010 14:53:28 -0000 on 25/08/2010 00:11 Marco van Tol said the following: > Would the inactive pages of the long running processes be swapped out in > favour of the arc? That would be great I think as long as that is made > clear at an obvious place so people don't get worried about swap usage > on a system that "shouldn't need to swap out anything". > > /usr/src/UPDATING springs to mind, together with quick hints on how to > see that the swapped out pages are harmless. :) I don't think there would be any significant impact on swapping. Remember that inactive pages also include clean pages that can be moved to cache and vnode pages that can be synced to disk. pagedaemon is smart enough to not trigger swapping too early. Makes sense? :) -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Wed Aug 25 16:40:28 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0D0101065697 for ; Wed, 25 Aug 2010 16:40:28 +0000 (UTC) (envelope-from marco@tolstoy.tols.org) Received: from tolstoy.tols.org (tolstoy.tols.org [IPv6:2a02:898:0:20::57:1]) by mx1.freebsd.org (Postfix) with ESMTP id 9291D8FC12 for ; Wed, 25 Aug 2010 16:40:27 +0000 (UTC) Received: from tolstoy.tols.org (localhost [127.0.0.1]) by tolstoy.tols.org (8.14.4/8.14.4) with ESMTP id o7PGeO9U031277 for ; Wed, 25 Aug 2010 16:40:24 GMT (envelope-from marco@tolstoy.tols.org) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.96.1 at tolstoy.tols.org Received: (from marco@localhost) by tolstoy.tols.org (8.14.4/8.14.4/Submit) id o7PGeOkJ031276 for freebsd-fs@freebsd.org; Wed, 25 Aug 2010 16:40:24 GMT (envelope-from marco) Date: Wed, 25 Aug 2010 16:40:24 +0000 From: Marco van Tol To: freebsd-fs@freebsd.org Message-ID: <20100825164024.GC24997@tolstoy.tols.org> Mail-Followup-To: freebsd-fs@freebsd.org References: <20100810214418.GA28288@tolstoy.tols.org> <20100811014919.GA52992@icarus.home.lan> <20100811192537.GA44635@tolstoy.tols.org> <20100811214302.GB44635@tolstoy.tols.org> <20100812205625.GA79515@server.vk2pj.dyndns.org> <20100824150035.GB99477@roberto-al.eurocontrol.fr> <20100824200527.GC11990@server.vk2pj.dyndns.org> <20100824211136.GA18104@tolstoy.tols.org> <4C752E65.3070701@icyb.net.ua> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4C752E65.3070701@icyb.net.ua> User-Agent: Mutt/1.4.2.3i X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on tolstoy.tols.org Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Aug 2010 16:40:28 -0000 On Wed, Aug 25, 2010 at 05:53:25PM +0300, Andriy Gapon wrote: > on 25/08/2010 00:11 Marco van Tol said the following: > > Would the inactive pages of the long running processes be swapped out in > > favour of the arc? That would be great I think as long as that is made > > clear at an obvious place so people don't get worried about swap usage > > on a system that "shouldn't need to swap out anything". > > > > /usr/src/UPDATING springs to mind, together with quick hints on how to > > see that the swapped out pages are harmless. :) > > I don't think there would be any significant impact on swapping. > Remember that inactive pages also include clean pages that can be > moved to cache > and vnode pages that can be synced to disk. > pagedaemon is smart enough to not trigger swapping too early. > Makes sense? :) It does, thanks a lot. :) Shameless plug: This book is, even tho a bit old, very helpfull: http://www.amazon.com/Design-Implementation-FreeBSD-Operating-System/dp/0201702452 Sidetrack: Are there any plans to have a similar book based on a more recent freebsd version released? -- Marco van Tol From owner-freebsd-fs@FreeBSD.ORG Thu Aug 26 10:52:09 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E7C9310656AD for ; Thu, 26 Aug 2010 10:52:08 +0000 (UTC) (envelope-from fbsd@dannysplace.net) Received: from mailgw.dannysplace.net (mailgw.dannysplace.net [204.109.56.184]) by mx1.freebsd.org (Postfix) with ESMTP id 6925D8FC0A for ; Thu, 26 Aug 2010 10:52:08 +0000 (UTC) Received: from [203.206.171.212] (helo=[192.168.10.10]) by mailgw.dannysplace.net with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.72 (FreeBSD)) (envelope-from ) id 1OoZaK-000NMA-Bi for freebsd-fs@freebsd.org; Thu, 26 Aug 2010 20:21:30 +1000 Message-ID: <4C76401C.5070807@dannysplace.net> Date: Thu, 26 Aug 2010 20:21:16 +1000 From: Danny Carroll User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.2.8) Gecko/20100802 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: freebsd-fs Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Authenticated-User: danny X-Authenticator: plain X-Exim-Version: 4.72 (build at 12-Jul-2010 18:31:29) X-Date: 2010-08-26 20:21:28 X-Connected-IP: 203.206.171.212:61486 X-Message-Linecount: 100 X-Body-Linecount: 89 X-Message-Size: 3993 X-Body-Size: 3540 X-Received-Count: 1 X-Recipient-Count: 1 X-Local-Recipient-Count: 1 X-Local-Recipient-Defer-Count: 0 X-Local-Recipient-Fail-Count: 0 X-SA-Exim-Connect-IP: 203.206.171.212 X-SA-Exim-Mail-From: fbsd@dannysplace.net X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on damka.dannysplace.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.3.1 X-SA-Exim-Version: 4.2 X-SA-Exim-Scanned: Yes (on mailgw.dannysplace.net) Subject: ZFS on root, DR techniques. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: fbsd@dannysplace.net List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Aug 2010 10:52:09 -0000 Hiya All, I'm busy building a machine that uses ZFS only. It's using gpart vdevs and I've been able to create a ZFS mirror root easily thanks to pjd@'s article at http://blogs.freebsdish.org/pjd/2010/08/06/from-sysinstall-to-zfs-only-configuration/ This machine will eventually go into a data centre and I am thinking a little about data recovery. With UFS it's quite trivial to do a dump/restore and I'd like to have a clear idea of how to do this with ZFS. I am sure smarter ppl than I have figured this out so feel free to chime in at any time and tell me where to find instructions from the the original zfs DR wheel (nothing is as yet in the handbook. Here are my inital thoughts - I've not had time check if this can actually be done (never used a fixit boot before): I have a (gzipped) file containing a snapshot of my daily DR located at "backuserver:/client-root-snapshot-daily.gz" I have a fresh piece of hardware with my mirrored root disks ready to go. I boot off the dvd1 iso and select fixit. I use gpart to label the drives as per the original instructions: dd if=/dev/zero of=/dev/ada0 count=79 gpart create -s GPT ada0 gpart add -b 34 -s 128 -t freebsd-boot ada0 gpart add -s 2g -t freebsd-swap -l swap0 ada0 gpart add -t freebsd-zfs -l system0 ada0 gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0 dd if=/dev/zero of=/dev/ada1 count=79 gpart create -s GPT ada1 gpart add -b 34 -s 128 -t freebsd-boot ada1 gpart add -s 2g -t freebsd-swap -l swap1 ada1 gpart add -t freebsd-zfs -l system1 ada1 gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1 I then re-create the swap mirror. gmirror label -F -h -b round-robin swap /dev/gpt/swap0 gmirror insert -h -p 1 swap /dev/gpt/swap1 Then I create the ZFS pool as it was: zpool create -O mountpoint=/mnt -O atime=off -O setuid=off -O canmount=off system /dev/gpt/system0 zpool attach system /dev/gpt/system1 /dev/gpt/system0 zfs create -o mountpoint=legacy -o setuid=on system/root zpool set bootfs=system/root system zfs create -o compress=lzjb system/tmp chmod 1777 /mnt/tmp zfs create -o canmount=off system/usr zfs create -o setuid=on system/usr/local zfs create -o compress=gzip system/usr/src zfs create -o compress=lzjb system/usr/obj zfs create -o compress=gzip system/usr/ports zfs create -o compress=off system/usr/ports/distfiles zfs create -o canmount=off system/var zfs create -o compress=gzip system/var/log zfs create -o compress=lzjb system/var/audit zfs create -o compress=lzjb system/var/tmp chmod 1777 /mnt/var/tmp zfs create -o canmount=off system/usr/home zfs create system/usr/home/pjd Then I receive the file from the backup server: ssh backuserver "gzcat client-root-snapshot-daily.gz" | zfs recv -F system Reboot and all is good. There are a few questions I have.... Does the fixit image contain enough to be able to do this? - can I enable networking? - are zfs and gpart utils already there? - if not, what about the livefs image? Do I really need to recreate the while ZFS tree again or will the snapshot do that? Should I be backing up the ZFS properties of each ZFS fs so I know what needs to be set on the newly created filesystems? Or is the snapshot file enough? Is there a better way to do this? I am very interested to hear what others think... Eventually I'd like to submit what I figure out to the doc maintainers for inclusion in the handbook perhaps. -D From owner-freebsd-fs@FreeBSD.ORG Thu Aug 26 11:15:57 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B97421065673 for ; Thu, 26 Aug 2010 11:15:57 +0000 (UTC) (envelope-from fbsd@dannysplace.net) Received: from mailgw.dannysplace.net (mailgw.dannysplace.net [204.109.56.184]) by mx1.freebsd.org (Postfix) with ESMTP id 89FD08FC1D for ; Thu, 26 Aug 2010 11:15:57 +0000 (UTC) Received: from [203.206.171.212] (helo=[192.168.10.10]) by mailgw.dannysplace.net with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.72 (FreeBSD)) (envelope-from ) id 1OoaR6-000OqZ-9Z for freebsd-fs@freebsd.org; Thu, 26 Aug 2010 21:16:00 +1000 Message-ID: <4C764CE4.7070402@dannysplace.net> Date: Thu, 26 Aug 2010 21:15:48 +1000 From: Danny Carroll User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.2.8) Gecko/20100802 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <4C76401C.5070807@dannysplace.net> In-Reply-To: <4C76401C.5070807@dannysplace.net> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Authenticated-User: danny X-Authenticator: plain X-Exim-Version: 4.72 (build at 12-Jul-2010 18:31:29) X-Date: 2010-08-26 21:16:00 X-Connected-IP: 203.206.171.212:55418 X-Message-Linecount: 16 X-Body-Linecount: 3 X-Message-Size: 580 X-Body-Size: 41 X-Received-Count: 1 X-Recipient-Count: 1 X-Local-Recipient-Count: 1 X-Local-Recipient-Defer-Count: 0 X-Local-Recipient-Fail-Count: 0 X-SA-Exim-Connect-IP: 203.206.171.212 X-SA-Exim-Mail-From: fbsd@dannysplace.net X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on damka.dannysplace.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.3.1 X-SA-Exim-Version: 4.2 X-SA-Exim-Scanned: Yes (on mailgw.dannysplace.net) Subject: Re: ZFS on root, DR techniques. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: fbsd@dannysplace.net List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Aug 2010 11:15:57 -0000 s/data recovery/disaster recovery/g -D From owner-freebsd-fs@FreeBSD.ORG Thu Aug 26 18:33:06 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BF4391065697 for ; Thu, 26 Aug 2010 18:33:06 +0000 (UTC) (envelope-from kungfujesus06@gmail.com) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 728448FC13 for ; Thu, 26 Aug 2010 18:33:06 +0000 (UTC) Received: by gwj23 with SMTP id 23so939911gwj.13 for ; Thu, 26 Aug 2010 11:33:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=/ymS755Z3lHwdBmzieZ6UzmP38646KFqrkMPZ993aYo=; b=LseMjzKztliG6ttw6lAntkd17+Hj/X0fj5PMZwgfOjmoR3IgCRjl6BOGpPCudjpXbi 1eoeBLhXfqOuSgh+M69s03GhCCmr9wzh+7ZtbARYFbsY6slQVlyb9ALjQR2DB9AP9fe5 QTsjBrO0loC/TWCYPm54MpJRqGSqJkE1mGTrY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=ta8Z2CFCcwn7I6gnaMk6tkesDxR80aFkSaIMMog5SdLo+FiuWzV2CKghSNwBniHI5Z 6Z9Pq+n/yInzkFWLL2d55R6EiC28TMT6T9c9Gv2/kPAPojlfnBK/3TQwbk0ukHyRSgYP ZmxLH2i2NgEIYrquDGifLYjr5VUAbuM++5iz0= MIME-Version: 1.0 Received: by 10.100.135.13 with SMTP id i13mr11426391and.34.1282846109891; Thu, 26 Aug 2010 11:08:29 -0700 (PDT) Received: by 10.231.10.140 with HTTP; Thu, 26 Aug 2010 11:08:29 -0700 (PDT) Date: Thu, 26 Aug 2010 14:08:29 -0400 Message-ID: From: Adam Stylinski To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Problem with zpool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Aug 2010 18:33:06 -0000 Ok so I was sending snapshots into the pool and I may have sent one that was bad because I got a "bad magic number" error at some point. At the time I figured it was my disk (ad6) which was going bad. It would not allow me no matter what I to offline this ad6 device, it claimed insufficient replicas existed. So I removed the device and put a device on the same port on the same controller to replace it. I then ran the replace command only to have it sit there pretending to replace but zpool would sit on top with status g_wait. I googled around and found a guy on one of the mailing lists with a similar bug, they said it was fixed in a revision to zfs (can't remember which mfc) but upgrading to fbsd 8.1 would fix the problem. This is a v13 pool, and now that I've upgraded to 8.1, I'm running a scrub which forced the resilver and it's claiming to "replace". Well as it turns out I had a cronjob for freebsd-update cron which happened to pop in for 8.0 some time during freebsd-update. So the result was I was on an 8.1 kernel but an 8.0 userland, which did allow me to scrub (I guess the ABIs didn't break this update between zpool and libzpool). It finished the scrub, but it still claims this: 8991447011275450347 UNAVAIL 0 10.0K 0 was /dev/ad6/old ada2 ONLINE 0 0 0 14.1G resilvered Meaning it's trying to write to the old device which it won't let me offline, while still resilvering to ada2. While it did tell me that it resilvered successfully with no checksum errors or read or write errors, it's still there. I am now scrubbing with 8.1's userland and kernel, do you guys think it will finally allow me to remove the device it's replacing? Also, it claims that there is corruption in the pool, but the files that are "affected" are 100% fine, as I've md5'd them against remote copies. I realize the metadata could be bad, so I'm not sure how to go about fixing that. Anyway, please don't tl;dr and let me know whatever advice you can give. I have some moderate level backups of some of the data (the entire pool has 2.7TB occupied), but I'd like to avoid a destroy and create process. So far this is what zpool status reports: [adam@nasbox ~]$ zpool status pool: share state: DEGRADED status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: http://www.sun.com/msg/ZFS-8000-8A scrub: resilver in progress for 0h9m, 2.16% done, 6h53m to go config: NAME STATE READ WRITE CKSUM share DEGRADED 0 0 0 raidz1 DEGRADED 0 0 0 ada1 ONLINE 0 0 0 60.9M resilvered ada3 ONLINE 0 0 0 60.9M resilvered replacing DEGRADED 0 0 0 8991447011275450347 UNAVAIL 0 10.0K 0 was /dev/ad6/old ada2 ONLINE 0 0 0 14.1G resilvered ada4 ONLINE 0 0 0 56.7M resilvered raidz1 ONLINE 0 0 0 da0 ONLINE 0 0 0 da2 ONLINE 0 0 0 da3 ONLINE 0 0 0 da1 ONLINE 0 0 0 raidz1 ONLINE 0 0 0 aacd0 ONLINE 0 0 0 aacd1 ONLINE 0 0 0 aacd2 ONLINE 0 0 0 aacd3 ONLINE 0 0 0 logs DEGRADED 0 0 0 ada0 ONLINE 0 0 0 From owner-freebsd-fs@FreeBSD.ORG Thu Aug 26 21:42:27 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4221010656A4 for ; Thu, 26 Aug 2010 21:42:27 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx21.fluidhosting.com [204.14.89.4]) by mx1.freebsd.org (Postfix) with ESMTP id C741F8FC08 for ; Thu, 26 Aug 2010 21:42:26 +0000 (UTC) Received: (qmail 13703 invoked by uid 399); 26 Aug 2010 21:15:46 -0000 Received: from localhost (HELO ?192.168.0.142?) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 26 Aug 2010 21:15:46 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Message-ID: <4C76D981.7080602@FreeBSD.org> Date: Thu, 26 Aug 2010 14:15:45 -0700 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 MIME-Version: 1.0 To: freebsd-fs@FreeBSD.org X-Enigmail-Version: 1.1.1 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Questions about FreeBSD and Linux on the same disk X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Aug 2010 21:42:27 -0000 Howdy, I'm looking to expand my horizons so I'm experimenting with Ubuntu. I was given the advice to use ext2 file systems so that I could mount the Linux disks from FreeBSD but it seems that we have support up through ext4? Or perhaps the port of efs2progs is necessary for that? I don't mind using ext2 if that's the best choice, but if I can use the newer (better?) option that's fine too. What I am looking for is the "safe" choice, a way to mount the partitions that a) won't crash FreeBSD, and b) won't cause data loss. This would be for both FreeBSD 7-stable and 9-current. On the other side, I'm reading up on mounting ufs2 from Linux, and it seems r/w support is still considered "dangerous." I can mount r/o just fine, but I'm wondering if anyone has actual experience with using r/w on a regular basis. In addition to just wanting to be able to access stuff that is on one OS' disk from the other on an occasional basis my grand scheme is to have all/part of a home directory that is shared between OS'; so that's my definition of "safe." So far it seems that the best/safest alternative is an ext2 partition, which is Ok, but I'd prefer to stick with ufs2 if possible since FreeBSD is still going to be my main platform. TIA, Doug -- Improve the effectiveness of your Internet presence with a domain name makeover! http://SupersetSolutions.com/ Computers are useless. They can only give you answers. -- Pablo Picasso From owner-freebsd-fs@FreeBSD.ORG Thu Aug 26 23:29:58 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8787B10656A5 for ; Thu, 26 Aug 2010 23:29:58 +0000 (UTC) (envelope-from mj@feral.com) Received: from ns1.feral.com (ns1.feral.com [192.67.166.1]) by mx1.freebsd.org (Postfix) with ESMTP id 530128FC13 for ; Thu, 26 Aug 2010 23:29:58 +0000 (UTC) Received: from [192.168.221.2] (remotevpn [192.168.221.2]) by ns1.feral.com (8.14.3/8.14.3) with ESMTP id o7QN0ftg058695 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Thu, 26 Aug 2010 16:00:41 -0700 (PDT) (envelope-from mj@feral.com) Message-ID: <4C76F210.5020207@feral.com> Date: Thu, 26 Aug 2010 16:00:32 -0700 From: Matthew Jacob Organization: Feral Software User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <4C76D981.7080602@FreeBSD.org> In-Reply-To: <4C76D981.7080602@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender DNS name whitelisted, not delayed by milter-greylist-4.2.6 (ns1.feral.com [192.168.221.1]); Thu, 26 Aug 2010 16:00:42 -0700 (PDT) Subject: Re: Questions about FreeBSD and Linux on the same disk X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Aug 2010 23:29:58 -0000 On 8/26/2010 2:15 PM, Doug Barton wrote: > Howdy, > > I'm looking to expand my horizons so I'm experimenting with Ubuntu. I > was given the advice to use ext2 file systems so that I could mount > the Linux disks from FreeBSD but it seems that we have support up > through ext4? Or perhaps the port of efs2progs is necessary for that? > I don't mind using ext2 if that's the best choice, but if I can use > the newer (better?) option that's fine too. What I am looking for is > the "safe" choice, a way to mount the partitions that a) won't crash > FreeBSD, and b) won't cause data loss. This would be for both FreeBSD > 7-stable and 9-current. I share systems on disks quite a lot. But my typical usage is have Linux be the primary system and use grub to chainboot the FreeBSD partitions. If I absolutely must have a shared filesystem between the two, MS-DOS is the safest choice. From owner-freebsd-fs@FreeBSD.ORG Thu Aug 26 23:52:56 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8644C106564A for ; Thu, 26 Aug 2010 23:52:56 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx21.fluidhosting.com [204.14.89.4]) by mx1.freebsd.org (Postfix) with ESMTP id 11D878FC0A for ; Thu, 26 Aug 2010 23:52:55 +0000 (UTC) Received: (qmail 24832 invoked by uid 399); 26 Aug 2010 23:52:55 -0000 Received: from localhost (HELO ?192.168.0.142?) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 26 Aug 2010 23:52:55 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Message-ID: <4C76FE56.10306@FreeBSD.org> Date: Thu, 26 Aug 2010 16:52:54 -0700 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <4C76D981.7080602@FreeBSD.org> <4C76F210.5020207@feral.com> In-Reply-To: <4C76F210.5020207@feral.com> X-Enigmail-Version: 1.2a1pre OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Subject: Re: Questions about FreeBSD and Linux on the same disk X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Aug 2010 23:52:56 -0000 On 8/26/2010 4:00 PM, Matthew Jacob wrote: > > I share systems on disks quite a lot. But my typical usage is have Linux > be the primary system and use grub to chainboot the FreeBSD partitions. > If I absolutely must have a shared filesystem between the two, MS-DOS is > the safest choice. Ok, that's doable since I'll have a large FAT32 disk that I will be using for shared data anyway, and on my previous dual-boot (win/freebsd) I was using symlinks in the freebsd $HOME to the fat32 drive without problems, so thanks for this answer. Meanwhile, I just posted the 2nd of my questions to -hackers about finding a boot manager that can handle booting linux from a logical volume (which grub can do) and multiple freebsds. From what you wrote above it sounds like you have the "multiple freebsd" problem licked, perhaps you could take a look at my post on -hackers and help me out there too? :-/ Pressing my luck, Doug From owner-freebsd-fs@FreeBSD.ORG Fri Aug 27 00:10:42 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0686C1065670 for ; Fri, 27 Aug 2010 00:10:42 +0000 (UTC) (envelope-from mj@feral.com) Received: from ns1.feral.com (ns1.feral.com [192.67.166.1]) by mx1.freebsd.org (Postfix) with ESMTP id AA5818FC0C for ; Fri, 27 Aug 2010 00:10:41 +0000 (UTC) Received: from [192.168.221.2] (remotevpn [192.168.221.2]) by ns1.feral.com (8.14.3/8.14.3) with ESMTP id o7R0AOZd059051 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Thu, 26 Aug 2010 17:10:40 -0700 (PDT) (envelope-from mj@feral.com) Message-ID: <4C770268.1010707@feral.com> Date: Thu, 26 Aug 2010 17:10:16 -0700 From: Matthew Jacob Organization: Feral Software User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <4C76D981.7080602@FreeBSD.org> <4C76F210.5020207@feral.com> <4C76FE56.10306@FreeBSD.org> In-Reply-To: <4C76FE56.10306@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender DNS name whitelisted, not delayed by milter-greylist-4.2.6 (ns1.feral.com [192.168.221.1]); Thu, 26 Aug 2010 17:10:40 -0700 (PDT) Subject: Re: Questions about FreeBSD and Linux on the same disk X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Aug 2010 00:10:42 -0000 Umm. I don't do much with the extended partitions, but I certainly have more than one FreeBSD system on the box- two on drive 0 and one on drive 2. title Fedora (2.6.32.16-141.fc12.x86_64) root (hd0,0) kernel /vmlinuz-2.6.32.16-141.fc12.x86_64 ro root=/dev/mapper/VolGroup-lv_root console=ttyS0,115200n8 SYSFONT=latarcyrheb-sun16 LANG=en_US.UTF-8 KEYTABLE=us initrd /initramfs-2.6.32.16-141.fc12.x86_64.img savedefault ... title FreeBSD rootnoverify (hd0,2) makeactive chainloader +1 savedefault title FreeBSD32 rootnoverify (hd0,1) makeactive chainloader +1 savedefault title FreeBSD32 8.X rootnoverify (hd2,0) makeactive chainloader +1 savedefault .... > above it sounds like you have the "multiple freebsd" problem licked, > perhaps you could take a look at my post on -hackers and help me out > there too? :-/ > > > Pressing my luck, > > Doug > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Fri Aug 27 01:33:06 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A65FD1065675 for ; Fri, 27 Aug 2010 01:33:06 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx21.fluidhosting.com [204.14.89.4]) by mx1.freebsd.org (Postfix) with ESMTP id 537E38FC1F for ; Fri, 27 Aug 2010 01:33:06 +0000 (UTC) Received: (qmail 17247 invoked by uid 399); 27 Aug 2010 01:33:05 -0000 Received: from localhost (HELO ?192.168.0.142?) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 27 Aug 2010 01:33:05 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Message-ID: <4C7715D0.10604@FreeBSD.org> Date: Thu, 26 Aug 2010 18:33:04 -0700 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 MIME-Version: 1.0 To: freebsd-fs@freebsd.org X-Enigmail-Version: 1.2a1pre OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Difference of opinion about my disk geometry X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Aug 2010 01:33:06 -0000 Howdy, What I've done for years in order to create a dual-boot between windows and FreeBSD, with one shared fat32 data partition; is to boot the FreeBSD install CD, create the 3 primary partitions, install FreeBSD, install Windows into the 1st primary partition, then boot the FreeBSD CD again and install the boot mgr. My rationale for this is that we (FreeBSD) know more about modern disk geometry than Windows XP does, and this method has always served me well. What I've done now is purchase a new HD for my laptop because the old one was starting to act dodgy. It's a 250 GB Hitachi Travelstar, SATA 300 in case anyone cares (which is the same as what was installed by Dell, except the old one was 100 GB). Since I have more space I'm attempting to experiment with more stuff as you may have seen from my other posts. However, FreeBSD disagrees with Windows and Linux about what the right geometry should look like. Depending on which OS I use to create partitions I get various errors about whether or not things end on cylinder and/or track boundaries. This does not seem like a good thing. Below are what the various OS' think about the disk. (Ignore the fact that the 3rd partition has an unknown type, that used to be a FreeBSD partition that seems to have been mangled by grub2, which I'm going to fix later.) When I run FreeBSD fdisk from sysinstall I get the following message: It is safe to use 484521/16/63 as the disk geometry blah blah blah, Do you want to change this? I've been saying no, but now I think what I want to do is say yes, and change it to 30401/255/63 which is what Windows and Linux think it is, and repartition the whole drive. Does that sound reasonable? Of course this prompts me to ask the questions of why are we looking at this differently than Windows and Linux, and what are the advantages/disadvantages to the 2 methods? Thanks again, Doug Windows: Sectors/Track 63 Size 232.88 GB (250,056,737,280 bytes) Total Cylinders 30,401 Total Sectors 488,392,065 Tracks/Cylinder 255 Linux: fdisk -l Disk /dev/sda: 250.1 GB, 250059350016 bytes 255 heads, 63 sectors/track, 30401 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Device Boot Start End Blocks Id System /dev/sda1 1 3264 26214016+ 7 HPFS/NTFS Partition 1 does not end on cylinder boundary. /dev/sda2 3264 18276 120587585+ f W95 Ext'd (LBA) /dev/sda3 * 18277 27414 73400166 b5 Unknown Partition 3 does not end on cylinder boundary. /dev/sda4 27414 30402 23996448 a5 FreeBSD Partition 4 does not end on cylinder boundary. /dev/sda5 3264 16318 104860255+ 6 FAT16 /dev/sda6 16319 16971 5242880 83 Linux /dev/sda7 16971 17232 2096128 82 Linux swap / Solaris /dev/sda8 17233 17493 2096128 83 Linux /dev/sda9 17494 18276 6288384 83 Linux fdisk -c -l Device Boot Start End Blocks Id System /dev/sda1 1 3264 26214016+ 7 HPFS/NTFS /dev/sda2 3264 18276 120587585+ f W95 Ext'd (LBA) /dev/sda3 * 18277 27414 73400166 b5 Unknown /dev/sda4 27414 30402 23996448 a5 FreeBSD /dev/sda5 3264 16318 104860255+ 6 FAT16 /dev/sda6 16319 16971 5242880 83 Linux /dev/sda7 16971 17232 2096128 82 Linux swap / Solaris /dev/sda8 17233 17493 2096128 83 Linux /dev/sda9 17494 18276 6288384 83 Linux FreeBSD: ******* Working on device /dev/ad0 ******* parameters extracted from in-core disklabel are: cylinders=484521 heads=16 sectors/track=63 (1008 blks/cyl) Figures below won't work with BIOS for partitions not in cyl 1 parameters to be used for BIOS calculations are: cylinders=484521 heads=16 sectors/track=63 (1008 blks/cyl) Media sector size is 512 Warning: BIOS sector numbering starts with sector 1 Information from DOS bootblock is: The data for partition 1 is: sysid 7 (0x07),(OS/2 HPFS, NTFS, QNX-2 (16 bit) or Advanced UNIX) start 63, size 52428033 (25599 Meg), flag 0 beg: cyl 0/ head 1/ sector 1; end: cyl 1023/ head 15/ sector 63 The data for partition 2 is: sysid 15 (0x0f),(Extended DOS (LBA)) start 52428157, size 241175171 (117761 Meg), flag 0 beg: cyl 1023/ head 254/ sector 63; end: cyl 1023/ head 254/ sector 63 The data for partition 3 is: sysid 181 (0xb5),(unknown) start 293603940, size 146800332 (71679 Meg), flag 80 (active) beg: cyl 1023/ head 255/ sector 63; end: cyl 1023/ head 15/ sector 63 The data for partition 4 is: sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD) start 440404272, size 47992896 (23434 Meg), flag 0 beg: cyl 1023/ head 255/ sector 63; end: cyl 1023/ head 15/ sector 63 From owner-freebsd-fs@FreeBSD.ORG Fri Aug 27 03:08:48 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5A4BC1065696 for ; Fri, 27 Aug 2010 03:08:48 +0000 (UTC) (envelope-from schumi.han@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 0FFBC8FC12 for ; Fri, 27 Aug 2010 03:08:47 +0000 (UTC) Received: by gxk24 with SMTP id 24so1095821gxk.13 for ; Thu, 26 Aug 2010 20:08:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=szQi7OtdZEl+jTN/3Tvned0GYnmxxxbbBuYRtCqXdyc=; b=PBynrsx/KdWsOYmYYBrrWkpgH5sLreu5Pw5WJ3kmMKuatyayg/Nh/yKNIal/ZRik6I im62oDo8Mxjcw2sN9H2I6gqkLO+psDInmO2S1IcxUOJl83g4Mr9YeDaDTec0y+FbldhN Obh9kGuO9uN6F+q2FovqW1OlOgxgOZ7lmeQD0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=vW7gkyC3l1szZ3JcjdO90eyngUxfpHspDhWpCNe1yPsr/6gaoVrot8qxeDLLViD8Xx kSxi6Im07X7pxS9devrD/oF7b8nED/CAm401efav2x5VtK9WEKurTUEelNyWX71I/N2k E7n7aHahkh2w4SpiHSvqnHek2to2D3qy/8Zis= MIME-Version: 1.0 Received: by 10.151.5.12 with SMTP id h12mr1153865ybi.73.1282877094959; Thu, 26 Aug 2010 19:44:54 -0700 (PDT) Received: by 10.151.8.14 with HTTP; Thu, 26 Aug 2010 19:44:54 -0700 (PDT) Date: Fri, 27 Aug 2010 10:44:54 +0800 Message-ID: From: Zhu Han To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: ZFS Porting Roadmap X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Aug 2010 03:08:48 -0000 Hi, Does anybody know is there a road map document on porting different zpool versions to FreeBSD? I want to know when will ZFS pool version 19 be ported to FreeBSD? Removal of slog device is important for our environment. This is the description of zpool version 19: http://hub.opensolaris.org/bin/view/Community+Group+zfs/19 Thank you for your help. best regards, hanzhu From owner-freebsd-fs@FreeBSD.ORG Fri Aug 27 06:02:39 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F5681065675; Fri, 27 Aug 2010 06:02:39 +0000 (UTC) (envelope-from bruce@cran.org.uk) Received: from muon.cran.org.uk (unknown [IPv6:2001:470:1f08:679::2]) by mx1.freebsd.org (Postfix) with ESMTP id 0A42E8FC23; Fri, 27 Aug 2010 06:02:38 +0000 (UTC) Received: from unknown (client-82-31-24-1.midd.adsl.virginmedia.com [82.31.24.1]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by muon.cran.org.uk (Postfix) with ESMTPSA id BD6035F15; Fri, 27 Aug 2010 06:02:34 +0000 (UTC) Date: Fri, 27 Aug 2010 07:02:33 +0100 From: Bruce Cran To: Doug Barton Message-ID: <20100827070233.000075a8@unknown> In-Reply-To: <4C7715D0.10604@FreeBSD.org> References: <4C7715D0.10604@FreeBSD.org> X-Mailer: Claws Mail 3.7.4cvs1 (GTK+ 2.16.0; i586-pc-mingw32msvc) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Difference of opinion about my disk geometry X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Aug 2010 06:02:39 -0000 On Thu, 26 Aug 2010 18:33:04 -0700 Doug Barton wrote: > Below are what the various OS' think about the disk. (Ignore the fact > that the 3rd partition has an unknown type, that used to be a FreeBSD > partition that seems to have been mangled by grub2, which I'm going > to fix later.) When I run FreeBSD fdisk from sysinstall I get the > following message: > It is safe to use 484521/16/63 as the disk geometry blah blah blah, > Do you want to change this? > I've been saying no, but now I think what I want to do is say yes, > and change it to 30401/255/63 which is what Windows and Linux think > it is, and repartition the whole drive. Does that sound reasonable? > > Of course this prompts me to ask the questions of why are we looking > at this differently than Windows and Linux, and what are the > advantages/disadvantages to the 2 methods? CHS is totally obsolete, and can be ignored for anything but really old computers I think - LBA has been used since about 2000. If you use the modern partitioning tool in Windows, diskpart.exe, you don't get told about the geometry at all; if you use gpart in FreeBSD you get told about the fwheads and fwsectors but partitions are specified in terms of an offset. -- Bruce Cran From owner-freebsd-fs@FreeBSD.ORG Fri Aug 27 07:28:37 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4C37E10656A4 for ; Fri, 27 Aug 2010 07:28:37 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id ECA198FC0A for ; Fri, 27 Aug 2010 07:28:36 +0000 (UTC) Received: from outgoing.leidinger.net (p57B3B97D.dip.t-dialin.net [87.179.185.125]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id D2DB88449AE; Fri, 27 Aug 2010 09:09:23 +0200 (CEST) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 78D451A1E; Fri, 27 Aug 2010 09:09:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1282892960; bh=U2VMweNHpyZRYb8URcUyLB+WDH0nUsgqPuVCqbXxEng=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=v9Sk6wWUJYbEb3hBgRxO87E+OHOFBGWER5Ww5LgciBqonZxsT2gyejnZ5y3QKXrMd QvQxVH6iOQwtFW25pgToGSdXcUEEcBlKjSOuxKBi7yI9aO9vxilISBYihh41DtHVZQ KI6HT74L6uArMiyrXg41pBQAV4LbhMnlEwC18kaLuqN6ALBtUWoCGX4vRc3I8uzNWX gMlvSL5GomOcxqM26oBQRxdPyNbFqzmLn4i6D5qjn0fWjm1ouivp1RvY3OKbitHzQo 1Zpj1INzdRtbqmUYHOOIrdOrqR0RMorL5/3AYACTl0gsIcKd6jC14PNxilKUSvxRZr gTrwwSKVO5tAw== Received: (from www@localhost) by webmail.leidinger.net (8.14.4/8.13.8/Submit) id o7R79JjM062602; Fri, 27 Aug 2010 09:09:20 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from pslux.ec.europa.eu (pslux.ec.europa.eu [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Fri, 27 Aug 2010 09:09:19 +0200 Message-ID: <20100827090919.20705h1jxj2ssmg4@webmail.leidinger.net> Date: Fri, 27 Aug 2010 09:09:19 +0200 From: Alexander Leidinger To: Zhu Han References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: D2DB88449AE.A62F7 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.023, required 6, autolearn=disabled, ALL_TRUSTED -1.00, DKIM_SIGNED 0.10, DKIM_VALID -0.10, DKIM_VALID_AU -0.10, TW_ZF 0.08) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1283497766.75165@beylDZQfTT9zxLNxDKeMDA X-EBL-Spam-Status: No X-Mailman-Approved-At: Fri, 27 Aug 2010 11:12:18 +0000 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS Porting Roadmap X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Aug 2010 07:28:37 -0000 Quoting Zhu Han (from Fri, 27 Aug 2010 10:44:54 +0800): > Hi, > > Does anybody know is there a road map document on porting different zpool > versions to FreeBSD? I want to know when will ZFS pool version 19 be ported > to FreeBSD? Removal of slog device is important for our environment. There is work in progress to port the most recent zfs version (including zfs diff support). In 9-current the zfs is at v15. There are patches available for 8.1 to move it to v15 (search the archive). There is also some work on the way to port some non-intrusive changes from more recent zfs versions to the v15 in 9-current, but those are changes which do not break compatibility. This means there is no chance to get slog-removal support before the most recent zfs version is imported, except you do the work yourself or pay someone (pjd@ or mm@) to do the work. Bye, Alexander. -- Last week a cop stopped me in my car. He asked me if I had a police record. I said, no, but I have the new DEVO album. Cops have no sense of humor. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-fs@FreeBSD.ORG Fri Aug 27 16:58:44 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A05CC106564A for ; Fri, 27 Aug 2010 16:58:44 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx21.fluidhosting.com [204.14.89.4]) by mx1.freebsd.org (Postfix) with ESMTP id 49AE68FC12 for ; Fri, 27 Aug 2010 16:58:44 +0000 (UTC) Received: (qmail 12490 invoked by uid 399); 27 Aug 2010 16:58:43 -0000 Received: from localhost (HELO ?192.168.0.142?) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 27 Aug 2010 16:58:43 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Message-ID: <4C77EEC2.80909@FreeBSD.org> Date: Fri, 27 Aug 2010 09:58:42 -0700 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 MIME-Version: 1.0 To: Bruce Cran References: <4C7715D0.10604@FreeBSD.org> <20100827070233.000075a8@unknown> In-Reply-To: <20100827070233.000075a8@unknown> X-Enigmail-Version: 1.1.1 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Difference of opinion about my disk geometry X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Aug 2010 16:58:44 -0000 On 08/26/2010 11:02 PM, Bruce Cran wrote: > CHS is totally obsolete, and can be ignored for anything but really old > computers I think - LBA has been used since about 2000. If you use the > modern partitioning tool in Windows, diskpart.exe, you don't get told > about the geometry at all; if you use gpart in FreeBSD you get told > about the fwheads and fwsectors but partitions are specified in terms > of an offset. Thanks to you and to Partha for the replies. I'd like to make it clear I am not arguing with either of you, I don't even know enough about disks to BE dangerous. :) I am simply trying to understand what's going on (and make sure I'm not doing something now that's going to cause my data to go away tomorrow). In Linux fdisk I get errors that the partitions don't end on cylinder boundaries. In FreeBSD sysinstall when I re-installed my 9-current partition last night I got the following error after finishing the fdisk step, "chunk ad0s2 does not start on a track boundary" (ad0s2 is the extended partition I created in windows). So neither OS seems to like the current situation, but if you're telling me that those errors are harmless I have no problems ignoring them. What I definitely think is true at this point is that my 25-year-old "knowledge" about this stuff is now worse than useless. :) So I need to learn how to to do it better. I took a look at the man page for gpart and it's way over my head. I was not impressed with the way that the disk slicer for the ubuntu installer worked. Biggest single objection was that it considers 1 gigabyte to be 1,000,000 bytes, but at least it said this somewhere obvious so I could do the math to make it "right" myself, and fwiw Nautilus and the gnome "Disk Utility" seem to have the same view of the world, even though 'df -h' shows the "real" Gbs. However the results seemed Ok. I did some digging on line and saw many references to gparted, which is included in the ubuntu live cd, and seemed quite easy to work with. So my current line of thinking is that when I get ready to redo this disk for the last time (which should be soon now that I have my grub situation sorted) that I'll boot the ubuntu live cd and slice the whole disk first with gparted, then install windows, the 2 FreeBSDs (and specify the 30401/255/63 geometry to sysinstall just in case), then Linux last so that it can install grub and I'll finally be done! If anyone sees a problem with that process I'd appreciate it if you let me know asap because at this point even I am tired of fiddling with this stuff. :) Doug -- Improve the effectiveness of your Internet presence with a domain name makeover! http://SupersetSolutions.com/ Computers are useless. They can only give you answers. -- Pablo Picasso From owner-freebsd-fs@FreeBSD.ORG Fri Aug 27 18:53:12 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E95521065673; Fri, 27 Aug 2010 18:53:11 +0000 (UTC) (envelope-from bruce@cran.org.uk) Received: from muon.cran.org.uk (brucec-2-pt.tunnel.tserv5.lon1.ipv6.he.net [IPv6:2001:470:1f08:679::2]) by mx1.freebsd.org (Postfix) with ESMTP id 90E2B8FC20; Fri, 27 Aug 2010 18:53:11 +0000 (UTC) Received: from unknown (client-82-31-24-1.midd.adsl.virginmedia.com [82.31.24.1]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by muon.cran.org.uk (Postfix) with ESMTPSA id 1B6725F15; Fri, 27 Aug 2010 18:53:06 +0000 (UTC) Date: Fri, 27 Aug 2010 19:53:07 +0100 From: Bruce Cran To: Doug Barton Message-ID: <20100827195307.0000619a@unknown> In-Reply-To: <4C77EEC2.80909@FreeBSD.org> References: <4C7715D0.10604@FreeBSD.org> <20100827070233.000075a8@unknown> <4C77EEC2.80909@FreeBSD.org> X-Mailer: Claws Mail 3.7.4cvs1 (GTK+ 2.16.0; i586-pc-mingw32msvc) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Difference of opinion about my disk geometry X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Aug 2010 18:53:12 -0000 On Fri, 27 Aug 2010 09:58:42 -0700 Doug Barton wrote: > What I definitely think is true at this point is that my 25-year-old > "knowledge" about this stuff is now worse than useless. :) So I need > to learn how to to do it better. I took a look at the man page for > gpart and it's way over my head. I was not impressed with the way > that the disk slicer for the ubuntu installer worked. Biggest single > objection was that it considers 1 gigabyte to be 1,000,000 bytes, but > at least it said this somewhere obvious so I could do the math to > make it "right" myself, and fwiw Nautilus and the gnome "Disk > Utility" seem to have the same view of the world, even though 'df -h' > shows the "real" Gbs. However the results seemed Ok. I did some > digging on line and saw many references to gparted, which is included > in the ubuntu live cd, and seemed quite easy to work with. Apparently it's Ubuntu's policy to use base 10 units for disks: https://wiki.ubuntu.com/UnitsPolicy gpart isn't very easy to learn, but quite straightforward to use once you understand it - but one problem is that the manpage leaves a lot of usability details out such as the ability to specify sizes in MB/GB and to omit sizes in order to use the rest of the space. It also doesn't mention initializing the freebsd container with a "bsd" (or MBR) scheme. For example, to setup a disk with 40GB for FreeBSD, 50GB for Windows and the rest for Linux: Create an mfs system to work on: truncate -s 100G mdfs mdconfig -a -f mdfs Init the disk with an MBR: gpart create -s mbr md0 Create a BSD container: gpart add -t freebsd -s 40G md0 Init with a BSD scheme: gpart create -s bsd md0s1 Use 1GB for /: gpart add -t freebsd-ufs -s 1G md0s1 4GB for swap: gpart add -t freebsd-swap -s 4G md0s1 2GB for /var: gpart add -t freebsd-ufs -s 2G md0s1 1GB for /tmp: gpart add -t freebsd-ufs -s 1G md0s1 The rest for /usr: gpart add -t freebsd-ufs md0s1 Back in the MBR, create an NTFS partition: gpart add -t ntfs md0 We also want Linux, so del and re-create: gpart delete -i 2 md0 This time, only use 50GB for NTFS: gpart add -t ntfs -s 50G md0 At this point it's not possible to create Linux partitions, because the "linux-data" and "linux-swap" types are only available if you're working on a GPT disk. At each stage you can also specify "-f x" e.g. "gpart create -f x -s mbr md0" in order to have the new layout stored in memory but not written to disk until you run "gpart commit md0". It seems the issue about cylinder alignment came about through compatibility issues with old versions of Windows (98/ME) and appartently (http://tldp.org/HOWTO/Large-Disk-HOWTO-6.html) sparc needs the boot partition to start on a cylinder boundary. It also seems that diskpart in Windows has some rules regarding cylinder alignment too (http://support.microsoft.com/kb/300415). -- Bruce Cran From owner-freebsd-fs@FreeBSD.ORG Sat Aug 28 02:18:22 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C98A010656A9 for ; Sat, 28 Aug 2010 02:18:22 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 74CA98FC17 for ; Sat, 28 Aug 2010 02:18:22 +0000 (UTC) Received: by gwj23 with SMTP id 23so1610011gwj.13 for ; Fri, 27 Aug 2010 19:18:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=fGPPGVmyjfKyTHu91EqKwqcSv9qNuclEMYFGihzwQ44=; b=qZ30KI+TULf3klDgXQjgEu2GovkQxoJTTz82FlBZb5Pc+5n14w7m7i35wv5fWIsaCj c9ij8qYtSRWDFYDIKkGJtutveG6wUyQvxcgY/Ngjy28+4A1e7StWKf3pXtqwv15D43CB Qs++OFM4yB3NBNBl5784+tp1zgPSveLLCpSWk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=EpVCcDBvi4Z+XJOsH0CaLmUESAWZzSE82sOOD5MkL3xdGE2W3XeCFgvQI8aHyJ25YC XvdqsUWJZAsPR+CGiulg6bJa5f/unN4LNSeGAUDnON8OWi2umqCBed9/1KG29oyz+zAc yQx54FpcW/GWP9xmxLUD/3j+QCUVmAu88yiEQ= Received: by 10.100.121.6 with SMTP id t6mr1779430anc.141.1282960012252; Fri, 27 Aug 2010 18:46:52 -0700 (PDT) Received: from centel.dataix.local ([99.181.137.20]) by mx.google.com with ESMTPS id e18sm4439913ana.15.2010.08.27.18.46.49 (version=SSLv3 cipher=RC4-MD5); Fri, 27 Aug 2010 18:46:50 -0700 (PDT) Sender: "J. Hellenthal" Message-ID: <4C786A88.20509@DataIX.net> Date: Fri, 27 Aug 2010 21:46:48 -0400 From: jhell User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.8) Gecko/20100806 Lightning/1.0b1 Thunderbird MIME-Version: 1.0 To: Adam Stylinski References: In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Problem with zpool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Aug 2010 02:18:22 -0000 On 08/26/2010 14:08, Adam Stylinski wrote: > scrub: resilver in progress for 0h9m, 2.16% done, 6h53m to go You probably want to wait for this to be done before continuing operation. > > replacing DEGRADED 0 0 0 > 8991447011275450347 UNAVAIL 0 10.0K 0 was /dev/ad6/old > ada2 ONLINE 0 0 0 14.1G resilvered > ada4 ONLINE 0 0 0 56.7M resilvered > With the above and the scrub: message these seems just fine. > > > logs DEGRADED 0 0 0 > ada0 ONLINE 0 0 0 Logs in DEGRADED state could be the cause of your whole problem. Add a spare to that thing and replace the drive if possible or just add another to it to see if it goes away. PS you wont be able to remove it but you will be able to offline it if you have enough replicas. Mirror this vdev as you can not raidzN a log device. for instance: zpool add topool log mirror ad0 ad1 mirror ad2 ad3 And don't forget! A raidz group with N disks of size X with P parity disks can hold approximately (N-P)*X bytes and can withstand P device(s) failing before data integrity is compromised. The minimum number of devices in a raidz group is one more than the number of parity disks. The recommended number is between 3 and 9 to help increase performance. Regards, -- jhell,v From owner-freebsd-fs@FreeBSD.ORG Sat Aug 28 04:43:55 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B72B01065697 for ; Sat, 28 Aug 2010 04:43:55 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-7.mit.edu (DMZ-MAILSEC-SCANNER-7.MIT.EDU [18.7.68.36]) by mx1.freebsd.org (Postfix) with ESMTP id 74D738FC17 for ; Sat, 28 Aug 2010 04:43:55 +0000 (UTC) X-AuditID: 12074424-b7b2bae000005b3f-05-4c7890741233 Received: from mailhub-auth-1.mit.edu ( [18.9.21.35]) by dmz-mailsec-scanner-7.mit.edu (Symantec Brightmail Gateway) with SMTP id 18.AB.23359.470987C4; Sat, 28 Aug 2010 00:28:36 -0400 (EDT) Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103]) by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id o7S4SrBr005533 for ; Sat, 28 Aug 2010 00:28:53 -0400 Received: from multics.mit.edu (MULTICS.MIT.EDU [18.187.1.73]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id o7S4SpAL018570 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Sat, 28 Aug 2010 00:28:52 -0400 (EDT) Received: (from kaduk@localhost) by multics.mit.edu (8.12.9.20060308) id o7S4SpB9000924; Sat, 28 Aug 2010 00:28:51 -0400 (EDT) Date: Sat, 28 Aug 2010 00:28:51 -0400 (EDT) From: Benjamin Kaduk To: freebsd-fs@freebsd.org Message-ID: User-Agent: Alpine 1.10 (GSO 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Brightmail-Tracker: AAAAAgB+j9UVxJa6 Subject: memory inconsistencies with OpenAFS on FreeBSD X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Aug 2010 04:43:55 -0000 Hi all, I've been working to make the OpenAFS network filesystem client usable on FreeBSD; it's currently in a mostly-working state. ( http://www.openafs.org/ ; I have a hackish packaging of it as a FreeBSD port at http://web.mit.edu/freebsd/openafs/openafs.shar . Note that it is broken on recent -current since it it fails to register its syscall properly; this is fixed in git master pending an update to syscalls.master.) Normal file operations from a shell work okay, I can write and edit a file with vi, etc.; copying /usr/src into and out of AFS proceeds nicely. However, if I proceed to the standard lazy man's filesystem stress test, buildworld, things don't get very far: >>> stage 1.1: legacy release compatibility shims [...] ===> tools/build (obj,includes,depend,all,install) [...] cc -O2 -pipe -std=gnu99 -I/afs/zone.mit.edu/user/kaduk/build/obj/afs/zone.mit. edu/user/kaduk/build/tmp/legacy/usr/include -c /afs/zone.mit.edu/user/kaduk/buil d/tools/build/dummy.c building static egacy library *** Signal 11 I also don't seem to be able to run executables from AFS: freebuild# ./my_mmap test4 elf_load_section: truncated ELF file Abort Trying to dig a bit deeper and get a smaller test case, rwatson suggested that I look at mmap-ing a file and reading/writing from it. I wrote a small test program, and reading from the mmaped file works okay. However, writing to the mmap-ed file does not seem to take effect (though my test program does modify the target file on local disk). (Also, when I do silly things like try to access unmapped memory which ought to generate a core dump, I get on the console: freebuild kernel: Failed to write core file for process my_mmap (error 14)) Where should I be looking to track down what's going on? I note that we do provide our own getpages and putpages vops, and this code hasn't been particularly loved since the FreeBSD 4.x days. Thanks for any suggestions, Ben Kaduk From owner-freebsd-fs@FreeBSD.ORG Sat Aug 28 08:35:59 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1B5161065696 for ; Sat, 28 Aug 2010 08:35:59 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id A95B98FC16 for ; Sat, 28 Aug 2010 08:35:58 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o7S8OsXa032703 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 28 Aug 2010 11:24:54 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o7S8OsGr053596; Sat, 28 Aug 2010 11:24:54 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o7S8OrZ6053595; Sat, 28 Aug 2010 11:24:53 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 28 Aug 2010 11:24:53 +0300 From: Kostik Belousov To: Benjamin Kaduk Message-ID: <20100828082453.GG2396@deviant.kiev.zoral.com.ua> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="41nqFSEcPhzBz76I" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.2 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_50, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-fs@freebsd.org Subject: Re: memory inconsistencies with OpenAFS on FreeBSD X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Aug 2010 08:35:59 -0000 --41nqFSEcPhzBz76I Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Aug 28, 2010 at 12:28:51AM -0400, Benjamin Kaduk wrote: > Hi all, >=20 > I've been working to make the OpenAFS network filesystem client usable on= =20 > FreeBSD; it's currently in a mostly-working state.=20 > ( http://www.openafs.org/ ; I have a hackish packaging of it as a FreeBSD= =20 > port at http://web.mit.edu/freebsd/openafs/openafs.shar . Note that it i= s=20 > broken on recent -current since it it fails to register its syscall=20 > properly; this is fixed in git master pending an update to=20 > syscalls.master.) >=20 > Normal file operations from a shell work okay, I can write and=20 > edit a file with vi, etc.; copying /usr/src into and out of AFS proceeds= =20 > nicely. > However, if I proceed to the standard lazy man's filesystem stress test,= =20 > buildworld, things don't get very far: > >>>stage 1.1: legacy release compatibility shims > [...] > =3D=3D=3D> tools/build (obj,includes,depend,all,install) > [...] > cc -O2 -pipe -std=3Dgnu99=20 > -I/afs/zone.mit.edu/user/kaduk/build/obj/afs/zone.mit. > edu/user/kaduk/build/tmp/legacy/usr/include -c=20 > /afs/zone.mit.edu/user/kaduk/buil > d/tools/build/dummy.c > building static egacy library > *** Signal 11 >=20 > I also don't seem to be able to run executables from AFS: > freebuild# ./my_mmap test4 > elf_load_section: truncated ELF file > Abort This sounds very much as missed vnode_pager_setsize() calls. VM tracks the file size as vnode vmobject size separately. I think this was done for historical reasons, but also it allows to not traverse the vop stack calling VOP_GETATTR each time when size is needed. >=20 >=20 > Trying to dig a bit deeper and get a smaller test case, rwatson suggested= =20 > that I look at mmap-ing a file and reading/writing from it. I wrote a=20 > small test program, and reading from the mmaped file works okay. However,= =20 > writing to the mmap-ed file does not seem to take effect (though my test= =20 > program does modify the target file on local disk). >=20 > (Also, when I do silly things like try to access unmapped memory which=20 > ought to generate a core dump, I get on the console: > freebuild kernel: Failed to write core file for process my_mmap (error=20 > 14)) >=20 > Where should I be looking to track down what's going on? >=20 > I note that we do provide our own getpages and putpages vops, and this=20 > code hasn't been particularly loved since the FreeBSD 4.x days. >=20 > Thanks for any suggestions, >=20 > Ben Kaduk > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" --41nqFSEcPhzBz76I Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkx4x9UACgkQC3+MBN1Mb4hfWgCg8XBS8Xp6xBovk09983KFTAKG DN8AnR4sYmguJR425zvFVNPAcbuYudMQ =eDx5 -----END PGP SIGNATURE----- --41nqFSEcPhzBz76I-- From owner-freebsd-fs@FreeBSD.ORG Sat Aug 28 08:55:26 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 330121065694; Sat, 28 Aug 2010 08:55:26 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id D16DE8FC18; Sat, 28 Aug 2010 08:55:25 +0000 (UTC) Received: by iwn36 with SMTP id 36so3821775iwn.13 for ; Sat, 28 Aug 2010 01:55:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=ox4CnHiZpq7uuXzbe/whASRUVym/3f1e4eFoewEiU/o=; b=Kh+YyoOrayzU6mdzi60K1uDL4AU1Edr6Sdvp8KlIe2BJ5VJDJAJ+ePVW7g81PSjtbJ buj341awHxWA431QQTKj+rDb5NEGV2Lrft9JKQC3eQvwtLBKpiZdBuEz5BESPuegDEJ4 UrERlDe69IeS2ok/ukTXox6523M7PaqBAoM1w= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=VY+9Cfa8wmqatlK2RZ4cvdBGoiK1h1l9SC/gnpsFoze7C2AEeEWRBxkXfs7Xjiq+vR zYOYMuJjS/We5veE8eKPi5TfV+n6nIwRl4iV5Yv5tSnGGDkpHcMXDGju/gRvEONy7aJV Srj9RQZsLjyz57nlzZNjUr3/NiJwctPhK+I98= Received: by 10.231.13.77 with SMTP id b13mr2093103iba.28.1282985661245; Sat, 28 Aug 2010 01:54:21 -0700 (PDT) Received: from centel.dataix.local ([99.181.137.20]) by mx.google.com with ESMTPS id e8sm4659329ibb.20.2010.08.28.01.54.18 (version=SSLv3 cipher=RC4-MD5); Sat, 28 Aug 2010 01:54:19 -0700 (PDT) Sender: "J. Hellenthal" Message-ID: <4C78CEB9.9010400@DataIX.net> Date: Sat, 28 Aug 2010 04:54:17 -0400 From: jhell User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.8) Gecko/20100806 Lightning/1.0b1 Thunderbird MIME-Version: 1.0 References: <4C76D981.7080602@FreeBSD.org> <4C76F210.5020207@feral.com> <4C76FE56.10306@FreeBSD.org> <4C770268.1010707@feral.com> In-Reply-To: <4C770268.1010707@feral.com> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, Matthew Jacob Subject: Re: Questions about FreeBSD and Linux on the same disk X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Aug 2010 08:55:26 -0000 On 08/26/2010 20:10, Matthew Jacob wrote: > > Umm. I don't do much with the extended partitions, but I certainly have > more than one FreeBSD system on the box- two on drive 0 and one on drive 2. > > title Fedora (2.6.32.16-141.fc12.x86_64) > root (hd0,0) > kernel /vmlinuz-2.6.32.16-141.fc12.x86_64 ro > root=/dev/mapper/VolGroup-lv_root console=ttyS0,115200n8 > SYSFONT=latarcyrheb-sun16 LANG=en_US.UTF-8 KEYTABLE=us > initrd /initramfs-2.6.32.16-141.fc12.x86_64.img > savedefault > ... > title FreeBSD > rootnoverify (hd0,2) > makeactive > chainloader +1 > savedefault > title FreeBSD32 > rootnoverify (hd0,1) > makeactive > chainloader +1 > savedefault > title FreeBSD32 8.X > rootnoverify (hd2,0) > makeactive > chainloader +1 > savedefault > .... > >> above it sounds like you have the "multiple freebsd" problem licked, >> perhaps you could take a look at my post on -hackers and help me out >> there too? :-/ >> >> Something you may also find of interest is the ``map'' command in grub that allows you to remap where drives are before you boot as well hide other drives so certain OS's can't see them. Explained in the grub info page. What I did with it.... Since I like the thought of Windows not being able to touch my FreeBSD install (mbr) and that I like my FreeBSD install to be on the first disk of the machine for sanity reasons. My install was like this. (from empty disks to multi-booting). * Install FreeBSD as usual on disk0 partitions however you create them. * (Optional) install additional FreeBSD's or Linux versions on other partitions. * Install grub from one of the above installs to disk0's mbr. * Time to install Windows! (Eeeek!) turn off disk0 in the bios so the machine & Windows can not see it. This way it thinks disk1 is disk0 and will install as such. Also installing its mbr/bootmbr to disk1 and not disk0. * After the install and update of your installed Windows onto disk1 pop back into the bios and turn disk0 back on or plug it back in or reverse whatever you have done before. * When your computer reboots after you re-enabled disk0 it will boot up to grub and allow you to boot into FreeBSD or Linux whatever you choose from the first 3 steps. * Now effectively what you have is two ways to boot either disk0 or disk1 using grub to boot either. And if disk0 fails then your installation will fail-over to disk1 and boot Windows without intervention. * Next step is to boot to the grub command line after editing your grub config for convienience, and add a section for your windows install that you can add commands to later on after testing. * Boot to the grub command line hit ``e'' on the Windows label to edit and ``a'' to add a entry to contain something like ``map (ad0) (ad1)'' and then repeat for an addition of another line below that ``map (ad1) (ad0)'' then use ``b'' to try to boot into Windows. * Once you find the right commands through testing jot them down and boot into where your grub config is and edit to your liking. * When you boot back into windows the next time you can turn off your actual disk0 from the Device Manager and be sure that windows will never touch it again. With this type of install you always have a OS that can boot without the need for pulling out some rescue disk and can be sure that one risky OS like Windows will not touch the MBR of another OS. PS: I am looking for my exact configuration used and if found I will post back. Regards, -- jhell,v