From owner-freebsd-fs@FreeBSD.ORG Sun Dec 17 16:22:33 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BBD6116A416; Sun, 17 Dec 2006 16:22:33 +0000 (UTC) (envelope-from nork@FreeBSD.org) Received: from sakura.ninth-nine.com (sakura.ninth-nine.com [219.127.74.120]) by mx1.FreeBSD.org (Postfix) with ESMTP id D1C5343CD4; Sun, 17 Dec 2006 16:22:31 +0000 (GMT) (envelope-from nork@FreeBSD.org) Received: from nadesico.ninth-nine.com (nadesico.ninth-nine.com [219.127.74.122]) by sakura.ninth-nine.com (8.13.8/8.13.8/NinthNine) with SMTP id kBHGMUs8044736; Mon, 18 Dec 2006 01:22:30 +0900 (JST) (envelope-from nork@FreeBSD.org) Date: Mon, 18 Dec 2006 01:22:29 +0900 From: Norikatsu Shigemura To: pjd@FreeBSD.org Message-Id: <20061218012229.11e8cb10.nork@FreeBSD.org> X-Mailer: Sylpheed version 2.3.0beta7 (GTK+ 2.10.6; i386-portbld-freebsd6.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.0.2 (sakura.ninth-nine.com [219.127.74.121]); Mon, 18 Dec 2006 01:22:30 +0900 (JST) Cc: freebsd-fs@FreeBSD.org Subject: ZFS cannot be compiled by changing sleepq_* X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Dec 2006 16:22:33 -0000 Hi pjd! In recently 7-current, by changing sleepq_* by kmacy, I cannot compile ZFS. cc -O2 -fno-strict-aliasing -pipe -march=pentium3 -D_SOLARIS_C_SOURCE -D_XOPEN_SOURCE=600 -D_XOPEN_SOURCE_EXTENDED=2 -D_XOPEN_VERSION=600 -D_POSIX_C_SOURCE=200112L -D__BSD_VISIBLE=1 -D_STDC_C99 -DZFS_NO_ZONE -DZFS_MPSAFE -Werror -D_KERNEL -DKLD_MODULE -std=c99 -nostdinc -I- -I/usr/src/sys/modules/zfs/../../i386/include -I/usr/src/sys/modules/zfs/../../compat/opensolaris -I/usr/src/sys/modules/zfs/../../contrib/opensolaris/uts/common/fs/zfs -I/usr/src/sys/modules/zfs/../../contrib/opensolaris/uts/common -I/usr/src/sys/modules/zfs/../.. -I/usr/src/sys/modules/zfs/../../i386/include -I/usr/src/sys/modules/zfs/../../../contrib/opensolaris/common/zfs -I/usr/src/sys/modules/zfs/../../../contrib/opensolaris/common -I/usr/include -I. -I@ -I@/contrib/altq -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -fno-common -g -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -ffreestanding -Wall -Wno-u! nknown-pragmas -Wno-missing-braces -Wno-sign-compare -Wno-parentheses -Wno-uninitialized -Wno-implicit-function-declaration -Wno-unused -Wno-trigraphs -Wno-char-subscripts -Wno-switch -c /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_condvar.c /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_condvar.c: In function `cv_wait_unlock': /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_condvar.c:128: error: too few arguments to function `sleepq_add' /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_condvar.c: In function `cv_timedwait': /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_condvar.c:165: error: too few arguments to function `sleepq_add' /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_condvar.c: In function `cv_signal': /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_condvar.c:189: error: too few arguments to function `sleepq_signal' /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_condvar.c: In function `cv_broadcast': /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_condvar.c:205: error: too few arguments to function `sleepq_broadcast' *** Error code 1 Stop in /usr/src/sys/modules/zfs. From owner-freebsd-fs@FreeBSD.ORG Mon Dec 18 14:08:06 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3E58116A407; Mon, 18 Dec 2006 14:08:06 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from smtp1.yandex.ru (smtp1.yandex.ru [213.180.223.87]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3512C43CA0; Mon, 18 Dec 2006 14:08:04 +0000 (GMT) (envelope-from bu7cher@yandex.ru) Received: from ns.kirov.so-cdu.ru ([81.18.142.225]:58631 "EHLO [127.0.0.1]" smtp-auth: "bu7cher" TLS-CIPHER: "DHE-RSA-AES256-SHA keybits 256/256 version TLSv1/SSLv3" TLS-PEER-CN1: ) by mail.yandex.ru with ESMTP id S2079203AbWLRNue (ORCPT + 1 other); Mon, 18 Dec 2006 16:50:34 +0300 X-Comment: RFC 2476 MSA function at smtp1.yandex.ru logged sender identity as: bu7cher Message-ID: <45869C9A.8090405@yandex.ru> Date: Mon, 18 Dec 2006 16:50:18 +0300 From: "Andrey V. Elsukov" User-Agent: Mozilla Thunderbird 1.5 (FreeBSD/20051231) MIME-Version: 1.0 To: Nico -telmich- Schottelius References: <20061202120228.GB27796@schottelius.org> In-Reply-To: <20061202120228.GB27796@schottelius.org> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, Robert Watson Subject: Re: ACL broken on all FreeBSD variants X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Dec 2006 14:08:06 -0000 Nico -telmich- Schottelius wrote: > Just choosed that provocant topic, because there seem to be no > reaction on all the reports I send about FreeBSD ACLs. > > Just wanted to know, whether anyone REALLY uses ACLs with default > entries. If so, does it really work with creating new files? Look here: http://www.freebsd.org/cgi/man.cgi?query=setfacl&sektion=1&apropos=0&manpath=FreeBSD+6.1-RELEASE ... -d The operations apply to the default ACL entries instead of access ACL entries. Currently *only directories* may have default ACL's. ... And here: http://www.onlamp.com/lpt/a/6185 ... Directories are more complex, as they can have up to three types of ACLs: * An access ACL affects access to the directory itself. * The default directory ACL sets the default permissions on any subdirectories created within the directory. * The default access ACL sets the default permissions on any files created within the directory. Note that if the default directory ACL is not set, subdirectories will also inherit this ACL. However, if the default directory ACL is set, that value will override the value of this ACL. The current FreeBSD implementation supports *only the first two types* of directory ACLs, so double-check the effective permissions on any files you create in directories containing ACLs. ... -- WBR, Andrey V. Elsukov From owner-freebsd-fs@FreeBSD.ORG Mon Dec 18 21:22:17 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 634BA16A40F; Mon, 18 Dec 2006 21:22:17 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id D4CEF43CAE; Mon, 18 Dec 2006 21:22:07 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id A8004456B1; Mon, 18 Dec 2006 21:54:49 +0100 (CET) Received: from localhost (dlr55.neoplus.adsl.tpnet.pl [83.24.47.55]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 1141045696; Mon, 18 Dec 2006 21:54:41 +0100 (CET) Date: Mon, 18 Dec 2006 21:54:29 +0100 From: Pawel Jakub Dawidek To: Norikatsu Shigemura Message-ID: <20061218205429.GB77687@garage.freebsd.pl> References: <20061218012229.11e8cb10.nork@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="NDin8bjvE/0mNLFQ" Content-Disposition: inline In-Reply-To: <20061218012229.11e8cb10.nork@FreeBSD.org> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=BAYES_00,RCVD_IN_NJABL_DUL, RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@FreeBSD.org Subject: Re: ZFS cannot be compiled by changing sleepq_* X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Dec 2006 21:22:17 -0000 --NDin8bjvE/0mNLFQ Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Dec 18, 2006 at 01:22:29AM +0900, Norikatsu Shigemura wrote: > Hi pjd! >=20 > In recently 7-current, by changing sleepq_* by kmacy, > I cannot compile ZFS. >=20 > cc -O2 -fno-strict-aliasing -pipe -march=3Dpentium3 -D_SOLARIS_C_SOURCE -= D_XOPEN_SOURCE=3D600 -D_XOPEN_SOURCE_EXTENDED=3D2 -D_XOPEN_VERSION=3D600 -D= _POSIX_C_SOURCE=3D200112L -D__BSD_VISIBLE=3D1 -D_STDC_C99 -DZFS_NO_ZONE -DZ= FS_MPSAFE -Werror -D_KERNEL -DKLD_MODULE -std=3Dc99 -nostdinc -I- -I/usr/s= rc/sys/modules/zfs/../../i386/include -I/usr/src/sys/modules/zfs/../../comp= at/opensolaris -I/usr/src/sys/modules/zfs/../../contrib/opensolaris/uts/com= mon/fs/zfs -I/usr/src/sys/modules/zfs/../../contrib/opensolaris/uts/common = -I/usr/src/sys/modules/zfs/../.. -I/usr/src/sys/modules/zfs/../../i386/incl= ude -I/usr/src/sys/modules/zfs/../../../contrib/opensolaris/common/zfs -I/u= sr/src/sys/modules/zfs/../../../contrib/opensolaris/common -I/usr/include -= I. -I@ -I@/contrib/altq -finline-limit=3D8000 --param inline-unit-growth=3D= 100 --param large-function-growth=3D1000 -fno-common -g -mno-align-long-str= ings -mpreferred-stack-boundary=3D2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2= -mno-sse3 -ffreestanding -Wall -Wno-u! > nknown-pragmas -Wno-missing-braces -Wno-sign-compare -Wno-parentheses -W= no-uninitialized -Wno-implicit-function-declaration -Wno-unused -Wno-trigra= phs -Wno-char-subscripts -Wno-switch -c /usr/src/sys/modules/zfs/../../comp= at/opensolaris/kern/opensolaris_condvar.c > /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_condva= r.c: In function `cv_wait_unlock': > /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_condva= r.c:128: error: too few arguments to function `sleepq_add' > /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_condva= r.c: In function `cv_timedwait': > /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_condva= r.c:165: error: too few arguments to function `sleepq_add' > /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_condva= r.c: In function `cv_signal': > /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_condva= r.c:189: error: too few arguments to function `sleepq_signal' > /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_condva= r.c: In function `cv_broadcast': > /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_condva= r.c:205: error: too few arguments to function `sleepq_broadcast' > *** Error code 1 >=20 > Stop in /usr/src/sys/modules/zfs. That's why there is a date in the patch name, so you can fetch FreeBSD source from exactly that date:) If you still want to compile it on recent HEAD, add '0' argument at the end of those functions calls: -sleepq_add(cvp, &mp->sx_object, cvp->cv_description, SLEEPQ_CONDVAR); +sleepq_add(cvp, &mp->sx_object, cvp->cv_description, SLEEPQ_CONDVAR, 0); -sleepq_add(cvp, &mp->sx_object, cvp->cv_description, SLEEPQ_CONDVAR); +sleepq_add(cvp, &mp->sx_object, cvp->cv_description, SLEEPQ_CONDVAR, 0); -sleepq_signal(cvp, SLEEPQ_CONDVAR, -1); +sleepq_signal(cvp, SLEEPQ_CONDVAR, -1, 0); -sleepq_broadcast(cvp, SLEEPQ_CONDVAR, -1); +sleepq_broadcast(cvp, SLEEPQ_CONDVAR, -1, 0); --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --NDin8bjvE/0mNLFQ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFFhwAFForvXbEpPzQRAtdwAJ9S/1Bhr4rsvfptJA6agQavusfiCQCgw5CJ ageEdF1emXMYHJYL0B5zcs0= =VUoc -----END PGP SIGNATURE----- --NDin8bjvE/0mNLFQ-- From owner-freebsd-fs@FreeBSD.ORG Mon Dec 18 23:03:01 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 696C516A403 for ; Mon, 18 Dec 2006 23:03:01 +0000 (UTC) (envelope-from aronesimi@yahoo.com) Received: from web58612.mail.re3.yahoo.com (web58612.mail.re3.yahoo.com [68.142.236.210]) by mx1.FreeBSD.org (Postfix) with SMTP id 90F3643CA0 for ; Mon, 18 Dec 2006 23:03:00 +0000 (GMT) (envelope-from aronesimi@yahoo.com) Received: (qmail 67557 invoked by uid 60001); 18 Dec 2006 18:36:22 -0000 Message-ID: <20061218183622.67555.qmail@web58612.mail.re3.yahoo.com> DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=1qIXLqznvNi1UoLgHcBt/9M/SqCSGEm57l5Sb7bN9ND1D9Rd35eO8OV1iE+S+pNqQGDQ5Rf975rmxt86ip44R7io121Vijt5RLRpdxE4Jac4OgcApXd2SqldXrU5hO/hV/g7xQvkAfKtkBsMQd4I8U87dOceCvmNu7ww4vGLNd4=; X-YMail-OSG: aZQ10eEVM1lGy6eUUac9JqvB7GTFWzE1fqPYE8uzL34EqcQWPdagBX_PVfO1.uSCoRwuh_Y1Fmnu8wTXk_K7b0C4jRu.IFw9lU8bNNV9HlkyhzYSPM2lXw-- Received: from [75.72.230.91] by web58612.mail.re3.yahoo.com via HTTP; Mon, 18 Dec 2006 10:36:19 PST Date: Mon, 18 Dec 2006 10:36:19 -0800 (PST) From: Arone Silimantia To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Mailman-Approved-At: Mon, 18 Dec 2006 23:12:40 +0000 Subject: quotas safe on >2 TB filesystems in 6.1-RELEASE ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Dec 2006 23:03:01 -0000 I am planning on building a 6.1-RELEASE system with a boot array smaller than 2 TB (and thus not interesting or dangerous in any way - also sysinstall compatible). But then after installation I am going to use CLI tools (all bigdisk safe, right ?) to partition and newfs a 9 TB array. I will not be using snapshots on this array. But I do absolutely need to run quotas (both user and group) on this 9 GB array. I also need to successfully use all quota admin tools (repquota, edquota, quota, etc.) Can I get an assurance that this is totally safe, sane, and fit to run in a mission critical, data critical environment ? Anyone doing it currently ? Any comments or warnings of _any kind_ much appreciated. __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-fs@FreeBSD.ORG Mon Dec 18 23:56:30 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6AAA716A47B for ; Mon, 18 Dec 2006 23:56:30 +0000 (UTC) (envelope-from nico-freebsd-fs@schottelius.org) Received: from schottelius.org (dslb-082-083-023-114.pools.arcor-ip.net [82.83.23.114]) by mx1.FreeBSD.org (Postfix) with SMTP id 068FF43C9F for ; Mon, 18 Dec 2006 23:56:28 +0000 (GMT) (envelope-from nico-freebsd-fs@schottelius.org) Received: (qmail 29408 invoked by uid 1000); 18 Dec 2006 23:29:17 -0000 Date: Tue, 19 Dec 2006 00:29:17 +0100 From: Nico -telmich- Schottelius To: "Andrey V. Elsukov" Message-ID: <20061218232917.GK4152@schottelius.org> References: <20061202120228.GB27796@schottelius.org> <45869C9A.8090405@yandex.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ibvzjYYg+QDzMCy1" Content-Disposition: inline In-Reply-To: <45869C9A.8090405@yandex.ru> User-Agent: echo $message | gpg -e $sender -s | netcat mailhost 25 X-Linux-Info: http://linux.schottelius.org/ X-Operating-System: Linux 2.6.19.1-hydrogenium Cc: freebsd-fs@freebsd.org, Robert Watson Subject: Re: ACL broken on all FreeBSD variants X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Dec 2006 23:56:30 -0000 --ibvzjYYg+QDzMCy1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Andrey V. Elsukov [Mon, Dec 18, 2006 at 04:50:18PM +0300]: > ... > Directories are more complex, as they can have up to three types of ACLs: >=20 > * An access ACL affects access to the directory itself. > * The default directory ACL sets the default permissions on any > subdirectories created within the directory. > * The default access ACL sets the default permissions on any > files created within the directory. Note that if the default directory > ACL is not set, subdirectories will also inherit this ACL. However, if > the default directory ACL is set, that value will override the value > of this ACL. >=20 > The current FreeBSD implementation supports *only the first two types* > of directory ACLs, so double-check the effective permissions on any > files you create in directories containing ACLs. > ... Thanks! Are there any plans to support the last type? This is the common use of ACLs in my situation, so currently we are hacking the stuff by executing chmod && chown each time files get updated. Recursively. Very dirty and not very performant, but the only solution that seems to be available with FreeBSD. Nico --=20 ``...if there's one thing about Linux users, they're do-ers, not whiners.'' (A quotation of Andy Patrizio I completely agree with) --ibvzjYYg+QDzMCy1 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) iD8DBQFFhyRNuL75KpiFGIwRAoQgAKDMZ6i2nFh+crx0+Rjol7OFNCHgUACdEH6q C1L9FVa6pKhvE3IvJBNJtzs= =hctv -----END PGP SIGNATURE----- --ibvzjYYg+QDzMCy1-- From owner-freebsd-fs@FreeBSD.ORG Tue Dec 19 14:22:07 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D5C8D16A416 for ; Tue, 19 Dec 2006 14:22:07 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [64.129.166.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9B58C43CB6 for ; Tue, 19 Dec 2006 14:20:29 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.8/8.13.8) with ESMTP id kBJDVtgh004302; Tue, 19 Dec 2006 07:31:55 -0600 (CST) (envelope-from anderson@centtech.com) Message-ID: <4587E9D7.9050709@centtech.com> Date: Tue, 19 Dec 2006 07:32:07 -0600 From: Eric Anderson User-Agent: Thunderbird 1.5.0.7 (X11/20061015) MIME-Version: 1.0 To: Arone Silimantia References: <20061218183622.67555.qmail@web58612.mail.re3.yahoo.com> In-Reply-To: <20061218183622.67555.qmail@web58612.mail.re3.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.88.4/2357/Tue Dec 19 03:14:19 2006 on mh1.centtech.com X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=8.0 tests=AWL,BAYES_00 autolearn=ham version=3.1.6 X-Spam-Checker-Version: SpamAssassin 3.1.6 (2006-10-03) on mh1.centtech.com Cc: freebsd-fs@freebsd.org Subject: Re: quotas safe on >2 TB filesystems in 6.1-RELEASE ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Dec 2006 14:22:07 -0000 On 12/18/06 12:36, Arone Silimantia wrote: > I am planning on building a 6.1-RELEASE system with a > boot array smaller than 2 TB (and thus not interesting > or dangerous in any way - also sysinstall compatible). > > But then after installation I am going to use CLI > tools (all bigdisk safe, right ?) to partition and > newfs a 9 TB array. If you really need to partition it, then do so, however I believe you'll get better performance on a RAID array if you simply newfs the raw disk (/dev/daX) instead of adding slices and such on it. Also, I highly recommend using GJOURNAL on this disk. What I do is create a small (4GB) fast LUN (RAID 10 or 0+1 is good) and use it as my journal device, and then create the remaining space as another LUN for the data, and use GJOURNAL on top. GJOURNAL is probably still considered slightly experimental, but I have over 25Tb (in a few different systems, under different loads), and have been very happy about it. With 9TB without any journaling, you might run into problems if you crash and need to fsck - the number of files you could have on the file system could well require more memory/time than you have available. > I will not be using snapshots on this array. > > But I do absolutely need to run quotas (both user and > group) on this 9 GB array. I also need to successfully > use all quota admin tools (repquota, edquota, quota, > etc.) > > Can I get an assurance that this is totally safe, > sane, and fit to run in a mission critical, data > critical environment ? Anyone doing it currently ? > > Any comments or warnings of _any kind_ much appreciated. I don't think anyone will say 'I promise it will work' of course, but I would start by using the latest 6-STABLE source since there have been quite a number of updates to file system related code since 6.1. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology An undefined problem has an infinite number of solutions. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Wed Dec 20 15:43:36 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A355A16A514 for ; Wed, 20 Dec 2006 15:43:36 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [64.129.166.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 18D5343CBF for ; Wed, 20 Dec 2006 15:43:22 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.8/8.13.8) with ESMTP id kBKFhEgv086915; Wed, 20 Dec 2006 09:43:14 -0600 (CST) (envelope-from anderson@centtech.com) Message-ID: <45895A1D.2010105@centtech.com> Date: Wed, 20 Dec 2006 09:43:25 -0600 From: Eric Anderson User-Agent: Thunderbird 1.5.0.7 (X11/20061015) MIME-Version: 1.0 To: Arone Silimantia References: <626700.55454.qm@web58612.mail.re3.yahoo.com> In-Reply-To: <626700.55454.qm@web58612.mail.re3.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.88.4/2360/Wed Dec 20 00:24:09 2006 on mh1.centtech.com X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=8.0 tests=AWL,BAYES_00 autolearn=ham version=3.1.6 X-Spam-Checker-Version: SpamAssassin 3.1.6 (2006-10-03) on mh1.centtech.com Cc: freebsd-fs@freebsd.org Subject: Re: quotas safe on >2 TB filesystems in 6.1-RELEASE ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Dec 2006 15:43:36 -0000 On 12/20/06 09:15, Arone Silimantia wrote: > Eric, > > Thanks for your comments and help - your posts on this > list are much appreciated. Comments in line below: > > > --- Eric Anderson wrote: > >> With 9TB without >> any journaling, you might run into problems if you >> crash and need to >> fsck - the number of files you could have on the >> file system could well >> require more memory/time than you have available. > > > Hmmm...the time required is more dependent on inodes > than on size of data / size of files, right ? > > My 9 TB dataset uses about 36 million inodes. > > Any comments on that number ? Large ? Pedestrian ? > Typical ? Sounds like you have a lot of larger files (~250k per file on average possibly), which helps the fsck times. 36 million inodes should be fsck'able with enough memory (maybe ~3gb ish? That's a wild guess). I have two 10Tb file systems, one has 180million inodes. I don't attempt to fsck it, because it would take a very long time, and I might possibly run out of memory (I have 8Gb of memory). I use GJOURNAL on these, and am very happy about it (thanks Pawel!). df snippet: Filesystem 1K-blocks Used Avail Capacity iused ifree %iused Mounted on /dev/label/vol10-data.journal 9925732858 8780187028 351487202 96% 180847683 1102147515 14% /vol10 /dev/label/vol11-data.journal 9925732858 2598987846 6532686384 28% 49422705 1233572493 4% /vol11 I also have several 2Tb partitions (set up prior to gjournal being available), that are *FULL*, and each have about 25-45million inodes on them. Those fsck in about 4-7hours each, using between 1Gb and 3Gb of memory to do so. > > I am hoping that that could be fsck'd (modern hitachi > SATA drives, raid-6, 3ware) in 48 hours ... or am I > way off ? Depends on the drives really, and maybe caching. The geom_cache module (still beta probably, and not in the src tree currently I don't believe) is said to improve fsck times. 48 hours is a long time, and I *think* it should complete within that time frame, but you'd really have to test it to be sure. >>> But I do absolutely need to run quotas (both user >> and >>> group) on this 9 GB array. I also need to >> successfully >>> use all quota admin tools (repquota, edquota, >> quota, >>> etc.) >>> >>> Can I get an assurance that this is totally safe, >>> sane, and fit to run in a mission critical, data >>> critical environment ? Anyone doing it currently >> ? >>> Any comments or warnings of _any kind_ much >> appreciated. >> >> I don't think anyone will say 'I promise it will >> work' of course, but I >> would start by using the latest 6-STABLE source >> since there have been >> quite a number of updates to file system related >> code since 6.1. > > > Ok, but all of the CLI tools (edquota, repquota, > quota, quotacheck, quotaon) are all known-good for > "bigdisk" ? > > And there is no known "quotas just don't work with > bigdisk" problems ? > > I was hoping someone out there was running quotas with > 6.1-RELEASE on a >2TB filesystem and could report favorably... I'm not certain. There were some bugs in quotas, that recently were fixed (Kris Kennaway I think reported them and saw the fixes into the tree), and prior to that I saw those consistently, so I stopped using them. I haven't tried since the fixes have been in place, and the fixes (if I recall correctly) had to do with background fsck (softupdates maybe) and not the size of the disk. You could try this in a mock-up environment, by creating a sparse file and use it with mdconfig, then newfs it, enabling quotas, mount, and then use a script to create massive amounts (36million-ish) files of about 200k-ish in size, with random users, in a similar fashion as your real data, and see if all goes well. You can also try your fsck that way too. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology An undefined problem has an infinite number of solutions. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Wed Dec 20 17:28:13 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E7D3F16A511 for ; Wed, 20 Dec 2006 17:28:13 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from home.quip.cz (grimm.quip.cz [213.220.192.218]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6034043D73 for ; Wed, 20 Dec 2006 17:26:14 +0000 (GMT) (envelope-from 000.fbsd@quip.cz) Received: from [192.168.1.2] (qwork.quip.test [192.168.1.2]) by home.quip.cz (Postfix) with ESMTP id 05C89628F; Wed, 20 Dec 2006 17:52:29 +0100 (CET) Message-ID: <45896A4D.7040103@quip.cz> Date: Wed, 20 Dec 2006 17:52:29 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 X-Accept-Language: cz, cs, en, en-us MIME-Version: 1.0 References: <626700.55454.qm@web58612.mail.re3.yahoo.com> <45895A1D.2010105@centtech.com> In-Reply-To: <45895A1D.2010105@centtech.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, Arone Silimantia Subject: Re: quotas safe on >2 TB filesystems in 6.1-RELEASE ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Dec 2006 17:28:14 -0000 Eric Anderson wrote: [...] >> Hmmm...the time required is more dependent on inodes >> than on size of data / size of files, right ? >> >> My 9 TB dataset uses about 36 million inodes. >> >> Any comments on that number ? Large ? Pedestrian ? Typical ? > > > Sounds like you have a lot of larger files (~250k per file on average > possibly), which helps the fsck times. 36 million inodes should be > fsck'able with enough memory (maybe ~3gb ish? That's a wild guess). I > have two 10Tb file systems, one has 180million inodes. I don't attempt > to fsck it, because it would take a very long time, and I might possibly > run out of memory (I have 8Gb of memory). I use GJOURNAL on these, and > am very happy about it (thanks Pawel!). How can I calculate required memory? Some time ago I have problems with background fsck (snapshots?) on FreeBSD 6.0 with 1.2TB SCSI array (kernel panic). Then I set background_fsck="NO" and foreground fsck is running well. But I don't know how much memory is needed for: /dev/da1s1d 1132570308 514379258 561562536 48% 12456 146363222 0% /array0 real memory = 2147401728 (2047 MB) avail memory = 2096431104 (1999 MB) # swapinfo Device 1K-blocks Used Avail Capacity /dev/da0s1b 4194304 0 4194304 0% Can swap be used in time of fscking disc, or only "real" memory is used? Miroslav Lachman From owner-freebsd-fs@FreeBSD.ORG Wed Dec 20 18:12:39 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DBC7F16A403 for ; Wed, 20 Dec 2006 18:12:38 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [64.129.166.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 870BB43C9F for ; Wed, 20 Dec 2006 18:12:12 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.8/8.13.8) with ESMTP id kBKIC5rE014468; Wed, 20 Dec 2006 12:12:06 -0600 (CST) (envelope-from anderson@centtech.com) Message-ID: <45897D01.2030809@centtech.com> Date: Wed, 20 Dec 2006 12:12:17 -0600 From: Eric Anderson User-Agent: Thunderbird 1.5.0.7 (X11/20061015) MIME-Version: 1.0 To: Arone Silimantia References: <496085.15364.qm@web58616.mail.re3.yahoo.com> In-Reply-To: <496085.15364.qm@web58616.mail.re3.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.88.4/2361/Wed Dec 20 09:30:05 2006 on mh1.centtech.com X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=8.0 tests=AWL,BAYES_00 autolearn=ham version=3.1.6 X-Spam-Checker-Version: SpamAssassin 3.1.6 (2006-10-03) on mh1.centtech.com Cc: freebsd-fs@freebsd.org Subject: Re: quotas safe on >2 TB filesystems in 6.1-RELEASE ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Dec 2006 18:12:39 -0000 On 12/20/06 10:48, Arone Silimantia wrote: > --- Eric Anderson wrote: > > >> I also have several 2Tb partitions (set up prior to >> gjournal being >> available), that are *FULL*, and each have about >> 25-45million inodes on >> them. Those fsck in about 4-7hours each, using >> between 1Gb and 3Gb of >> memory to do so. > > > That's encouraging. What do you have your > kern.maxdsiz set to ? 1024000000 ? I used to set it very high - 2.5GB if I recall. > How about on the systems with the big arrays you were > talking about ? Or do you not bother with a large > maxdsiz there since you don't bother with fsck anyway > ? I don't set it anymore, and since switching to amd64 platform on this machine, I don't need it. I still use non-GJOURNALed 2Tb file systems, and two 10Tb GJOURNALed file systems on this system. With GJOURNAL, fsck takes a few seconds, and I'm online. >>> Ok, but all of the CLI tools (edquota, repquota, >>> quota, quotacheck, quotaon) are all known-good for >>> "bigdisk" ? >>> >>> And there is no known "quotas just don't work with >>> bigdisk" problems ? >>> >>> I was hoping someone out there was running quotas >> with >>> 6.1-RELEASE on a >2TB filesystem and could report >> favorably... >> >> I'm not certain. There were some bugs in quotas, >> that recently were >> fixed (Kris Kennaway I think reported them and saw >> the fixes into the >> tree), and prior to that I saw those consistently, >> so I stopped using >> them. I haven't tried since the fixes have been in >> place, and the fixes >> (if I recall correctly) had to do with background >> fsck (softupdates >> maybe) and not the size of the disk. > > > Yes, I was hoping Kris would respond to this thread > and give a thumbs up or down on quotas in 6.1-RELEASE > for >2TB disks. They work great with my 2 TB arrays, > but I need to go bigger now, and I don't know that I > can wait for 6.2-RELEASE to arrive... You don't have to wait for 6.2-RELEASE to start using the latest STABLE version of FreeBSD. Just download the most recent 6.2-RC image and go for it. Once 6.2-RELEASE comes out, you can just cvsup and rebuild, reboot, and you're rolling. The 6-STABLE branch is just that - stable. It isn't beta by any means, and I really recommend going past 6.1 to 6-STABLE. > I do not run snapshots in any form, and do not do > background fsck. Ok, well, the quota issues are easy to check with a quick sparse file backed disk.. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology An undefined problem has an infinite number of solutions. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Wed Dec 20 15:15:24 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CA28816A403 for ; Wed, 20 Dec 2006 15:15:24 +0000 (UTC) (envelope-from aronesimi@yahoo.com) Received: from web58612.mail.re3.yahoo.com (web58612.mail.re3.yahoo.com [68.142.236.210]) by mx1.FreeBSD.org (Postfix) with SMTP id 7160A43CB8 for ; Wed, 20 Dec 2006 15:15:18 +0000 (GMT) (envelope-from aronesimi@yahoo.com) Received: (qmail 56750 invoked by uid 60001); 20 Dec 2006 15:15:17 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=G0F1ygwdDhJGpmqmjFxRa+Q/iYCNi4C/SaWPkKdWwjB8D9jzqgn4BNW54hekzrDmvw/NxqQMi2vyk5ch+FHrTT12Gq3mJm0k4+crEb6qMwd/xOfz2QxqUjXSxBXupFlAkYYoufumtKHOJeNQS2iFBiikdDQX+aZ5/+LtVJi92bo=; X-YMail-OSG: wkyEY2IVM1mJfXsTw38ez5lLSFywIOIFAu4U7t7f40FtAbbUkjhVWsT8EShsMq71G9ZgzSPU6LcxHxlUd2qk3kxipR8DnzzOQNL1jPy9V2SHOZelH4.zeqjyp.mlfvMU3jQwoA_aNkRuuHVqV0ofmgGdoQTt0TO16ij_nCLq Received: from [75.72.230.91] by web58612.mail.re3.yahoo.com via HTTP; Wed, 20 Dec 2006 07:15:17 PST Date: Wed, 20 Dec 2006 07:15:17 -0800 (PST) From: Arone Silimantia To: Eric Anderson In-Reply-To: <4587E9D7.9050709@centtech.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <626700.55454.qm@web58612.mail.re3.yahoo.com> X-Mailman-Approved-At: Thu, 21 Dec 2006 12:33:46 +0000 Cc: freebsd-fs@freebsd.org Subject: Re: quotas safe on >2 TB filesystems in 6.1-RELEASE ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Dec 2006 15:15:24 -0000 Eric, Thanks for your comments and help - your posts on this list are much appreciated. Comments in line below: --- Eric Anderson wrote: > With 9TB without > any journaling, you might run into problems if you > crash and need to > fsck - the number of files you could have on the > file system could well > require more memory/time than you have available. Hmmm...the time required is more dependent on inodes than on size of data / size of files, right ? My 9 TB dataset uses about 36 million inodes. Any comments on that number ? Large ? Pedestrian ? Typical ? I am hoping that that could be fsck'd (modern hitachi SATA drives, raid-6, 3ware) in 48 hours ... or am I way off ? > > But I do absolutely need to run quotas (both user > and > > group) on this 9 GB array. I also need to > successfully > > use all quota admin tools (repquota, edquota, > quota, > > etc.) > > > > Can I get an assurance that this is totally safe, > > sane, and fit to run in a mission critical, data > > critical environment ? Anyone doing it currently > ? > > > > Any comments or warnings of _any kind_ much > appreciated. > > I don't think anyone will say 'I promise it will > work' of course, but I > would start by using the latest 6-STABLE source > since there have been > quite a number of updates to file system related > code since 6.1. Ok, but all of the CLI tools (edquota, repquota, quota, quotacheck, quotaon) are all known-good for "bigdisk" ? And there is no known "quotas just don't work with bigdisk" problems ? I was hoping someone out there was running quotas with 6.1-RELEASE on a >2TB filesystem and could report favorably... __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-fs@FreeBSD.ORG Wed Dec 20 16:49:00 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CE7D116A407 for ; Wed, 20 Dec 2006 16:49:00 +0000 (UTC) (envelope-from aronesimi@yahoo.com) Received: from web58616.mail.re3.yahoo.com (web58616.mail.re3.yahoo.com [68.142.236.250]) by mx1.FreeBSD.org (Postfix) with SMTP id 233E643CC0 for ; Wed, 20 Dec 2006 16:48:36 +0000 (GMT) (envelope-from aronesimi@yahoo.com) Received: (qmail 17496 invoked by uid 60001); 20 Dec 2006 16:48:36 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=ltZTGikafzidyphGw+oMvALVBCiHCIHGasa6+V9HTNds7zcOb31Rg13JQeBNbURKkKqX5vdvj3/HFPydrA/bWLM2Jy4Oc/hSf+vUudi0lvymyrdtHHj3aXnxpQnGXvRIBudxEF3aoQ3Vq5e+Fub5Wv8hlLBieyPvTVxeIrPg+gQ=; X-YMail-OSG: PWudGIgVM1m5kROMxyWRBMsBY31cYCxfchMdJ1Pq Received: from [75.72.230.91] by web58616.mail.re3.yahoo.com via HTTP; Wed, 20 Dec 2006 08:48:36 PST Date: Wed, 20 Dec 2006 08:48:36 -0800 (PST) From: Arone Silimantia To: Eric Anderson In-Reply-To: <45895A1D.2010105@centtech.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <496085.15364.qm@web58616.mail.re3.yahoo.com> X-Mailman-Approved-At: Thu, 21 Dec 2006 12:33:57 +0000 Cc: freebsd-fs@freebsd.org Subject: Re: quotas safe on >2 TB filesystems in 6.1-RELEASE ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Dec 2006 16:49:00 -0000 --- Eric Anderson wrote: > I also have several 2Tb partitions (set up prior to > gjournal being > available), that are *FULL*, and each have about > 25-45million inodes on > them. Those fsck in about 4-7hours each, using > between 1Gb and 3Gb of > memory to do so. That's encouraging. What do you have your kern.maxdsiz set to ? 1024000000 ? How about on the systems with the big arrays you were talking about ? Or do you not bother with a large maxdsiz there since you don't bother with fsck anyway ? > > Ok, but all of the CLI tools (edquota, repquota, > > quota, quotacheck, quotaon) are all known-good for > > "bigdisk" ? > > > > And there is no known "quotas just don't work with > > bigdisk" problems ? > > > > I was hoping someone out there was running quotas > with > > 6.1-RELEASE on a >2TB filesystem and could report > favorably... > > I'm not certain. There were some bugs in quotas, > that recently were > fixed (Kris Kennaway I think reported them and saw > the fixes into the > tree), and prior to that I saw those consistently, > so I stopped using > them. I haven't tried since the fixes have been in > place, and the fixes > (if I recall correctly) had to do with background > fsck (softupdates > maybe) and not the size of the disk. Yes, I was hoping Kris would respond to this thread and give a thumbs up or down on quotas in 6.1-RELEASE for >2TB disks. They work great with my 2 TB arrays, but I need to go bigger now, and I don't know that I can wait for 6.2-RELEASE to arrive... I do not run snapshots in any form, and do not do background fsck. Thanks! __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-fs@FreeBSD.ORG Sat Dec 23 02:55:42 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0DD8916A416; Sat, 23 Dec 2006 02:55:42 +0000 (UTC) (envelope-from nork@FreeBSD.org) Received: from sakura.ninth-nine.com (sakura.ninth-nine.com [219.127.74.120]) by mx1.freebsd.org (Postfix) with ESMTP id 878DB13C455; Sat, 23 Dec 2006 02:55:41 +0000 (UTC) (envelope-from nork@FreeBSD.org) Received: from nadesico.ninth-nine.com (nadesico.ninth-nine.com [219.127.74.122]) by sakura.ninth-nine.com (8.13.8/8.13.8/NinthNine) with SMTP id kBN2Hcp3055185; Sat, 23 Dec 2006 11:17:38 +0900 (JST) (envelope-from nork@FreeBSD.org) Date: Sat, 23 Dec 2006 11:17:38 +0900 From: Norikatsu Shigemura To: pjd@FreeBSD.org Message-Id: <20061223111738.6292605a.nork@FreeBSD.org> X-Mailer: Sylpheed version 2.3.0beta7 (GTK+ 2.10.6; i386-portbld-freebsd6.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.0.2 (sakura.ninth-nine.com [219.127.74.121]); Sat, 23 Dec 2006 11:17:38 +0900 (JST) Cc: freebsd-fs@FreeBSD.org Subject: [REPORT] Some panics on ZFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Dec 2006 02:55:42 -0000 Hi pjd! I noticed some problems on ZFS, so I report to you:-). 1. After using ZFS, when kldunload zfs, memory leaked about 8KB. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ZFS filesystem version 3 ZFS storage pool version 3 Warning: memory type solaris leaked memory on destroy (116 allocations, 7328 bytes leaked). ZFS filesystem version 3 ZFS storage pool version 3 Warning: memory type solaris leaked memory on destroy (116 allocations, 7328 bytes leaked). ZFS filesystem version 3 ZFS storage pool version 3 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - I think that this is not critical:-). 2. When not-kldload zfs, 'mount -t zfs tank /mnt' causes panic. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - panic: mutex Giant owned at /usr/src/sys/modules/zfs/../../contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c:232 cpuid = 0 KDB: enter: panic [thread pid 1126 tid 100147 ] Stopped at kdb_enter+0x30: leave db> bt Tracing pid 1126 tid 100147 td 0x851bed80 kdb_enter(806b5d1c,0,806b4fe7,fa2e96ac,851bed80,...) at kdb_enter+0x30 panic(806b4fe7,806b50bc,84c5e8e6,e8,852bb800,...) at panic+0x14e _mtx_assert(8072af88,2,84c5e8e6,e8,fa2e96f8,...) at _mtx_assert+0xfb vdev_geom_open(852bb800,fa2e9720,fa2e9728,175,806b653d,...) at vdev_geom_open+0x77 vdev_open(852bb800,0,0,852bb000,852bb000,...) at vdev_open+0x180 vdev_root_open(852bb000,fa2e9784,fa2e978c,806b653d,122,...) at vdev_root_open+0x42 vdev_open(852bb000,84c57fa7,0,0,804a89f0,...) at vdev_open+0x180 spa_load(1,0,2c,84868340,6,...) at spa_load+0x1d6 spa_open_common(84c57c13,0,84c57c13,fa2e9870,804bdbe1,...) at spa_open_common+0x15f dsl_dir_open_spa(0,84868340,84c57e3b,fa2e99d0,fa2e99d4,...) at dsl_dir_open_spa+0x62 dsl_dir_open(84868340,84c57e3b,fa2e99d0,fa2e99d4,e34,...) at dsl_dir_open+0x2e dsl_prop_get(84868340,84c5d09d,8,1,fa2e9a50,...) at dsl_prop_get+0x29 dsl_prop_get_integer(84868340,84c5d09d,fa2e9a50,0,1c7,...) at dsl_prop_get_integer+0x38 zfs_mount(853c57d4,851bed80,806bebc5,3ce,0,...) at zfs_mount+0x13a vfs_domount(851bed80,84bab450,84edc170,0,84bab9d0,...) at vfs_domount+0x77a vfs_donmount(851bed80,0,850e0b80,850e0b80,3,...) at vfs_donmount+0x4c9 nmount(851bed80,fa2e9d00,c,c,fa2e9d38,...) at nmount+0xaa syscall(fa2e9d38) at syscall+0x2e3 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x2, esp = 0x207, ebp = 0x7fbfd948 --- db> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Other case too, mount -a with /etc/fstab like following line: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - tank /mnt zfs rw 0 0 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - But 'kldload zfs; mount -t zfs tank /mnt' didn't panic. f(?_?)? I have a workaround, so I don't think critical problem:-). 3. When I'm testing with iozone, ZFS panic-ed. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x4 fault code = supervisor read, page not present instruction pointer = 0x20:0x84bef72b stack pointer = 0x28:0xfa0378c0 frame pointer = 0x28:0xfa037900 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 17326 (iozone) [thread pid 17326 tid 100174 ] Stopped at arc_release+0xab: cmpl %ecx,0x4(%eax) db> bt Tracing pid 17326 tid 100174 td 0x84ef3000 arc_release(a782935c,b36901e0,36,8072a3a0,0,...) at arc_release+0xab dbuf_dirty(b36901e0,d6922a00,e28c000,0,4000,...) at dbuf_dirty+0x3ec dmu_write(84697b80,1909b,0,e28c000,0,...) at dmu_write+0x8f zfs_write(fa037b90,0,806b3987,0,0,...) at zfs_write+0x6c2 VOP_WRITE_APV(84c61f00,fa037b90,84ef3000,806bfde7,23e,...) at VOP_WRITE_APV+0x14c vn_write(88e8b990,fa037c58,87c39e00,0,84ef3000,...) at vn_write+0x240 dofilewrite(84ef3000,3,88e8b990,fa037c58,ffffffff,...) at dofilewrite+0x85 kern_writev(84ef3000,3,fa037c58,8710000,30000,...) at kern_writev+0x60 write(84ef3000,fa037d00,c,34a,84ef3000,...) at write+0x4f syscall(fa037d38) at syscall+0x2e3 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (138674176), eip = 0x2, esp = 0x212, ebp = 0x401c8520 --- db> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Other case, but I think that this is current problem:-). - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - panic: mi_switch: switch in a critical section cpuid = 0 KDB: enter: panic [thread pid 2262 tid 100131 ] Stopped at kdb_enter+0x30: leave db> db> db> bt Tracing pid 2262 tid 100131 td 0x84f95510 kdb_enter(806b5d1c,0,806b66d6,f9fe3b7c,84f95510,...) at kdb_enter+0x30 panic(806b66d6,9,806b6699,15a,84f8d510,...) at panic+0x14e mi_switch(1,0,806b98ac,285,80730ed4,...) at mi_switch+0xdb turnstile_wait(8072ab08,84f8d510,0,166,84f8d512,...) at turnstile_wait+0x4c5 _mtx_lock_sleep(8072ab08,84f95510,0,806b653d,171,...) at _mtx_lock_sleep+0x18a _mtx_lock_flags(8072ab08,0,806b653d,171,93,...) at _mtx_lock_flags+0xc8 _sx_assert(85198cac,4,84f41a97,36,0,...) at _sx_assert+0x10b _sx_xunlock(85198cac,84f41a97,36,93,85198cac,...) at _sx_xunlock+0x28 cv_timedwait(85198d64,85198cac,1388,5f,0,...) at cv_timedwait+0xd2 txg_thread_wait(85198d64,5,85198d5c,85198d54,85198d64,...) at txg_thread_wait+0x3f txg_timelimit_thread(85198c00,f9fe3d38,806b2c0a,328,84f99900,...) at txg_timelimit_thread+0x75 fork_exit(84f083c0,85198c00,f9fe3d38) at fork_exit+0xd1 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xf9fe3d6c, ebp = 0 --- db> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - I saw some memory modified after free. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Memory modified after free 0x8f695c00(1020) val=deadc09e @ 0x8f695db0 panic: Most recently used by solaris cpuid = 1 KDB: enter: panic [thread pid 1412 tid 100240 ] Stopped at kdb_enter+0x30: leave db> bt Tracing pid 1412 tid 100240 td 0x85209360 kdb_enter(806b5d1c,1,806cfcfe,fa0df808,85209360,...) at kdb_enter+0x30 panic(806cfcfe,84c73b15,3fc,deadc09e,8f695db0,...) at panic+0x14e mtrash_ctor(8f695c00,400,0,2,0,...) at mtrash_ctor+0x65 uma_zalloc_arg(810675a0,0,2,8106ea00,88035000,...) at uma_zalloc_arg+0x12b malloc(240,84c7ab20,2,fa0df89c,84c0149e,...) at malloc+0xda kmem_alloc(240,2,88035000,8b261a00,fa0df8e4,...) at kmem_alloc+0x21 kmem_zalloc(240,2,84944840,fa0df8cc,804bdbe1,...) at kmem_zalloc+0x1e zio_create(77,0,8b261a00,90b82000,4000,...) at zio_create+0x2d zio_write(8b56ac00,88035000,6,2,1,...) at zio_write+0x77 arc_write(8b56ac00,88035000,6,2,1,...) at arc_write+0x105 dbuf_sync(90aaa000,8b56ac00,87dfd000,0,84c72bf8,...) at dbuf_sync+0x3ca dnode_sync(888ca478,0,8b56ac00,87dfd000,8b56ac00,...) at dnode_sync+0x51b dmu_objset_sync_dnodes(87dfd000,400,84c72b11,806b4d34,ae,...) at dmu_objset_sync_dnodes+0x97 dmu_objset_sync(8481c000,87dfd000,77,0,fa0dfbac,...) at dmu_objset_sync+0x60 dsl_dataset_sync(87be7600,87dfd000,0,850a25cc,850a2584,...) at dsl_dataset_sync+0x24 dsl_pool_sync(850a2400,77,0,1,0,...) at dsl_pool_sync+0x8f spa_sync(88035000,77,0,850a255c,850a2554,...) at spa_sync+0x453 txg_sync_thread(850a2400,fa0dfd38,806b2c0a,328,85234240,...) at txg_sync_thread+0x1d7 fork_exit(84c39d20,850a2400,fa0dfd38) at fork_exit+0xd1 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xfa0dfd6c, ebp = 0 --- db> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 4. Workaround for "kmem_malloc(X): kmem_map too small: Y total allocated" a. write "options KVA_PAGES=512" to kernel config b. write vm.kmem_size=1610612736 to /boot/loader.conf c. use over 1.5GB physical memory:-) I don't catch this type panic:-). But top(1) reported ... - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Mem: 35M Active, 20M Inact, 1401M Wired, 39M Buf, 44M Free - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Wow! From owner-freebsd-fs@FreeBSD.ORG Sat Dec 23 14:42:02 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 586E516A40F; Sat, 23 Dec 2006 14:42:02 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.freebsd.org (Postfix) with ESMTP id 4243E13C44E; Sat, 23 Dec 2006 14:41:59 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 2AB37487FE; Sat, 23 Dec 2006 15:24:20 +0100 (CET) Received: from localhost (cvl74.internetdsl.tpnet.pl [83.19.93.74]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id B7FDE45681; Sat, 23 Dec 2006 15:23:58 +0100 (CET) Date: Sat, 23 Dec 2006 15:23:32 +0100 From: Pawel Jakub Dawidek To: Norikatsu Shigemura Message-ID: <20061223142332.GA99259@garage.freebsd.pl> References: <20061223111738.6292605a.nork@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="RnlQjJ0d97Da+TV1" Content-Disposition: inline In-Reply-To: <20061223111738.6292605a.nork@FreeBSD.org> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=0.5 required=3.0 tests=BAYES_00,RCVD_IN_XBL autolearn=no version=3.0.4 Cc: freebsd-fs@FreeBSD.org Subject: Re: [REPORT] Some panics on ZFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Dec 2006 14:42:02 -0000 --RnlQjJ0d97Da+TV1 Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Dec 23, 2006 at 11:17:38AM +0900, Norikatsu Shigemura wrote: > Hi pjd! >=20 > I noticed some problems on ZFS, so I report to you:-). >=20 > 1. After using ZFS, when kldunload zfs, memory leaked about 8KB. > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -= - > ZFS filesystem version 3 > ZFS storage pool version 3 > Warning: memory type solaris leaked memory on destroy (116 allocations, 7= 328 bytes leaked). > ZFS filesystem version 3 > ZFS storage pool version 3 > Warning: memory type solaris leaked memory on destroy (116 allocations, = 7328 bytes leaked). > ZFS filesystem version 3 > ZFS storage pool version 3 > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -= - This one is already fixed. > 2. When not-kldload zfs, 'mount -t zfs tank /mnt' causes panic. > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -= - > panic: mutex Giant owned at /usr/src/sys/modules/zfs/../../contrib/openso= laris/uts/common/fs/zfs/vdev_geom.c:232 > cpuid =3D 0 > KDB: enter: panic > [thread pid 1126 tid 100147 ] > Stopped at kdb_enter+0x30: leave > db> bt > Tracing pid 1126 tid 100147 td 0x851bed80 > kdb_enter(806b5d1c,0,806b4fe7,fa2e96ac,851bed80,...) at kdb_enter+0x30 > panic(806b4fe7,806b50bc,84c5e8e6,e8,852bb800,...) at panic+0x14e > _mtx_assert(8072af88,2,84c5e8e6,e8,fa2e96f8,...) at _mtx_assert+0xfb > vdev_geom_open(852bb800,fa2e9720,fa2e9728,175,806b653d,...) at vdev_geom_= open+0x77 > vdev_open(852bb800,0,0,852bb000,852bb000,...) at vdev_open+0x180 > vdev_root_open(852bb000,fa2e9784,fa2e978c,806b653d,122,...) at vdev_root_= open+0x42 > vdev_open(852bb000,84c57fa7,0,0,804a89f0,...) at vdev_open+0x180 > spa_load(1,0,2c,84868340,6,...) at spa_load+0x1d6 > spa_open_common(84c57c13,0,84c57c13,fa2e9870,804bdbe1,...) at spa_open_co= mmon+0x15f > dsl_dir_open_spa(0,84868340,84c57e3b,fa2e99d0,fa2e99d4,...) at dsl_dir_op= en_spa+0x62 > dsl_dir_open(84868340,84c57e3b,fa2e99d0,fa2e99d4,e34,...) at dsl_dir_open= +0x2e > dsl_prop_get(84868340,84c5d09d,8,1,fa2e9a50,...) at dsl_prop_get+0x29 > dsl_prop_get_integer(84868340,84c5d09d,fa2e9a50,0,1c7,...) at dsl_prop_ge= t_integer+0x38 > zfs_mount(853c57d4,851bed80,806bebc5,3ce,0,...) at zfs_mount+0x13a > vfs_domount(851bed80,84bab450,84edc170,0,84bab9d0,...) at vfs_domount+0x7= 7a > vfs_donmount(851bed80,0,850e0b80,850e0b80,3,...) at vfs_donmount+0x4c9 > nmount(851bed80,fa2e9d00,c,c,fa2e9d38,...) at nmount+0xaa > syscall(fa2e9d38) at syscall+0x2e3 > Xint0x80_syscall() at Xint0x80_syscall+0x20 > --- syscall (7, FreeBSD ELF32, wait4), eip =3D 0x2, esp =3D 0x207, ebp = =3D 0x7fbfd948 --- > db>=20 > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -= - >=20 > Other case too, mount -a with /etc/fstab like following line: > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -= - > tank /mnt zfs rw 0 0 > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -= - >=20 > But 'kldload zfs; mount -t zfs tank /mnt' didn't panic. f(?_?)? It can be easly fixed, but bascially in zfs world you should mount file systems after loading zfs.ko via: # zfs mount -a and activate zvols: # zfs volinit > 3. When I'm testing with iozone, ZFS panic-ed. > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -= - > Fatal trap 12: page fault while in kernel mode > cpuid =3D 0; apic id =3D 00 > fault virtual address =3D 0x4 > fault code =3D supervisor read, page not present > instruction pointer =3D 0x20:0x84bef72b > stack pointer =3D 0x28:0xfa0378c0 > frame pointer =3D 0x28:0xfa037900 > code segment =3D base 0x0, limit 0xfffff, type 0x1b > =3D DPL 0, pres 1, def32 1, gran 1 > processor eflags =3D interrupt enabled, resume, IOPL =3D 0 > current process =3D 17326 (iozone) > [thread pid 17326 tid 100174 ] > Stopped at arc_release+0xab: cmpl %ecx,0x4(%eax) > db> bt > Tracing pid 17326 tid 100174 td 0x84ef3000 > arc_release(a782935c,b36901e0,36,8072a3a0,0,...) at arc_release+0xab > dbuf_dirty(b36901e0,d6922a00,e28c000,0,4000,...) at dbuf_dirty+0x3ec > dmu_write(84697b80,1909b,0,e28c000,0,...) at dmu_write+0x8f > zfs_write(fa037b90,0,806b3987,0,0,...) at zfs_write+0x6c2 > VOP_WRITE_APV(84c61f00,fa037b90,84ef3000,806bfde7,23e,...) at VOP_WRITE_A= PV+0x14c > vn_write(88e8b990,fa037c58,87c39e00,0,84ef3000,...) at vn_write+0x240 > dofilewrite(84ef3000,3,88e8b990,fa037c58,ffffffff,...) at dofilewrite+0x85 > kern_writev(84ef3000,3,fa037c58,8710000,30000,...) at kern_writev+0x60 > write(84ef3000,fa037d00,c,34a,84ef3000,...) at write+0x4f > syscall(fa037d38) at syscall+0x2e3 > Xint0x80_syscall() at Xint0x80_syscall+0x20 > --- syscall (138674176), eip =3D 0x2, esp =3D 0x212, ebp =3D 0x401c8520 -= -- > db>=20 > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -= - Interesting, I'll try to reproduce it. > Other case, but I think that this is current problem:-). > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -= - > panic: mi_switch: switch in a critical section > cpuid =3D 0 > KDB: enter: panic > [thread pid 2262 tid 100131 ] > Stopped at kdb_enter+0x30: leave > db>=20 > db>=20 > db> bt > Tracing pid 2262 tid 100131 td 0x84f95510 > kdb_enter(806b5d1c,0,806b66d6,f9fe3b7c,84f95510,...) at kdb_enter+0x30 > panic(806b66d6,9,806b6699,15a,84f8d510,...) at panic+0x14e > mi_switch(1,0,806b98ac,285,80730ed4,...) at mi_switch+0xdb > turnstile_wait(8072ab08,84f8d510,0,166,84f8d512,...) at turnstile_wait+0x= 4c5 > _mtx_lock_sleep(8072ab08,84f95510,0,806b653d,171,...) at _mtx_lock_sleep+= 0x18a > _mtx_lock_flags(8072ab08,0,806b653d,171,93,...) at _mtx_lock_flags+0xc8 > _sx_assert(85198cac,4,84f41a97,36,0,...) at _sx_assert+0x10b > _sx_xunlock(85198cac,84f41a97,36,93,85198cac,...) at _sx_xunlock+0x28 > cv_timedwait(85198d64,85198cac,1388,5f,0,...) at cv_timedwait+0xd2 > txg_thread_wait(85198d64,5,85198d5c,85198d54,85198d64,...) at txg_thread_= wait+0x3f > txg_timelimit_thread(85198c00,f9fe3d38,806b2c0a,328,84f99900,...) at txg_= timelimit_thread+0x75 > fork_exit(84f083c0,85198c00,f9fe3d38) at fork_exit+0xd1 > fork_trampoline() at fork_trampoline+0x8 > --- trap 0x1, eip =3D 0, esp =3D 0xf9fe3d6c, ebp =3D 0 --- > db>=20 > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -= - Not necessarily, it could be a bug in my Solaris compatible convar(9) based on sx(9) locks. > I saw some memory modified after free. > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -= - > Memory modified after free 0x8f695c00(1020) val=3Ddeadc09e @ 0x8f695db0 > panic: Most recently used by solaris >=20 > cpuid =3D 1 > KDB: enter: panic > [thread pid 1412 tid 100240 ] > Stopped at kdb_enter+0x30: leave > db> bt =20 > Tracing pid 1412 tid 100240 td 0x85209360 > kdb_enter(806b5d1c,1,806cfcfe,fa0df808,85209360,...) at kdb_enter+0x30 > panic(806cfcfe,84c73b15,3fc,deadc09e,8f695db0,...) at panic+0x14e > mtrash_ctor(8f695c00,400,0,2,0,...) at mtrash_ctor+0x65 > uma_zalloc_arg(810675a0,0,2,8106ea00,88035000,...) at uma_zalloc_arg+0x12b > malloc(240,84c7ab20,2,fa0df89c,84c0149e,...) at malloc+0xda > kmem_alloc(240,2,88035000,8b261a00,fa0df8e4,...) at kmem_alloc+0x21 > kmem_zalloc(240,2,84944840,fa0df8cc,804bdbe1,...) at kmem_zalloc+0x1e > zio_create(77,0,8b261a00,90b82000,4000,...) at zio_create+0x2d > zio_write(8b56ac00,88035000,6,2,1,...) at zio_write+0x77 > arc_write(8b56ac00,88035000,6,2,1,...) at arc_write+0x105 > dbuf_sync(90aaa000,8b56ac00,87dfd000,0,84c72bf8,...) at dbuf_sync+0x3ca > dnode_sync(888ca478,0,8b56ac00,87dfd000,8b56ac00,...) at dnode_sync+0x51b > dmu_objset_sync_dnodes(87dfd000,400,84c72b11,806b4d34,ae,...) at dmu_objs= et_sync_dnodes+0x97 > dmu_objset_sync(8481c000,87dfd000,77,0,fa0dfbac,...) at dmu_objset_sync+0= x60 > dsl_dataset_sync(87be7600,87dfd000,0,850a25cc,850a2584,...) at dsl_datase= t_sync+0x24 > dsl_pool_sync(850a2400,77,0,1,0,...) at dsl_pool_sync+0x8f > spa_sync(88035000,77,0,850a255c,850a2554,...) at spa_sync+0x453 > txg_sync_thread(850a2400,fa0dfd38,806b2c0a,328,85234240,...) at txg_sync_= thread+0x1d7 > fork_exit(84c39d20,850a2400,fa0dfd38) at fork_exit+0xd1 > fork_trampoline() at fork_trampoline+0x8 > --- trap 0x1, eip =3D 0, esp =3D 0xfa0dfd6c, ebp =3D 0 --- > db> > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -= - Are you able to setup memguard(9)? Or maybe you can give me procedure to reproduce it? > 4. Workaround for "kmem_malloc(X): kmem_map too small: Y total allocated" > a. write "options KVA_PAGES=3D512" to kernel config > b. write vm.kmem_size=3D1610612736 to /boot/loader.conf > c. use over 1.5GB physical memory:-) > I don't catch this type panic:-). But top(1) reported ... I implement quite good work-around few days ago, which should fix make it much harder to trigger this panic (I'm not able to do it anymore, but I don't think what I got is the real fix). Thanks for our feedback! I'll try to reproduce iozone panic, I'd also like to reproduce "Memory modified after free" and "mi_switch: switch in a critical section" panics, can you tell me how to do it? --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --RnlQjJ0d97Da+TV1 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFFjTvkForvXbEpPzQRAoTYAJ4sz+N1LacUQDK4rMlSNNfAp/13SwCfaeCJ 4Iv2K7aG/kwi2Sc57iTxvgc= =NxzQ -----END PGP SIGNATURE----- --RnlQjJ0d97Da+TV1--