From owner-freebsd-fs@FreeBSD.ORG Sun Sep 30 08:28:37 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BFD0E16A421; Sun, 30 Sep 2007 08:28:37 +0000 (UTC) (envelope-from howard@leadmon.net) Received: from ibm.leadmon.net (ibm.leadmon.net [207.114.24.13]) by mx1.freebsd.org (Postfix) with ESMTP id 7A79F13C47E; Sun, 30 Sep 2007 08:28:37 +0000 (UTC) (envelope-from howard@leadmon.net) Received: from HDLDesktop (hdl-desktop.leadmon.net [207.114.24.8]) (authenticated bits=0) by ibm.leadmon.net (8.14.1/8.14.1/LNSG+SCOP+NJABL+SBL+DSBL+SORBS+CBL+RHSBL) with ESMTP id l8U8EeHA019971; Sun, 30 Sep 2007 04:14:43 -0400 (EDT) (envelope-from howard@leadmon.net) X-DKIM: Sendmail DKIM Filter v2.2.1 ibm.leadmon.net l8U8EeHA019971 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/simple; d=leadmon.net; s=default; t=1191140083; bh=4erW1qSwDflfX0Ajmm1NNg+426c=; h=X-DomainKeys: DomainKey-Signature:Authentication-Results:X-SenderID:From:To:Cc: References:Subject:Date:Organization:Message-ID:MIME-Version: Content-Type:Content-Transfer-Encoding:X-Mailer:X-MimeOLE: Thread-Index:In-Reply-To; b=k8Kiwz/Q9JYdwvR8y4/XpfHKOIRtbV604w8ZUu LQFN8NYLLqoshYT3qXgMqW1mdssVq5VjaijWn9hooqZUD0dLYYISESZZ0L8imNoqjQc BqJdxjWQWXOoUowSQiaSE2vkQLTsyLgtBSmgNeZSbefMVw3AGDi0R6f581ald433aA= X-DomainKeys: Sendmail DomainKeys Filter v0.6.0 ibm.leadmon.net l8U8EeHA019971 DomainKey-Signature: a=rsa-sha1; s=default; d=leadmon.net; c=nofws; q=dns; h=authentication-results:x-senderid:from:to:cc:references: subject:date:organization:message-id:mime-version:content-type: content-transfer-encoding:x-mailer:x-mimeole:thread-index:in-reply-to; b=tBwy6zCfODV3dOk45iRuIut1b8yyENvCg6VVCX9kZSYt71gfzN/9rfQ1csxO/nnpE b+e/+waBy+7gLtb93ExueHXsotd6BLhYRgB3emM3XwYcAdBO8CatIpA9lN94VNXkNDo cq/dEA5m/FcwTSzFof8zf3CdVm9lPZRurWEvsD4= Authentication-Results: ibm.leadmon.net from=howard@leadmon.net; sender-id=pass; spf=pass X-SenderID: Sendmail Sender-ID Filter v0.2.14 ibm.leadmon.net l8U8EeHA019971 From: "Howard Leadmon" To: "'Kris Kennaway'" References: <46F7EDD7.6060904@psg.com> <46F84E94.8070706@FreeBSD.org> Date: Sun, 30 Sep 2007 04:14:35 -0400 Organization: Leadmon Networking Message-ID: <019301c80339$f34e6700$081872cf@Leadmon.local> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3138 Thread-Index: Acf/BpGKaAqvhuP0Qzi5T4HygVoevgEMolBg In-Reply-To: <46F84E94.8070706@FreeBSD.org> X-Virus-Scanned: ClamAV 0.91.2/4435/Sun Sep 30 03:32:39 2007 on ibm.leadmon.net X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: RE: zfs in production? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Sep 2007 08:28:37 -0000 Is ZFS available in amd64 yet? If so, is there any way to load a new server and use ZFS instead of UFS2 from the start? As I just have one RAID array on the servers I use, so not like I add additional drives in the future? Also, and maybe nobody knows, but I can't seem to get back on the -current mail list for some reason. I have subscribed to -fs, -stable, -performance, -i386, and even -amd64, all without a hitch, then I try and subscribe to -current and I never get any response at all back, I have sent in requests multiple time, and nothing. I can't figure out why I can't get on -current for love or money.. Just my luck.. --- Howard Leadmon > -----Original Message----- > From: owner-freebsd-fs@freebsd.org [mailto:owner-freebsd-fs@freebsd.org] On > Behalf Of Kris Kennaway > Sent: Monday, September 24, 2007 7:56 PM > To: Randy Bush > Cc: freebsd-fs@freebsd.org > Subject: Re: zfs in production? > > Randy Bush wrote: > > we are thinking of using zfs on a production server, using gmirror for > > booting and then following http://wiki.freebsd.org/ZFSOnRoot for the rest. > > > > but we would like to hear from folk using zfs in production for any > > length of time, as we do not really have the resources to be pioneers. > > > > thanks. > > > > randy > > _______________________________________________ > > freebsd-fs@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > > > > > I use it on a couple of heavily loaded servers. The only issues are > those I have posted about on current before. > > Kris > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Sun Sep 30 08:41:41 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C6E0616A41B; Sun, 30 Sep 2007 08:41:41 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: from hyperion.scode.org (cl-1361.ams-04.nl.sixxs.net [IPv6:2001:960:2:550::2]) by mx1.freebsd.org (Postfix) with ESMTP id 9A74E13C478; Sun, 30 Sep 2007 08:41:41 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: by hyperion.scode.org (Postfix, from userid 1001) id 17BB6239476; Sun, 30 Sep 2007 10:41:39 +0200 (CEST) Date: Sun, 30 Sep 2007 10:41:39 +0200 From: Peter Schuller To: Howard Leadmon Message-ID: <20070930084138.GA57688@hyperion.scode.org> References: <46F7EDD7.6060904@psg.com> <46F84E94.8070706@FreeBSD.org> <019301c80339$f34e6700$081872cf@Leadmon.local> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="BXVAT5kNtrzKuDFl" Content-Disposition: inline In-Reply-To: <019301c80339$f34e6700$081872cf@Leadmon.local> User-Agent: Mutt/1.5.16 (2007-06-09) Cc: freebsd-fs@freebsd.org Subject: Re: zfs in production? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Sep 2007 08:41:41 -0000 --BXVAT5kNtrzKuDFl Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable > Is ZFS available in amd64 yet?=20 Yes. In fact amd64 is preferred. > If so, is there any way to load a new server > and use ZFS instead of UFS2 from the start? If you do not intend to use ZFS for the root file system or similar, you can just install as usual making sure to reserve space/drives as appropriate. Then begin using ZFS after upgrading to -CURRENT. If you want root-on-ZFS you'll have to either install one one device and then do some acrobatics to convert the root file system, or boot =66rom some external media and perform a more custom installation (e.g., install a modern -CURRENT on a USB stick, then use 'make installworld', 'make distribution' etc to set up your system like you would a jail). > As I just have one RAID array on > the servers I use, so not like I add additional drives in the future? Note that you will loose some functionality (self-healing) if using ZFS on hardware RAID. --=20 / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller ' Key retrieval: Send an E-Mail to getpgpkey@scode.org E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org --BXVAT5kNtrzKuDFl Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFG/2FCDNor2+l1i30RAvrwAKCH6kFA7gIuUk1KOEPz9MNnQOuIZgCgoh+8 XzeX3eBUEll+Sk0TFe0HsUE= =oMLf -----END PGP SIGNATURE----- --BXVAT5kNtrzKuDFl-- From owner-freebsd-fs@FreeBSD.ORG Sun Sep 30 12:51:08 2007 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 93E5916A419 for ; Sun, 30 Sep 2007 12:51:08 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail18.syd.optusnet.com.au (mail18.syd.optusnet.com.au [211.29.132.199]) by mx1.freebsd.org (Postfix) with ESMTP id 4373A13C480 for ; Sun, 30 Sep 2007 12:51:07 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from besplex.bde.org (c220-239-235-248.carlnfd3.nsw.optusnet.com.au [220.239.235.248]) by mail18.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id l8UCowHe021789 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Sun, 30 Sep 2007 22:51:05 +1000 Date: Sun, 30 Sep 2007 22:50:58 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: fs@freebsd.org Message-ID: <20070930213402.Y1820@besplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Subject: fix for ffs_fsync() always writing to the disk X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Sep 2007 12:51:08 -0000 Please review this fix. Sync updates of inodes were apparently broken for about 15 years until they were fixed accidentally on 1998/03/08 in ffs_inode.c rev.1.36. ffs_fsync() asks for a sync update, but ffs_update() used to return immediately if IN_MODIFIED (or a timestamp flag) is not set. This early return is wrong in the sync case, since quite often, a previous call to ffs_fsync() will have cleared IN_MODIFIED and all timestamp flags when it copied the changes to the buffer; it wrote the buffer using a delayed write, but that write might not have completed. Now ffs_update() never returns early in the sync case (except for ro file systems and errors), so it always falls through to the code that writes the buffer synchronously. Thus fsync(2) on an ffs file system always does a synchronous write even if the file is only open ro and has not been written to since the file system was mounted. (It's interesting that fsync(1) only opens the file ro and that write permission is not required for fsync(2)). The early return was removed in ffs_inode.c rev.1.36, apparently just to support complications in the soft updates case, and this old bug got fixed accidentally for both the soft-updates case and the non-soft-updates cases as a side effect. In the soft updates case, IN_MODIFIED (or a timestamp flag) doesn't indicate all modifications, and a bwrite() is necessary for cleaning up the dependencies. The log message for 1.36 is null :-(. The bug has lived for another 9 years in file systems that were cloned from ffs before 1998. Detection of null changes (mainly for the sync case) gives a free optimization for the delayed-sync. Null changes can happen quite often because of limited resolution in timestamps. This is only a minor opimization because delayed sync is usually fast enough. Optimizing away null fsync()s is also a minor optimization because fsync() is rarely used. %%% Index: ffs_inode.c =================================================================== RCS file: /home/ncvs/src/sys/ufs/ffs/ffs_inode.c,v retrieving revision 1.92 diff -u -0 -r1.92 ffs_inode.c --- ffs_inode.c 7 Apr 2004 03:47:20 -0000 1.92 +++ ffs_inode.c 30 Sep 2007 12:41:03 -0000 @@ -79,0 +81,2 @@ + struct ufs1_dinode *din1p; + struct ufs2_dinode *din2p; @@ -114,7 +126,40 @@ - if (ip->i_ump->um_fstype == UFS1) - *((struct ufs1_dinode *)bp->b_data + - ino_to_fsbo(fs, ip->i_number)) = *ip->i_din1; - else - *((struct ufs2_dinode *)bp->b_data + - ino_to_fsbo(fs, ip->i_number)) = *ip->i_din2; - if (waitfor && !DOINGASYNC(vp)) { + /* + * Avoid physical writes of null changes if possible. Determining + * when this is possible is nontrivial. When (waitfor == 0), the + * simple check of IN_MODIFIED above suffices. Otherwise, we have + * to check for changes in both the inode and the buffer, since a + * previous (waitfor == 0) call here may have moved all changes to + * the buffer but the buffer might not have been written yet. + * + * Soft updates gives the additional complications that + * softdep_update_inodeblock() may have caused changes to the inode + * that are not reflected in IN_MODIFIED, and that a bwrite() is + * unavoidable for null changes if the buffer has dependencies, + * because soft updates is depending on us to call bwrite() so that + * it can remove the dependencies in its callback from bdone(). + */ + if (ip->i_ump->um_fstype == UFS1) { + din1p = (struct ufs1_dinode *)bp->b_data + + ino_to_fsbo(fs, ip->i_number); + if (bcmp(din1p, ip->i_din1, sizeof(*din1p)) == 0) { + if (waitfor == 0 || + ((bp->b_flags & B_DELWRI) == 0 && + LIST_FIRST(&bp->b_dep) == NULL)) { + bqrelse(bp); + return (0); + } + } else + *din1p = *ip->i_din1; + } else { + din2p = (struct ufs2_dinode *)bp->b_data + + ino_to_fsbo(fs, ip->i_number); + if (bcmp(din2p, ip->i_din2, sizeof(*din2p)) == 0) { + if (waitfor == 0 || + ((bp->b_flags & B_DELWRI) == 0 && + LIST_FIRST(&bp->b_dep) == NULL)) { + bqrelse(bp); + return (0); + } + } else + *din2p = *ip->i_din2; + } %%% This is ugly due to its large comment and complications for ffs1/2 and soft updates. Are the complications for soft updates complete? Bruce From owner-freebsd-fs@FreeBSD.ORG Sun Sep 30 19:24:02 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 07A0516A418 for ; Sun, 30 Sep 2007 19:24:02 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.freebsd.org (Postfix) with ESMTP id 92DA913C448 for ; Sun, 30 Sep 2007 19:24:01 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id E5BAC45E8F; Sun, 30 Sep 2007 21:23:59 +0200 (CEST) Received: from localhost (154.81.datacomsa.pl [195.34.81.154]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id A313D45684; Sun, 30 Sep 2007 21:23:55 +0200 (CEST) Date: Sun, 30 Sep 2007 21:23:52 +0200 From: Pawel Jakub Dawidek To: Andrei Kolu Message-ID: <20070930192352.GC30362@garage.freebsd.pl> References: <20070801122122.GA59065@harmless.hu> <1185979074.1264.14.camel@herring.rabson.org> <200708311115.19057.antik@bsd.ee> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="f0KYrhQ4vYSV2aJu" Content-Disposition: inline In-Reply-To: <200708311115.19057.antik@bsd.ee> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-2.6 required=3.0 tests=BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS - Port mapper failure - RPC: Timed out X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Sep 2007 19:24:02 -0000 --f0KYrhQ4vYSV2aJu Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 31, 2007 at 11:15:18AM +0300, Andrei Kolu wrote: > I followed this quick guide http://wiki.freebsd.org/ZFSQuickStartGuide an= d=20 > encountered following problem: >=20 > # zfs snapshot tank/ufs@31082007 > # mount > /dev/ad4s1a on / (ufs, local) > devfs on /dev (devfs, local) > /dev/ad4s1e on /tmp (ufs, local, soft-updates) > /dev/ad4s1f on /usr (ufs, local, soft-updates) > /dev/ad4s1d on /var (ufs, local, soft-updates) > tank on /tank (zfs, local, noatime) > tank/usr on /tank/usr (zfs, local, noatime) > tank/usr/ports on /tank/usr/ports (zfs, local, noatime) > tank/usr/ports/distfiles on /tank/usr/ports/distfiles (zfs, local, noatim= e) > /dev/zvol/tank/ufs on /ufs (ufs, local) >=20 > # mount -r /dev/zvol/tank/ufs@31082007 /ufs31082007 > mount: /ufs31082007: No such file or directory > # mkdir /ufs31082007 > # mount -r /dev/zvol/tank/ufs@31082007 /ufs31082007 > mount_nfs: path@server syntax is deprecated, use server:path Our mount(8) have problems with understading @. Try: mount -t ufs -r /dev/zvol/tank/ufs@31082007 /ufs31082007 --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --f0KYrhQ4vYSV2aJu Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFG//fIForvXbEpPzQRAgdDAJ4kJN6uddhoIBWrOw0NnADfUa2x9wCggy+d 4Vr6V8txwuN0VN4/r/y3q5Q= =dwAH -----END PGP SIGNATURE----- --f0KYrhQ4vYSV2aJu-- From owner-freebsd-fs@FreeBSD.ORG Mon Oct 1 08:14:09 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1624116A552 for ; Mon, 1 Oct 2007 08:14:09 +0000 (UTC) (envelope-from liste@encephalon.de) Received: from digitiminimi.de (digitiminimi.de [217.172.44.71]) by mx1.freebsd.org (Postfix) with ESMTP id BFAFA13C459 for ; Mon, 1 Oct 2007 08:14:08 +0000 (UTC) (envelope-from liste@encephalon.de) Received: from localhost (digitiminimi.de [217.172.44.71]) by digitiminimi.de (Postfix) with ESMTP id E99CB9B5B7 for ; Mon, 1 Oct 2007 10:15:52 +0200 (CEST) X-Virus-Scanned: amavisd-new at digitiminimi.de Received: from digitiminimi.de ([217.172.44.71]) by localhost (main.digitiminimi.de [217.172.44.71]) (amavisd-new, port 10024) with ESMTP id 9VJhKfLN+B5Y for ; Mon, 1 Oct 2007 10:15:46 +0200 (CEST) Received: from cojote.suedfac.com (sffwd2.suedfactoring.de [212.202.224.253]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by digitiminimi.de (Postfix) with ESMTP id 828319B49E for ; Mon, 1 Oct 2007 10:15:29 +0200 (CEST) Message-ID: <4700A791.6020606@encephalon.de> Date: Mon, 01 Oct 2007 09:53:53 +0200 From: "Axel S. Gruner" User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 7bit Subject: zpool does not work X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2007 08:14:09 -0000 Hi, FreeBSD CURRENT from Fri Sep 28: #zpool create zeta raidz da1 da2 da3 da4 Assertion failed: (P???P???), function rv == 0, file /usr/src/cddl/sbin/zpool/../../../contrib/opensolaris/cmd/zpool/zpool_vdev.c, line 131. Abort (core dumped) What could be the Problem? Cheers, Axel From owner-freebsd-fs@FreeBSD.ORG Mon Oct 1 09:11:16 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9EB7E16A421 for ; Mon, 1 Oct 2007 09:11:16 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.freebsd.org (Postfix) with ESMTP id 3875413C43E for ; Mon, 1 Oct 2007 09:11:15 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 913C245E8F; Mon, 1 Oct 2007 11:11:13 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id BB0E2456AB; Mon, 1 Oct 2007 11:11:08 +0200 (CEST) Date: Mon, 1 Oct 2007 11:11:06 +0200 From: Pawel Jakub Dawidek To: "Axel S. Gruner" Message-ID: <20071001091106.GB35764@garage.freebsd.pl> References: <4700A791.6020606@encephalon.de> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="WYTEVAkct0FjGQmd" Content-Disposition: inline In-Reply-To: <4700A791.6020606@encephalon.de> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: zpool does not work X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2007 09:11:16 -0000 --WYTEVAkct0FjGQmd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Oct 01, 2007 at 09:53:53AM +0200, Axel S. Gruner wrote: > Hi, >=20 > FreeBSD CURRENT from Fri Sep 28: >=20 > #zpool create zeta raidz da1 da2 da3 da4 > Assertion failed: (P???P???), function rv =3D=3D 0, file > /usr/src/cddl/sbin/zpool/../../../contrib/opensolaris/cmd/zpool/zpool_vde= v.c, > line 131. > Abort (core dumped) >=20 > What could be the Problem? It fails on geom_gettree(). Something seems to be wrong with your GEOM topology, most likely some strange provider name that use characters reserved for XML. Can you paste the output of 'sysctl -b kern.geom.confxml'? --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --WYTEVAkct0FjGQmd Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHALmqForvXbEpPzQRArsKAKDtGji9h5ks8KgDDhTEP2KbH2yjfgCfaF8I H9LG5D4uDOn7k/cq7J6lvSk= =TH0h -----END PGP SIGNATURE----- --WYTEVAkct0FjGQmd-- From owner-freebsd-fs@FreeBSD.ORG Mon Oct 1 09:23:05 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3D01A16A420 for ; Mon, 1 Oct 2007 09:23:05 +0000 (UTC) (envelope-from liste@encephalon.de) Received: from digitiminimi.de (digitiminimi.de [217.172.44.71]) by mx1.freebsd.org (Postfix) with ESMTP id DC69E13C48E for ; Mon, 1 Oct 2007 09:23:03 +0000 (UTC) (envelope-from liste@encephalon.de) Received: from localhost (digitiminimi.de [217.172.44.71]) by digitiminimi.de (Postfix) with ESMTP id 3025A9B497; Mon, 1 Oct 2007 11:44:38 +0200 (CEST) X-Virus-Scanned: amavisd-new at digitiminimi.de Received: from digitiminimi.de ([217.172.44.71]) by localhost (main.digitiminimi.de [217.172.44.71]) (amavisd-new, port 10024) with ESMTP id 7E0EslQIZRXd; Mon, 1 Oct 2007 11:44:31 +0200 (CEST) Received: from cojote.suedfac.com (sffwd2.suedfactoring.de [212.202.224.253]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by digitiminimi.de (Postfix) with ESMTP id EEF539B495; Mon, 1 Oct 2007 11:44:22 +0200 (CEST) Message-ID: <4700BC65.1000303@encephalon.de> Date: Mon, 01 Oct 2007 11:22:45 +0200 From: "Axel S. Gruner" User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4700A791.6020606@encephalon.de> <20071001091106.GB35764@garage.freebsd.pl> In-Reply-To: <20071001091106.GB35764@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 8bit Cc: freebsd-fs@freebsd.org Subject: Re: zpool does not work X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2007 09:23:05 -0000 Pawel Jakub Dawidek schrieb: > On Mon, Oct 01, 2007 at 09:53:53AM +0200, Axel S. Gruner wrote: >> Hi, >> >> FreeBSD CURRENT from Fri Sep 28: >> >> #zpool create zeta raidz da1 da2 da3 da4 >> Assertion failed: (P???P???), function rv == 0, file >> /usr/src/cddl/sbin/zpool/../../../contrib/opensolaris/cmd/zpool/zpool_vdev.c, >> line 131. >> Abort (core dumped) >> >> What could be the Problem? > > It fails on geom_gettree(). Something seems to be wrong with your GEOM > topology, most likely some strange provider name that use characters > reserved for XML. > > Can you paste the output of 'sysctl -b kern.geom.confxml'? Thanks for the quick reply. Here it is: ZFS::VDEV ZFS::ZVOL FD fd0 1 r0w0e0 fd0 0 0 ACD acd0 1 r0w0e0 acd0 8796093020160 2048 DEV msdosfs/ 5 r0w0e0 ntfs/Lokaler Datentr0/00ger 5 r0w0e0 da4s8 4 r0w0e0 da4s7 4 r0w0e0 da4s6 4 r0w0e0 da4s5 4 r0w0e0 da3s8 4 r0w0e0 da3s7 4 r0w0e0 da3s6 4 r0w0e0 da3s5 4 r0w0e0 da0s1d 4 r0w0e0 da0s1c 4 r0w0e0 da0s1b 4 r0w0e0 da0s1a 4 r0w0e0 da4s2 3 r0w0e0 da4s1 3 r0w0e0 da3s2 3 r0w0e0 da3s1 3 r0w0e0 da1s1 3 r0w0e0 da0s1 3 r0w0e0 da4 2 r0w0e0 da3 2 r0w0e0 da2 2 r0w0e0 da1 2 r0w0e0 da0 2 r0w0e0 acd0 2 r0w0e0 fd0 2 r0w0e0 BSD da0s1 3 512 32256 32256 r3w3e3 r1w1e1 da0s1d 68033657344 512 3 68033657344 132878237 5368709120 10485760 7 r0w0e0 da0s1c 73402366464 512 2 73402366464 143363997 0 0 0 r1w1e0 da0s1b 4294967296 512 1 4294967296 8388608 1073741824 2097152 1 r1w1e1 da0s1a 1073741824 512 0 1073741824 2097152 0 0 7 PART LABEL da3s6 4 r0w0e0 r0w0e0 msdosfs/ 49319424 512 0 49319424 96327 0 0 da3s5 4 r0w0e0 r0w0e0 ntfs/Lokaler Datentr0/00ger 35648331264 512 0 35648331264 69625647 0 0 VFS ffs.da0s1d 4 r1w1e1 ffs.da0s1a 4 r1w1e1 MBR da4 2 r0w0e0 r0w0e0 da4s2 36692974080 512 1 36692974080 71665965 36701199360 71682030 5 r0w0e0 da4s1 36701167104 512 0 36701167104 71681967 32256 63 7 da3 2 r0w0e0 r0w0e0 da3s2 36692974080 512 1 36692974080 71665965 36701199360 71682030 5 r0w0e0 da3s1 36701167104 512 0 36701167104 71681967 32256 63 7 da1 2 r0w0e0 r0w0e0 da1s1 73402366464 512 0 73402366464 143363997 32256 63 165 da0 2 r3w3e4 r3w3e3 da0s1 73402366464 512 0 73402366464 143363997 32256 63 165 MBREXT da4s2 3 r0w0e0 r0w0e0 da4s8 789594624 512 3 789594624 1542177 35903379456 70123788 131 r0w0e0 da4s7 205599744 512 2 205599744 401562 35697747456 69722163 130 r0w0e0 da4s6 49319424 512 1 49319424 96327 35648395776 69625773 11 r0w0e0 da4s5 35648331264 512 0 35648331264 69625647 32256 63 7 da3s2 3 r0w0e0 r0w0e0 da3s8 789594624 512 3 789594624 1542177 35903379456 70123788 131 r0w0e0 da3s7 205599744 512 2 205599744 401562 35697747456 69722163 130 r0w0e0 da3s6 49319424 512 1 49319424 96327 35648395776 69625773 11 r0w0e0 da3s5 35648331264 512 0 35648331264 69625647 32256 63 7 MD DISK da4 1 r0w0e0 da4 73407488000 512 255 63 da3 1 r0w0e0 da3 73407488000 512 255 63 da2 1 r0w0e0 da2 73407488000 512 255 63 da1 1 r0w0e0 da1 73407488000 512 255 63 da0 1 r3w3e4 da0 73407488000 512 255 63 SWAP swap 4 r1w1e0 From owner-freebsd-fs@FreeBSD.ORG Mon Oct 1 09:31:54 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8750A16A417 for ; Mon, 1 Oct 2007 09:31:54 +0000 (UTC) (envelope-from liste@encephalon.de) Received: from digitiminimi.de (digitiminimi.de [217.172.44.71]) by mx1.freebsd.org (Postfix) with ESMTP id 2287213C447 for ; Mon, 1 Oct 2007 09:31:54 +0000 (UTC) (envelope-from liste@encephalon.de) Received: from localhost (digitiminimi.de [217.172.44.71]) by digitiminimi.de (Postfix) with ESMTP id 93E869B496; Mon, 1 Oct 2007 11:53:28 +0200 (CEST) X-Virus-Scanned: amavisd-new at digitiminimi.de Received: from digitiminimi.de ([217.172.44.71]) by localhost (main.digitiminimi.de [217.172.44.71]) (amavisd-new, port 10024) with ESMTP id v1Iyx+IVb5p7; Mon, 1 Oct 2007 11:53:22 +0200 (CEST) Received: from cojote.suedfac.com (ns2.suedfactoring.de [212.202.224.253]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by digitiminimi.de (Postfix) with ESMTP id DBC4E9B406; Mon, 1 Oct 2007 11:53:21 +0200 (CEST) Message-ID: <4700BE80.5080109@encephalon.de> Date: Mon, 01 Oct 2007 11:31:44 +0200 From: "Axel S. Gruner" User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4700A791.6020606@encephalon.de> <20071001091106.GB35764@garage.freebsd.pl> <4700BC65.1000303@encephalon.de> In-Reply-To: <4700BC65.1000303@encephalon.de> Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: zpool does not work X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2007 09:31:54 -0000 Hi, Axel S. Gruner schrieb: > Pawel Jakub Dawidek schrieb: >> On Mon, Oct 01, 2007 at 09:53:53AM +0200, Axel S. Gruner wrote: >>> Hi, >>> >>> FreeBSD CURRENT from Fri Sep 28: >>> >>> #zpool create zeta raidz da1 da2 da3 da4 >>> Assertion failed: (P???P???), function rv == 0, file >>> /usr/src/cddl/sbin/zpool/../../../contrib/opensolaris/cmd/zpool/zpool_vdev.c, >>> line 131. >>> Abort (core dumped) >>> >>> What could be the Problem? >> It fails on geom_gettree(). Something seems to be wrong with your GEOM >> topology, most likely some strange provider name that use characters >> reserved for XML. >> >> Can you paste the output of 'sysctl -b kern.geom.confxml'? I think this could be the problem: ntfs/Lokaler Datentr0/00ger Two of the disks are NTFS formatted. After fdisk these disks, i can create the zpool. It would be nice if i can also create a zpool on these disk without first fdisk/format these. Or, maybe an error message which points me to the problem. Cheers, Axel From owner-freebsd-fs@FreeBSD.ORG Mon Oct 1 10:24:43 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D3F9016A421; Mon, 1 Oct 2007 10:24:43 +0000 (UTC) (envelope-from db@nipsi.de) Received: from fop.bsdsystems.de (mx.bsdsystems.de [88.198.57.43]) by mx1.freebsd.org (Postfix) with ESMTP id 8D8D813C455; Mon, 1 Oct 2007 10:24:43 +0000 (UTC) (envelope-from db@nipsi.de) Received: from [172.16.1.13] (e176099133.adsl.alicedsl.de [85.176.99.133]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by fop.bsdsystems.de (Postfix) with ESMTP id 882A8259C8; Mon, 1 Oct 2007 11:56:29 +0200 (CEST) Message-ID: <4700C441.30100@nipsi.de> Date: Mon, 01 Oct 2007 11:56:17 +0200 From: db User-Agent: Thunderbird 2.0.0.6 (Windows/20070728) MIME-Version: 1.0 To: "Axel S. Gruner" References: <4700A791.6020606@encephalon.de> <20071001091106.GB35764@garage.freebsd.pl> <4700BC65.1000303@encephalon.de> <4700BE80.5080109@encephalon.de> In-Reply-To: <4700BE80.5080109@encephalon.de> Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, Pawel Jakub Dawidek Subject: Re: zpool does not work X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2007 10:24:43 -0000 We reported this months ago... :-( http://www.freebsd.org/cgi/query-pr.cgi?pr=104389&cat= could someone please verify and commit this fix? best regards, Dennis Axel S. Gruner schrieb: > Hi, > > Axel S. Gruner schrieb: > >> Pawel Jakub Dawidek schrieb: >> >>> On Mon, Oct 01, 2007 at 09:53:53AM +0200, Axel S. Gruner wrote: >>> >>>> Hi, >>>> >>>> FreeBSD CURRENT from Fri Sep 28: >>>> >>>> #zpool create zeta raidz da1 da2 da3 da4 >>>> Assertion failed: (P???P???), function rv == 0, file >>>> /usr/src/cddl/sbin/zpool/../../../contrib/opensolaris/cmd/zpool/zpool_vdev.c, >>>> line 131. >>>> Abort (core dumped) >>>> >>>> What could be the Problem? >>>> >>> It fails on geom_gettree(). Something seems to be wrong with your GEOM >>> topology, most likely some strange provider name that use characters >>> reserved for XML. >>> >>> Can you paste the output of 'sysctl -b kern.geom.confxml'? >>> > > I think this could be the problem: > > ntfs/Lokaler Datentr0/00ger > > Two of the disks are NTFS formatted. After fdisk these disks, i can > create the zpool. > > It would be nice if i can also create a zpool on these disk without > first fdisk/format these. Or, maybe an error message which points me to > the problem. > > Cheers, > > Axel > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Mon Oct 1 11:08:24 2007 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5C2DA16A417 for ; Mon, 1 Oct 2007 11:08:24 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 3FE9B13C455 for ; Mon, 1 Oct 2007 11:08:24 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.1/8.14.1) with ESMTP id l91B8OW4064451 for ; Mon, 1 Oct 2007 11:08:24 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.1/8.14.1/Submit) id l91B8NWk064447 for freebsd-fs@FreeBSD.org; Mon, 1 Oct 2007 11:08:23 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 1 Oct 2007 11:08:23 GMT Message-Id: <200710011108.l91B8NWk064447@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to you X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2007 11:08:24 -0000 Current FreeBSD problem reports Critical problems Serious problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o kern/114856 fs [ntfs] [patch] Bug in NTFS allows bogus file modes. o kern/116170 fs Kernel panic when mounting /tmp 4 problems total. Non-critical problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/114847 fs [ntfs] [patch] dirmask support for NTFS ala MSDOSFS 1 problem total. From owner-freebsd-fs@FreeBSD.ORG Mon Oct 1 12:05:18 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C9D7616A41A; Mon, 1 Oct 2007 12:05:18 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id 77FE813C480; Mon, 1 Oct 2007 12:05:18 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (localhost [127.0.0.1]) by spam.des.no (Postfix) with ESMTP id 2D74C2102; Mon, 1 Oct 2007 14:05:09 +0200 (CEST) X-Spam-Tests: AWL X-Spam-Learn: disabled X-Spam-Score: 0.0/3.0 X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on tim.des.no Received: from ds4.des.no (des.no [80.203.243.180]) by smtp.des.no (Postfix) with ESMTP id 9B9CC2155; Mon, 1 Oct 2007 14:05:08 +0200 (CEST) Received: by ds4.des.no (Postfix, from userid 1001) id 5D5C6844A8; Mon, 1 Oct 2007 14:00:24 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: db References: <4700A791.6020606@encephalon.de> <20071001091106.GB35764@garage.freebsd.pl> <4700BC65.1000303@encephalon.de> <4700BE80.5080109@encephalon.de> <4700C441.30100@nipsi.de> Date: Mon, 01 Oct 2007 14:00:24 +0200 In-Reply-To: <4700C441.30100@nipsi.de> (db@nipsi.de's message of "Mon\, 01 Oct 2007 11\:56\:17 +0200") Message-ID: <861wcf82p3.fsf@ds4.des.no> User-Agent: Gnus/5.110006 (No Gnus v0.6) Emacs/22.1 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, Pawel Jakub Dawidek Subject: Re: zpool does not work X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2007 12:05:18 -0000 db writes: > We reported this months ago... :-( > > http://www.freebsd.org/cgi/query-pr.cgi?pr=3D104389&cat=3D > > could someone please verify and commit this fix? Both patches in the PR are incorrect. The second patch is better than the first, but still gets the logic wrong and completely misses the point of using an sbuf in the first place. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-fs@FreeBSD.ORG Mon Oct 1 12:16:47 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0F91016A417 for ; Mon, 1 Oct 2007 12:16:47 +0000 (UTC) (envelope-from rmilkowski@task.gda.pl) Received: from bojer.task.gda.pl (bojer.task.gda.pl [153.19.250.250]) by mx1.freebsd.org (Postfix) with ESMTP id B122313C49D for ; Mon, 1 Oct 2007 12:16:46 +0000 (UTC) (envelope-from rmilkowski@task.gda.pl) Received: from localhost (localhost [127.0.0.1]) by bojer.task.gda.pl (Postfix) with ESMTP id 8C2FABF43; Mon, 1 Oct 2007 13:57:47 +0200 (CEST) X-Virus-Scanned: amavisd-new at task.gda.pl Received: from bojer.task.gda.pl ([127.0.0.1]) by localhost (bojer.task.gda.pl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hnaiER+YdaTn; Mon, 1 Oct 2007 13:57:06 +0200 (CEST) Received: from localhost (e1-1.ns500-1.ts.milt.as9105.net [212.74.112.53]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by bojer.task.gda.pl (Postfix) with ESMTP id 19BACBF03; Mon, 1 Oct 2007 13:57:06 +0200 (CEST) Date: Mon, 1 Oct 2007 12:57:05 +0100 From: Robert Milkowski X-Mailer: The Bat! (v3.71.03) Professional Organization: CI TASK http://www.task.gda.pl X-Priority: 3 (Normal) Message-ID: <14950433.20071001125705@task.gda.pl> To: Neil Perrin In-Reply-To: <46FC316A.8010004@Sun.COM> References: <20070920115621.GF4517@garage.freebsd.pl> <8B5FB4B1-2398-491C-95F4-E79361606916@stromnet.se> <20070921142540.GB5690@garage.freebsd.pl> <18167.56892.707010.947405@gargle.gargle.HOWL> <46FC316A.8010004@Sun.COM> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, zfs-discuss@opensolaris.org, Roch - PAE Subject: Re[2]: [zfs-discuss] ZFS (and quota) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Robert Milkowski List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2007 12:16:47 -0000 Hello Neil, Thursday, September 27, 2007, 11:40:42 PM, you wrote: NP> Roch - PAE wrote: >> Pawel Jakub Dawidek writes: >> > I'm CCing zfs-discuss@opensolaris.org, as this doesn't look like >> > FreeBSD-specific problem. >> > >> > It looks there is a problem with block allocation(?) when we are near >> > quota limit. tank/foo dataset has quota set to 10m: >> > >> > Without quota: >> > >> > FreeBSD: >> > # dd if=/dev/zero of=/tank/test bs=512 count=20480 >> > time: 0.7s >> > >> > Solaris: >> > # dd if=/dev/zero of=/tank/test bs=512 count=20480 >> > time: 4.5s >> > >> > With quota: >> > >> > FreeBSD: >> > # dd if=/dev/zero of=/tank/foo/test bs=512 count=20480 >> > dd: /tank/foo/test: Disc quota exceeded >> > time: 306.5s >> > >> > Solaris: >> > # dd if=/dev/zero of=/tank/foo/test bs=512 count=20480 >> > write: Disc quota exceeded >> > time: 602.7s >> > >> > CPU is almost entirely idle, but disk activity seems to be high. >> > >> >> >> Yes, as we are near quota limit, each transaction group >> will accept a small amount as to not overshoot the limit. >> >> I don't know if we have the optimal strategy yet. >> >> -r NP> Aside from the quota perf issue, has any analysis been done as to NP> why FreeBSD is over 6X faster than Solaris without quotas? NP> Do other perf tests show a similar disparity? NP> Is there a difference in dd itself? NP> I assume that it was identical hardware and pool config. Good question. It should have been cached on Solaris so it should definitely be below 1s range. -- Best regards, Robert mailto:rmilkowski@task.gda.pl http://milek.blogspot.com From owner-freebsd-fs@FreeBSD.ORG Mon Oct 1 12:35:08 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3CD2016A418 for ; Mon, 1 Oct 2007 12:35:08 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.freebsd.org (Postfix) with ESMTP id 8BEDC13C457 for ; Mon, 1 Oct 2007 12:35:07 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id A401E45E90; Mon, 1 Oct 2007 14:35:05 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 6F22645683; Mon, 1 Oct 2007 14:34:59 +0200 (CEST) Date: Mon, 1 Oct 2007 14:34:57 +0200 From: Pawel Jakub Dawidek To: Robert Milkowski Message-ID: <20071001123457.GE35764@garage.freebsd.pl> References: <20070920115621.GF4517@garage.freebsd.pl> <8B5FB4B1-2398-491C-95F4-E79361606916@stromnet.se> <20070921142540.GB5690@garage.freebsd.pl> <18167.56892.707010.947405@gargle.gargle.HOWL> <46FC316A.8010004@Sun.COM> <14950433.20071001125705@task.gda.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="dFWYt1i2NyOo1oI9" Content-Disposition: inline In-Reply-To: <14950433.20071001125705@task.gda.pl> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org, Neil Perrin , Roch - PAE , zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] ZFS (and quota) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2007 12:35:08 -0000 --dFWYt1i2NyOo1oI9 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Oct 01, 2007 at 12:57:05PM +0100, Robert Milkowski wrote: > Hello Neil, >=20 > Thursday, September 27, 2007, 11:40:42 PM, you wrote: >=20 >=20 > NP> Roch - PAE wrote: > >> Pawel Jakub Dawidek writes: > >> > I'm CCing zfs-discuss@opensolaris.org, as this doesn't look like > >> > FreeBSD-specific problem. > >> >=20 > >> > It looks there is a problem with block allocation(?) when we are ne= ar > >> > quota limit. tank/foo dataset has quota set to 10m: > >> >=20 > >> > Without quota: > >> >=20 > >> > FreeBSD: > >> > # dd if=3D/dev/zero of=3D/tank/test bs=3D512 count=3D20480 > >> > time: 0.7s > >> >=20 > >> > Solaris: > >> > # dd if=3D/dev/zero of=3D/tank/test bs=3D512 count=3D20480 > >> > time: 4.5s > >> >=20 > >> > With quota: > >> >=20 > >> > FreeBSD: > >> > # dd if=3D/dev/zero of=3D/tank/foo/test bs=3D512 count=3D20480 > >> > dd: /tank/foo/test: Disc quota exceeded > >> > time: 306.5s > >> >=20 > >> > Solaris: > >> > # dd if=3D/dev/zero of=3D/tank/foo/test bs=3D512 count=3D20480 > >> > write: Disc quota exceeded > >> > time: 602.7s > >> >=20 > >> > CPU is almost entirely idle, but disk activity seems to be high. > >> >=20 > >>=20 > >>=20 > >> Yes, as we are near quota limit, each transaction group > >> will accept a small amount as to not overshoot the limit. > >>=20 > >> I don't know if we have the optimal strategy yet. > >>=20 > >> -r >=20 > NP> Aside from the quota perf issue, has any analysis been done as to > NP> why FreeBSD is over 6X faster than Solaris without quotas? > NP> Do other perf tests show a similar disparity? > NP> Is there a difference in dd itself? > NP> I assume that it was identical hardware and pool config. (I don't see this e-mail in my ZFS inbox, that's why I'm replaying to Robert's e-mail.) Just to clarify. This was entirely different hardware. My e-mail was __only__ about quota performance in ZFS. Please, do not try to use those results for any other purpose. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --dFWYt1i2NyOo1oI9 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHAOlxForvXbEpPzQRAoIIAKCp+7TKzdo00Q5SkDg0DrQFe0kn8QCg3DlO 5S3v2IR2pS9pn4+cckSD3WQ= =pxmb -----END PGP SIGNATURE----- --dFWYt1i2NyOo1oI9-- From owner-freebsd-fs@FreeBSD.ORG Mon Oct 1 12:35:21 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A47C616A418 for ; Mon, 1 Oct 2007 12:35:21 +0000 (UTC) (envelope-from db@nipsi.de) Received: from fop.bsdsystems.de (mx.bsdsystems.de [88.198.57.43]) by mx1.freebsd.org (Postfix) with ESMTP id 5FE2A13C469 for ; Mon, 1 Oct 2007 12:35:21 +0000 (UTC) (envelope-from db@nipsi.de) Received: from [172.16.1.13] (e176099039.adsl.alicedsl.de [85.176.99.39]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by fop.bsdsystems.de (Postfix) with ESMTP id 4AC85259C8; Mon, 1 Oct 2007 14:35:20 +0200 (CEST) Message-ID: <4700E97A.1040704@nipsi.de> Date: Mon, 01 Oct 2007 14:35:06 +0200 From: db User-Agent: Thunderbird 2.0.0.6 (Windows/20070728) MIME-Version: 1.0 To: =?UTF-8?B?RGFnLUVybGluZyBTbcO4cmdyYXY=?= References: <4700A791.6020606@encephalon.de> <20071001091106.GB35764@garage.freebsd.pl> <4700BC65.1000303@encephalon.de> <4700BE80.5080109@encephalon.de> <4700C441.30100@nipsi.de> <861wcf82p3.fsf@ds4.des.no> In-Reply-To: <861wcf82p3.fsf@ds4.des.no> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-fs@freebsd.org Subject: Re: zpool does not work X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2007 12:35:21 -0000 Dag-Erling Smørgrav schrieb: > db writes: > >> We reported this months ago... :-( >> >> http://www.freebsd.org/cgi/query-pr.cgi?pr=104389&cat= >> >> could someone please verify and commit this fix? >> > > Both patches in the PR are incorrect. The second patch is better than > the first, but still gets the logic wrong and completely misses the > point of using an sbuf in the first place. > > DES > I meant this one http://www.freebsd.org/cgi/query-pr.cgi?prp=104389-3-diff regards, Dennis From owner-freebsd-fs@FreeBSD.ORG Mon Oct 1 13:52:13 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 957CB16A419 for ; Mon, 1 Oct 2007 13:52:13 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id 4E56613C480 for ; Mon, 1 Oct 2007 13:52:13 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (localhost [127.0.0.1]) by spam.des.no (Postfix) with ESMTP id B72642084; Mon, 1 Oct 2007 15:52:04 +0200 (CEST) X-Spam-Tests: AWL X-Spam-Learn: disabled X-Spam-Score: 0.0/3.0 X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on tim.des.no Received: from ds4.des.no (des.no [80.203.243.180]) by smtp.des.no (Postfix) with ESMTP id 441B92082; Mon, 1 Oct 2007 15:52:04 +0200 (CEST) Received: by ds4.des.no (Postfix, from userid 1001) id 264B6844A4; Mon, 1 Oct 2007 15:52:04 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Dennis Berger References: <4700A791.6020606@encephalon.de> <20071001091106.GB35764@garage.freebsd.pl> <4700BC65.1000303@encephalon.de> <4700BE80.5080109@encephalon.de> <4700C441.30100@nipsi.de> <861wcf82p3.fsf@ds4.des.no> <4700E97A.1040704@nipsi.de> Date: Mon, 01 Oct 2007 15:52:04 +0200 In-Reply-To: <4700E97A.1040704@nipsi.de> (db@nipsi.de's message of "Mon\, 01 Oct 2007 14\:35\:06 +0200") Message-ID: <86ve9r3ptn.fsf@ds4.des.no> User-Agent: Gnus/5.110006 (No Gnus v0.6) Emacs/22.1 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: zpool does not work X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2007 13:52:13 -0000 Dennis Berger writes: > Dag-Erling Sm=C3=B8rgrav writes: > > Both patches in the PR are incorrect. The second patch is better than > > the first, but still gets the logic wrong and completely misses the > > point of using an sbuf in the first place. > I meant this one > http://www.freebsd.org/cgi/query-pr.cgi?prp=3D104389-3-diff "still gets the logic wrong and completely misses the point of using an sbuf in the first place" DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-fs@FreeBSD.ORG Tue Oct 2 12:52:01 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9417216A468 for ; Tue, 2 Oct 2007 12:52:01 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.freebsd.org (Postfix) with ESMTP id 259E613C458 for ; Tue, 2 Oct 2007 12:52:00 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id A986845F45; Tue, 2 Oct 2007 14:51:58 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 0314745F43; Tue, 2 Oct 2007 14:51:51 +0200 (CEST) Date: Tue, 2 Oct 2007 14:51:48 +0200 From: Pawel Jakub Dawidek To: db Message-ID: <20071002125148.GI1693@garage.freebsd.pl> References: <4700A791.6020606@encephalon.de> <20071001091106.GB35764@garage.freebsd.pl> <4700BC65.1000303@encephalon.de> <4700BE80.5080109@encephalon.de> <4700C441.30100@nipsi.de> <861wcf82p3.fsf@ds4.des.no> <4700E97A.1040704@nipsi.de> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="LHvWgpbS7VDUdu2f" Content-Disposition: inline In-Reply-To: <4700E97A.1040704@nipsi.de> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org, Dag-Erling Sm??rgrav Subject: Re: zpool does not work X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2007 12:52:01 -0000 --LHvWgpbS7VDUdu2f Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Oct 01, 2007 at 02:35:06PM +0200, db wrote: > Dag-Erling Sm??rgrav schrieb: > >db writes: > > =20 > >>We reported this months ago... :-( > >> > >>http://www.freebsd.org/cgi/query-pr.cgi?pr=3D104389&cat=3D > >> > >>could someone please verify and commit this fix? > >> =20 > > > >Both patches in the PR are incorrect. The second patch is better than > >the first, but still gets the logic wrong and completely misses the > >point of using an sbuf in the first place. > > > >DES > > =20 > I meant this one > http://www.freebsd.org/cgi/query-pr.cgi?prp=3D104389-3-diff I fully agree that this should be fixed, but you should use sbuf() directly instead of sprintf()s. Could you another patch with this fixed? It will also eliminate direct memory allocation. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --LHvWgpbS7VDUdu2f Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHAj7kForvXbEpPzQRAv/MAJwJ7b6Y5x92t1kwJC1VsoPAE5JqigCeMo5K gIx1Ye6aHLewM5oRNWHQBrY= =UZ2k -----END PGP SIGNATURE----- --LHvWgpbS7VDUdu2f-- From owner-freebsd-fs@FreeBSD.ORG Wed Oct 3 18:50:14 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C14B516A418 for ; Wed, 3 Oct 2007 18:50:14 +0000 (UTC) (envelope-from softsearch@gmail.com) Received: from fk-out-0910.google.com (fk-out-0910.google.com [209.85.128.189]) by mx1.freebsd.org (Postfix) with ESMTP id 4A85A13C45D for ; Wed, 3 Oct 2007 18:50:14 +0000 (UTC) (envelope-from softsearch@gmail.com) Received: by fk-out-0910.google.com with SMTP id b27so5475436fka for ; Wed, 03 Oct 2007 11:50:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:date:from:reply-to:organization:x-priority:message-id:to:subject:mime-version:content-type:content-transfer-encoding; bh=PSvjf3V9BkoItlRB1o3SIjM1+mHIOqLEJNwBVCMJXhg=; b=RIInBn9PaeZsTqZUMW/WBoCNgni9UI1oJ0j4rFBycvHr62DG8Gm2Y5R7qw41oBWoBAAcrCr7ZBS80Rp9Zm457H9/jgMUQJMgza6o7+Jz/42k464q1bORz890bEqs/rFEV4W73HHruV3Ta48ZGwza9YBrOcvTVVUw0EffluN00AM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:date:from:reply-to:organization:x-priority:message-id:to:subject:mime-version:content-type:content-transfer-encoding; b=lhiV6OumV33EKrSCSRxGY7vo044ZgU75WCgwQIxHQG4OvuNV4YNzMnZHzKrBntT3nudMeOXcMve3H4rpnVWYjLlm7KBWNbQTlca9k8SZLxXkpK0H9esYiTxsqdSwQGJL+3khLkQOQhwV/2IofUpMAdg6lTGWxLdDf7nisI4DqVw= Received: by 10.82.158.12 with SMTP id g12mr1362513bue.1191435886551; Wed, 03 Oct 2007 11:24:46 -0700 (PDT) Received: from ?81.200.123.77? ( [81.200.123.77]) by mx.google.com with ESMTPS id k5sm4635778nfh.2007.10.03.11.24.43 (version=SSLv3 cipher=OTHER); Wed, 03 Oct 2007 11:24:45 -0700 (PDT) Date: Wed, 3 Oct 2007 22:24:07 +0400 From: Michael Monashev Organization: SoftSearch.ru X-Priority: 3 (Normal) Message-ID: <682935622.20071003222407@gmail.com> To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: zfs cache size X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Michael Monashev List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2007 18:50:14 -0000 Hi, freebsd-fs. How can I see/change zfs cache size? -- Michael Monashev From owner-freebsd-fs@FreeBSD.ORG Wed Oct 3 18:52:26 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 91D9916A419 for ; Wed, 3 Oct 2007 18:52:26 +0000 (UTC) (envelope-from softsearch@gmail.com) Received: from fk-out-0910.google.com (fk-out-0910.google.com [209.85.128.189]) by mx1.freebsd.org (Postfix) with ESMTP id 06FCA13C468 for ; Wed, 3 Oct 2007 18:52:25 +0000 (UTC) (envelope-from softsearch@gmail.com) Received: by fk-out-0910.google.com with SMTP id b27so5476082fka for ; Wed, 03 Oct 2007 11:52:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:date:from:reply-to:organization:x-priority:message-id:to:subject:mime-version:content-type:content-transfer-encoding; bh=CRr9VumsVdd1SP2cnQQhtD0YVCT+w4aIWJ9z95Wogek=; b=VokdoWmPKSBuYQXbPI/U/A7nLBis7+0GEEMAKX+CAIHvBH6FsavaCehn14/dqfOOw1nUpmetr3iYDBBvugBVfTrDY0samVMgdT+9E1OhnxqW/qW2ch6dJxDaz6Ux2p5p2EvKVVZxlBEG+/3YU4rmsAIWVbGuHq68IEaXjIna3OM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:date:from:reply-to:organization:x-priority:message-id:to:subject:mime-version:content-type:content-transfer-encoding; b=XPuRBQki6O3SA43xUEfz0fXHvw+qY4Ivf9qFszFW4OnrmsuwbkftQl5w9PcuFuLUQWJlmbXouCM/gPRI7QbaSVJBu9g17xk/dhc7yymuj7rBTeiHTsJltHkHmIzWne8tNG8V1van5a70tO/Ifh/YYWDuY0x7t2lXArnMMsbhcF4= Received: by 10.82.172.10 with SMTP id u10mr21706707bue.1191435889299; Wed, 03 Oct 2007 11:24:49 -0700 (PDT) Received: from ?81.200.123.77? ( [81.200.123.77]) by mx.google.com with ESMTPS id h6sm4607019nfh.2007.10.03.11.24.47 (version=SSLv3 cipher=OTHER); Wed, 03 Oct 2007 11:24:48 -0700 (PDT) Date: Wed, 3 Oct 2007 22:24:10 +0400 From: Michael Monashev Organization: SoftSearch.ru X-Priority: 3 (Normal) Message-ID: <787432347.20071003222410@gmail.com> To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: zfs and top X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Michael Monashev List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2007 18:52:26 -0000 Hi, freebsd-fs. I have spurious state in top for IO-heavy process: last pid: 2318; load averages: 0.18, 0.13, 0.12 up 0+06:02:44 19:57:20 57 processes: 2 running, 55 sleeping CPU states: 0.4% user, 0.0% nice, 5.1% system, 1.8% interrupt, 92.6% idle Mem: 281M Active, 1761M Inact, 1673M Wired, 56M Cache, 214M Buf, 141M Free Swap: 4096M Total, 1344K Used, 4095M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 2052 www 1 96 0 43272K 38208K zfs:(& 1 0:16 0.10% nginx 2054 www 1 96 0 43272K 38220K zfs:(& 1 0:15 0.05% nginx 2059 www 1 96 0 43272K 38224K zfs:(& 1 0:15 0.05% nginx 2062 www 1 96 0 43272K 38220K zfs:(& 0 0:14 0.05% nginx 2053 www 1 96 0 43272K 38220K zfs:(& 0 0:16 0.00% nginx 2058 www 1 96 0 43272K 38208K zfs:(& 1 0:16 0.00% nginx 2065 www 1 96 0 43272K 38220K zfs:(& 1 0:16 0.00% nginx 2063 www 1 96 0 43272K 38224K zfs:(& 0 0:16 0.00% nginx 2050 www 1 96 0 43272K 38208K zfs:(& 0 0:16 0.00% nginx 2048 www 1 96 0 43272K 38212K zfs:(& 0 0:16 0.00% nginx 2051 www 1 96 0 43272K 38208K zfs:(& 0 0:16 0.00% nginx 2060 www 1 96 0 43272K 38212K zfs:(& 1 0:15 0.00% nginx 2055 www 1 96 0 43272K 38224K zfs:(& 0 0:15 0.00% nginx 2057 www 1 96 0 43272K 38224K zfs:(& 1 0:15 0.00% nginx 2046 www 1 96 0 43272K 38208K zfs:(& 0 0:15 0.00% nginx 2061 www 1 96 0 43272K 38216K zfs:(& 0 0:14 0.00% nginx 2056 www 1 96 0 43272K 38208K zfs:(& 0 0:14 0.00% nginx 2049 www 1 96 0 43272K 38204K CPU0 0 0:14 0.00% nginx 2047 www 1 96 0 43272K 38208K zfs:(& 1 0:13 0.00% nginx 2064 www 1 96 0 43272K 38208K zfs:(& 1 0:13 0.00% nginx I patch ps ( thanx Ruslan Ermilov ): /usr/src/bin/ps/keyword.c - {"wchan", "WCHAN", NULL, LJUST, wchan, NULL, 6, 0, CHAR, NULL, 0}, + {"wchan", "WCHAN", NULL, LJUST, wchan, NULL, 16, 0, CHAR, NULL, 0}, But WCHAN is still spurious :-( > ps ax -o pid,ppid,%cpu,vsz,wchan,command | egrep '(nginx|PID)' PID PPID %CPU VSZ WCHAN COMMAND 846 1 0.0 36104 pause nginx: master process /usr/local/sbin/nginx 2046 846 0.0 43272 zfs:(&zi nginx: worker process (nginx) 2047 846 0.0 43272 zfs:(&zi nginx: worker process (nginx) 2048 846 0.0 43272 zfs:(&zi nginx: worker process (nginx) 2049 846 0.0 43272 zfs:(&zi nginx: worker process (nginx) 2050 846 0.0 43272 zfs:(&zi nginx: worker process (nginx) 2051 846 0.0 43272 zfs:(&zi nginx: worker process (nginx) 2052 846 0.0 43272 zfs:(&zi nginx: worker process (nginx) 2053 846 0.0 43272 zfs:(&zi nginx: worker process (nginx) 2054 846 0.0 43272 zfs:(&zi nginx: worker process (nginx) 2055 846 0.0 43272 zfs:(&zi nginx: worker process (nginx) 2056 846 0.0 43272 zfs:(&zi nginx: worker process (nginx) 2057 846 0.0 43272 zfs:(&zi nginx: worker process (nginx) 2058 846 0.0 43272 zfs:(&zi nginx: worker process (nginx) 2059 846 0.0 43272 zfs:(&zi nginx: worker process (nginx) 2060 846 0.0 43272 zfs:(&zi nginx: worker process (nginx) 2061 846 0.0 43272 zfs:(&zi nginx: worker process (nginx) 2062 846 0.0 43272 zfs nginx: worker process (nginx) 2063 846 0.0 43272 zfs:(&zi nginx: worker process (nginx) 2064 846 0.0 43272 zfs:(&zi nginx: worker process (nginx) 2065 846 0.0 43272 zfs:(&zi nginx: worker process (nginx) > How to see factual process state? -- Michael Monashev From owner-freebsd-fs@FreeBSD.ORG Thu Oct 4 09:02:50 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4AAD016A41B for ; Thu, 4 Oct 2007 09:02:50 +0000 (UTC) (envelope-from undelivery@le20news.com) Received: from ns34011.ovh.net (ns34011.ovh.net [213.251.169.25]) by mx1.freebsd.org (Postfix) with SMTP id 5BE0E13C481 for ; Thu, 4 Oct 2007 09:02:48 +0000 (UTC) (envelope-from undelivery@le20news.com) Received: (qmail 357 invoked by uid 0); 4 Oct 2007 08:17:19 -0000 Message-ID: <20071004081719.24220.qmail@ns34011.ovh.net> To: freebsd-fs@freebsd.org Date: Thu, 04 Oct 2007 10:05:19 +0200 From: "Emmanuel" X-Mailer-ListID: 45 Content-Transfer-Encoding: 7bit MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: =?iso-8859-1?q?Vins_de_Prestige_=E0_prix_imbattables_=3A_CLOS_DE?= =?iso-8859-1?q?_VOUGEOT_Grand_Cru_Vieilles_Vignes_1996?= X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: infos@le20news.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2007 09:02:50 -0000 [logo_le20_MAJ.gif] [1][chrono_anim.gif] Si vous n'arrivez pas à lire ce message, [2]cliquez ici 0 805 100 995 (appel gratuit depuis un fixe) [3]accueil [4]qui sommes nous ? [5]nous contacter [6][header_011007_CV.jpg] [7]Les meilleurs vins de Bourgogne, Vallée du Rhône et Provence [8]Côtes du Rhône - Sablet Blanc - Le Souverain prix par bouteille, vendu par 6 2006 Blanc Vallée du Rhône 5,65 EUR prix public [DEL: :DEL] [DEL: 9,00 EUR :DEL] économisez 38% [9]Côtes du Rhône Village - Séguret Rosé - Le Souverain prix par bouteille, vendu par 6 2006 Rosé Vallée du Rhône 5,90 EUR prix public [DEL: :DEL] [DEL: 9,00 EUR :DEL] économisez 35% [10]Coteaux d'Aix-en-Provence Rosé Terre d'Aixpressions - Les Vignerons du Roy René prix par bouteille, vendu par 3 2006 Rosé Vallée du Rhône 5,90 EUR prix public [DEL: :DEL] [DEL: 8,00 EUR :DEL] [11]Côtes du Rhône Villages Beaumes de Venise - Remy FERBRAS prix par bouteille, vendu par 6 2001 Rouge Vallée du Rhône 6,35 EUR prix public [DEL: :DEL] [DEL: 10,00 EUR :DEL] économisez 37% [12]Côtes du Rhône CAIRANNE - Remy FERBRAS prix par bouteille, vendu par 6 2001 Rouge Vallée du Rhône 6,51 EUR prix public [DEL: :DEL] [DEL: 8,00 EUR :DEL] [13]Lirac - Les Garrigues - Domaine Les Garriques prix par bouteille, vendu par 6 2004 Rouge Vallée du Rhône 6,95 EUR prix public [DEL: :DEL] [DEL: 12,00 EUR :DEL] économisez 43% [14]Costières-de-Nimes Hippolyte - Château Grande Cassagne prix par bouteille, vendu par 6 2005 Blanc Vallée du Rhône 6,95 EUR prix public [DEL: :DEL] [DEL: 13,00 EUR :DEL] économisez 47% [15]Côtes du Rhône - Domaine MAZURD prix par bouteille, vendu par 6 2004 Rouge Vallée du Rhône 7,00 EUR prix public [DEL: :DEL] [DEL: 7,50 EUR :DEL] [16]Côtes de Provence Rosé Cuvée Saint Honorat - Château Les Mesclances prix par bouteille, vendu par 6 2006 Rosé Vallée du Rhône 7,00 EUR prix public [DEL: :DEL] [DEL: 9,00 EUR :DEL] [17]Côtes du Rhône Villages Plan de Dieu - Château Saint-Jean prix par bouteille, vendu par 6 2004 Rouge Vallée du Rhône 7,37 EUR prix public [DEL: :DEL] [DEL: 10,00 EUR :DEL] [18]BOURGOGNE ALIGOTE - Antoine de Courchelle prix par bouteille, vendu par 6 2004 Blanc Bourgogne 7,50 EUR prix public [DEL: :DEL] [DEL: 11,00 EUR :DEL] économisez 32% [19]Côtes du Rhône Cuvée Rouge Plaisir - Domaine MAZURD prix par bouteille, vendu par 6 2000 Rouge Vallée du Rhône 7,55 EUR prix public [DEL: :DEL] [DEL: 10,00 EUR :DEL] [20]Côtes du Rhône Blanc sec Cuvée Saint Christophe - Domaine de Cassan prix par bouteille, vendu par 6 2005 Blanc Vallée du Rhône 7,95 EUR prix public [DEL: :DEL] [DEL: 10,00 EUR :DEL] [21]Côtes du Rhône CAIRANNE TEMPTATION - Cave de Cairanne prix par bouteille, vendu par 6 1996 Rouge Vallée du Rhône 7,95 EUR prix public [DEL: :DEL] [DEL: 27,00 EUR :DEL] économisez 71% [22]Côtes du Rhône Villages Beaumes de Venise - Château Redortier prix par bouteille, vendu par 3 2003 Rouge Vallée du Rhône 8,00 EUR prix public [DEL: :DEL] [DEL: 11,00 EUR :DEL] [23]Vin Biologique Côtes du Rhône Villages VISAN Le Garrigon - COUSTON Frère et soeur prix par bouteille, vendu par 6 2004 Rouge Vallée du Rhône 8,50 EUR prix public [DEL: :DEL] [DEL: 11,00 EUR :DEL] [24]Côtes du Rhône - Domaine de la Valériane prix par bouteille, vendu par 6 2004 Rouge Vallée du Rhône 8,60 EUR prix public [DEL: :DEL] [DEL: 10,00 EUR :DEL] [25]Lirac La Fermade - Domaine MABY prix par bouteille, vendu par 6 2005 Rouge Vallée du Rhône 9,00 EUR prix public [DEL: :DEL] [DEL: 12,00 EUR :DEL] [26]Vin Biologique Côtes du Rhône Le Claux ROCHE BUISSIERE - Antoine et Pierre JOLY 2005 Rouge Vallée du Rhône 9,00 EUR prix public [DEL: :DEL] [DEL: 10,00 EUR :DEL] [27]TAVEL Rosé - Château de Trinquevedel prix par bouteille, vendu par 6 2006 Rosé Vallée du Rhône 9,24 EUR prix public [DEL: :DEL] [DEL: 13,00 EUR :DEL] [28]Rasteau - Domaine Rabasse Charavin - Corinne COUTURIER prix par bouteille, vendu par 3 2003 Rouge Vallée du Rhône 9,24 EUR prix public [DEL: :DEL] [DEL: 11,00 EUR :DEL] [29]Gigondas - Domaine St François Xavier prix par bouteille, vendu par 6 2005 Rouge Vallée du Rhône 9,84 EUR prix public [DEL: :DEL] [DEL: 14,00 EUR :DEL] économisez 30% [30]Côtes du Rhône - Sablet - Domaine de Verquière prix par bouteille, vendu par 6 2001 Rouge Vallée du Rhône 9,95 EUR prix public [DEL: :DEL] [DEL: 12,00 EUR :DEL] [31]BANDOL Rosé - Château de la Noblesse prix par bouteille, vendu par 6 2005 Rosé Vallée du Rhône 9,95 EUR prix public [DEL: :DEL] [DEL: 14,00 EUR :DEL] [32]Crozes Hermitage Blanc - CAVE DE TAIN L'HERMITAGE prix par bouteille, vendu par 3 2005 Blanc Vallée du Rhône 9,95 EUR prix public [DEL: :DEL] [DEL: 16,00 EUR :DEL] économisez 38% [33]Rasteau - Domaine Rabasse Charavin - Corinne COUTURIER 2001 Rouge Vallée du Rhône 10,00 EUR prix public [DEL: :DEL] [DEL: 11,00 EUR :DEL] [34]TAVEL Rosé - Château d'Acqueria prix par bouteille, vendu par 6 2006 Rosé Vallée du Rhône 10,00 EUR prix public [DEL: :DEL] [DEL: 13,00 EUR :DEL] [35]Vacqueyras Château des Roques - Château des Roques prix par bouteille, vendu par 6 2003 Rouge Vallée du Rhône 10,87 EUR prix public [DEL: :DEL] [DEL: 14,00 EUR :DEL] [36]Côtes de Provence Blanc Cuvée Spéciale - Château de Berne prix par bouteille, vendu par 6 2004 Blanc Vallée du Rhône 11,20 EUR prix public [DEL: :DEL] [DEL: 12,20 EUR :DEL] [37]Côtes de Provence Rouge Cuvée Spéciale - Château de Berne prix par bouteille, vendu par 6 2002 Rouge Vallée du Rhône 11,60 EUR prix public [DEL: :DEL] [DEL: 15,00 EUR :DEL] [38]Crozes Hermitage Rouge - CAVE DE TAIN L'HERMITAGE prix par bouteille, vendu par 6 2004 Rouge Vallée du Rhône 11,79 EUR prix public [DEL: :DEL] [DEL: 16,00 EUR :DEL] [39]Gigondas - Domaine de Cassan prix par bouteille, vendu par 6 2004 Rouge Vallée du Rhône 11,80 EUR prix public [DEL: :DEL] [DEL: 14,00 EUR :DEL] [40]BOURGOGNE Rouge Pinot Noir - Patrick Girardin prix par bouteille, vendu par 6 2005 Rouge Bourgogne 11,90 EUR prix public [DEL: :DEL] [DEL: 15,00 EUR :DEL] [41]BANDOL Blanc Château de la Noblesse - Château de la Noblesse prix par bouteille, vendu par 3 2006 Blanc Vallée du Rhône 12,00 EUR prix public [DEL: :DEL] [DEL: 14,00 EUR :DEL] [42]Vacqueyras Cuvée des deux Frères - Château de Montmirail prix par bouteille, vendu par 6 2004 Rouge Vallée du Rhône 12,67 EUR prix public [DEL: :DEL] [DEL: 14,00 EUR :DEL] [43]Gigondas - Domaine de Cassan prix par bouteille, vendu par 6 2005 Rouge Vallée du Rhône 12,80 EUR prix public [DEL: :DEL] [DEL: 14,00 EUR :DEL] [44]Muscat Beaumes de Venise - Domaine de La Pigeade prix par bouteille, vendu par 2 2006 Blanc Vallée du Rhône 12,95 EUR prix public [DEL: :DEL] [DEL: 19,00 EUR :DEL] économisez 32% [45]Côtes du Rhône Villages CAIRANNE - Domaine Rabasse Charavin - Corinne COUTURIER 1999 Rouge Vallée du Rhône 12,95 EUR prix public [DEL: :DEL] [DEL: 14,00 EUR :DEL] [46]SAINT-VERAN A la Côte - Pierre et Véronique GIROUX prix par bouteille, vendu par 6 2005 Blanc Bourgogne 13,45 EUR prix public [DEL: :DEL] [DEL: 16,00 EUR :DEL] [47]Vin Biologique Vacqueyras - Christine et Eric Saurel prix par bouteille, vendu par 3 2003 Rouge Vallée du Rhône 13,80 EUR prix public [DEL: :DEL] [DEL: 15,00 EUR :DEL] [48]SAINT-AMOUR - Clovis LESIEUTRE Rouge Vallée du Rhône 14,00 EUR prix public [DEL: :DEL] [DEL: 19,00 EUR :DEL] [49]BOURGOGNE Blanc Chardonnay - Château Gaillard 2005 Blanc Bourgogne 14,00 EUR prix public [DEL: :DEL] [DEL: 19,00 EUR :DEL] [50]MARSANNAY - Charles AUDOIN prix par bouteille, vendu par 6 1998 Rouge Bourgogne 14,00 EUR prix public [DEL: :DEL] [DEL: 17,00 EUR :DEL] [51]SANTENAY 1er Cru Beaurepaire Blanc - Mestre Père et fils 1998 Rouge Bourgogne 15,00 EUR prix public [DEL: :DEL] [DEL: 22,00 EUR :DEL] économisez 32% [52]Gigondas Château Raspail - Château Raspail prix par bouteille, vendu par 6 2001 Rouge Vallée du Rhône 15,60 EUR prix public [DEL: :DEL] [DEL: 17,00 EUR :DEL] [53]Gigondas Cuvée Vieilles Vignes - Domaine Du Grand Montmirail prix par bouteille, vendu par 3 1999 Rouge Vallée du Rhône 15,80 EUR prix public [DEL: :DEL] [DEL: 18,00 EUR :DEL] [54]Rasteau - Domaine Rabasse Charavin - Corinne COUTURIER 1998 Rouge Vallée du Rhône 16,00 EUR prix public [DEL: :DEL] [DEL: 22,00 EUR :DEL] [55]ALOXE CORTON - B. DESCHAMPS-GORDO 2003 Rouge Bourgogne 17,00 EUR prix public [DEL: :DEL] [DEL: 19,00 EUR :DEL] [56]Saint-Joseph - La Source - Ferraton Père & Fils 2004 Rouge Vallée du Rhône 17,50 EUR prix public [DEL: :DEL] [DEL: 22,00 EUR :DEL] [57]Vin Biologique Gigondas - Christine et Eric Saurel prix par bouteille, vendu par 3 2003 Rouge Vallée du Rhône 17,90 EUR prix public [DEL: :DEL] [DEL: 22,00 EUR :DEL] [58]AUXEY-DURESSES - Bernard Delagrange prix par bouteille, vendu par 3 1990 Rouge Bourgogne 18,00 EUR prix public [DEL: :DEL] [DEL: 24,00 EUR :DEL] [59]MEURSAULT - Alexandre GAUVIN prix par bouteille, vendu par 3 2000 Blanc Bourgogne 19,00 EUR prix public [DEL: :DEL] [DEL: 31,00 EUR :DEL] économisez 39% [60]Châteauneuf-du-Pape - LES GRANDES SERRES prix par bouteille, vendu par 3 2003 Rouge Vallée du Rhône 19,25 EUR prix public [DEL: :DEL] [DEL: 33,00 EUR :DEL] économisez 42% [61]Côtes du Rhône CAIRANNE - Antique Séminaros prix par bouteille, vendu par 6 1998 Rouge Vallée du Rhône 19,28 EUR prix public [DEL: :DEL] [DEL: 27,00 EUR :DEL] [62]AUXEY-DURESSES - Bernard Delagrange prix par bouteille, vendu par 3 1982 1985 1988 1989 Rouge Bourgogne 19,80 EUR prix public [DEL: :DEL] [DEL: 27,00 EUR :DEL] [63]Rasteau - Domaine Rabasse Charavin - Corinne COUTURIER 1996 Rouge Vallée du Rhône 19,90 EUR prix public [DEL: :DEL] [DEL: 25,00 EUR :DEL] [64]Saint-Joseph Esprit de Granit Rouge - Cave de Tain l'Hermitage prix par bouteille, vendu par 3 2005 Rouge Vallée du Rhône 19,90 EUR prix public [DEL: :DEL] [DEL: 24,00 EUR :DEL] [65]Vin Biologique Châteauneuf-du-Pape Les vieilles vignes - Domaine de Villeneuve prix par bouteille, vendu par 3 2004 Rouge Vallée du Rhône 19,95 EUR prix public [DEL: :DEL] [DEL: 29,00 EUR :DEL] économisez 32% [66]Brouilly - Clovis LESIEUTRE Rouge Vallée du Rhône 20,00 EUR prix public [DEL: :DEL] [DEL: 22,00 EUR :DEL] [67]Châteauneuf-du-Pape - LES GRANDES SERRES prix par bouteille, vendu par 3 2000 Rouge Vallée du Rhône 21,18 EUR prix public [DEL: :DEL] [DEL: 33,00 EUR :DEL] économisez 36% [68]Côtes du Rhône CAIRANNE - Antique Séminaros prix par bouteille, vendu par 3 1996 Rouge Vallée du Rhône 21,20 EUR prix public [DEL: :DEL] [DEL: 27,00 EUR :DEL] [69]Châteauneuf-du-Pape - LES GRANDES SERRES prix par bouteille, vendu par 3 2001 2005 Blanc Vallée du Rhône 21,60 EUR prix public [DEL: :DEL] [DEL: 33,00 EUR :DEL] économisez 35% [70]MEURSAULT Monatine - Domaine ROUGEOT prix par bouteille, vendu par 3 1999 Blanc Bourgogne 22,00 EUR prix public [DEL: :DEL] [DEL: 31,00 EUR :DEL] économisez 30% [71]PULIGNY-MONTRACHET - Masson prix par bouteille, vendu par 2 2001 Blanc Bourgogne 22,50 EUR prix public [DEL: :DEL] [DEL: 30,00 EUR :DEL] [72]AUXEY-DURESSES - Bernard Delagrange prix par bouteille, vendu par 3 1979 Rouge Bourgogne 22,50 EUR prix public [DEL: :DEL] [DEL: 33,00 EUR :DEL] économisez 32% [73]Gigondas - Clovis LESIEUTRE 2004 Rouge Vallée du Rhône 23,00 EUR prix public [DEL: :DEL] [DEL: 25,00 EUR :DEL] [74]VOLNAY-SANTENOTS 1er CRU - Domaine ROUGEOT prix par bouteille, vendu par 3 1995 Rouge Bourgogne 24,00 EUR prix public [DEL: :DEL] [DEL: 41,00 EUR :DEL] économisez 42% [75]VOLNAY - Christophe Thilloux 1996 Rouge Bourgogne 24,20 EUR prix public [DEL: :DEL] [DEL: 29,00 EUR :DEL] [76]BEAUNE DU CHATEAU 1er CRU - Bouchard Père et fils prix par bouteille, vendu par 3 2005 Rouge Bourgogne 24,40 EUR prix public [DEL: :DEL] [DEL: 35,00 EUR :DEL] économisez 31% [77]VOLNAY-SANTENOTS 1er CRU - Domaine ROUGEOT prix par bouteille, vendu par 3 1991 1992 Rouge Bourgogne 24,50 EUR prix public [DEL: :DEL] [DEL: 41,00 EUR :DEL] économisez 41% [78]Côtes du Rhône Villages CAIRANNE - Domaine Rabasse Charavin - Corinne COUTURIER 1995 Rouge Vallée du Rhône 24,80 EUR prix public [DEL: :DEL] [DEL: 27,00 EUR :DEL] [79]PULIGNY-MONTRACHET 1er CRU Champs Gains - Stéphan Maroslavac prix par bouteille, vendu par 6 2001 Blanc Bourgogne 25,00 EUR prix public [DEL: :DEL] [DEL: 33,00 EUR :DEL] [80]MEURSAULT - Delagrange 2002 Rouge Bourgogne 25,00 EUR prix public [DEL: :DEL] [DEL: 28,00 EUR :DEL] [81]VOLNAY 1er CRU - Jean-Claude FROMONT 2000 Rouge Bourgogne 25,30 EUR prix public [DEL: :DEL] [DEL: 33,00 EUR :DEL] [82]VOSNE-ROMANEE - Marson & Natier prix par bouteille, vendu par 3 1997 Rouge Bourgogne 26,40 EUR prix public [DEL: :DEL] [DEL: 35,00 EUR :DEL] [83]BEAUNE - Bernard Delagrange 1978 Rouge Bourgogne 26,40 EUR prix public [DEL: :DEL] [DEL: 59,00 EUR :DEL] économisez 56% [84]POMMARD - Christophe Thilloux prix par bouteille, vendu par 3 1996 Rouge Bourgogne 28,60 EUR prix public [DEL: :DEL] [DEL: 32,00 EUR :DEL] [85]NUITS SAINT GEORGES - Martin-Dufour prix par bouteille, vendu par 3 2001 Rouge Bourgogne 28,60 EUR prix public [DEL: :DEL] [DEL: 35,00 EUR :DEL] [86]Châteauneuf-du-Pape - Domaine La Mereuille 2004 Rouge Vallée du Rhône 29,00 EUR prix public [DEL: :DEL] [DEL: 33,00 EUR :DEL] [87]CHARMES-CHAMBERTIN Grand Cru - Domaine des Varoilles 1992 Rouge Bourgogne 29,95 EUR prix public [DEL: :DEL] [DEL: 75,00 EUR :DEL] économisez 61% [88]VOLNAY 1er Cru Carelle sous la Chapelle - Nicolas Potel prix par bouteille, vendu par 3 2001 Rouge Bourgogne 30,00 EUR prix public [DEL: :DEL] [DEL: 38,00 EUR :DEL] [89]GEVREY-CHAMBERTIN 1er Cru - Domaine des Varoilles 1991 Rouge Bourgogne 31,00 EUR prix public [DEL: :DEL] [DEL: 45,00 EUR :DEL] économisez 32% [90]CHAMBOLLE-MUSIGNY - Paul Misset prix par bouteille, vendu par 3 1996 Rouge Bourgogne 32,70 EUR prix public [DEL: :DEL] [DEL: 40,00 EUR :DEL] [91]HERMITAGE Blanc - Cave de Tain l'Hermitage prix par bouteille, vendu par 2 2001 Blanc Vallée du Rhône 34,00 EUR prix public [DEL: :DEL] [DEL: 45,00 EUR :DEL] [92]HERMITAGE Rouge - Cave de Tain l'Hermitage prix par bouteille, vendu par 2 2001 Rouge Vallée du Rhône 34,00 EUR prix public [DEL: :DEL] [DEL: 45,00 EUR :DEL] [93]Condrieu - Côte Chatillon - François GERARD prix par bouteille, vendu par 3 2005 Blanc Vallée du Rhône 34,75 EUR prix public [DEL: :DEL] [DEL: 45,00 EUR :DEL] [94]VOSNE-ROMANEE 1999 - Paul Misset prix par bouteille, vendu par 6 2002 Rouge Bourgogne 34,98 EUR prix public [DEL: :DEL] [DEL: 65,00 EUR :DEL] économisez 47% [95]Châteauneuf-du-Pape - Domaine du Haut des Terres Blanches 1977 Rouge Vallée du Rhône 35,00 EUR prix public [DEL: :DEL] [DEL: 70,00 EUR :DEL] économisez 51% [96]VOLNAY - Delagrange 1978 Rouge Bourgogne 35,20 EUR prix public [DEL: :DEL] [DEL: 72,00 EUR :DEL] économisez 52% [97]Condrieu - Les Mandouls - Ferraton Père & Fils 2005 Blanc Vallée du Rhône 36,00 EUR prix public [DEL: :DEL] [DEL: 44,00 EUR :DEL] [98]CORTON RENARDES GRAND CRU VIEILLE VIGNE - Ludovic Belin 2004 Rouge Bourgogne 37,00 EUR prix public [DEL: :DEL] [DEL: 45,00 EUR :DEL] [99]VOLNAY 1er Cru - Delagrange 1979 Rouge Bourgogne 37,20 EUR prix public [DEL: :DEL] [DEL: 82,00 EUR :DEL] économisez 55% [100]CHARMES-CHAMBERTIN - Hubert de Beaumont prix par bouteille, vendu par 3 2004 Rouge Bourgogne 38,00 EUR prix public [DEL: :DEL] [DEL: 76,00 EUR :DEL] économisez 50% [101]VOLNAY 1er Cru - Delagrange 1978 Rouge Bourgogne 38,20 EUR prix public [DEL: :DEL] [DEL: 83,00 EUR :DEL] économisez 54% [102]NUITS SAINT GEORGES 1er CRU Aux Argillats - Martin-Dufour prix par bouteille, vendu par 3 1999 Rouge Bourgogne 38,90 EUR prix public [DEL: :DEL] [DEL: 47,00 EUR :DEL] [103]VOLNAY Clos du village - Delagrange 1979 Rouge Bourgogne 41,80 EUR prix public [DEL: :DEL] [DEL: 85,00 EUR :DEL] économisez 51% [104]CÔTE-RÔTIE - Domaine de Rosiers 2004 Rouge Vallée du Rhône 42,00 EUR prix public [DEL: :DEL] [DEL: 48,00 EUR :DEL] [105]NUITS SAINT GEORGES 1er CRU Aux Argillats - Martin-Dufour 1999 Rouge Bourgogne 42,00 EUR prix public [DEL: :DEL] [DEL: 47,00 EUR :DEL] [106]CÔTE-RÔTIE - Domaine de Rosiers 1999 2000 Rouge Vallée du Rhône 43,00 EUR prix public [DEL: :DEL] [DEL: 54,00 EUR :DEL] [107]NUITS SAINT GEORGES 1er CRU Aux Murgers - Paul Misset prix par bouteille, vendu par 3 1993 1996 Rouge Bourgogne 44,00 EUR prix public [DEL: :DEL] [DEL: 65,00 EUR :DEL] économisez 33% [108]VOLNAY 1er Cru - Bouchard Père et fils prix par bouteille, vendu par 2 2005 Rouge Bourgogne 47,00 EUR prix public [DEL: :DEL] [DEL: 75,00 EUR :DEL] économisez 38% [109]Châteauneuf-du-Pape - Domaine du Haut des Terres Blanches 1972 Rouge Vallée du Rhône 48,95 EUR prix public [DEL: :DEL] [DEL: 70,00 EUR :DEL] économisez 31% [110]Châteauneuf-du-Pape - Domaine du Haut des Terres Blanches 1971 Rouge Vallée du Rhône 49,60 EUR prix public [DEL: :DEL] [DEL: 70,00 EUR :DEL] économisez 30% [111]VOSNE-ROMANEE 1er Cru Souchot - Labouré Roi prix par bouteille, vendu par 2 1983 Rouge Bourgogne 52,80 EUR prix public [DEL: :DEL] [DEL: 65,00 EUR :DEL] [112]VOSNE-ROMANEE 1er Cru Suchot - Chantal LESCURE 1982 Rouge Bourgogne 52,80 EUR prix public [DEL: :DEL] [DEL: 65,00 EUR :DEL] [113]NUITS SAINT GEORGES Clos des Argillières - Bouchard Père et fils prix par bouteille, vendu par 2 2005 Rouge Bourgogne 59,00 EUR prix public [DEL: :DEL] [DEL: 85,00 EUR :DEL] économisez 31% [114]BEAUNE Montée Rouge - Gaston Boisseaux 1971 1979 Rouge Bourgogne 59,40 EUR prix public [DEL: :DEL] [DEL: 85,00 EUR :DEL] économisez 31% [115]CLOS VOUGEOT Grand Cru - Hubert de Beaumont 2004 Rouge Bourgogne 64,00 EUR prix public [DEL: :DEL] [DEL: 95,00 EUR :DEL] économisez 33% [116]CLOS DE LA ROCHE Grand Cru - CHANSON Père et Fils 1982 Rouge Bourgogne 67,00 EUR prix public [DEL: :DEL] [DEL: 80,00 EUR :DEL] [117]CHARMES-CHAMBERTIN Grand Cru - CHANSON Père et Fils 1983 1987 Rouge Bourgogne 75,00 EUR prix public [DEL: :DEL] [DEL: 95,00 EUR :DEL] [118]RUCHOTTES-CHAMBERTIN - P. Misserey 2003 Rouge Bourgogne 75,00 EUR prix public [DEL: :DEL] [DEL: 120,00 EUR :DEL] économisez 38% [119]BEAUNE Montée Rouge - Gaston Boisseaux 1966 Rouge Bourgogne 77,00 EUR prix public [DEL: :DEL] [DEL: 110,00 EUR :DEL] économisez 30% [120]CLOS SAINT DENIS Grand Cru - CHANSON Père et Fils 1973 Rouge Bourgogne 89,00 EUR prix public [DEL: :DEL] [DEL: 110,00 EUR :DEL] [121]Domaine de la Gaffeliere Saint-Emilion - Domaine de la Gaffeliere 1970 Rouge Bordeaux 90,00 EUR prix public [DEL: :DEL] [DEL: 139,00 EUR :DEL] économisez 36% [122]CHAMBERTIN Clos de Bèze Grand Cru - CHANSON Père et Fils 1991 1994 Rouge Bourgogne 92,00 EUR prix public [DEL: :DEL] [DEL: 110,00 EUR :DEL] [123]RICHEBOURG Grand Cru - Léonce Bocquet 1984 Rouge Bourgogne 121,00 EUR prix public [DEL: :DEL] [DEL: 210,00 EUR :DEL] économisez 43% [124]MAGNUM CÔTE-RÔTIE Brune et Blonde 2001 - E. GUIGAL 2003 Rouge Vallée du Rhône 127,60 EUR prix public [DEL: :DEL] [DEL: 150,00 EUR :DEL] [125]RICHEBOURG Grand Cru - Domaine Marc ROUGEOT-DUPIN 1991 Rouge Bourgogne 139,00 EUR prix public [DEL: :DEL] [DEL: 190,00 EUR :DEL] [126]CLOS DE VOUGEOT Grand Cru - Paul Misset 1983 Rouge Bourgogne 139,00 EUR prix public [DEL: :DEL] [DEL: 195,00 EUR :DEL] [127]Bâtard-Montrachet Grand Cru - Domaine Fleurot-Larose 1980 Blanc Bourgogne 219,00 EUR prix public [DEL: :DEL] [DEL: 250,00 EUR :DEL] [128]Champagne POMMERY - POMMERY 1955 Blanc Champagne 295,00 EUR prix public [DEL: :DEL] [DEL: 320,00 EUR :DEL] [129]Magnum Château Vieux Fortin Saint Emilion Grand Cru - Château Vieux Fortin 1950 Rouge Bordeaux 399,00 EUR prix public [DEL: :DEL] [DEL: 650,00 EUR :DEL] économisez 39% [130]Et beaucoup d'autres Grands vins de Bourgogne, Vallée du Rhône, et Champagnes à découvrir ... Offre valable dans la limite des stocks disponibles [131]Expéditions & retours [132]Remarques sur la confidentialité [133]Conditions d'utilisation [134]nous contacter "A tout moment, vous disposez d'un droit d'accès, de modification, de rectification et de suppression des données qui vous concernent" (art 34 de la loi Informatique et Libertés du 6 Janvier 1978). Pour vous désinscrire de nos mailing lists, [135]cliquez ici L'abus d'alcool est dangereux pour la santé, consommez avec modération. le20.fr - Copyright © 2005 le20.fr, 17 rue Berlier, 21000 DIJON - SARL AU CAPITAL DE 15 000 euros - RCS DIJON 480 509 769 [sendopen.php?MemberID=2857274&SendID=1550&Type=Send] References Visible links 1. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12472 2. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12469 3. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12472 4. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12344 5. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12470 6. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12472 7. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12472 8. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12465 9. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12464 10. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12458 11. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12467 12. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12390 13. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12407 14. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12462 15. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12417 16. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12463 17. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12415 18. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12429 19. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12416 20. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12349 21. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12369 22. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12393 23. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12358 24. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12380 25. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12382 26. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12460 27. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12455 28. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12401 29. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12413 30. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12381 31. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12466 32. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12385 33. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12454 34. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12461 35. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12392 36. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12356 37. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12357 38. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12389 39. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12384 40. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12432 41. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12456 42. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12414 43. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12423 44. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12394 45. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12350 46. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12450 47. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12359 48. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12370 49. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12374 50. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12448 51. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12447 52. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12383 53. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12363 54. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12355 55. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12444 56. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12412 57. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12362 58. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12379 59. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12438 60. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12406 61. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12409 62. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12378 63. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12354 64. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12457 65. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12366 66. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12375 67. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12408 68. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12410 69. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12422 70. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12430 71. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12371 72. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12437 73. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12372 74. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12395 75. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12419 76. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12420 77. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12400 78. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12351 79. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12402 80. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12376 81. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12405 82. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12433 83. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12442 84. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12386 85. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12445 86. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12373 87. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12348 88. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12377 89. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12352 90. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12425 91. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12388 92. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12387 93. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12396 94. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12426 95. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12397 96. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12452 97. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12411 98. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12353 99. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12436 100. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12443 101. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12435 102. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12418 103. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12421 104. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12439 105. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12391 106. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12346 107. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12424 108. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12451 109. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12398 110. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12399 111. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12428 112. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12453 113. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12446 114. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12440 115. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12364 116. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12360 117. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12368 118. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12431 119. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12441 120. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12365 121. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12434 122. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12367 123. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12449 124. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12427 125. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12404 126. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12347 127. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12361 128. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12403 129. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12459 130. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12472 131. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12345 132. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12468 133. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12471 134. http://www.le20newsletters.com/users/link.php?UserID=2857274&Newsletter=156&List=45&LinkType=Send&LinkID=12470 135. http://www.le20newsletters.com/users/unsub.php?Mem=2857274&ConfirmCode=1168a2bf3fea33c1ed3ece1647939697 Hidden links: 136. mailto:infos@le20.fr 137. mailto:infos@le20.fr 138. mailto:infos@le20.fr 139. mailto:infos@le20.fr From owner-freebsd-fs@FreeBSD.ORG Thu Oct 4 10:59:20 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1918016A417 for ; Thu, 4 Oct 2007 10:59:20 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.freebsd.org (Postfix) with ESMTP id AF97A13C457 for ; Thu, 4 Oct 2007 10:59:18 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 5978045F42; Thu, 4 Oct 2007 12:59:17 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 6CD2A45B26; Thu, 4 Oct 2007 12:59:12 +0200 (CEST) Date: Thu, 4 Oct 2007 12:59:06 +0200 From: Pawel Jakub Dawidek To: Michael Monashev Message-ID: <20071004105906.GD19113@garage.freebsd.pl> References: <682935622.20071003222407@gmail.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="pY3vCvL1qV+PayAL" Content-Disposition: inline In-Reply-To: <682935622.20071003222407@gmail.com> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: zfs cache size X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2007 10:59:20 -0000 --pY3vCvL1qV+PayAL Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Oct 03, 2007 at 10:24:07PM +0400, Michael Monashev wrote: > Hi, freebsd-fs. >=20 > How can I see/change zfs cache size? You can see ARC statistics by doing: # sysctl kstat and ARC size by doing: # sysctl vfs.zfs | grep arc You can also tune those values (vfs.zfs.arc_{min,max}), but from /boot/loader.conf. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --pY3vCvL1qV+PayAL Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHBMd6ForvXbEpPzQRAtX1AKDucqctMwdtPJp6Jrr32VXzMkZfnACgq//B iAHMJTnNBkUbJlo4CWyS7pk= =lPmx -----END PGP SIGNATURE----- --pY3vCvL1qV+PayAL-- From owner-freebsd-fs@FreeBSD.ORG Thu Oct 4 14:48:48 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C331416A469 for ; Thu, 4 Oct 2007 14:48:48 +0000 (UTC) (envelope-from softsearch@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.242]) by mx1.freebsd.org (Postfix) with ESMTP id 84C4A13C481 for ; Thu, 4 Oct 2007 14:48:48 +0000 (UTC) (envelope-from softsearch@gmail.com) Received: by an-out-0708.google.com with SMTP id c14so36329anc for ; Thu, 04 Oct 2007 07:48:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=yW5dHVkE13tw2h44FzMunQ6yjB19NhY47jdWDna6IwE=; b=cSzFvy+xS9gmeIeyDqO7CvdS211DFTJoLGErvOIPWVEWeCujshSuwfMkEdNpFrgo5n7F6RaZbCtH5KR5ZxC0Eo8mfP45Avmlwy02o3mPKV4M7ekB4iyBUcr8ppifvQNcqFRqaBy2l5nY1jUZwy8Hy7oTCd7JrrVYUMMvukWRFes= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=tClwRkI4vkDr708c0md6NLjGnfWCRNnIHo7WXgyXocYzwlCrYSUpvVWEyo5eoBrKytEtw0f1qRnuXHuS/hZqZhVFMD0VgCjp5xBWja9CW8dCS/gCjtJN0NxvO+/f1zEh2cxsFe3uBGvHMJPLr4hxc4NKaLZyh1Gs0UklFmLK9N0= Received: by 10.142.79.15 with SMTP id c15mr2568412wfb.1191509327027; Thu, 04 Oct 2007 07:48:47 -0700 (PDT) Received: by 10.142.199.15 with HTTP; Thu, 4 Oct 2007 07:48:46 -0700 (PDT) Message-ID: Date: Thu, 4 Oct 2007 18:48:46 +0400 From: "=?KOI8-R?B?7cnIwcnMIO3PzsHbo9c=?=" To: freebsd-fs@freebsd.org In-Reply-To: <20071004105906.GD19113@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <682935622.20071003222407@gmail.com> <20071004105906.GD19113@garage.freebsd.pl> Subject: Re: zfs cache size X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2007 14:48:48 -0000 Sorry for my English. -spurious +incomplete -- Michael From owner-freebsd-fs@FreeBSD.ORG Fri Oct 5 00:01:02 2007 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 31C3C16A468; Fri, 5 Oct 2007 00:01:02 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.freebsd.org (Postfix) with ESMTP id C5FF113C4B7; Fri, 5 Oct 2007 00:01:01 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id A541F45F44; Fri, 5 Oct 2007 02:01:00 +0200 (CEST) Received: from localhost (154.81.datacomsa.pl [195.34.81.154]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 3F52E456AB; Fri, 5 Oct 2007 02:00:55 +0200 (CEST) Date: Fri, 5 Oct 2007 02:00:46 +0200 From: Pawel Jakub Dawidek To: freebsd-fs@FreeBSD.org Message-ID: <20071005000046.GC92272@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="bKyqfOwhbdpXa4YI" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-2.6 required=3.0 tests=BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-current@FreeBSD.org Subject: ZFS kmem_map too small. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2007 00:01:02 -0000 --bKyqfOwhbdpXa4YI Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi. We'are about to branch RELENG_7 and I'd like to start discussion with folks that experience 'kmem_map too small' panic with the latest HEAD. I'm trying hard to reproduce it and I can't, so I need to gather more info how you are able to provoke this panic. What I did was to rsync 200 FreeBSD src trees from one directory to another on the same ZFS file system. It worked fine. The system I'm using is i386 and the only tuning I did is bigger kmem_map. From my /boot/loader.conf: vm.kmem_size=3D629145600 vm.kmem_size_max=3D629145600 The machine is dual core Pentium D 3GHz with 1GB of RAM. My pool is: lcf:root:/tank/0# zpool status pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 ad4 ONLINE 0 0 0 ad5 ONLINE 0 0 0 ad6 ONLINE 0 0 0 ad7 ONLINE 0 0 0 errors: No known data errors If you can still see those panic, please let me know as soon as possible and try to describe what your workload looks like, how to reproduce it, etc. I'd really like ZFS to be rock-stable for 7.0 even on i386. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --bKyqfOwhbdpXa4YI Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHBX6uForvXbEpPzQRAnb3AJ9Qa9S4pmIeA1RRbfq/f8krB/jTQACgsjBL PJp2ZwshSM4CPetRugv6Id8= =ONnA -----END PGP SIGNATURE----- --bKyqfOwhbdpXa4YI-- From owner-freebsd-fs@FreeBSD.ORG Fri Oct 5 00:46:57 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5291C16A421 for ; Fri, 5 Oct 2007 00:46:57 +0000 (UTC) (envelope-from amdmi3@amdmi3.ru) Received: from cp65.agava.net (cp65.agava.net [89.108.66.215]) by mx1.freebsd.org (Postfix) with ESMTP id 0F34713C45D for ; Fri, 5 Oct 2007 00:46:57 +0000 (UTC) (envelope-from amdmi3@amdmi3.ru) Received: from [213.148.20.85] (helo=nexii.panopticon) by cp65.agava.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.44 (FreeBSD)) id 1IdbLD-0005r3-69 for freebsd-fs@freebsd.org; Fri, 05 Oct 2007 04:46:55 +0400 Received: from hades.panopticon (hades.panopticon [192.168.0.2]) by nexii.panopticon (Postfix) with ESMTP id 2553917041 for ; Fri, 5 Oct 2007 04:48:15 +0400 (MSD) Received: by hades.panopticon (Postfix, from userid 1000) id B15324122; Fri, 5 Oct 2007 04:48:20 +0400 (MSD) Date: Fri, 5 Oct 2007 04:48:20 +0400 From: Dmitry Marakasov To: freebsd-fs@freebsd.org Message-ID: <20071005004820.GA29814@hades.panopticon> Mail-Followup-To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline User-Agent: Mutt/1.5.16 (2007-06-09) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - cp65.agava.net X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [26 6] X-AntiAbuse: Sender Address Domain - amdmi3.ru X-Source: X-Source-Args: X-Source-Dir: Subject: Very slow writes on flash + msdosfs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2007 00:46:57 -0000 Hi! I have USB flash: da0 at umass-sim0 bus 0 target 0 lun 0 da0: Removable Direct Access SCSI-2 device da0: 40.000MB/s transfers da0: 1999MB (4095998 512 byte sectors: 255H 63S/T 254C) It attaches to ehci and I get pretty good read/write speed: % dd if=/dev/zero of=/dev/da0 bs=1m dd: /dev/da0: short write on character device dd: /dev/da0: end of device 2000+0 records in 1999+1 records out 2097150976 bytes transferred in 178.132363 secs (11772993 bytes/sec) % dd if=/dev/da0 of=/dev/zero bs=1m count=100 100+0 records in 100+0 records out 104857600 bytes transferred in 7.240738 secs (14481618 bytes/sec) But newfs_msdos ran on it is very slow: % time newfs_msdos -F32 -LAMDmi3 -k 0xffff /dev/da0s1 /dev/da0s1: 4072456 sectors in 509057 FAT32 clusters (4096 bytes/cluster) bps=512 spc=8 res=32 nft=2 mid=0xf0 spt=63 hds=255 hid=0 bsec=4080447 bspf=3978 rdcl=2 infs=1 bkbs=0xffff newfs_msdos -F32 -LAMDmi3 -k 0xffff /dev/da0s1 0,02s user 0,21s system 0% cpu 2:54,37 total when it runs, gkrellm shows 23kb/s writes. Writing a 1.0Mb directory with 340 files takes 1.5 minutes (up to 300k/sec writes => seems like much more data is actually written that it's needed). Larger files behave somewhat better (up to 3 MB/s). Btw, dd if=/dev/zero of=/dev/da0 bs=512 show the same 23k/s speed as newfs_msdosfs. So where is the problem? Why's there no caching and why's there 1 sector writes? PS. I use 6.1, has the situation changed in -CURRENT? -- Best regards, Dmitry Marakasov mailto:amdmi3@amdmi3.ru From owner-freebsd-fs@FreeBSD.ORG Fri Oct 5 01:41:47 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 207E716A421; Fri, 5 Oct 2007 01:41:47 +0000 (UTC) (envelope-from mcdouga9@egr.msu.edu) Received: from mx.egr.msu.edu (surfnturf.egr.msu.edu [35.9.37.164]) by mx1.freebsd.org (Postfix) with ESMTP id E877C13C474; Fri, 5 Oct 2007 01:41:46 +0000 (UTC) (envelope-from mcdouga9@egr.msu.edu) Received: from localhost (localhost.egr.msu.edu [127.0.0.1]) by mx.egr.msu.edu (Postfix) with ESMTP id B2B812EBAD3; Thu, 4 Oct 2007 21:25:22 -0400 (EDT) X-Virus-Scanned: amavisd-new at egr.msu.edu Received: from mx.egr.msu.edu ([127.0.0.1]) by localhost (surfnturf.egr.msu.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id spMiCM3CMdMx; Thu, 4 Oct 2007 21:25:22 -0400 (EDT) Received: from localhost (daemon.egr.msu.edu [35.9.44.65]) by mx.egr.msu.edu (Postfix) with ESMTP id 895122EBAD0; Thu, 4 Oct 2007 21:25:22 -0400 (EDT) Received: by localhost (Postfix, from userid 21281) id 859EA33C3D; Thu, 4 Oct 2007 21:25:22 -0400 (EDT) Date: Thu, 4 Oct 2007 21:25:22 -0400 From: Adam McDougall To: Pawel Jakub Dawidek Message-ID: <20071005012522.GX4338@egr.msu.edu> References: <20071005000046.GC92272@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20071005000046.GC92272@garage.freebsd.pl> User-Agent: Mutt/1.5.16 (2007-06-09) Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: ZFS kmem_map too small. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2007 01:41:47 -0000 On Fri, Oct 05, 2007 at 02:00:46AM +0200, Pawel Jakub Dawidek wrote: Hi. We'are about to branch RELENG_7 and I'd like to start discussion with folks that experience 'kmem_map too small' panic with the latest HEAD. I'm trying hard to reproduce it and I can't, so I need to gather more info how you are able to provoke this panic. What I did was to rsync 200 FreeBSD src trees from one directory to another on the same ZFS file system. It worked fine. The system I'm using is i386 and the only tuning I did is bigger kmem_map. From my /boot/loader.conf: vm.kmem_size=629145600 vm.kmem_size_max=629145600 The machine is dual core Pentium D 3GHz with 1GB of RAM. My pool is: lcf:root:/tank/0# zpool status pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 ad4 ONLINE 0 0 0 ad5 ONLINE 0 0 0 ad6 ONLINE 0 0 0 ad7 ONLINE 0 0 0 errors: No known data errors If you can still see those panic, please let me know as soon as possible and try to describe what your workload looks like, how to reproduce it, etc. I'd really like ZFS to be rock-stable for 7.0 even on i386. I have a athlon x2 with 2G ram running amd64 -current build not more than a few weeks old and I started getting kmem panics at ~350M kmem, bumped kmem to 1G but it paniced at 1G after a few days, bumped it to 1.5G and it has been running since then, but the 1G crash was probably on Oct 1. Nightly it has a few systems ssh into it over the internet and run rsync as a system backup method. I would login and get some details but at present ssh is not responding properly and I'm offsite. I can check on it tomorrow. It is not a critical system so I can make adjustments to it if I should try to make it crash. I think I have 3 x 250gb sata disks in the zpool. I don't think I've tuned anything except kmem, and that was only since a month or so ago. I updated it near then, and Before that, it used to get panics that might have been unrelated to zfs (strange ones like kernel trap 9, others?). The systems that are being backed up to it are live login/webservers with a healthy amount of (probably mostly small) files. I know some of the files are excluded from rsync but the largest host being backed up does have approx 75 million inodes. Whatever gets backed up from it runs in one shot at 5am. From owner-freebsd-fs@FreeBSD.ORG Fri Oct 5 02:04:21 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 24E2A16A421; Fri, 5 Oct 2007 02:04:21 +0000 (UTC) (envelope-from dan@dan.emsphone.com) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by mx1.freebsd.org (Postfix) with ESMTP id E088F13C4C2; Fri, 5 Oct 2007 02:04:13 +0000 (UTC) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.14.1/8.14.1) id l9523kxN066686; Thu, 4 Oct 2007 21:03:46 -0500 (CDT) (envelope-from dan) Date: Thu, 4 Oct 2007 21:03:46 -0500 From: Dan Nelson To: Pawel Jakub Dawidek Message-ID: <20071005020346.GA36686@dan.emsphone.com> References: <20071005000046.GC92272@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20071005000046.GC92272@garage.freebsd.pl> X-OS: FreeBSD 7.0-CURRENT User-Agent: Mutt/1.5.16 (2007-06-09) Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: ZFS kmem_map too small. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2007 02:04:21 -0000 In the last episode (Oct 05), Pawel Jakub Dawidek said: > We'are about to branch RELENG_7 and I'd like to start discussion with > folks that experience 'kmem_map too small' panic with the latest > HEAD. > > I'm trying hard to reproduce it and I can't, so I need to gather more > info how you are able to provoke this panic. > > What I did was to rsync 200 FreeBSD src trees from one directory to > another on the same ZFS file system. It worked fine. > > The system I'm using is i386 and the only tuning I did is bigger > kmem_map. From my /boot/loader.conf: > > vm.kmem_size=629145600 > vm.kmem_size_max=629145600 I'm running on a 1gb i386 system with vfs.zfs.arc_min="64M" vfs.zfs.arc_max="192M" vm.kmem_size_min="640M" vm.kmem_size_max="640M" and no panics. Set arc_max any higher (256M for example) and I end up panicing on things like reading a 500M mailbox in mutt, etc. Instead of playing with huge amounts of tiny files like the src tree, try with a small amount of large files like ports/distfiles, or create a couple dozen 500MB files and copy them from place to place. I'd really like to set arc_max up to 512M at least. Sort of feels like a waste having all that RAM and not being able to use it for disk cache. Is there any way to make the arc truly dynamic, being able to shrink and grow as needed just like the old buffer cache? Why does it have to live in kmem? -- Dan Nelson dnelson@allantgroup.com From owner-freebsd-fs@FreeBSD.ORG Fri Oct 5 04:43:33 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9075E16A420; Fri, 5 Oct 2007 04:43:33 +0000 (UTC) (envelope-from thierry@herbelot.com) Received: from postfix1-g20.free.fr (postfix1-g20.free.fr [212.27.60.42]) by mx1.freebsd.org (Postfix) with ESMTP id 3AF0D13C44B; Fri, 5 Oct 2007 04:43:33 +0000 (UTC) (envelope-from thierry@herbelot.com) Received: from smtp4-g19.free.fr (smtp4-g19.free.fr [212.27.42.30]) by postfix1-g20.free.fr (Postfix) with ESMTP id 713681AF14CA; Fri, 5 Oct 2007 06:14:59 +0200 (CEST) Received: from smtp4-g19.free.fr (localhost.localdomain [127.0.0.1]) by smtp4-g19.free.fr (Postfix) with ESMTP id 72D633EA0BD; Fri, 5 Oct 2007 06:14:57 +0200 (CEST) Received: from herbelot.dyndns.org (bne75-4-82-227-159-103.fbx.proxad.net [82.227.159.103]) by smtp4-g19.free.fr (Postfix) with ESMTP id 3FFCF3EA0BC; Fri, 5 Oct 2007 06:14:56 +0200 (CEST) Received: from diversion.herbelot.nom (diversion.herbelot.nom [192.168.2.6]) by herbelot.dyndns.org (8.14.0/8.14.0) with ESMTP id l954Etx2011522; Fri, 5 Oct 2007 06:14:56 +0200 (CEST) From: Thierry Herbelot To: freebsd-current@freebsd.org Date: Fri, 5 Oct 2007 06:14:48 +0200 User-Agent: KMail/1.9.7 References: <20071005000046.GC92272@garage.freebsd.pl> In-Reply-To: <20071005000046.GC92272@garage.freebsd.pl> X-Warning: Windows can lose your files X-Op-Sys: Le FriBi de la mort qui tue X-Org: TfH&Co X-MailScanner: Found to be clean MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-15" Content-Transfer-Encoding: 8bit Content-Disposition: inline Message-Id: <200710050614.49466.thierry@herbelot.com> Cc: freebsd-fs@freebsd.org, Pawel Jakub Dawidek Subject: Re: ZFS kmem_map too small. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: thierry@herbelot.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2007 04:43:33 -0000 Le Friday 05 October 2007, Pawel Jakub Dawidek a écrit : > Hi. > > We'are about to branch RELENG_7 and I'd like to start discussion with > folks that experience 'kmem_map too small' panic with the latest HEAD. > > I'm trying hard to reproduce it and I can't, so I need to gather more > info how you are able to provoke this panic. Hello, I can usually see it with my old, *384MB-only*, bi-celeron machine by just make-ing the world with two parallel jobs (it seems to be fine with -j1). this is with a quite recent i386 -current, storing most of its file in a mirrored ZFS : % zpool status pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror ONLINE 0 0 0 ad0s2d ONLINE 0 0 0 ad2s2d ONLINE 0 0 0 errors: No known data errors % mount /dev/mirror/gm0a on / (ufs, local, soft-updates) devfs on /dev (devfs, local) tank on /tank (zfs, local, noatime) tank/files1 on /tank/files1 (zfs, local, noatime) tank/files3 on /tank/files3 (zfs, local, noatime) tank/tmp on /tmp (zfs, local, noatime) tank/usr on /usr (zfs, local, noatime) tank/var on /var (zfs, local, noatime) tank/files1 is used for the src tree, and tank/files3 for the obj tree. TfH From owner-freebsd-fs@FreeBSD.ORG Fri Oct 5 10:07:43 2007 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D2D0C16A419; Fri, 5 Oct 2007 10:07:43 +0000 (UTC) (envelope-from darrenr@freebsd.org) Received: from out1.smtp.messagingengine.com (out1.smtp.messagingengine.com [66.111.4.25]) by mx1.freebsd.org (Postfix) with ESMTP id 9C63913C45A; Fri, 5 Oct 2007 10:07:43 +0000 (UTC) (envelope-from darrenr@freebsd.org) Received: from compute1.internal (compute1.internal [10.202.2.41]) by out1.messagingengine.com (Postfix) with ESMTP id 5F31E2F29B; Fri, 5 Oct 2007 05:48:35 -0400 (EDT) Received: from heartbeat2.messagingengine.com ([10.202.2.161]) by compute1.internal (MEProxy); Fri, 05 Oct 2007 05:48:35 -0400 X-Sasl-enc: MG+xuwluGC3t99xD6iLhZ3UJsK2Z/FZwXgPBVDgX/D/5 1191577714 Received: from [192.168.1.239] (64-142-85-108.dsl.dynamic.sonic.net [64.142.85.108]) by mail.messagingengine.com (Postfix) with ESMTP id 015F4BF7D; Fri, 5 Oct 2007 05:48:33 -0400 (EDT) Message-ID: <47060857.2070001@freebsd.org> Date: Fri, 05 Oct 2007 02:48:07 -0700 From: Darren Reed Organization: FreeBSD User-Agent: Thunderbird 1.5.0.5 (Windows/20060719) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <20071005000046.GC92272@garage.freebsd.pl> In-Reply-To: <20071005000046.GC92272@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org Subject: Re: ZFS kmem_map too small. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: darrenr@freebsd.org List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2007 10:07:43 -0000 Pawel Jakub Dawidek wrote: > Hi. > > We'are about to branch RELENG_7 and I'd like to start discussion with > folks that experience 'kmem_map too small' panic with the latest HEAD. > > I'm trying hard to reproduce it and I can't, so I need to gather more > info how you are able to provoke this panic. Hmm, the first and only time it happened to me was when running cvsup on the rsync'd freebsd CVS tree after creating a snapshot of it. I've repeated that exercise since (and many times before) with no problems. That was when I mistakingly had the ARC max set to be maybe 10MB or so smaller than the kmem_map size (this was a mistake.) I've since grown that gap to 100MB. Tips on how to reproduce it? Make the kmem_map size small and set the ARC's maximum size to be the same as it ? :) Darren From owner-freebsd-fs@FreeBSD.ORG Fri Oct 5 10:26:23 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A126016A418 for ; Fri, 5 Oct 2007 10:26:23 +0000 (UTC) (envelope-from kometen@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.178]) by mx1.freebsd.org (Postfix) with ESMTP id 5017713C481 for ; Fri, 5 Oct 2007 10:26:23 +0000 (UTC) (envelope-from kometen@gmail.com) Received: by py-out-1112.google.com with SMTP id u77so964008pyb for ; Fri, 05 Oct 2007 03:26:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=ulrmhaL1zATV9WL3qsXQispaJl1/pWRUm7rxhNukS2Y=; b=J1BS30qy6suoR/F056N5ihO3jB8/nwwmtP+ofrX8MRiuG82S72ghHxEQJKoWsd1Hx/bxr83X4Gt5AnC/u5WMfJqjLSUMTKWH8Eqz3Gmll46ZtNGjDiLEdt2CSl5C9k+QuAOm9msYT2OqaIGrskWWwGmgaMw1mSFdUGVyjGmaTgU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=TpxY9+J2jYe+uQvhWj7MYQIz4Po0s5lxKCvO8KBuQphbITp8RToN9vF5GJyr/j+Jo0XrQNuNviFR/PTjvdchIFVts6xvCIFGfeCyshJ5Tc4MdZZ/4LzhGXL9e2Jd1W+oyU//fGZYDAOCwNFRbNPJ0YWV0odaD2WqnF2I5sjqnCM= Received: by 10.65.35.6 with SMTP id n6mr17852734qbj.1191579599276; Fri, 05 Oct 2007 03:19:59 -0700 (PDT) Received: by 10.65.52.9 with HTTP; Fri, 5 Oct 2007 03:19:59 -0700 (PDT) Message-ID: Date: Fri, 5 Oct 2007 12:19:59 +0200 From: "Claus Guttesen" To: "Pawel Jakub Dawidek" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20071005000046.GC92272@garage.freebsd.pl> Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: ZFS kmem_map too small. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2007 10:26:23 -0000 > I have a server which gets these 'kmem_map too small' stops. I have an > Dell PE2850 and is running current as of Sept. 26'th 2007. The error > is easy to trigger. I have copied 90 GB of jpeg-files with a size of > approx. 1 MB and a thumbnail at 4 KB. These files are distributed in > approx. 700 folders. The files resides on it's own partition. Forgot to mention: amd64, 4 GB ram, second gen. zzzeon. qlogic hba's. zetta~%>more /boot/loader.conf: #vm.kmem_size=512M #vm.kmem_size=640M vm.kmem_size=768M #vm.kmem_size_max=512M #vm.kmem_size_max=640M vm.kmem_size_max=1024M vfs.zfs.arc_max=1536M zetta~%>more /etc/sysctl.conf: kern.maxvnodes=400000 net.inet.tcp.log_debug=0 -- regards Claus When lenity and cruelty play for a kingdom, the gentlest gamester is the soonest winner. Shakespeare From owner-freebsd-fs@FreeBSD.ORG Fri Oct 5 10:40:05 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 626CE16A421 for ; Fri, 5 Oct 2007 10:40:05 +0000 (UTC) (envelope-from kometen@gmail.com) Received: from nz-out-0506.google.com (nz-out-0506.google.com [64.233.162.225]) by mx1.freebsd.org (Postfix) with ESMTP id 1357A13C461 for ; Fri, 5 Oct 2007 10:40:04 +0000 (UTC) (envelope-from kometen@gmail.com) Received: by nz-out-0506.google.com with SMTP id l8so360124nzf for ; Fri, 05 Oct 2007 03:40:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=hVvTzAoda6u8Sid3sVHLAKLEvca85q9pGEtB7wi82cM=; b=CFVGTNJ5H/k50F23Lxw6i1rqCwglgDuhOamnYg63nOlMnDb+uzDqozWmFBMMWRCiNDclkyt4PAejirzfDUVSN1SCPmi3DMWSr3j/YI1HcfTtMpANYaKSFRHnjepoeRMw5iSrA3DjciChW79hFANFIJicvTh2FYpETVnzA0vJhHo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=HQLDjIWvcm2IDg1bsjtngct3YYxVlXFoL/cAN7AHhAL/LwHP51QVBtQs5HqpPECf3rywX6G0AZyC5+ZfbHcOAEvYiY18NtVUx8DjnNk1RXedkHkojF6ZBUONkH9Xyq200Doi9Dz/9qZ03OfEaBcSv5fwGWjbfUEJGz9nTzbDzeU= Received: by 10.64.153.4 with SMTP id a4mr7975387qbe.1191579351317; Fri, 05 Oct 2007 03:15:51 -0700 (PDT) Received: by 10.65.52.9 with HTTP; Fri, 5 Oct 2007 03:15:51 -0700 (PDT) Message-ID: Date: Fri, 5 Oct 2007 12:15:51 +0200 From: "Claus Guttesen" To: "Pawel Jakub Dawidek" In-Reply-To: <20071005000046.GC92272@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20071005000046.GC92272@garage.freebsd.pl> Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: ZFS kmem_map too small. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2007 10:40:05 -0000 > We'are about to branch RELENG_7 and I'd like to start discussion with > folks that experience 'kmem_map too small' panic with the latest HEAD. > > I'm trying hard to reproduce it and I can't, so I need to gather more > info how you are able to provoke this panic. > > What I did was to rsync 200 FreeBSD src trees from one directory to > another on the same ZFS file system. It worked fine. I have a server which gets these 'kmem_map too small' stops. I have an Dell PE2850 and is running current as of Sept. 26'th 2007. The error is easy to trigger. I have copied 90 GB of jpeg-files with a size of approx. 1 MB and a thumbnail at 4 KB. These files are distributed in approx. 700 folders. The files resides on it's own partition. It crashed while I copied the files down to the server and the server crashed copying the files on to another partiton on the same server. Everytime it crashed with a 'kmem_map too small' it seemed to want slightly more than was set aside in vm.kmem_size. So I adjusted it upwards from 512M, 640M and finally 768M. vm.kmem_size_max was set at 512M, 640M and finally 1024M (1 GB). It crashed with alot of file i/o. When I stopped copying files the server has been stable. It did *not* occur with a kernel build at May 25'th 2007. So now I will either downgrade or move the zfs-partitions to (open)solaris. We are getting closer to our xmas-season and I need to have a rock-solid zfs-server. -- regards Claus When lenity and cruelty play for a kingdom, the gentlest gamester is the soonest winner. Shakespeare From owner-freebsd-fs@FreeBSD.ORG Fri Oct 5 13:40:19 2007 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A55D016A419; Fri, 5 Oct 2007 13:40:19 +0000 (UTC) (envelope-from cb@severious.net) Received: from ion.gank.org (ion.gank.org [69.55.238.164]) by mx1.freebsd.org (Postfix) with ESMTP id 82AE713C44B; Fri, 5 Oct 2007 13:40:19 +0000 (UTC) (envelope-from cb@severious.net) Received: by ion.gank.org (Postfix, from userid 1001) id 3D2DA1109C; Fri, 5 Oct 2007 08:40:19 -0500 (CDT) Date: Fri, 5 Oct 2007 08:40:17 -0500 From: Craig Boston To: Pawel Jakub Dawidek Message-ID: <20071005133958.GA1123@nowhere> Mail-Followup-To: Craig Boston , Pawel Jakub Dawidek , freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org References: <20071005000046.GC92272@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20071005000046.GC92272@garage.freebsd.pl> User-Agent: Mutt/1.4.2.3i Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org Subject: Re: ZFS kmem_map too small. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2007 13:40:19 -0000 On Fri, Oct 05, 2007 at 02:00:46AM +0200, Pawel Jakub Dawidek wrote: > If you can still see those panic, please let me know as soon as possible > and try to describe what your workload looks like, how to reproduce it, > etc. I'd really like ZFS to be rock-stable for 7.0 even on i386. Have two of them for you. My amd64 system is offline for an indeterminate amount of time, so I don't know what its status is. #1) The one I can reproduce it on the most reliably is my old fileserver. It only has 512MB RAM, which I know is below the officially supported limit, but it _used_ to work perfectly (even if a bit slow) with the settings tweaked. hw.machine_arch: i386 hw.realmem: 536309760 vm.kmem_size: 306184192 vfs.zfs.arc_max: 167772160 kern.maxvnodes: 25000 About 1.5-2 months ago it was working fine with the autotuned arc_max value -- I don't remember exactly what it was. Once the panics started I've been slowly bumping it down, but no matter how low I set it, it can't seem to survive a buildworld & buildkernel. I don't have the ZIL disabled on this one, as I'm running PostgreSQL on it and prefer fsync() to work. #2) This one panics far less frequently and is more difficult to trigger. My latest ARC size reduction seems to have it almost stable, but with the autotuned value would panic within the day. Besides the kmem_map too small panic, I sometimes get a "could not allocate kthread stack" panic, which MAY be related. Or it may not. hw.machine_arch: i386 hw.realmem: 2145959936 (2GB) vm.kmem_size: 671088640 vfs.zfs.arc_max: 300000000 kern.maxvnodes: 75000 The other odd thing about this one: vm.kvm_size: 1069543424 vm.kvm_free: 0 I don't know if I should be concerned that kvm_free is consistently zero on this system, but it's a bit strange as all the others I work on have >0. Please let me know if there's any information I can provide that would be useful. Craig From owner-freebsd-fs@FreeBSD.ORG Fri Oct 5 14:48:13 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 63A4B16A41A; Fri, 5 Oct 2007 14:48:13 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.freebsd.org (Postfix) with ESMTP id EBC6113C4A5; Fri, 5 Oct 2007 14:48:12 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 4472545F5E; Fri, 5 Oct 2007 16:48:10 +0200 (CEST) Received: from localhost (154.81.datacomsa.pl [195.34.81.154]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 28BDC45F42; Fri, 5 Oct 2007 16:48:06 +0200 (CEST) Date: Fri, 5 Oct 2007 16:47:59 +0200 From: Pawel Jakub Dawidek To: Claus Guttesen Message-ID: <20071005144759.GB98210@garage.freebsd.pl> References: <20071005000046.GC92272@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="M9NhX3UHpAaciwkO" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-2.6 required=3.0 tests=BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: ZFS kmem_map too small. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2007 14:48:13 -0000 --M9NhX3UHpAaciwkO Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Oct 05, 2007 at 12:19:59PM +0200, Claus Guttesen wrote: > > I have a server which gets these 'kmem_map too small' stops. I have an > > Dell PE2850 and is running current as of Sept. 26'th 2007. The error > > is easy to trigger. I have copied 90 GB of jpeg-files with a size of > > approx. 1 MB and a thumbnail at 4 KB. These files are distributed in > > approx. 700 folders. The files resides on it's own partition. >=20 > Forgot to mention: >=20 > amd64, 4 GB ram, second gen. zzzeon. qlogic hba's. >=20 > zetta~%>more /boot/loader.conf: > #vm.kmem_size=3D512M > #vm.kmem_size=3D640M > vm.kmem_size=3D768M > #vm.kmem_size_max=3D512M > #vm.kmem_size_max=3D640M > vm.kmem_size_max=3D1024M > vfs.zfs.arc_max=3D1536M That's bad for sure. arc_max can't be larger than kmem_size. I should tune it down automatically in that case. Try something like 700-800MB for arc_max or increase kmem_size to 2GB. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --M9NhX3UHpAaciwkO Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHBk6fForvXbEpPzQRAhNrAKC7peLBE+I5heDfrch/+gkAnHg1CQCeLv5j 8U8jfxdTKZzOEomFa04LUio= =sgzk -----END PGP SIGNATURE----- --M9NhX3UHpAaciwkO-- From owner-freebsd-fs@FreeBSD.ORG Fri Oct 5 18:01:43 2007 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 41E8416A420; Fri, 5 Oct 2007 18:01:43 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.freebsd.org (Postfix) with ESMTP id BAD7413C46A; Fri, 5 Oct 2007 18:01:42 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 56DE545F44; Fri, 5 Oct 2007 20:01:41 +0200 (CEST) Received: from localhost (154.81.datacomsa.pl [195.34.81.154]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 655FA456AB; Fri, 5 Oct 2007 20:01:34 +0200 (CEST) Date: Fri, 5 Oct 2007 20:01:19 +0200 From: Pawel Jakub Dawidek To: Darren Reed Message-ID: <20071005180119.GE98210@garage.freebsd.pl> References: <20071005000046.GC92272@garage.freebsd.pl> <47060857.2070001@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="CdrF4e02JqNVZeln" Content-Disposition: inline In-Reply-To: <47060857.2070001@freebsd.org> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-2.6 required=3.0 tests=BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org Subject: Re: ZFS kmem_map too small. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2007 18:01:43 -0000 --CdrF4e02JqNVZeln Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Oct 05, 2007 at 02:48:07AM -0700, Darren Reed wrote: > Pawel Jakub Dawidek wrote: > > Hi. > >=20 > > We'are about to branch RELENG_7 and I'd like to start discussion with > > folks that experience 'kmem_map too small' panic with the latest HEAD. > >=20 > > I'm trying hard to reproduce it and I can't, so I need to gather more > > info how you are able to provoke this panic. >=20 > Hmm, the first and only time it happened to me was when running cvsup > on the rsync'd freebsd CVS tree after creating a snapshot of it. I've > repeated that exercise since (and many times before) with no problems. >=20 > That was when I mistakingly had the ARC max set to be maybe 10MB or > so smaller than the kmem_map size (this was a mistake.) I've since > grown that gap to 100MB. >=20 > Tips on how to reproduce it? Make the kmem_map size small and set the > ARC's maximum size to be the same as it ? :) I'm not really interested in fixing configuration bugs, although we should do as much as we can to not allow bugus tunnings. I'm most interested in 'kmem_map too small' panic with default configuration, ie. no other changes than vm.kmem_size/vm.kmem_size_max. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --CdrF4e02JqNVZeln Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHBnvvForvXbEpPzQRAgBVAKC9Mc5wGMyVvTO4KJhR02b7R0YZagCgkyxI 09PL4bks02BwfknklC1ZNWM= =Ws0m -----END PGP SIGNATURE----- --CdrF4e02JqNVZeln-- From owner-freebsd-fs@FreeBSD.ORG Fri Oct 5 20:14:22 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8276516A417 for ; Fri, 5 Oct 2007 20:14:22 +0000 (UTC) (envelope-from mday@apple.com) Received: from mail-out4.apple.com (mail-out4.apple.com [17.254.13.23]) by mx1.freebsd.org (Postfix) with ESMTP id 64AF013C44B for ; Fri, 5 Oct 2007 20:14:22 +0000 (UTC) (envelope-from mday@apple.com) Received: from relay11.apple.com (relay11.apple.com [17.128.113.48]) by mail-out4.apple.com (Postfix) with ESMTP id 2475B14C1946; Fri, 5 Oct 2007 12:55:01 -0700 (PDT) Received: from relay11.apple.com (unknown [127.0.0.1]) by relay11.apple.com (Symantec Mail Security) with ESMTP id 08D3E2805B; Fri, 5 Oct 2007 12:55:01 -0700 (PDT) X-AuditID: 11807130-a43c3bb000004daf-5a-470696946e63 Received: from doomsday.apple.com (doomsday.apple.com [17.202.43.217]) by relay11.apple.com (Apple SCV relay) with ESMTP id DC9FA2804D; Fri, 5 Oct 2007 12:55:00 -0700 (PDT) Message-Id: <5F743816-CE50-489E-AB48-C278212791C5@apple.com> From: Mark Day To: Dmitry Marakasov In-Reply-To: <20071005004820.GA29814@hades.panopticon> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v911) Date: Fri, 5 Oct 2007 12:55:00 -0700 References: <20071005004820.GA29814@hades.panopticon> X-Mailer: Apple Mail (2.911) X-Brightmail-Tracker: AAAAAA== Cc: freebsd-fs@freebsd.org Subject: Re: Very slow writes on flash + msdosfs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2007 20:14:22 -0000 On Mac OS X (where msdosfs is derived from FreeBSD), I fixed it by just buffering up the I/O into 128KB chunks. Simple but effective. The diffs are below, based on the Mac OS X sources. But I'll bet it's not too hard to apply to FreeBSD; I don't think the newfs_msdos sources have diverged too much. I hope this helps. -Mark Index: newfs_msdos.tproj/newfs_msdos.c =================================================================== --- newfs_msdos.tproj/newfs_msdos.c (revision 26754) +++ newfs_msdos.tproj/newfs_msdos.c (revision 26759) @@ -73,6 +73,7 @@ #define DOSMAGIC 0xaa55 /* DOS magic number */ #define MINBPS 128 /* minimum bytes per sector */ +#define MAXBPS 4096 /* maximum bytes per sector */ #define MAXSPC 128 /* maximum sectors per cluster */ #define MAXNFT 16 /* maximum number of FATs */ #define DEFBLK 4096 /* default block size */ @@ -81,6 +82,16 @@ #define RESFTE 2 /* reserved FAT entries */ /* + * The size of our in-memory I/O buffer. This is the size of the writes we + * do to the device (except perhaps a few odd sectors at the end). + * + * This must be a multiple of the sector size. Larger is generally faster, + * but some old devices have bugs if you ask them to do more than 128KB + * per I/O. + */ +#define IO_BUFFER_SIZE (128*1024) + +/* * [2873845] FAT12 volumes can have 1..4084 clusters. FAT16 can have * 4085..65524 clusters. FAT32 is 65525 clusters or more. * Since many other implementations are off by 1, 2, 4, 8, 10, or 16, @@ -312,7 +323,8 @@ struct bsxbpb *bsxbpb; struct bsx *bsx; struct de *de; - u_int8_t *img; + u_int8_t *io_buffer; /* The buffer for sectors being constructed/written */ + u_int8_t *img; /* Current sector within io_buffer */ const char *fname, *dtype, *bname; ssize_t n; time_t now; @@ -445,6 +457,9 @@ if (bpb.bps < MINBPS) errx(1, "bytes/sector (%u) is too small; minimum is %u", bpb.bps, MINBPS); + if (bpb.bps > MAXBPS) + errx(1, "bytes/sector (%u) is too large; maximum is %u", + bpb.bps, MAXBPS); if (!(fat = opt_F)) { if (opt_f) fat = 12; @@ -702,12 +717,13 @@ bpb.sec = 0; } print_bpb(&bpb); - if (!opt_N) { + if (!opt_N) { gettimeofday(&tv, NULL); now = tv.tv_sec; tm = localtime(&now); - if (!(img = malloc(bpb.bps))) + if (!(io_buffer = malloc(IO_BUFFER_SIZE))) err(1, NULL); + img = io_buffer; dir = bpb.res + (bpb.spf ? bpb.spf : bpb.bspf) * bpb.nft; for (lsn = 0; lsn < dir + (fat == 32 ? bpb.spc : rds); lsn++) { x = lsn; @@ -808,9 +824,21 @@ (u_int)tm->tm_mday; mk2(de->date, x); } - if ((n = write(fd, img, bpb.bps)) == -1) + img += bpb.bps; + if (img >= (io_buffer + IO_BUFFER_SIZE)) { + /* We filled the I/O buffer, so write it out now */ + if ((n = write(fd, io_buffer, IO_BUFFER_SIZE)) == -1) + err(1, "%s", fname); + if (n != IO_BUFFER_SIZE) + errx(1, "%s: can't write sector %u", fname, lsn); + img = io_buffer; + } + } + if (img != io_buffer) { + /* The I/O buffer was partially full; write it out before exit */ + if ((n = write(fd, io_buffer, img-io_buffer)) == -1) err(1, "%s", fname); - if (n != bpb.bps) + if (n != (img-io_buffer)) errx(1, "%s: can't write sector %u", fname, lsn); } } On Oct 4, 2007, at 5:48 PM, Dmitry Marakasov wrote: > Hi! > > I have USB flash: > > da0 at umass-sim0 bus 0 target 0 lun 0 > da0: Removable Direct Access SCSI-2 device > da0: 40.000MB/s transfers > da0: 1999MB (4095998 512 byte sectors: 255H 63S/T 254C) > > It attaches to ehci and I get pretty good read/write speed: > > % dd if=/dev/zero of=/dev/da0 bs=1m > dd: /dev/da0: short write on character device > dd: /dev/da0: end of device > 2000+0 records in > 1999+1 records out > 2097150976 bytes transferred in 178.132363 secs (11772993 bytes/sec) > > % dd if=/dev/da0 of=/dev/zero bs=1m count=100 > 100+0 records in > 100+0 records out > 104857600 bytes transferred in 7.240738 secs (14481618 bytes/sec) > > But newfs_msdos ran on it is very slow: > > % time newfs_msdos -F32 -LAMDmi3 -k 0xffff /dev/da0s1 > /dev/da0s1: 4072456 sectors in 509057 FAT32 clusters (4096 bytes/ > cluster) > bps=512 spc=8 res=32 nft=2 mid=0xf0 spt=63 hds=255 hid=0 > bsec=4080447 bspf=3978 rdcl=2 infs=1 bkbs=0xffff > newfs_msdos -F32 -LAMDmi3 -k 0xffff /dev/da0s1 0,02s user 0,21s > system 0% cpu 2:54,37 total > > when it runs, gkrellm shows 23kb/s writes. > > Writing a 1.0Mb directory with 340 files takes 1.5 minutes (up to > 300k/sec > writes => seems like much more data is actually written that it's > needed). > > Larger files behave somewhat better (up to 3 MB/s). > > Btw, dd if=/dev/zero of=/dev/da0 bs=512 show the same 23k/s speed as > newfs_msdosfs. > > So where is the problem? Why's there no caching and why's there 1 > sector > writes? > > PS. I use 6.1, has the situation changed in -CURRENT? > > -- > Best regards, > Dmitry Marakasov mailto:amdmi3@amdmi3.ru > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Fri Oct 5 23:00:58 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 558E116A417 for ; Fri, 5 Oct 2007 23:00:58 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail09.syd.optusnet.com.au (mail09.syd.optusnet.com.au [211.29.132.190]) by mx1.freebsd.org (Postfix) with ESMTP id D3D2F13C46A for ; Fri, 5 Oct 2007 23:00:57 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from besplex.bde.org (c220-239-235-248.carlnfd3.nsw.optusnet.com.au [220.239.235.248]) by mail09.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id l95N0nHg032384 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 6 Oct 2007 09:00:52 +1000 Date: Sat, 6 Oct 2007 09:00:49 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Dmitry Marakasov In-Reply-To: <20071005004820.GA29814@hades.panopticon> Message-ID: <20071006080406.S689@besplex.bde.org> References: <20071005004820.GA29814@hades.panopticon> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org Subject: Re: Very slow writes on flash + msdosfs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2007 23:00:58 -0000 On Fri, 5 Oct 2007, Dmitry Marakasov wrote: > I have USB flash: > ... > But newfs_msdos ran on it is very slow: > > % time newfs_msdos -F32 -LAMDmi3 -k 0xffff /dev/da0s1 > /dev/da0s1: 4072456 sectors in 509057 FAT32 clusters (4096 bytes/cluster) > bps=512 spc=8 res=32 nft=2 mid=0xf0 spt=63 hds=255 hid=0 bsec=4080447 bspf=3978 rdcl=2 infs=1 bkbs=0xffff > newfs_msdos -F32 -LAMDmi3 -k 0xffff /dev/da0s1 0,02s user 0,21s system 0% cpu 2:54,37 total Try 512 bytes/cluster for real slowness. > ... > Writing a 1.0Mb directory with 340 files takes 1.5 minutes (up to 300k/sec > writes => seems like much more data is actually written that it's > needed). > > Larger files behave somewhat better (up to 3 MB/s). > > Btw, dd if=/dev/zero of=/dev/da0 bs=512 show the same 23k/s speed as > newfs_msdosfs. > > So where is the problem? Why's there no caching and why's there 1 sector > writes? Old versions of msdosfs don't implement clustering. > PS. I use 6.1, has the situation changed in -CURRENT? Yes. However, clustering won't help much for small files, due to BSD's fundamental design error of per-vnode buffering. With 340 files in a 1.0MB directory, the average file size is about 3K. This is smaller than the block size of 4K, so for most files clustering will have no effect, and there will be about 340 write accesses for the data alone, even though the data is written asynchronously (except in the sync-mount case where everythiung is written synchronously -- this will be much slower). There will be at least another 340 write accesses for writing the directory entry synchronously on creation of the files. There will be a few more accesses for writes to the FAT and writes to the the directory entry for completion of writes to the files. Another 340 or so accesses may be required after bugs in synchronous update of metadata a fixed (currently, all FAT updates are asynchronous, but many should be synchronous unless the file system is mounted sync). This gives a minimum of 2 or 3 writes per file so if the write speed is 23k/sec for 512-blocks = 46 transactions/sec, then writing 340 files will take a minimum of 15-22 seconds. I don't know why it would take 1.5 minutes with 4K-clusters, but it would take about that long with 512-clusters. Async mounts would reduce the minimum number of writes per file to about 1 (for the data block). msdosfs doesn't implement them yet. Bruce From owner-freebsd-fs@FreeBSD.ORG Sat Oct 6 02:09:46 2007 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6351A16A41A; Sat, 6 Oct 2007 02:09:46 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id 23DCA13C481; Sat, 6 Oct 2007 02:09:46 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (localhost [127.0.0.1]) by spam.des.no (Postfix) with ESMTP id C467220BC; Sat, 6 Oct 2007 04:09:36 +0200 (CEST) X-Spam-Tests: AWL X-Spam-Learn: disabled X-Spam-Score: -0.0/3.0 X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on tim.des.no Received: from ds4.des.no (des.no [80.203.243.180]) by smtp.des.no (Postfix) with ESMTP id 3AFCC20BB; Sat, 6 Oct 2007 04:09:36 +0200 (CEST) Received: by ds4.des.no (Postfix, from userid 1001) id E8F8B84486; Sat, 6 Oct 2007 04:09:35 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Pawel Jakub Dawidek References: <20071005000046.GC92272@garage.freebsd.pl> Date: Sat, 06 Oct 2007 04:09:35 +0200 In-Reply-To: <20071005000046.GC92272@garage.freebsd.pl> (Pawel Jakub Dawidek's message of "Fri\, 5 Oct 2007 02\:00\:46 +0200") Message-ID: <86r6k9ouxs.fsf@ds4.des.no> User-Agent: Gnus/5.110006 (No Gnus v0.6) Emacs/22.1 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org Subject: Re: ZFS kmem_map too small. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Oct 2007 02:09:46 -0000 Pawel Jakub Dawidek writes: > We'are about to branch RELENG_7 and I'd like to start discussion with > folks that experience 'kmem_map too small' panic with the latest HEAD. I've been getting it recently (twice in the last ten days). Intel E6600 with 4 GB RAM running an amd64 kernel from September 25. After the first panic, I increased kmem_size_max to 2G so that I now have: des@ds4 ~% sysctl -h vm | grep kmem vm.kmem_size_scale: 3 vm.kmem_size_max: 2,147,483,648 vm.kmem_size_min: 0 vm.kmem_size: 1,381,412,864 After the second panic, I set kmem_size_max to 4G and kmem_size to 2G in loader.conf, but I haven't rebooted yet. No backtraces because the watchdog kicked in and rebooted the machine. I now have tace_on_panic=3D1 and debugger_on_panic=3D0, so next time I should get at least a backtrace, and a dump if I'm lucky. Both panics occurred during the nightly rsync of my imap spool from my mail server to my file server, where it is kept in a compressed dataset which I snapshot every day. The spool currently contains about 1.5 M files with an average size of slightly less than 8 kB. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-fs@FreeBSD.ORG Sat Oct 6 13:52:39 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1B25116A468 for ; Sat, 6 Oct 2007 13:52:39 +0000 (UTC) (envelope-from kometen@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.186]) by mx1.freebsd.org (Postfix) with ESMTP id 848F013C49D for ; Sat, 6 Oct 2007 13:52:38 +0000 (UTC) (envelope-from kometen@gmail.com) Received: by nf-out-0910.google.com with SMTP id b2so643946nfb for ; Sat, 06 Oct 2007 06:52:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=5YjOiuQKmjGE3IfMCS+jpR+C5yFup6sWxmg+4FIkMg4=; b=iOwFTnK5MqIniV5Ph+RXWixb+e0BV7/IacDIsL3vefvYo5EwL0ihSMy6IfHIlYvm+iBI2gp1A5rmGTRAY04pC2SnVK5xKTknkq1Xc+cm9eIp1kG8m1p/svI0uTCm2+WfQPO7okYNZCfzpD926zlbaiNQQBRDmVF0N+l7Qmp6+Z4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=qL1lh0nRqtenS70RJ9pTxsn28e7CUV0zjQ6bQY9GtRvV4qozMVE5jiHUZTVPtop7os5bdWcmu9T75Rt4w1h56b/ECmncdic6GVD5i3jAMdt0m6fzY2Sa59CwOulOKjRvNB9Vv0fLbpWSGvnXNUIIBtzwUGtUfNiz3+O6GFBhI3k= Received: by 10.78.142.14 with SMTP id p14mr5268927hud.1191678756774; Sat, 06 Oct 2007 06:52:36 -0700 (PDT) Received: by 10.78.146.10 with HTTP; Sat, 6 Oct 2007 06:52:36 -0700 (PDT) Message-ID: Date: Sat, 6 Oct 2007 15:52:36 +0200 From: "Claus Guttesen" To: "Ivan Voras" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20071005000046.GC92272@garage.freebsd.pl> Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: ZFS kmem_map too small. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Oct 2007 13:52:39 -0000 > > What I did was to rsync 200 FreeBSD src trees from one directory to > > another on the same ZFS file system. It worked fine. > > It looks like most problems (including mine I sent you before) are when > rsync (and possibly NFS?) are run over the network. How much kernel > memory does a heavily loaded network stack (multiple parallel > connections & TCP streams) consume? Just for the TCP send & receive > buffers I'd guess at least something like 128K*number_of_connections. In my case I rsync'ed both across the network and locally within one zpool. Both times I got the 'kmem_map too small' error. -- regards Claus When lenity and cruelty play for a kingdom, the gentlest gamester is the soonest winner. Shakespeare From owner-freebsd-fs@FreeBSD.ORG Sat Oct 6 18:18:23 2007 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D05C616A46B; Sat, 6 Oct 2007 18:18:23 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (mail.bitblocks.com [64.142.15.60]) by mx1.freebsd.org (Postfix) with ESMTP id A3BF313C4C1; Sat, 6 Oct 2007 18:18:23 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (localhost.bitblocks.com [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id E1D575B52; Sat, 6 Oct 2007 10:46:14 -0700 (PDT) To: Pawel Jakub Dawidek In-reply-to: Your message of "Fri, 05 Oct 2007 20:01:19 +0200." <20071005180119.GE98210@garage.freebsd.pl> Date: Sat, 06 Oct 2007 10:46:14 -0700 From: Bakul Shah Message-Id: <20071006174614.E1D575B52@mail.bitblocks.com> Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org Subject: Re: ZFS kmem_map too small. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Oct 2007 18:18:23 -0000 I have an athlon 64 x2 w/ 2GB of memory running in 64 bit mode. The system dies in a couple of days. This machine does cvsup mirroring every few hours + occassional portupgrade + makes. But I have not been able to panic it deterministically. Its loader.conf has vfs.zfs.zil_disable=1 vfs.zfs.prefetch_disable=1 No other changes from defaults for anything relating to vm or zfs. The only way to make this more stable was to revert arc.c to R1.10. There two differences: R1.11 arc_reclaim_needed() returns 1 when 80% of kmem is used, while R1.10 does so at 50% of kmem. Second, R1.11 sets arc_c_max to 3/4 of kmem_size while R1.10 sets it to 1/2 of kmem_size. > > Tips on how to reproduce it? Make the kmem_map size small and set the > > ARC's maximum size to be the same as it ? :) > > I'm not really interested in fixing configuration bugs, although we > should do as much as we can to not allow bugus tunnings. IMHO bad config can result in suboptimal performance but a panic seems a bit drastic. Particularly as things that used to work a month ago now don't. It would be nice if configurations were sanitized during initialization or if that is not possible take away a tuning knob altogether (or leave it under debugging). Still, fiddling with limits to make the panic go away seems to somehow miss the point as I always worry it will show up under other conditions. May be there a way to ensure that kmem_map is never too small or may be zfs can reserve a few resources for its own use so that it can get out of a tight spot? Thanks!