From owner-freebsd-fs@FreeBSD.ORG Sun Aug 8 00:06:49 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CD21C1065674 for ; Sun, 8 Aug 2010 00:06:49 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 83EB48FC08 for ; Sun, 8 Aug 2010 00:06:49 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEAE+RXUyDaFvO/2dsb2JhbACDFZ4nsESQdoEmgyFzBIk7 X-IronPort-AV: E=Sophos;i="4.55,336,1278302400"; d="scan'208";a="87671541" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 07 Aug 2010 20:06:45 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 9F54B9210F; Sat, 7 Aug 2010 20:06:47 -0400 (EDT) Date: Sat, 7 Aug 2010 20:06:47 -0400 (EDT) From: Rick Macklem To: Markus Gebert Message-ID: <163511110.410136.1281226007565.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [24.65.230.102] X-Mailer: Zimbra 6.0.7_GA_2476.RHEL4 (ZimbraWebClient - SAF3 (Mac)/6.0.7_GA_2473.RHEL4_64) Cc: freebsd-fs@freebsd.org Subject: Re: NFS problem: file doesn't appear in file listing, but can be accessed directly X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Aug 2010 00:06:50 -0000 > On 07.08.2010, at 02:35, Rick Macklem wrote: > > I agree, this must be some kind of server issue then. Still, I wonder > why the solaris client didn't show the problem... > > 2 things: 1 - the solaris client uses ReaddirPlus and not Readdir RPCs by default (I can't remember if that default can be overridden?). 2 - different size requests. The Readdir RPC request specifies the maximum size of a reply (including all XDR) and ReaddirPlus both that and a maximum size for the directory information (which is not defined by the RFC, so there is confusion over what that size actually is a?). When the solaris client specifies a different maximum sizes, that probably avoids the bug. (Since most "ls" listings for directories work, the bug is probably a miscalculation of this limit that results in the entry being missed, but that's just a guess??) rick ps: I don't recall the previous emails mentioning that the "ls" was ok for a solaris client. From owner-freebsd-fs@FreeBSD.ORG Sun Aug 8 10:39:20 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1B7D8106566C for ; Sun, 8 Aug 2010 10:39:20 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id AA05E8FC13 for ; Sun, 8 Aug 2010 10:39:19 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id EF4AC45CD9; Sun, 8 Aug 2010 12:39:17 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id E690545C8C; Sun, 8 Aug 2010 12:39:12 +0200 (CEST) Date: Sun, 8 Aug 2010 12:39:04 +0200 From: Pawel Jakub Dawidek To: Steven Hartland Message-ID: <20100808103904.GB2037@garage.freebsd.pl> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ZoaI/ZTpAVc4A5k6" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@FreeBSD.ORG Subject: Re: ZFS diff support on FreeBSD X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Aug 2010 10:39:20 -0000 --ZoaI/ZTpAVc4A5k6 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Aug 01, 2010 at 07:58:13PM +0100, Steven Hartland wrote: > I cant seem to track down the correlation of various ZFS features > that are present in Solaris and the corresponding FreeBSD versions. >=20 > I'm interested in the zfs diff command. Could anyone tell me where > this info is, or do they know off hand when we're likely to see this > added to the FreeBSD implementation of zfs? >=20 > http://arc.opensolaris.org/caselog/PSARC/2010/105/20100328_tim.haley We're are working on getting most recent ZFS into FreeBSD, but this particular functionality is not in OpenSolaris yet. At this point I also can't give you any dates. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --ZoaI/ZTpAVc4A5k6 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxeiUgACgkQForvXbEpPzSI7gCgtmV9TUJfrt106nJvgOuvmk5h 90EAoK2iRQoWucWuSUjaXxRne1H990Jc =Mhpd -----END PGP SIGNATURE----- --ZoaI/ZTpAVc4A5k6-- From owner-freebsd-fs@FreeBSD.ORG Sun Aug 8 15:17:15 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2A7F4106568D for ; Sun, 8 Aug 2010 15:17:15 +0000 (UTC) (envelope-from thomas@gibfest.dk) Received: from mail.tyknet.dk (mail.tyknet.dk [213.150.42.155]) by mx1.freebsd.org (Postfix) with ESMTP id D78698FC19 for ; Sun, 8 Aug 2010 15:17:14 +0000 (UTC) Received: from [10.10.1.200] (1503026959.dong.dbnet.dk [89.150.95.15]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.tyknet.dk (Postfix) with ESMTPSA id E5302638DFC for ; Sun, 8 Aug 2010 17:17:13 +0200 (CEST) X-DKIM: OpenDKIM Filter v1.1.2 mail.tyknet.dk E5302638DFC DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=gibfest.dk; s=default; t=1281280633; bh=InoxNyR6aGbVCiBzhkg91/AQF5hyNAUY0ZxDprXe23M=; h=Message-ID:Date:From:MIME-Version:To:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=orcO+0+vfWygp41h1H/4YIvxqKxXj7VSixhc1PVcnOkGSbcijys2zyemR36A7J0Vx K+BWxeaeFKOaOSGmO81Hy63ERcM4Ldm9+IKXQRm3Nj83EJw6Jr2dCy2apb3gBaRjO9 hqytArgK9j7maWofE9UC9qodeU/IPhQZPY5OcLSA= Message-ID: <4C5ECA78.6010803@gibfest.dk> Date: Sun, 08 Aug 2010 17:17:12 +0200 From: Thomas Steen Rasmussen User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.1.11) Gecko/20100711 Lightning/1.0b1 Thunderbird/3.0.6 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <4C57E20E.2030908@gibfest.dk> <20100806135001.GF1710@garage.freebsd.pl> In-Reply-To: <20100806135001.GF1710@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: HAST initial sync speed X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Aug 2010 15:17:15 -0000 On 06-08-2010 15:50, Pawel Jakub Dawidek wrote: > On Tue, Aug 03, 2010 at 11:31:58AM +0200, Thomas Rasmussen wrote: > >> Hello list, >> >> I finally got my ZFS/HAST setup up and running, or trying to at least. >> I am wondering how fast the initial HAST sync normally is - I created >> these 4 HAST providers yesterday on 4 146 gig drives, and they still each >> have over 90 gigabytes 'dirty' today. The machines are powerful (dell >> r710) and are otherwise idle, and they are connected to the same gigabit >> switch. >> >> I can supply details about any part of the configuration if needed, but I >> just wanted to ask if you guys believe something is wrong here. I can't help >> but think, if the initial sync takes 24+ hours, then if I ever need to >> replace one of the servers, I will be without redundancy until the new >> server reaches 0 'dirty' bytes, correct ? >> > Correct, but synchronizartion should take much, much less time. > Is dirty count actually decreasing? > > Hello, Yes it was decreasing steadily but very slowly. It finished between thursday evening and friday morning, and the dirty count is now 0. All in all it took over 72 hours. It was transferring around 20mbits while doing this. However, if I copied a large file to the primary HAST node, it would use up a lot more bandwidth. It is like HAST was synchronizing the "empty space" with lower priority or something. Does that make any sense ? The servers are not in production so I can perform any testing needed. Thank you for your reply. Regards Thomas Steen Rasmussen From owner-freebsd-fs@FreeBSD.ORG Sun Aug 8 16:26:52 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1FF3E106567D for ; Sun, 8 Aug 2010 16:26:52 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 1959D8FC20 for ; Sun, 8 Aug 2010 16:26:40 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 40A1C45C9C; Sun, 8 Aug 2010 18:26:38 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 0AA2C45C99; Sun, 8 Aug 2010 18:26:32 +0200 (CEST) Date: Sun, 8 Aug 2010 18:26:24 +0200 From: Pawel Jakub Dawidek To: Steven Hartland Message-ID: <20100808162624.GE2037@garage.freebsd.pl> References: <20100808103904.GB2037@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="/2994txjAzEdQwm5" Content-Disposition: inline In-Reply-To: <20100808103904.GB2037@garage.freebsd.pl> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@FreeBSD.ORG Subject: Re: ZFS diff support on FreeBSD X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Aug 2010 16:26:52 -0000 --/2994txjAzEdQwm5 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Aug 08, 2010 at 12:39:04PM +0200, Pawel Jakub Dawidek wrote: > On Sun, Aug 01, 2010 at 07:58:13PM +0100, Steven Hartland wrote: > > I cant seem to track down the correlation of various ZFS features > > that are present in Solaris and the corresponding FreeBSD versions. > >=20 > > I'm interested in the zfs diff command. Could anyone tell me where > > this info is, or do they know off hand when we're likely to see this > > added to the FreeBSD implementation of zfs? > >=20 > > http://arc.opensolaris.org/caselog/PSARC/2010/105/20100328_tim.haley >=20 > We're are working on getting most recent ZFS into FreeBSD, but this > particular functionality is not in OpenSolaris yet. At this point I also > can't give you any dates. BTW. zfs diff was just committed to OpenSolaris. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --/2994txjAzEdQwm5 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxe2q8ACgkQForvXbEpPzSd+wCg2dtQfMA+1eu4LsNlXR9+zUqD epEAoMGW6r3IZka0I+G+QvauYHM69fmr =qBwd -----END PGP SIGNATURE----- --/2994txjAzEdQwm5-- From owner-freebsd-fs@FreeBSD.ORG Sun Aug 8 17:53:09 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8C7611065675; Sun, 8 Aug 2010 17:53:09 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 636E78FC14; Sun, 8 Aug 2010 17:53:09 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o78Hr9HH001839; Sun, 8 Aug 2010 17:53:09 GMT (envelope-from mm@freefall.freebsd.org) Received: (from mm@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o78Hr9SJ001835; Sun, 8 Aug 2010 17:53:09 GMT (envelope-from mm) Date: Sun, 8 Aug 2010 17:53:09 GMT Message-Id: <201008081753.o78Hr9SJ001835@freefall.freebsd.org> To: mm@FreeBSD.org, freebsd-fs@FreeBSD.org, mm@FreeBSD.org From: mm@FreeBSD.org Cc: Subject: Re: kern/148655: [zfs] Booting from a degraded raidz no longer works in 8-STABLE [regression] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Aug 2010 17:53:09 -0000 Synopsis: [zfs] Booting from a degraded raidz no longer works in 8-STABLE [regression] Responsible-Changed-From-To: freebsd-fs->mm Responsible-Changed-By: mm Responsible-Changed-When: Sun Aug 8 17:53:09 UTC 2010 Responsible-Changed-Why: I am taking this PR. http://www.freebsd.org/cgi/query-pr.cgi?pr=148655 From owner-freebsd-fs@FreeBSD.ORG Sun Aug 8 23:25:40 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BBB3C1065673 for ; Sun, 8 Aug 2010 23:25:40 +0000 (UTC) (envelope-from cryx-freebsd@h3q.com) Received: from mail.h3q.com (mail.h3q.com [213.73.89.199]) by mx1.freebsd.org (Postfix) with ESMTP id DE08A8FC13 for ; Sun, 8 Aug 2010 23:25:39 +0000 (UTC) Received: (qmail 18191 invoked from network); 8 Aug 2010 22:58:57 -0000 Received: from mail.h3q.com (HELO mail.h3q.com) (cryx) by mail.h3q.com with AES256-SHA encrypted SMTP; 8 Aug 2010 22:58:57 -0000 Message-ID: <4C5F36B0.3070505@h3q.com> Date: Mon, 09 Aug 2010 00:58:56 +0200 From: Philipp Wuensche User-Agent: Postbox 1.1.5 (Macintosh/20100613) MIME-Version: 1.0 To: Andriy Gapon References: <4C4C15DE.1030802@icyb.net.ua> In-Reply-To: <4C4C15DE.1030802@icyb.net.ua> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: zfs loader: allow access to any filesystem in a pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Aug 2010 23:25:40 -0000 Andriy Gapon wrote: > This is a ZFS boot patch that I would like to present you for review and testing: > http://people.freebsd.org/~avg/zfsboot.diff > > Currently in zfsloader you can only switch between pools, but you can only > access one predetermined filesystem within a pool - wither root dataset of a > pool or a filesystem pointed to by bootfs pool property. > With this patch it is possible to access any filesystem in a pool and, thus, to > boot kernel and modules from an alternative filesystem. This is another > stepping stone in so called Boot Environments support. This is great, exactly what I need for my Boot Environment stuff: http://anonsvn.h3q.com/projects/freebsd-patches/wiki/manageBE greetings, philipp From owner-freebsd-fs@FreeBSD.ORG Sun Aug 8 23:50:05 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F1AE81065674 for ; Sun, 8 Aug 2010 23:50:04 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by mx1.freebsd.org (Postfix) with ESMTP id 9F1CE8FC0A for ; Sun, 8 Aug 2010 23:50:04 +0000 (UTC) Received: by qyk11 with SMTP id 11so1722377qyk.13 for ; Sun, 08 Aug 2010 16:50:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=+Vr8iuy/bwHB+HiV9SJFo4AGJHp3UEKti+ou9oaddVQ=; b=XpTJsVM3phMsd9VJZp/+w3I2O17GaxYPCAGJBLrn+dqMwv+imTNyAW6LeUum8JLVid lhcoza6pSjjfYP6ZoVKtSQvyDhc/DNu1F21dmMDYUtbi02XQartfDr1VBiqdNEvqI1Js 8RArrf0ZMCI1NVnr3vqKA5zUyaxJbPgjPN3N0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=ZFgow74/2SqylqrZNfjgFM29R8l83pTfybfdlGDmoR2nHb2xLJ7dmvVqTPzn/TfA1x vH8IXz+8dp+0jBuzc+gPEAbz0n4dT1gm7uc4wAjJQZvhvrSTEgysFCK7pyhCjjm125ZJ 0ETv7uE7Ggb9Kzk9uZgee8ELDlginXeUHd5zY= MIME-Version: 1.0 Received: by 10.224.104.153 with SMTP id p25mr8153339qao.98.1281309663049; Sun, 08 Aug 2010 16:21:03 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.229.251.6 with HTTP; Sun, 8 Aug 2010 16:21:03 -0700 (PDT) In-Reply-To: <20100613144240.GV13238@deviant.kiev.zoral.com.ua> References: <20100603143501.GA3176@a91-153-117-195.elisa-laajakaista.fi> <20100613144240.GV13238@deviant.kiev.zoral.com.ua> Date: Mon, 9 Aug 2010 01:21:03 +0200 X-Google-Sender-Auth: krODq5mtkRxwjsvi6157yuZ27fc Message-ID: From: Attilio Rao To: Kostik Belousov Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, Jaakko Heinonen Subject: Re: syncer vnode leak because of nmount() race X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Aug 2010 23:50:05 -0000 I think that the patch looks good. Did you test it? Otherwise you could ask to gianni@ to test it out. Thanks, Attilio 2010/6/13 Kostik Belousov : > On Fri, Jun 04, 2010 at 05:00:49PM +0200, Attilio Rao wrote: >> 2010/6/3 Jaakko Heinonen : >> > >> > Hi, >> > >> > I have been playing with devfs and stress2 recently and I was able to >> > trigger a syncer vnode leak with devfs.sh. As far as I can tell it is = a >> > nmount(2) (initial) mount race against update mount. The code in >> > vfs_domount() is protected with vfs_busy()/vfs_unbusy() but it doesn't >> > protect against update mounts. The protection by Giant is defeated >> > because devfs_root() may sleep due to sx_xlock(). vfs_domount() calls >> > VFS_ROOT() before allocating the syncer vnode. VI_MOUNT vnode flag >> > doesn't provide sufficient protection against update mounts either. >> >> Thanks for this bug report. >> I think that, luckilly, it is not a very common condition to have the >> mount still in flight and get updates... :) >> However, I think that the real breakage here is that the check on >> mnt->mnt_syncer is done lockless and it is unsafe. It might really be >> protected by the mount interlock but that's tricky because >> vfs_allocate_syncvnode() sleeps. Also re-checking the condition (after >> the allocation takes place) is not too simple here. > Is it =C2=A0? I think that the patch below would fix the issue, by > syncronizing mnt_syncer by syncer mutex. > > On the other hand, I prefer the jh' patch, because it seemingly disallows > parallel updates of the mount point. I believe that mp->mnt_vnodecovered > should be stable after the call to vfs_busy() succeeded. > > >> I found also this bug when rewriting the syncer and I resolved that by >> using a separate flag for that (in my case it was simpler and more >> beneficial actually for some other reasons, but you may do the same >> thing with a mnt_kern_flag entry). >> If you have no time I can work actively on the patch, even if I'm >> confident, luckilly, this problem is very hard to happen in the real >> life. >> >> Additively, note that vfs_busy() here is not intended to protect >> against such situations but against unmount. >> Also, I'm very unsure about what Giant protection might bring here. >> IMHO we might probabilly remove it at all as long as all the sleeping >> point present make it completely unuseful if anything (and I don't see >> a reason why it may be needed). >> >> > PS. vfs_unbusy(9) manual page is out of date after r184554 and IMO >> > =C2=A0 =C2=A0vfs_busy(9) manual page is misleading because it talks ab= out >> > =C2=A0 =C2=A0synchronizing access to a mount point. >> >> May you be more precise on what is misleading please? > > diff --git a/sys/kern/vfs_mount.c b/sys/kern/vfs_mount.c > index 088d939..053bd68 100644 > --- a/sys/kern/vfs_mount.c > +++ b/sys/kern/vfs_mount.c > @@ -1034,14 +1034,10 @@ vfs_domount( > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0else > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0mp->mnt_kern_flag &=3D ~MNTK_ASYNC; > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0MNT_IUNLOCK(mp); > - =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if ((mp->mnt_flag & MN= T_RDONLY) =3D=3D 0) { > - =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 if (mp->mnt_syncer =3D=3D NULL) > - =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 error =3D vfs_allocate_syncvnode(mp); > - =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 } else { > - =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 if (mp->mnt_syncer !=3D NULL) > - =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 vrele(mp->mnt_syncer); > - =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 mp->mnt_syncer =3D NULL; > - =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 } > + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if ((mp->mnt_flag & MN= T_RDONLY) =3D=3D 0) > + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 error =3D vfs_allocate_syncvnode(mp); > + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 else > + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 vfs_deallocate_syncvnode(mp); > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0vfs_unbusy(mp); > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0VI_LOCK(vp); > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0vp->v_iflag &=3D ~= VI_MOUNT; > diff --git a/sys/kern/vfs_subr.c b/sys/kern/vfs_subr.c > index ff6744a..6e4bb12 100644 > --- a/sys/kern/vfs_subr.c > +++ b/sys/kern/vfs_subr.c > @@ -3400,12 +3400,31 @@ vfs_allocate_syncvnode(struct mount *mp) > =C2=A0 =C2=A0 =C2=A0 =C2=A0/* XXX - vn_syncer_add_to_worklist() also grab= s and drops sync_mtx. */ > =C2=A0 =C2=A0 =C2=A0 =C2=A0mtx_lock(&sync_mtx); > =C2=A0 =C2=A0 =C2=A0 =C2=A0sync_vnode_count++; > + =C2=A0 =C2=A0 =C2=A0 if (mp->mnt_syncer =3D=3D NULL) { > + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 mp->mnt_syncer =3D vp; > + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 vp =3D NULL; > + =C2=A0 =C2=A0 =C2=A0 } > =C2=A0 =C2=A0 =C2=A0 =C2=A0mtx_unlock(&sync_mtx); > =C2=A0 =C2=A0 =C2=A0 =C2=A0BO_UNLOCK(bo); > - =C2=A0 =C2=A0 =C2=A0 mp->mnt_syncer =3D vp; > + =C2=A0 =C2=A0 =C2=A0 if (vp !=3D NULL) > + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 vgone(vp); > =C2=A0 =C2=A0 =C2=A0 =C2=A0return (0); > =C2=A0} > > +void > +vfs_deallocate_syncvnode(struct mount *mp) > +{ > + =C2=A0 =C2=A0 =C2=A0 struct vnode *vp; > + > + =C2=A0 =C2=A0 =C2=A0 mtx_lock(&sync_mtx); > + =C2=A0 =C2=A0 =C2=A0 vp =3D mp->mnt_syncer; > + =C2=A0 =C2=A0 =C2=A0 if (vp !=3D NULL) > + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 mp->mnt_syncer =3D NUL= L; > + =C2=A0 =C2=A0 =C2=A0 mtx_unlock(&sync_mtx); > + =C2=A0 =C2=A0 =C2=A0 if (vp !=3D NULL) > + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 vrele(vp); > +} > + > =C2=A0/* > =C2=A0* Do a lazy sync of the filesystem. > =C2=A0*/ > @@ -3484,15 +3503,16 @@ sync_reclaim(struct vop_reclaim_args *ap) > > =C2=A0 =C2=A0 =C2=A0 =C2=A0bo =3D &vp->v_bufobj; > =C2=A0 =C2=A0 =C2=A0 =C2=A0BO_LOCK(bo); > - =C2=A0 =C2=A0 =C2=A0 vp->v_mount->mnt_syncer =3D NULL; > + =C2=A0 =C2=A0 =C2=A0 mtx_lock(&sync_mtx); > + =C2=A0 =C2=A0 =C2=A0 if (vp->v_mount->mnt_syncer =3D=3D vp) > + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 vp->v_mount->mnt_synce= r =3D NULL; > =C2=A0 =C2=A0 =C2=A0 =C2=A0if (bo->bo_flag & BO_ONWORKLST) { > - =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 mtx_lock(&sync_mtx); > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0LIST_REMOVE(bo, bo= _synclist); > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0syncer_worklist_le= n--; > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0sync_vnode_count--= ; > - =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 mtx_unlock(&sync_mtx); > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0bo->bo_flag &=3D ~= BO_ONWORKLST; > =C2=A0 =C2=A0 =C2=A0 =C2=A0} > + =C2=A0 =C2=A0 =C2=A0 mtx_unlock(&sync_mtx); > =C2=A0 =C2=A0 =C2=A0 =C2=A0BO_UNLOCK(bo); > > =C2=A0 =C2=A0 =C2=A0 =C2=A0return (0); > diff --git a/sys/sys/mount.h b/sys/sys/mount.h > index 20dcf64..dcff206 100644 > --- a/sys/sys/mount.h > +++ b/sys/sys/mount.h > @@ -731,6 +731,7 @@ int vfs_busy(struct mount *, int); > =C2=A0int =C2=A0 =C2=A0vfs_export =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /* process mount export info */ > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(struct mount *, struct export_a= rgs *); > =C2=A0int =C2=A0 =C2=A0vfs_allocate_syncvnode(struct mount *); > +void =C2=A0 vfs_deallocate_syncvnode(struct mount *); > =C2=A0int =C2=A0 =C2=A0vfs_donmount(struct thread *td, int fsflags, struc= t uio *fsoptions); > =C2=A0void =C2=A0 vfs_getnewfsid(struct mount *); > =C2=A0struct cdev *vfs_getrootfsid(struct mount *); > --=20 Peace can only be achieved by understanding - A. Einstein From owner-freebsd-fs@FreeBSD.ORG Mon Aug 9 11:06:55 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7CB0C106564A for ; Mon, 9 Aug 2010 11:06:55 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 502708FC30 for ; Mon, 9 Aug 2010 11:06:55 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o79B6taB048999 for ; Mon, 9 Aug 2010 11:06:55 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o79B6sY0048997 for freebsd-fs@FreeBSD.org; Mon, 9 Aug 2010 11:06:54 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 9 Aug 2010 11:06:54 GMT Message-Id: <201008091106.o79B6sY0048997@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Aug 2010 11:06:55 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/149297 fs [zfs] zpool iostat fails to show 666GB o kern/149173 fs [patch] [zfs] make OpenSolaris installa o kern/149022 fs [hang] File system operations hangs with suspfs state o kern/149015 fs [zfs] [patch] misc fixes for ZFS code to build on Glib o kern/149014 fs [zfs] [patch] declarations in ZFS libraries/utilities o kern/149013 fs [zfs] [patch] make ZFS makefiles use the libraries fro o kern/148504 fs [zfs] ZFS' zpool does not allow replacing drives to be o kern/148490 fs [zfs]: zpool attach - resilver bidirectionally, and re o kern/148368 fs [zfs] ZFS hanging forever on 8.1-PRERELEASE o bin/148296 fs [zfs] [loader] [patch] Very slow probe in /usr/src/sys o kern/148204 fs [nfs] UDP NFS causes overload o kern/148138 fs [zfs] zfs raidz pool commands freeze o kern/147903 fs [zfs] [panic] Kernel panics on faulty zfs device o kern/147881 fs [zfs] [patch] ZFS "sharenfs" doesn't allow different " o kern/147790 fs [zfs] zfs set acl(mode|inherit) fails on existing zfs o kern/147420 fs [ufs] [panic] ufs_dirbad, nullfs, jail panic (corrupt o kern/147292 fs [nfs] [patch] readahead missing in nfs client options o kern/146941 fs [zfs] [panic] Kernel Double Fault - Happens constantly o kern/146786 fs [zfs] zpool import hangs with checksum errors o kern/146708 fs [ufs] [panic] Kernel panic in softdep_disk_write_compl o kern/146528 fs [zfs] Severe memory leak in ZFS on i386 o kern/146502 fs [nfs] FreeBSD 8 NFS Client Connection to Server o kern/146375 fs [nfs] [patch] Typos in macro variables names in sys/fs o kern/145778 fs [zfs] [panic] panic in zfs_fuid_map_id (known issue fi s kern/145712 fs [zfs] cannot offline two drives in a raidz2 configurat o kern/145411 fs [xfs] [panic] Kernel panics shortly after mounting an o kern/145309 fs [disklabel]: Editing disk label invalidates the whole o kern/145272 fs [zfs] [panic] Panic during boot when accessing zfs on o kern/145246 fs [ufs] dirhash in 7.3 gratuitously frees hashes when it o kern/145238 fs [zfs] [panic] kernel panic on zpool clear tank o kern/145229 fs [zfs] Vast differences in ZFS ARC behavior between 8.0 o kern/145189 fs [nfs] nfsd performs abysmally under load o kern/144929 fs [ufs] [lor] vfs_bio.c + ufs_dirhash.c o kern/144458 fs [nfs] [patch] nfsd fails as a kld p kern/144447 fs [zfs] sharenfs fsunshare() & fsshare_main() non functi o kern/144416 fs [panic] Kernel panic on online filesystem optimization s kern/144415 fs [zfs] [panic] kernel panics on boot after zfs crash o kern/144234 fs [zfs] Cannot boot machine with recent gptzfsboot code o kern/143825 fs [nfs] [panic] Kernel panic on NFS client o kern/143345 fs [ext2fs] [patch] extfs minor header cleanups to better o kern/143212 fs [nfs] NFSv4 client strange work ... o kern/143184 fs [zfs] [lor] zfs/bufwait LOR o kern/142924 fs [ext2fs] [patch] Small cleanup for the inode struct in o kern/142914 fs [zfs] ZFS performance degradation over time o kern/142878 fs [zfs] [vfs] lock order reversal o kern/142597 fs [ext2fs] ext2fs does not work on filesystems with real o kern/142489 fs [zfs] [lor] allproc/zfs LOR o kern/142466 fs Update 7.2 -> 8.0 on Raid 1 ends with screwed raid [re o kern/142401 fs [ntfs] [patch] Minor updates to NTFS from NetBSD o kern/142306 fs [zfs] [panic] ZFS drive (from OSX Leopard) causes two o kern/142068 fs [ufs] BSD labels are got deleted spontaneously o kern/141897 fs [msdosfs] [panic] Kernel panic. msdofs: file name leng o kern/141463 fs [nfs] [panic] Frequent kernel panics after upgrade fro o kern/141305 fs [zfs] FreeBSD ZFS+sendfile severe performance issues ( o kern/141091 fs [patch] [nullfs] fix panics with DIAGNOSTIC enabled o kern/141086 fs [nfs] [panic] panic("nfs: bioread, not dir") on FreeBS o kern/141010 fs [zfs] "zfs scrub" fails when backed by files in UFS2 o kern/140888 fs [zfs] boot fail from zfs root while the pool resilveri o kern/140661 fs [zfs] [patch] /boot/loader fails to work on a GPT/ZFS- o kern/140640 fs [zfs] snapshot crash o kern/140134 fs [msdosfs] write and fsck destroy filesystem integrity o kern/140068 fs [smbfs] [patch] smbfs does not allow semicolon in file o kern/139725 fs [zfs] zdb(1) dumps core on i386 when examining zpool c o kern/139715 fs [zfs] vfs.numvnodes leak on busy zfs o bin/139651 fs [nfs] mount(8): read-only remount of NFS volume does n o kern/139597 fs [patch] [tmpfs] tmpfs initializes va_gen but doesn't u o kern/139564 fs [zfs] [panic] 8.0-RC1 - Fatal trap 12 at end of shutdo o kern/139407 fs [smbfs] [panic] smb mount causes system crash if remot o kern/139363 fs [nfs] diskless root nfs mount from non FreeBSD server o kern/138790 fs [zfs] ZFS ceases caching when mem demand is high o kern/138421 fs [ufs] [patch] remove UFS label limitations o kern/138202 fs mount_msdosfs(1) see only 2Gb f kern/137037 fs [zfs] [hang] zfs rollback on root causes FreeBSD to fr o kern/136968 fs [ufs] [lor] ufs/bufwait/ufs (open) o kern/136945 fs [ufs] [lor] filedesc structure/ufs (poll) o kern/136944 fs [ffs] [lor] bufwait/snaplk (fsync) o kern/136873 fs [ntfs] Missing directories/files on NTFS volume o kern/136865 fs [nfs] [patch] NFS exports atomic and on-the-fly atomic o kern/136470 fs [nfs] Cannot mount / in read-only, over NFS o kern/135546 fs [zfs] zfs.ko module doesn't ignore zpool.cache filenam o kern/135469 fs [ufs] [panic] kernel crash on md operation in ufs_dirb o kern/135050 fs [zfs] ZFS clears/hides disk errors on reboot o kern/134491 fs [zfs] Hot spares are rather cold... o kern/133676 fs [smbfs] [panic] umount -f'ing a vnode-based memory dis o kern/133614 fs [panic] panic: ffs_truncate: read-only filesystem o kern/133174 fs [msdosfs] [patch] msdosfs must support utf-encoded int f kern/133150 fs [zfs] Page fault with ZFS on 7.1-RELEASE/amd64 while w o kern/132960 fs [ufs] [panic] panic:ffs_blkfree: freeing free frag o kern/132397 fs reboot causes filesystem corruption (failure to sync b o kern/132331 fs [ufs] [lor] LOR ufs and syncer o kern/132237 fs [msdosfs] msdosfs has problems to read MSDOS Floppy o kern/132145 fs [panic] File System Hard Crashes o kern/131441 fs [unionfs] [nullfs] unionfs and/or nullfs not combineab o kern/131360 fs [nfs] poor scaling behavior of the NFS server under lo o kern/131342 fs [nfs] mounting/unmounting of disks causes NFS to fail o bin/131341 fs makefs: error "Bad file descriptor" on the mount poin o kern/130920 fs [msdosfs] cp(1) takes 100% CPU time while copying file o kern/130229 fs [iconv] usermount fails on fs that need iconv o kern/130210 fs [nullfs] Error by check nullfs o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129488 fs [smbfs] Kernel "bug" when using smbfs in smbfs_smb.c: o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/129059 fs [zfs] [patch] ZFS bootloader whitelistable via WITHOUT f kern/128829 fs smbd(8) causes periodic panic on 7-RELEASE o kern/127420 fs [gjournal] [panic] Journal overflow on gmirrored gjour o kern/127029 fs [panic] mount(8): trying to mount a write protected zi o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file o kern/125895 fs [ffs] [panic] kernel: panic: ffs_blkfree: freeing free s kern/125738 fs [zfs] [request] SHA256 acceleration in ZFS p kern/124621 fs [ext3] [patch] Cannot mount ext2fs partition f bin/124424 fs [zfs] zfs(8): zfs list -r shows strange snapshots' siz o kern/123939 fs [msdosfs] corrupts new files o kern/122380 fs [ffs] ffs_valloc:dup alloc (Soekris 4801/7.0/USB Flash o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o bin/121898 fs [nullfs] pwd(1)/getcwd(2) fails with Permission denied o bin/121779 fs [ufs] snapinfo(8) (and related tools?) only work for t o bin/121366 fs [zfs] [patch] Automatic disk scrubbing from periodic(8 o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha f kern/120991 fs [panic] [fs] [snapshot] System crashes when manipulati o kern/120483 fs [ntfs] [patch] NTFS filesystem locking changes o kern/120482 fs [ntfs] [patch] Sync style changes between NetBSD and F f kern/119735 fs [zfs] geli + ZFS + samba starting on boot panics 7.0-B o kern/118912 fs [2tb] disk sizing/geometry problem with large array o kern/118713 fs [minidump] [patch] Display media size required for a k o bin/118249 fs mv(1): moving a directory changes its mtime o kern/118107 fs [ntfs] [panic] Kernel panic when accessing a file at N o bin/117315 fs [smbfs] mount_smbfs(8) and related options can't mount o kern/117314 fs [ntfs] Long-filename only NTFS fs'es cause kernel pani o kern/117158 fs [zfs] zpool scrub causes panic if geli vdevs detach on o bin/116980 fs [msdosfs] [patch] mount_msdosfs(8) resets some flags f o conf/116931 fs lack of fsck_cd9660 prevents mounting iso images with p kern/116608 fs [msdosfs] [patch] msdosfs fails to check mount options o kern/116583 fs [ffs] [hang] System freezes for short time when using o kern/116170 fs [panic] Kernel panic when mounting /tmp o kern/115645 fs [snapshots] [panic] lockmgr: thread 0xc4c00d80, not ex o bin/115361 fs [zfs] mount(8) gets into a state where it won't set/un o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o kern/113852 fs [smbfs] smbfs does not properly implement DFS referral o bin/113838 fs [patch] [request] mount(8): add support for relative p o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/111843 fs [msdosfs] Long Names of files are incorrectly created o kern/111782 fs [ufs] dump(8) fails horribly for large filesystems s bin/111146 fs [2tb] fsck(8) fails on 6T filesystem o kern/109024 fs [msdosfs] mount_msdosfs: msdosfs_iconv: Operation not o kern/109010 fs [msdosfs] can't mv directory within fat32 file system o bin/107829 fs [2TB] fdisk(8): invalid boundary checking in fdisk / w o kern/106107 fs [ufs] left-over fsck_snapshot after unfinished backgro o kern/106030 fs [ufs] [panic] panic in ufs from geom when a dead disk o kern/104406 fs [ufs] Processes get stuck in "ufs" state under persist o kern/104133 fs [ext2fs] EXT2FS module corrupts EXT2/3 filesystems o kern/103035 fs [ntfs] Directories in NTFS mounted disc images appear o kern/101324 fs [smbfs] smbfs sometimes not case sensitive when it's s o kern/99290 fs [ntfs] mount_ntfs ignorant of cluster sizes o kern/97377 fs [ntfs] [patch] syntax cleanup for ntfs_ihash.c o kern/95222 fs [iso9660] File sections on ISO9660 level 3 CDs ignored o kern/94849 fs [ufs] rename on UFS filesystem is not atomic o kern/94769 fs [ufs] Multiple file deletions on multi-snapshotted fil o kern/94733 fs [smbfs] smbfs may cause double unlock o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D o kern/92272 fs [ffs] [hang] Filling a filesystem while creating a sna f kern/91568 fs [ufs] [panic] writing to UFS/softupdates DVD media in o kern/91134 fs [smbfs] [patch] Preserve access and modification time a kern/90815 fs [smbfs] [patch] SMBFS with character conversions somet o kern/88657 fs [smbfs] windows client hang when browsing a samba shar o kern/88266 fs [smbfs] smbfs does not implement UIO_NOCOPY and sendfi o kern/87859 fs [smbfs] System reboot while umount smbfs. o bin/86765 fs [patch] bsdlabel(8) assigning wrong fs type. o kern/86587 fs [msdosfs] rm -r /PATH fails with lots of small files o kern/85326 fs [smbfs] [panic] saving a file via samba to an overquot o kern/84589 fs [2TB] 5.4-STABLE unresponsive during background fsck 2 o kern/80088 fs [smbfs] Incorrect file time setting on NTFS mounted vi o kern/73484 fs [ntfs] Kernel panic when doing `ls` from the client si o bin/73019 fs [ufs] fsck_ufs(8) cannot alloc 607016868 bytes for ino o kern/71774 fs [ntfs] NTFS cannot "see" files on a WinXP filesystem o kern/68978 fs [panic] [ufs] crashes with failing hard disk, loose po o kern/65920 fs [nwfs] Mounted Netware filesystem behaves strange o kern/65901 fs [smbfs] [patch] smbfs fails fsx write/truncate-down/tr o kern/61503 fs [smbfs] mount_smbfs does not work as non-root o kern/55617 fs [smbfs] Accessing an nsmb-mounted drive via a smb expo o kern/51685 fs [hang] Unbounded inode allocation causes kernel to loc o kern/51583 fs [nullfs] [patch] allow to work with devices and socket o kern/36566 fs [smbfs] System reboot with dead smb mount and umount o kern/33464 fs [ufs] soft update inconsistencies after system crash o kern/18874 fs [2TB] 32bit NFS servers export wrong negative values t 189 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon Aug 9 12:53:06 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 511281065676; Mon, 9 Aug 2010 12:53:06 +0000 (UTC) (envelope-from idaho@bydgoszcz.wsinf.edu.pl) Received: from mail.bydgoszcz.wsinf.edu.pl (onm164.internetdsl.tpnet.pl [83.0.218.164]) by mx1.freebsd.org (Postfix) with ESMTP id 0CB748FC1D; Mon, 9 Aug 2010 12:53:05 +0000 (UTC) Received: from [192.168.138.150] (localhost [127.0.0.1]) by mail.bydgoszcz.wsinf.edu.pl (Postfix) with ESMTP id E6804FFD7; Mon, 9 Aug 2010 14:31:22 +0200 (CEST) Message-ID: <4C5FF51A.3060401@bydgoszcz.wsinf.edu.pl> Date: Mon, 09 Aug 2010 14:31:22 +0200 From: =?UTF-8?B?xYF1a2FzeiBXxIVzaWtvd3NraQ==?= User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 MIME-Version: 1.0 To: Martin Matuska References: <201008051630.o75GUBig092787@freefall.freebsd.org> In-Reply-To: <201008051630.o75GUBig092787@freefall.freebsd.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org Subject: Re: Re: kern/148655: [zfs] Booting from a degraded raidz no longer works in 8-STABLE [regression] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Aug 2010 12:53:06 -0000 W dniu 20:59, Martin Matuska pisze: > A proposed patch is attached. > > The function vdev_read_phys() (sys/boot/zfs/zfsimpl.c, #325) does call > vdev->v_phys_read() without checking if that function is registered. > > This check should be done in vdev_read_phys before doing anything else. > > vdev_create initializes vdev->v_phys_read as 0 and unavailable vdevs > keep this value. Works great for raidz1. Thank you. -- Best regards, Lukasz Wasikowski From owner-freebsd-fs@FreeBSD.ORG Mon Aug 9 17:27:20 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EC259106566C; Mon, 9 Aug 2010 17:27:20 +0000 (UTC) (envelope-from jh@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id C35038FC1B; Mon, 9 Aug 2010 17:27:20 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o79HRKWm027703; Mon, 9 Aug 2010 17:27:20 GMT (envelope-from jh@freefall.freebsd.org) Received: (from jh@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o79HRKS5027699; Mon, 9 Aug 2010 17:27:20 GMT (envelope-from jh) Date: Mon, 9 Aug 2010 17:27:20 GMT Message-Id: <201008091727.o79HRKS5027699@freefall.freebsd.org> To: jh@FreeBSD.org, freebsd-fs@FreeBSD.org, jh@FreeBSD.org From: jh@FreeBSD.org Cc: Subject: Re: bin/86765: [patch] bsdlabel(8) assigning wrong fs type. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Aug 2010 17:27:21 -0000 Synopsis: [patch] bsdlabel(8) assigning wrong fs type. Responsible-Changed-From-To: freebsd-fs->jh Responsible-Changed-By: jh Responsible-Changed-When: Mon Aug 9 17:27:20 UTC 2010 Responsible-Changed-Why: Take. http://www.freebsd.org/cgi/query-pr.cgi?pr=86765 From owner-freebsd-fs@FreeBSD.ORG Mon Aug 9 20:49:24 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 705D71065670 for ; Mon, 9 Aug 2010 20:49:24 +0000 (UTC) (envelope-from andriy@irbisnet.com) Received: from smtp105.rog.mail.re2.yahoo.com (smtp105.rog.mail.re2.yahoo.com [206.190.36.83]) by mx1.freebsd.org (Postfix) with SMTP id E45EC8FC0C for ; Mon, 9 Aug 2010 20:49:23 +0000 (UTC) Received: (qmail 23947 invoked from network); 9 Aug 2010 20:22:43 -0000 Received: from smtp.irbisnet.com (andriy@99.235.226.221 with login) by smtp105.rog.mail.re2.yahoo.com with SMTP; 09 Aug 2010 13:22:42 -0700 PDT X-Yahoo-SMTP: dz9sigaswBA5kWoYWVTZrGHmIs2vaKgG1w-- X-YMail-OSG: i2FJq5AVM1k2LH7igqSYmiA5BEYNUdGfWNFc4xzWXCS2pWM QDrDmK9Ne_qQvMYtbLV0IJM4BCcAG6NEZlkpPlOf_bh__.escq6m.WVD6fqq t.7OXoxrud4WjO.1Vzydkszl2c05M4RDC77ZSDtsgRZwbat3_PdsF.kD5JUY Jja3KXK_ni8SSNOVcuodD_kL9FoSZz7QLG0j7.WVpnX_7k8LDhejjTLEob.3 YhbA0CyB1UHtK.XYDj4L5HX21Q.R5xE2a8k3Hc.NcuBSRBtC.CG5jM8sdeIS NQA0dRZdZHKbfk7V6cEEwY.SWK5IJMoxFTa1pJxU.UkmUxleWBukW1p6uoHI Ffigue_7oAJUlHhlaSG0ySH3L82KfAyr7MBh5EkMSHqY3eIoOLbhpAqiQoSQ OOFNUCX2hZn6rWFfx4RD6sVq8PztWyOfzvb9GRCu78F5Y X-Yahoo-Newman-Property: ymail-3 Received: by smtp.irbisnet.com (Postfix, from userid 80) id 82DE31143F; Mon, 9 Aug 2010 16:22:42 -0400 (EDT) To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Date: Mon, 09 Aug 2010 16:22:42 -0400 From: Andriy Bakay Message-ID: X-Sender: andriy@irbisnet.com User-Agent: RoundCube Webmail/0.4-beta Subject: Re: Converting sysinstalled FreeBSD into ZFS-only server. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Aug 2010 20:49:24 -0000 1. In your post you are using dedicated partition for swap. Did it provide any advantages versus swap on ZFS volume? 2. You are suggesting to set 'vm_kmem_size' value to 150% of RAM. What pros. and cons. against following formula: vm_kmem_size = RAM / 2 vfs.zfs.arc_max = vm_kmem_size - 512M 3. Could you be more verbose about your ZFS layout, what major advantages it provide against for example the following: http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot. I found it very interesting but I need more info. Regards, Andriy From owner-freebsd-fs@FreeBSD.ORG Mon Aug 9 22:14:28 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AB55B1065672 for ; Mon, 9 Aug 2010 22:14:28 +0000 (UTC) (envelope-from marco@tolstoy.tols.org) Received: from tolstoy.tols.org (tolstoy.tols.org [IPv6:2a02:898:0:20::57:1]) by mx1.freebsd.org (Postfix) with ESMTP id 47DE18FC17 for ; Mon, 9 Aug 2010 22:14:27 +0000 (UTC) Received: from tolstoy.tols.org (localhost [127.0.0.1]) by tolstoy.tols.org (8.14.4/8.14.4) with ESMTP id o79MENXb012947 for ; Mon, 9 Aug 2010 22:14:23 GMT (envelope-from marco@tolstoy.tols.org) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.96.1 at tolstoy.tols.org Received: (from marco@localhost) by tolstoy.tols.org (8.14.4/8.14.4/Submit) id o79MEN4m012946 for freebsd-fs@freebsd.org; Mon, 9 Aug 2010 22:14:23 GMT (envelope-from marco) Date: Mon, 9 Aug 2010 22:14:23 +0000 From: Marco van Tol To: freebsd-fs@freebsd.org Message-ID: <20100809221423.GF4964@tolstoy.tols.org> Mail-Followup-To: freebsd-fs@freebsd.org References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on tolstoy.tols.org Subject: Re: Converting sysinstalled FreeBSD into ZFS-only server. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Aug 2010 22:14:28 -0000 On Mon, Aug 09, 2010 at 04:22:42PM -0400, Andriy Bakay wrote: > > 1. In your post you are using dedicated partition for swap. Did it provide > any advantages versus swap on ZFS volume? For this one I think the answer is that it is very common to set dumpdev in /etc/rc.conf to your swap device. Dumping core to a software raid is, uhm, suboptimal. :-) > 2. You are suggesting to set 'vm_kmem_size' value to 150% of RAM. What > pros. and cons. against following formula: > > vm_kmem_size = RAM / 2 > vfs.zfs.arc_max = vm_kmem_size - 512M > > 3. Could you be more verbose about your ZFS layout, what major advantages > it provide against for example the following: > http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot. I found it very interesting > but I need more info. Me 2. Marco van Tol -- The hardest thing in the world to understand is the income tax - Albert Einstein From owner-freebsd-fs@FreeBSD.ORG Tue Aug 10 02:14:23 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7F064106566B; Tue, 10 Aug 2010 02:14:23 +0000 (UTC) (envelope-from thomas@gibfest.dk) Received: from mail.tyknet.dk (mail.tyknet.dk [213.150.42.155]) by mx1.freebsd.org (Postfix) with ESMTP id D21858FC12; Tue, 10 Aug 2010 02:14:22 +0000 (UTC) Received: from [10.32.67.59] (fw.int.webpartner.dk [213.150.34.98]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.tyknet.dk (Postfix) with ESMTPSA id 97C7F638DFC; Tue, 10 Aug 2010 04:14:21 +0200 (CEST) X-DKIM: OpenDKIM Filter v1.1.2 mail.tyknet.dk 97C7F638DFC DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=gibfest.dk; s=default; t=1281406461; bh=SlJEfu7lrQh/DiPOb7yaCqsqLnz+uXDlmjMao/JHJzg=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=I0/STZWabsqQpZQY6v7eLquKorJkKIOzZshzNWvphXPHBjDnx4Mx24bHJxH+VugCb YfoxlCS6mkSvSX55TyLZ88+stoQJM0Go7lONRVQgcC//KbKRLgvYezjglWTmKUDigb qC6JKoCstad4zjnQjQv9aXox6zm1de/67Wgcdu/0= Message-ID: <4C60B5FD.9080603@gibfest.dk> Date: Tue, 10 Aug 2010 04:14:21 +0200 From: Thomas Steen Rasmussen User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.11) Gecko/20100711 Lightning/1.0b1 Thunderbird/3.0.6 MIME-Version: 1.0 To: pjd@FreeBSD.org References: <4C57E20E.2030908@gibfest.dk> <20100806135001.GF1710@garage.freebsd.pl> <4C5ECA78.6010803@gibfest.dk> In-Reply-To: <4C5ECA78.6010803@gibfest.dk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: HAST initial sync speed X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Aug 2010 02:14:23 -0000 On 08-08-2010 17:17, Thomas Steen Rasmussen wrote: > On 06-08-2010 15:50, Pawel Jakub Dawidek wrote: >> On Tue, Aug 03, 2010 at 11:31:58AM +0200, Thomas Rasmussen wrote: >> >>> Hello list, >>> >>> I finally got my ZFS/HAST setup up and running, or trying to at least. >>> I am wondering how fast the initial HAST sync normally is - I created >>> these 4 HAST providers yesterday on 4 146 gig drives, and they still each >>> have over 90 gigabytes 'dirty' today. The machines are powerful (dell >>> r710) and are otherwise idle, and they are connected to the same gigabit >>> switch. >>> >>> I can supply details about any part of the configuration if needed, but I >>> just wanted to ask if you guys believe something is wrong here. I can't help >>> but think, if the initial sync takes 24+ hours, then if I ever need to >>> replace one of the servers, I will be without redundancy until the new >>> server reaches 0 'dirty' bytes, correct ? >>> >> Correct, but synchronizartion should take much, much less time. >> Is dirty count actually decreasing? >> >> > Hello, > > Yes it was decreasing steadily but very slowly. It finished between thursday > evening and friday morning, and the dirty count is now 0. All in all it took over > 72 hours. It was transferring around 20mbits while doing this. However, if I > copied a large file to the primary HAST node, it would use up a lot more > bandwidth. It is like HAST was synchronizing the "empty space" with lower > priority or something. Does that make any sense ? The servers are not in > production so I can perform any testing needed. Thank you for your reply. > > Regards > > Thomas Steen Rasmussen > Hello again, I just wanted to include the configs here for completeness: /etc/hast.conf: ----------------------------- resource hasthd4 { local /dev/label/hd4 on server1 { remote 192.168.0.15 } on server2 { remote 192.168.0.14 } } resource hasthd5 { local /dev/label/hd5 on server1 { remote 192.168.0.15 } on server2 { remote 192.168.0.14 } } resource hasthd6 { local /dev/label/hd6 on server1 { remote 192.168.0.15 } on server2 { remote 192.168.0.14 } } resource hasthd7 { local /dev/label/hd7 on server1 { remote 192.168.0.15 } on server2 { remote 192.168.0.14 } } ----------------------------- To create the setup I ran the following commands on both servers: glabel label ssd0 /dev/mfid1 glabel label ssd1 /dev/mfid2 glabel label hd4 /dev/mfid3 glabel label hd5 /dev/mfid4 glabel label hd6 /dev/mfid5 glabel label hd7 /dev/mfid6 And on server2: [root@server2 ~]# hastctl create hasthd4 [root@server2 ~]# hastctl create hasthd5 [root@server2 ~]# hastctl create hasthd6 [root@server2 ~]# hastctl create hasthd7 [root@server2 ~]# /etc/rc.d/hastd start [root@server2 ~]# hastctl role secondary all And on server1: [root@server1 ~]# hastctl create hasthd4 [root@server1 ~]# hastctl create hasthd5 [root@server1 ~]# hastctl create hasthd6 [root@server1 ~]# hastctl create hasthd7 [root@server1 ~]# /etc/rc.d/hastd start [root@server1 ~]# hastctl role primary all This made the HAST devices appear on server1 under /dev/hast/ Then I created the ZFS filesystem on top, on server1: zpool create hatank raidz2 /dev/hast/hasthd4 /dev/hast/hasthd5 /dev/hast/hasthd6 /dev/hast/hasthd7 cache /dev/label/ssd0 /dev/label/ssd1 This resulted in the following "hastctl status" output, on server1: hasthd4: role: primary provname: hasthd4 localpath: /dev/label/hd4 extentsize: 2097152 keepdirty: 64 remoteaddr: 192.168.0.15 replication: memsync status: complete dirty: 146051956736 bytes hasthd5: role: primary provname: hasthd5 localpath: /dev/label/hd5 extentsize: 2097152 keepdirty: 64 remoteaddr: 192.168.0.15 replication: memsync status: complete dirty: 146045665280 bytes hasthd6: role: primary provname: hasthd6 localpath: /dev/label/hd6 extentsize: 2097152 keepdirty: 64 remoteaddr: 192.168.0.15 replication: memsync status: complete dirty: 146047762432 bytes hasthd7: role: primary provname: hasthd7 localpath: /dev/label/hd7 extentsize: 2097152 keepdirty: 64 remoteaddr: 192.168.0.15 replication: memsync status: complete dirty: 146047762432 bytes -------------------------------------------------- The problem again is simply that the initial synchronization took way too long. If I copy a large file to the primary HAST server now it syncs very quickly. I am open for any input, I obviously can't really use HAST before this problem is solved. Thank you again. Thomas Steen Rasmussen From owner-freebsd-fs@FreeBSD.ORG Tue Aug 10 02:45:12 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B5CDE1065673 for ; Tue, 10 Aug 2010 02:45:12 +0000 (UTC) (envelope-from andriy@irbisnet.com) Received: from smtp102.rog.mail.re2.yahoo.com (smtp102.rog.mail.re2.yahoo.com [206.190.36.80]) by mx1.freebsd.org (Postfix) with SMTP id 54D968FC16 for ; Tue, 10 Aug 2010 02:45:11 +0000 (UTC) Received: (qmail 30071 invoked from network); 10 Aug 2010 02:45:11 -0000 Received: from smtp.irbisnet.com (andriy@99.235.226.221 with login) by smtp102.rog.mail.re2.yahoo.com with SMTP; 09 Aug 2010 19:45:11 -0700 PDT X-Yahoo-SMTP: dz9sigaswBA5kWoYWVTZrGHmIs2vaKgG1w-- X-YMail-OSG: RPsDAwQVM1nekv1wL60y.djpqEcADECc8p99k78DJQFfoFf PwjUwK82hfK8ywInywPMK6xpDlAYV.h3oXso1WQ.bm4EO4vh.Boovm5V_BH8 ENXtJP3WWEJWQIsvWCKyw9P_WoYFSQ.SJJwTkqkTWOa4QLlTLwlLoTAEwZBU BKv1Bqu6.ehLP30WNSgCfnmxN9qam7PInvg.a2sI_1ekknKFfAnJiLyTJ1gu FkGX3ZLAVK6UDOLRuPgsxTGiDXdKYvcJjKJltThl8z.V30b0Mv8n_6_XRlvM O9ufSbT5whdAv1JzzlF15YEYMTTGEFkQJ4ioL.j1zngZHRNwwRfZz7n8r7o. sg4qaM8Si_oWxRoLd8QltN93rMpGBytS3tjdwVxSotI99IRVqy5QQmSnVKQc c7zuesGDB1Rn5qXfOLWUffGfpK1YhRVJ9Y9lRBYdJ19QA X-Yahoo-Newman-Property: ymail-3 Received: from prime.irbisnet.com (prime.irbisnet.vpn [10.78.76.4]) by smtp.irbisnet.com (Postfix) with ESMTPSA id A5DC11140C for ; Mon, 9 Aug 2010 22:45:10 -0400 (EDT) Message-ID: <4C60BD34.7020805@irbisnet.com> Date: Mon, 09 Aug 2010 22:45:08 -0400 From: Andriy Bakay User-Agent: Thunderbird 2.0.0.23 (X11/20091223) MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <20100809221423.GF4964@tolstoy.tols.org> In-Reply-To: <20100809221423.GF4964@tolstoy.tols.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: Converting sysinstalled FreeBSD into ZFS-only server. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Aug 2010 02:45:12 -0000 > On Mon, Aug 09, 2010 at 04:22:42PM -0400, Andriy Bakay wrote: >> 1. In your post you are using dedicated partition for swap. Did it provide >> any advantages versus swap on ZFS volume? > > For this one I think the answer is that it is very common to set dumpdev > in /etc/rc.conf to your swap device. Dumping core to a software raid > is, uhm, suboptimal. :-) > Yes, I know ZFS volume cannot be used as dumpdev (sorry I should put it in my question) and ZFS volume did not guaranteed to be contiguous (which could be performance consideration), but beside this, what else? >> 2. You are suggesting to set 'vm_kmem_size' value to 150% of RAM. What >> pros. and cons. against following formula: >> >> vm_kmem_size = RAM / 2 >> vfs.zfs.arc_max = vm_kmem_size - 512M >> >> 3. Could you be more verbose about your ZFS layout, what major advantages >> it provide against for example the following: >> http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot. I found it very interesting >> but I need more info. > > Me 2. > > Marco van Tol > From owner-freebsd-fs@FreeBSD.ORG Tue Aug 10 07:55:45 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8FB551065672 for ; Tue, 10 Aug 2010 07:55:45 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id D6AE88FC18 for ; Tue, 10 Aug 2010 07:55:44 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 75ACD45C9C; Tue, 10 Aug 2010 09:55:42 +0200 (CEST) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 9AEDA45685; Tue, 10 Aug 2010 09:55:37 +0200 (CEST) Date: Tue, 10 Aug 2010 09:55:28 +0200 From: Pawel Jakub Dawidek To: Thomas Steen Rasmussen Message-ID: <20100810075528.GA1754@garage.freebsd.pl> References: <4C57E20E.2030908@gibfest.dk> <20100806135001.GF1710@garage.freebsd.pl> <4C5ECA78.6010803@gibfest.dk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="IS0zKkzwUGydFO0o" Content-Disposition: inline In-Reply-To: <4C5ECA78.6010803@gibfest.dk> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: HAST initial sync speed X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Aug 2010 07:55:45 -0000 --IS0zKkzwUGydFO0o Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Aug 08, 2010 at 05:17:12PM +0200, Thomas Steen Rasmussen wrote: > On 06-08-2010 15:50, Pawel Jakub Dawidek wrote: > >Correct, but synchronizartion should take much, much less time. > >Is dirty count actually decreasing? > > > > =20 > Hello, >=20 > Yes it was decreasing steadily but very slowly. It finished between=20 > thursday > evening and friday morning, and the dirty count is now 0. All in all it= =20 > took over > 72 hours. It was transferring around 20mbits while doing this. However,= =20 > if I > copied a large file to the primary HAST node, it would use up a lot more > bandwidth. It is like HAST was synchronizing the "empty space" with lower > priority or something. Does that make any sense ? The servers are not in > production so I can perform any testing needed. Thank you for your reply. It does make sense, but HAST is not doing this:) Could you start with veryfing if synchronization is so slow also when you have only one resource configured? --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --IS0zKkzwUGydFO0o Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxhBfAACgkQForvXbEpPzRbyACglMxEZZ7Ursm7C5pOfBOgGIFi yXEAn3DRUPwKiaoh4glXai0uYRd0M1Rd =GuRp -----END PGP SIGNATURE----- --IS0zKkzwUGydFO0o-- From owner-freebsd-fs@FreeBSD.ORG Tue Aug 10 09:36:00 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 047C71065678 for ; Tue, 10 Aug 2010 09:36:00 +0000 (UTC) (envelope-from mgamsjager@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 9143D8FC28 for ; Tue, 10 Aug 2010 09:35:57 +0000 (UTC) Received: by wyj26 with SMTP id 26so13683088wyj.13 for ; Tue, 10 Aug 2010 02:35:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:cc:content-type; bh=2C7kyjDAUE5cN0SWplBq4Ak8297fab2umzczPjWYkDs=; b=bgaO1+qrNnZ7AknEkEYKAtW3aqtJx+wEBODcfJ0romXsa//5LW8ikzDCXEJytqXejN 2l3GsfyqvmmUnO/7+Dx40CC2vfEXROQ097vVfDD0KskZyqyUc61iy0Lo1h9rweXS9MKN JsGnKpoYL0ML3urKT8dNY4mVMfAWVRGpRHeiM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; b=raxICRNapSRnrEjohZwPLmdGqy5Kfy1WR0zNbkGt5UTq9S0FFwUkr9AOnVwHr2Dkno 7Y5kiuibZ3IXJa9LU9Tfc7hFzK2h5ULRWcmeHWm79691w/KOvHIk+pVGVah7gAIQ5lw6 Naw9rH1EZPF4jHdQXq0fV5lptMOIM32vmyb1U= Received: by 10.227.135.15 with SMTP id l15mr14867452wbt.203.1281431600267; Tue, 10 Aug 2010 02:13:20 -0700 (PDT) MIME-Version: 1.0 Received: by 10.227.148.3 with HTTP; Tue, 10 Aug 2010 02:12:49 -0700 (PDT) In-Reply-To: References: From: Matthias Gamsjager Date: Tue, 10 Aug 2010 11:12:49 +0200 Message-ID: To: Andriy Bakay Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-fs@freebsd.org Subject: Re: Converting sysinstalled FreeBSD into ZFS-only server. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Aug 2010 09:36:00 -0000 about point 2: well most users who did had problems with random zfs crashes did fix it by using 150%. check the freebsd forum about this. sorry if I can't give any hard foundations about this claim but it does help. On Mon, Aug 9, 2010 at 10:22 PM, Andriy Bakay wrote: > > 1. In your post you are using dedicated partition for swap. Did it provide > any advantages versus swap on ZFS volume? > 2. You are suggesting to set 'vm_kmem_size' value to 150% of RAM. What > pros. and cons. against following formula: > > vm_kmem_size = RAM / 2 > vfs.zfs.arc_max = vm_kmem_size - 512M > > 3. Could you be more verbose about your ZFS layout, what major advantages > it provide against for example the following: > http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot. I found it very interesting > but I need more info. > > Regards, > Andriy > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Tue Aug 10 11:00:54 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D36221065675 for ; Tue, 10 Aug 2010 11:00:54 +0000 (UTC) (envelope-from alexvpetrov@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 669BF8FC12 for ; Tue, 10 Aug 2010 11:00:54 +0000 (UTC) Received: by eyh6 with SMTP id 6so4353224eyh.13 for ; Tue, 10 Aug 2010 04:00:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:subject:date :user-agent:mime-version:content-type:content-transfer-encoding :message-id; bh=qzBR52XRClf98R5wyW9L239km1HKVcFbg6iTc9eL2Io=; b=GxLYasLE/IJWz3b4TcCV9IY3aFqPwl8AFYI76xKwKlI4jn7M0r9rOV/z3eA6i73Ijw IhrR7C/IVt861hTQ39ZOndury7oQCygVedvCfjB6fpt301C60FTvosfD0KmsBtjsSqp3 IuZGfiA8jS4GNQMHKjIH3V6Px1bSfaiASYyuc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:mime-version:content-type :content-transfer-encoding:message-id; b=YITcr1WRvat5a1el6Bjr+w8Uzpk4QBw9pV7OYz+dHD8ZWPWEkcKoXhyZNMwQiGq874 h4qcs5m2v2CvfQerKSittI8axIdLQGXB6tS9mKTKwTjBof0zs2N1NyykYmoaZjDJzHGt TgqMQDwoqn9HtH3eOxf6qeInfC076+ATBkTso= Received: by 10.213.105.207 with SMTP id u15mr13362856ebo.83.1281436641487; Tue, 10 Aug 2010 03:37:21 -0700 (PDT) Received: from alex.super ([91.195.101.86]) by mx.google.com with ESMTPS id u9sm9513204eeh.23.2010.08.10.03.37.19 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 10 Aug 2010 03:37:20 -0700 (PDT) From: "Alex V. Petrov" To: freebsd-fs@freebsd.org Date: Tue, 10 Aug 2010 18:37:17 +0800 User-Agent: KMail/1.13.5 (FreeBSD/8.1-STABLE; KDE/4.4.5; amd64; ; ) MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201008101837.17311.alexvpetrov@gmail.com> Subject: zpool - low speed write (moved from freebsd-stable) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Aug 2010 11:00:54 -0000 Hi All! $ dd if=/dev/random of=/tank/test bs=3M count=1000 1000+0 records in 1000+0 records out 3145728000 bytes transferred in 298.153293 secs (10550707 bytes/sec) What, i think, very-very low :-( FreeBSD alex.super 8.1-STABLE FreeBSD 8.1-STABLE #76: Mon Aug 2 20:19:09 KRAST 2010 alex@alex.super:/usr/obj/usr/src/sys/ALEX amd64 real memory = 4294967296 (4096 MB) avail memory = 4098732032 (3908 MB) zpool status -v pool: tank state: ONLINE scrub: scrub completed after 7h12m with 0 errors on Tue Aug 3 04:54:14 2010 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 ada4 ONLINE 0 0 0 ada2 ONLINE 0 0 0 ada3 ONLINE 0 0 0 errors: No known data errors zpool history History for 'tank': 2009-07-16.19:46:24 zpool create tank ad12 2009-12-13.14:58:46 zpool add tank ad8 2010-04-24.01:59:41 zpool upgrade tank 2010-05-09.02:16:34 zpool add tank ada3 2010-05-25.17:57:12 zpool scrub tank 2010-06-27.16:02:45 zpool scrub tank 2010-08-02.21:41:53 zpool scrub tank ada2 at ahcich2 bus 0 scbus2 target 0 lun 0 ada2: ATA-8 SATA 2.x device ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada2: Command Queueing enabled ada2: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C) ada3 at ahcich3 bus 0 scbus3 target 0 lun 0 ada3: ATA-8 SATA 2.x device ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada3: Command Queueing enabled ada3: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C) ada4 at ahcich4 bus 0 scbus4 target 0 lun 0 ada4: ATA-8 SATA 2.x device ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada4: Command Queueing enabled smartd daily output: Checking health of /dev/ada2: OK Checking health of /dev/ada3: OK Checking health of /dev/ada4: OK /boot/loader.conf: ahci_load="YES" sem_load="YES" snd_hda_load="YES" nvidia_load="YES" linux_load="YES" wlan_xauth_load="YES" vboxdrv_load="YES" atapicam_load="YES" coretemp_load="YES" aio_load="YES" **************************************** Next without "ahci_load="YES"" zpool destroy tank zpool create tank ad8 ad10 ad12 dd if=/dev/zero of=/tank/test.zero bs=3M count=1000 1000+0 records in 1000+0 records out 3145728000 bytes transferred in 83.219741 secs (37800262 bytes/sec) zpool offline tank ad10 cannot offline ad10: no valid replicas zpool destroy tank zpool create tank ad8 ad12 dd if=/dev/zero of=/tank/test.zero bs=3M count=1000 1000+0 records in 1000+0 records out 3145728000 bytes transferred in 126.873102 secs (24794286 bytes/sec) dd if=/dev/zero of=/tank/test.zero bs=64k count=1000000 1000000+0 records in 1000000+0 records out 65536000000 bytes transferred in 1735.680273 secs (37758106 bytes/sec) zpool destroy tank zpool create tank ad8 dd if=/dev/zero of=/tank/test.zero bs=3M count=1000 1000+0 records in 1000+0 records out 3145728000 bytes transferred in 39.550739 secs (79536517 bytes/sec) dd if=/dev/zero of=/tank/test.zero bs=64k count=100000 100000+0 records in 100000+0 records out 6553600000 bytes transferred in 90.810344 secs (72167990 bytes/sec) 1-disk zpool faster multy-disks? **************************************** zpool destroy tank At one time 3 dd dd if=/dev/zero of=/dev/ad8 bs=64k count=1000000 1000000+0 records in 1000000+0 records out 65536000000 bytes transferred in 605.173955 secs (108292830 bytes/sec) dd if=/dev/zero of=/dev/ad10 bs=64k count=1000000 1000000+0 records in 1000000+0 records out 65536000000 bytes transferred in 759.946393 secs (86237662 bytes/sec) dd if=/dev/zero of=/dev/ad12 bs=64k count=1000000 1000000+0 records in 1000000+0 records out 65536000000 bytes transferred in 605.139062 secs (108299074 bytes/sec) Disks ad4 ad6 ad8 ad10 ad12 da0 da1 KB/t 0,00 16,00 64,00 64,00 64,00 0,00 0,00 tps 0 1 1667 1329 1646 0 0 MB/s 0,00 0,02 104 83,04 103 0,00 0,00 %busy 0 0 93 94 93 0 0 ie this problem is not the controller ----- Alex V. Petrov From owner-freebsd-fs@FreeBSD.ORG Tue Aug 10 11:14:03 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A5EDB106564A for ; Tue, 10 Aug 2010 11:14:03 +0000 (UTC) (envelope-from martin@lispworks.com) Received: from lwfs1-cam.cam.lispworks.com (mail.lispworks.com [193.34.186.230]) by mx1.freebsd.org (Postfix) with ESMTP id 4539E8FC14 for ; Tue, 10 Aug 2010 11:14:02 +0000 (UTC) Received: from higson.cam.lispworks.com (IDENT:U2FsdGVkX1/HdUq3R8YWX0Y6QiVAbUgJslEIf6969yk@higson [192.168.1.7]) by lwfs1-cam.cam.lispworks.com (8.14.3/8.14.3) with ESMTP id o7AB33iR056068; Tue, 10 Aug 2010 12:03:03 +0100 (BST) (envelope-from martin@lispworks.com) Received: from higson.cam.lispworks.com by higson.cam.lispworks.com (8.13.1) id o7AB33Uj024473; Tue, 10 Aug 2010 12:03:03 +0100 Received: (from martin@localhost) by higson.cam.lispworks.com (8.13.1/8.13.1/Submit) id o7AB33R4024470; Tue, 10 Aug 2010 12:03:03 +0100 Date: Tue, 10 Aug 2010 12:03:03 +0100 Message-Id: <201008101103.o7AB33R4024470@higson.cam.lispworks.com> From: Martin Simmons To: freebsd-fs@freebsd.org In-reply-to: (message from Andriy Bakay on Mon, 09 Aug 2010 16:22:42 -0400) References: Subject: Re: Converting sysinstalled FreeBSD into ZFS-only server. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Aug 2010 11:14:03 -0000 >>>>> On Mon, 09 Aug 2010 16:22:42 -0400, Andriy Bakay said: > > 3. Could you be more verbose about your ZFS layout, what major advantages > it provide against for example the following: > http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot. I found it very interesting > but I need more info. Are you referring to mounting "/" from the inner dataset called system/root (rather that using the root dataset like in GPTZFSBoot)? I think an inner dataset is better for "/" because it gives you more flexibility later. In particular, you can do zfs destroy on it (which is impossible on the root dataset). You can also have alternate roots for different OS versions. __Martin From owner-freebsd-fs@FreeBSD.ORG Tue Aug 10 12:23:05 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1BF421065674 for ; Tue, 10 Aug 2010 12:23:05 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) by mx1.freebsd.org (Postfix) with ESMTP id C2AED8FC1C for ; Tue, 10 Aug 2010 12:23:04 +0000 (UTC) Received: by people.fsn.hu (Postfix, from userid 1001) id 29AC73CC53C; Tue, 10 Aug 2010 14:23:02 +0200 (CEST) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000065, version=1.2.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MF-ACE0E1EA [pR: 15.8024] X-CRM114-CacheID: sfid-20100810_14224_A68C04F6 X-CRM114-Status: Good ( pR: 15.8024 ) X-DSPAM-Result: Whitelisted X-DSPAM-Processed: Tue Aug 10 14:23:01 2010 X-DSPAM-Confidence: 0.7005 X-DSPAM-Probability: 0.0000 X-DSPAM-Signature: 4c6144a5659061149830581 X-DSPAM-Factors: 27, From*Attila Nagy , 0.00087, wrote, 0.00230, wrote, 0.00230, FreeBSD, 0.00243, >+>, 0.00449, >+the, 0.00922, >+the, 0.00922, >+is, 0.01000, see+if, 0.01000, see+if, 0.01000, problem+is, 0.01000, >+Well, 0.01000, Subject*file, 0.01000, Subject*file, 0.01000, Subject+Re, 0.01000, wrote+>>, 0.01000, wrote+>>, 0.01000, Subject*can+be, 0.99000, Date*14+22, 0.99000, mount, 0.01000, fixes, 0.01000, client+and, 0.01000, Subject*but, 0.99000, 16+36, 0.01000, Nagy, 0.01000, Nagy, 0.01000, X-Spambayes-Classification: ham; 0.00 Message-ID: <4C614497.4080600@fsn.hu> Date: Tue, 10 Aug 2010 14:22:47 +0200 From: Attila Nagy User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.23) Gecko/20090817 Thunderbird/2.0.0.23 Mnenhy/0.7.6.0 MIME-Version: 1.0 To: Rick Macklem References: <507225020.401114.1281141334084.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <507225020.401114.1281141334084.JavaMail.root@erie.cs.uoguelph.ca> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: NFS problem: file doesn't appear in file listing, but can be accessed directly X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Aug 2010 12:23:05 -0000 On 08/07/2010 02:35 AM, Rick Macklem wrote: >> Subject: Re: NFS problem: file doesn't appear in file listing, but can be accessed directly >> On 03.08.2010, at 16:36, Attila Nagy wrote: >> >>>> You can try replacing the client and server with the experimental >>>> ones and see if that fixes the problem. >>>> For the client: mount with "-t newnfs" instead of "-t nfs" >>>> For the server: start both mountd and nfsd with the "-e" option >>> Sure. It works with the newnfs client, so it must be either a weird >>> interaction between the FreeBSD server and client (old), or a client >>> bug. >> Have you tried comparing on-the-wire NFS requests and responses >> between newnfs and legacy nfs clients for your test case? Maybe you >> can rule out the former like this. Or at least prove that the server >> actually responds correctly to the READDIR request. >> > Well, I looked at a packet capture emailed to me by Attila Nagy and > the filename shows up in the Lookup near the end (the "ls -la"), but > is not in any of the readdir replies (a search from the start of the > capture in wireshark only finds it at the Lookup). Therefore, I think > the problem is w.r.t. the server. (He did this failed case with the > "newnfs", so both clients see the problem.) > > Attila, could you by any chance try switching to the experimental > server ("-e" on both mountd and nfsd) and see if the problem persists? Sure. Switched to -e for both mountd and nfsd and I can see the same. Any ideas about what should I provide to investigate this further? Thanks, From owner-freebsd-fs@FreeBSD.ORG Tue Aug 10 12:32:05 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EFC6D1065675 for ; Tue, 10 Aug 2010 12:32:05 +0000 (UTC) (envelope-from andriy@irbisnet.com) Received: from smtp102.rog.mail.re2.yahoo.com (smtp102.rog.mail.re2.yahoo.com [206.190.36.80]) by mx1.freebsd.org (Postfix) with SMTP id 90F648FC17 for ; Tue, 10 Aug 2010 12:32:05 +0000 (UTC) Received: (qmail 36808 invoked from network); 10 Aug 2010 12:32:04 -0000 Received: from smtp.irbisnet.com (andriy@99.235.226.221 with login) by smtp102.rog.mail.re2.yahoo.com with SMTP; 10 Aug 2010 05:32:04 -0700 PDT X-Yahoo-SMTP: dz9sigaswBA5kWoYWVTZrGHmIs2vaKgG1w-- X-YMail-OSG: qYGpWikVM1na7RuhWi332cKd_sNRDnCSXqjnfR.rmgN.kfS JMjmdyQPddsseBAz0cKOQV_pJFtcyXnmdWlMMEXlUGDILCtatp2MKUmztpvg AV_j7yYRsIX43e4qTYQ13vL6ACwq6SBd9s12G2RYmtnM41KXOyPG4W0RCBnr hDKo0XSsagLDeiqe79CZvBWvELOUL7uTMcMh3gLJOo7pFOzKN8u0POvDC69. Fg150SJh4aUCGFCBtiSFNV2mOcYeQ.ie6omvvFM56COFC4h.dC9EUHBBHU1F tZ6LCoS1GwNSGTeWx1c3x1Vaof8eS99LRRVRWyMx40NFvzrPmpZAzI3tXLtX LqpI6xWDAProej1MnZJzqLjU4yM9QHi7lqe9HGD7kPDrlHcou2ncrM_PbL_7 fVT1LJxu08nJwZ_tgakXZ3eRLog154xN9HBRnCYyN5SYbWJNs4jACTiWH21g ZXwMf X-Yahoo-Newman-Property: ymail-3 Received: from genie.irbisnet.lan (genie.irbisnet.lan [192.168.0.2]) by smtp.irbisnet.com (Postfix) with ESMTPA id A6FE21140C for ; Tue, 10 Aug 2010 08:32:01 -0400 (EDT) Message-Id: From: Andriy Bakay To: freebsd-fs@freebsd.org In-Reply-To: <201008101103.o7AB33R4024470@higson.cam.lispworks.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v936) Date: Tue, 10 Aug 2010 08:31:29 -0400 References: <201008101103.o7AB33R4024470@higson.cam.lispworks.com> X-Mailer: Apple Mail (2.936) Subject: Re: Converting sysinstalled FreeBSD into ZFS-only server. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Aug 2010 12:32:06 -0000 On 10-Aug-10, at 7:03 AM, Martin Simmons wrote: >>>>>> On Mon, 09 Aug 2010 16:22:42 -0400, Andriy Bakay said: >> >> 3. Could you be more verbose about your ZFS layout, what major >> advantages >> it provide against for example the following: >> http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot. I found it very >> interesting >> but I need more info. > > Are you referring to mounting "/" from the inner dataset called > system/root > (rather that using the root dataset like in GPTZFSBoot)? > > I think an inner dataset is better for "/" because it gives you more > flexibility later. In particular, you can do zfs destroy on it > (which is > impossible on the root dataset). You can also have alternate roots > for > different OS versions. Good point about "/", thanks. But what about this one: # zfs create -o canmount=off system/usr # zfs create -o canmount=off system/var Maybe Pawel could comment out. > > __Martin > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Tue Aug 10 12:48:13 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2D3951065679 for ; Tue, 10 Aug 2010 12:48:13 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) by mx1.freebsd.org (Postfix) with ESMTP id D20248FC1C for ; Tue, 10 Aug 2010 12:48:12 +0000 (UTC) Received: by people.fsn.hu (Postfix, from userid 1001) id C46B73CC6D3; Tue, 10 Aug 2010 14:48:10 +0200 (CEST) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MF-ACE0E1EA [pR: 11.7601] X-CRM114-CacheID: sfid-20100810_14475_32427208 X-CRM114-Status: Good ( pR: 11.7601 ) X-DSPAM-Result: Whitelisted X-DSPAM-Processed: Tue Aug 10 14:48:10 2010 X-DSPAM-Confidence: 0.7619 X-DSPAM-Probability: 0.0000 X-DSPAM-Signature: 4c614a8a694271149370396 X-DSPAM-Factors: 27, From*Attila Nagy , 0.00085, wrote, 0.00230, wrote, 0.00230, FreeBSD, 0.00243, FreeBSD, 0.00243, >+It, 0.01000, both+the, 0.01000, I+haven't, 0.01000, ps, 0.01000, but+I've, 0.01000, Subject*file, 0.01000, Subject*file, 0.01000, I've+tried, 0.01000, wrote+>>, 0.01000, wrote+>>, 0.01000, Subject*can+be, 0.99000, things+>, 0.01000, tested, 0.01000, >+1, 0.01000, >+2, 0.01000, Subject*but, 0.99000, default, 0.01000, default, 0.01000, why+the, 0.01000, by+>, 0.01000, at+02, 0.01000, X-Spambayes-Classification: ham; 0.00 Message-ID: <4C614A7D.8030509@fsn.hu> Date: Tue, 10 Aug 2010 14:47:57 +0200 From: Attila Nagy User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.23) Gecko/20090817 Thunderbird/2.0.0.23 Mnenhy/0.7.6.0 MIME-Version: 1.0 To: Rick Macklem References: <163511110.410136.1281226007565.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <163511110.410136.1281226007565.JavaMail.root@erie.cs.uoguelph.ca> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: NFS problem: file doesn't appear in file listing, but can be accessed directly X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Aug 2010 12:48:13 -0000 On 08/08/2010 02:06 AM, Rick Macklem wrote: >> On 07.08.2010, at 02:35, Rick Macklem wrote: >> >> I agree, this must be some kind of server issue then. Still, I wonder >> why the solaris client didn't show the problem... >> >> > 2 things: > 1 - the solaris client uses ReaddirPlus and not Readdir RPCs by > default (I can't remember if that default can be overridden?). I don't know whether this counts or not, but I've tried with an old Solaris (x86) 8 (uname says it's Generic_108529-06). > ps: I don't recall the previous emails mentioning that the "ls" > was ok for a solaris client. > It was in 4C582968.9000303@fsn.hu: "I've mounted the share from an old Solaris box, which sees the file." This means, when I issue an ls | grep problematic_fname, I can see the filename. BTW, this turned out to be incorrect, because I've tested with the same (one of the missing) filename. Doing a find . in the directory on both FreeBSD and Solaris client shows some differences. So what I have now is: both the normal and the experimental NFSD show the problem on both FreeBSD and Solaris clients (only the missing file names are different). On Solaris there are 78 files missing, on FreeBSD there are 193, I haven't checked that they are persistent across reboots or not. From owner-freebsd-fs@FreeBSD.ORG Tue Aug 10 15:13:35 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 053B41065672; Tue, 10 Aug 2010 15:13:35 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D03668FC0A; Tue, 10 Aug 2010 15:13:34 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o7AFDYQ7051671; Tue, 10 Aug 2010 15:13:34 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o7AFDYff051667; Tue, 10 Aug 2010 15:13:34 GMT (envelope-from linimon) Date: Tue, 10 Aug 2010 15:13:34 GMT Message-Id: <201008101513.o7AFDYff051667@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/149495: [zfs] chflags sappend on zfs not working right X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Aug 2010 15:13:35 -0000 Old Synopsis: chflags sappend on zfs not working right New Synopsis: [zfs] chflags sappend on zfs not working right Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Tue Aug 10 15:13:17 UTC 2010 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=149495 From owner-freebsd-fs@FreeBSD.ORG Tue Aug 10 21:44:24 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0254F1065673 for ; Tue, 10 Aug 2010 21:44:24 +0000 (UTC) (envelope-from marco@tolstoy.tols.org) Received: from tolstoy.tols.org (tolstoy.tols.org [IPv6:2a02:898:0:20::57:1]) by mx1.freebsd.org (Postfix) with ESMTP id 4DE468FC12 for ; Tue, 10 Aug 2010 21:44:22 +0000 (UTC) Received: from tolstoy.tols.org (localhost [127.0.0.1]) by tolstoy.tols.org (8.14.4/8.14.4) with ESMTP id o7ALiIPP029653 for ; Tue, 10 Aug 2010 21:44:18 GMT (envelope-from marco@tolstoy.tols.org) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.96.1 at tolstoy.tols.org Received: (from marco@localhost) by tolstoy.tols.org (8.14.4/8.14.4/Submit) id o7ALiIdI029652 for freebsd-fs@freebsd.org; Tue, 10 Aug 2010 21:44:18 GMT (envelope-from marco) Date: Tue, 10 Aug 2010 21:44:18 +0000 From: Marco van Tol To: freebsd-fs@freebsd.org Message-ID: <20100810214418.GA28288@tolstoy.tols.org> Mail-Followup-To: freebsd-fs@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on tolstoy.tols.org Subject: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Aug 2010 21:44:24 -0000 Hi there, What you will find here is a short description of my attempts to find the optimal settings for zfs memory, followed by what I ended up with at this point. Then I started wondering what I was missing, because I think what I'm doing might be plain stupid. Because noone else seems to be doing it this way. I hope it makes a tiny bit of sense. If it doesn't just pretend you didn't read it. ;) All the way at the bottom of the mail you will find some details about the hardware / software. I have been trying, like many others, what would be best practice regarding zfs, kmem_size, arc_max and what not and had trouble finding that reliable magic post that just summed up the magic formula. So I went ahead and mixed a couple of advises I had seen during my search and at some point ended up on a machine with 2 Gigabytes of physical memory with the settings: vm.kmem_size: 3G vfs.zfs.arc_max: 750M The 3G vm.kmem_size on a machine with 2G physical memory comes from something I had read about fragmented kernel memory which would warrant this setting. Now, these settings annoyed me a lot because, it used to be that page cache (and with that I mean active/inactive memory) was auto tuned. That it would (in short) take as much available memory as possible, and just release it when an application needed it. I'm describing it simple here because I am by no means a filesystem and/or kernel expert. At some point I stumbled upon a blog post from Kip Macy, and a reply from Richard Elling at: http://daemonflux.blogspot.com/2010/01/zfs-arc-page-cache-and-1970s-buffer.html Somewhere around this time I started to think that an auto-tuned arc might still be possible given that ZFS releases memory when there is high demand for it. So I googled for "freebsd zfs memory pressure". After reading through a couple of hits I felt like just trying it and ended up with the settings: physical memory: 2G vm.kmem_size: 3G vfs.zfs.arc_max: 1792M Then I setup a couple of terms: - top -I # To monitor active/inactive/wired/free - sysctl kstat.zfs.misc.arcstats.size # (In a loop) To monitor the arc size - # And a term to do some tests while watching those values # test 1 - tar the filesystem to grow the arc from reads -> tar -cf /dev/null / Sure enough the arc grew rapid to some max value. After a little while the values where: 12M Active, 18M Inact, 1789M Wired, 144K Cache, 156M Free kstat.zfs.misc.arcstats.size: 1697754720 Nothing to worry about so far # Test 2 - do a bunch of writes to see if that makes things different -> bash> for i in {1..10} ; do dd if=/dev/zero of=/zfspath/file$i \ bs=1m count=1024 ; done And again the arc would grow and shrink a little bit, leaving the values: 8308K Active, 22M Inact, 1710M Wired, 136K Cache, 235M Free kstat.zfs.misc.arcstats.size: 1596992192 Still nothing to worry about # Test 3 - let an application demand some memory and see what happens. -> perl -e '$x="x"x500_000_000' After perl completed the values were: 5112K Active, 5884K Inact, 932M Wired, 14M Cache, 1019M Free kstat.zfs.misc.arcstats.size: 817991496 No differences in swap usage worth mentioning, I think somewhere around 5 megabytes. Top mentioned pages going into swap very briefly. (Side info: I had run a too large value with the perl step before, which left me with 35MB swap) # Test 4 - All of test 1, 2 and 3 at the same time, for a (probably # lame) attempt at a mixed environment. For test 1 (tar) I excluded # the files where I was writing the files for test 2 (dd). Test 3 ran # in a sleepless loop with the value 1_000_000_000 instead of the 500 # megs I used in the original test 3. Ending values: 21M Active, 7836K Inact, 1672M Wired, 4140K Cache, 272M Free kstat.zfs.misc.arcstats.size: 1570260528 Swap usage didn't change in the running top I was watching. ------ All in all this looks like a close attempt at zfs memory being auto tuned while using maximum amount of memory. The only problem is, nobody else is doing it like this so its very likely that this is not the smart thing to do. What are the first problems the zfs people can think off with a setup like this? Thanks in advance! Marco van Tol ------ Machine Details: zpool status pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz1 ONLINE 0 0 0 gpt/tank_s0 ONLINE 0 0 0 gpt/tank_s1 ONLINE 0 0 0 gpt/tank_s2 ONLINE 0 0 0 gpt/tank_s3 ONLINE 0 0 0 errors: No known data errors hw.machine: amd64 hw.model: Intel(R) Atom(TM) CPU 330 @ 1.60GHz hw.ncpu: 2 hw.physmem: 2135396352 hw.pagesize: 4096 vm.kmem_size: 3221225472 vfs.zfs.version.spa: 14 vfs.zfs.version.zpl: 3 vfs.zfs.prefetch_disable: 1 vfs.zfs.zil_disable: 0 vfs.zfs.zio.use_uma: 0 vfs.zfs.vdev.cache.size: 10485760 vfs.zfs.arc_min: 234881024 vfs.zfs.arc_max: 1879048192 gpart show => 34 976773101 ada0 GPT (466G) 34 128 1 freebsd-boot (64K) 162 4194304 2 freebsd-swap (2.0G) 4194466 972578669 3 freebsd-zfs (464G) => 34 976773101 ada1 GPT (466G) 34 128 1 freebsd-boot (64K) 162 4194304 2 freebsd-swap (2.0G) 4194466 972578669 3 freebsd-zfs (464G) => 34 976773101 ada2 GPT (466G) 34 128 1 freebsd-boot (64K) 162 4194304 2 freebsd-swap (2.0G) 4194466 972578669 3 freebsd-zfs (464G) => 34 976773101 ada3 GPT (466G) 34 128 1 freebsd-boot (64K) 162 4194304 2 freebsd-swap (2.0G) 4194466 972578669 3 freebsd-zfs (464G) swap: /dev/gpt/swap0 none swap sw 0 0 /dev/gpt/swap1 none swap sw 0 0 /dev/gpt/swap2 none swap sw 0 0 /dev/gpt/swap3 none swap sw 0 0 zfs list NAME USED AVAIL REFER MOUNTPOINT tank 83.7G 1.24T 28.4K legacy tank/dirvish 21.6G 106G 21.6G /dirvish tank/home 21.4G 1.24T 21.4G /home tank/mm 30.1G 120G 30.1G /mm tank/root 745M 279M 745M legacy tank/tmp 126K 4.00G 126K /tmp tank/usr 2.95G 13.1G 2.95G /usr tank/var 115M 3.89G 115M /var -- A male gynecologist is like an auto mechanic who never owned a car. - Carrie Snow From owner-freebsd-fs@FreeBSD.ORG Tue Aug 10 22:14:39 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D73D51065690; Tue, 10 Aug 2010 22:14:39 +0000 (UTC) (envelope-from thomas@gibfest.dk) Received: from mail.tyknet.dk (mail.tyknet.dk [213.150.42.155]) by mx1.freebsd.org (Postfix) with ESMTP id 8E2D08FC2E; Tue, 10 Aug 2010 22:14:39 +0000 (UTC) Received: from [10.32.67.58] (fw.int.webpartner.dk [213.150.34.98]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.tyknet.dk (Postfix) with ESMTPSA id 60F3A638DFC; Wed, 11 Aug 2010 00:14:38 +0200 (CEST) X-DKIM: OpenDKIM Filter v1.1.2 mail.tyknet.dk 60F3A638DFC DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=gibfest.dk; s=default; t=1281478478; bh=KCauY8mK0nMzJD7c0NAXP1v/F/gVe6FF4WihnfJfW3A=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=w9iJUcMOKHGcQ3afjGR23vAle2yOXcxyOCpwDUCuE6J6ljEfOKv4jKNoDZIniS7pJ VXPtA28+hxsN7LaW/WY3IHt4GWpeEoXaj795Q19q6WYsnAWuSnTLt4MqZ8lXuowUlP 5Oa3YXVKkR+OBaHwTIHuddga2c6BLTSjWnxDQFOI= Message-ID: <4C61CF4D.4060009@gibfest.dk> Date: Wed, 11 Aug 2010 00:14:37 +0200 From: Thomas Steen Rasmussen User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.11) Gecko/20100711 Lightning/1.0b1 Thunderbird/3.0.6 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4C57E20E.2030908@gibfest.dk> <20100806135001.GF1710@garage.freebsd.pl> <4C5ECA78.6010803@gibfest.dk> <20100810075528.GA1754@garage.freebsd.pl> In-Reply-To: <20100810075528.GA1754@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: HAST initial sync speed X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Aug 2010 22:14:39 -0000 On 10-08-2010 09:55, Pawel Jakub Dawidek wrote: > On Sun, Aug 08, 2010 at 05:17:12PM +0200, Thomas Steen Rasmussen wrote: > >> On 06-08-2010 15:50, Pawel Jakub Dawidek wrote: >> >>> Correct, but synchronizartion should take much, much less time. >>> Is dirty count actually decreasing? >>> >>> >>> >> Hello, >> >> Yes it was decreasing steadily but very slowly. It finished between >> thursday >> evening and friday morning, and the dirty count is now 0. All in all it >> took over >> 72 hours. It was transferring around 20mbits while doing this. However, >> if I >> copied a large file to the primary HAST node, it would use up a lot more >> bandwidth. It is like HAST was synchronizing the "empty space" with lower >> priority or something. Does that make any sense ? The servers are not in >> production so I can perform any testing needed. Thank you for your reply. >> > It does make sense, but HAST is not doing this:) Could you start with > veryfing if synchronization is so slow also when you have only one > resource configured? > > Hello, I can now confirm that synchronization is also slow with only one HAST resource. It is currently running with 2.5 megabits per second... around 300kilobytes per sec is written to the disk according to gstat. An iperf test shows a network potential of about a gigabit (wire speed). Thomas Steen Rasmussen From owner-freebsd-fs@FreeBSD.ORG Wed Aug 11 01:16:15 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B494C1065670 for ; Wed, 11 Aug 2010 01:16:15 +0000 (UTC) (envelope-from universite@ukr.net) Received: from otrada.od.ua (unknown [IPv6:2001:5c0:1000:b::599b]) by mx1.freebsd.org (Postfix) with ESMTP id 6124C8FC12 for ; Wed, 11 Aug 2010 01:16:13 +0000 (UTC) Received: from [10.0.0.10] (phenom.otrada.od.ua [10.0.0.10]) (authenticated bits=0) by otrada.od.ua (8.14.3/8.14.3) with ESMTP id o7B1G6tZ066799 for ; Wed, 11 Aug 2010 04:16:07 +0300 (EEST) (envelope-from universite@ukr.net) X-Authentication-Warning: otrada.od.ua: Host phenom.otrada.od.ua [10.0.0.10] claimed to be [10.0.0.10] Message-ID: <4C61F9CB.1010506@ukr.net> Date: Wed, 11 Aug 2010 04:15:55 +0300 From: "Vladislav V. Prodan" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <20100810214418.GA28288@tolstoy.tols.org> In-Reply-To: <20100810214418.GA28288@tolstoy.tols.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-0.4 required=5.0 tests=ALL_TRUSTED,AWL autolearn=failed version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on mary-teresa.otrada.od.ua X-Virus-Scanned: clamav-milter 0.95.3 at mary-teresa.otrada.od.ua X-Virus-Status: Clean Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Aug 2010 01:16:15 -0000 11.08.2010 0:44, Marco van Tol wrote: > physical memory: 2G > vm.kmem_size: 3G > vfs.zfs.arc_max: 1792M Please, insert into /boot/loader.conf only this options: vm.kmem_size="999M" vm.kmem_size_max="999M" vfs.zfs.arc_max="160M" vfs.zfs.prefetch_disable=1 And after reboot, please run dd... From owner-freebsd-fs@FreeBSD.ORG Wed Aug 11 01:49:23 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 768B11065670 for ; Wed, 11 Aug 2010 01:49:23 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta12.westchester.pa.mail.comcast.net (qmta12.westchester.pa.mail.comcast.net [76.96.59.227]) by mx1.freebsd.org (Postfix) with ESMTP id D5B2B8FC08 for ; Wed, 11 Aug 2010 01:49:22 +0000 (UTC) Received: from omta24.westchester.pa.mail.comcast.net ([76.96.62.76]) by qmta12.westchester.pa.mail.comcast.net with comcast id smP11e0051ei1Bg5CppN1t; Wed, 11 Aug 2010 01:49:22 +0000 Received: from koitsu.dyndns.org ([98.248.41.155]) by omta24.westchester.pa.mail.comcast.net with comcast id sppM1e0013LrwQ23kppM20; Wed, 11 Aug 2010 01:49:22 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id B91B29B425; Tue, 10 Aug 2010 18:49:19 -0700 (PDT) Date: Tue, 10 Aug 2010 18:49:19 -0700 From: Jeremy Chadwick To: freebsd-fs@freebsd.org Message-ID: <20100811014919.GA52992@icarus.home.lan> References: <20100810214418.GA28288@tolstoy.tols.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100810214418.GA28288@tolstoy.tols.org> User-Agent: Mutt/1.5.20 (2009-06-14) Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Aug 2010 01:49:23 -0000 On Tue, Aug 10, 2010 at 09:44:18PM +0000, Marco van Tol wrote: > [...] > > All in all this looks like a close attempt at zfs memory being auto > tuned while using maximum amount of memory. The only problem is, nobody > else is doing it like this so its very likely that this is not the smart > thing to do. I'm not sure what "nobody else is doing it like this" means, but Solaris 10 behaves exactly as you describe -- the ARC takes up as much memory as it can. When an application or other piece of the kernel wants memory which the ARC can free up, it releases memory. There's no tuning required on Solaris; it "just works". At my day job (where we use Solaris 10), when we introduced ZFS into the picture, many of our memory usage monitors began firing indicating lack of free memory due to ARC usage. We had to add some code to our check_system_memory monitor which called kstat to examine the zfs:0:arcstats:size and zfs:0:arcstats:c_min properties and then add that to the overall amount of memory available. Code in question: my $zfsarcstatssize = "zfs:0:arcstats:size"; my $zfsarcstatsmin = "zfs:0:arcstats:c_min"; my $zfsarcsize; my $zfsarcmin = 0; if ($OS eq 'solaris' and -x $kstat) { my @dump = `$kstat -p $zfsarcstatssize $zfsarcstatsmin`; unless ($?) { foreach my $dump (@dump) { chomp $dump; my ($label, $size) = split(/\t/, $dump); if ($label eq $zfsarcstatssize) { $zfsarcsize = sprintf("%.2f", $size/1024/1024); } if ($label eq $zfsarcstatsmin) { $zfsarcmin = sprintf("%.2f", $size/1024/1024); } } $real_available += $zfsarcsize - $zfsarcmin if (defined $zfsarcsize and $zfsarcsize > $zfsarcmin); } } I believe OpenSolaris behaves the same as Solaris 10 in this regard. Their VM model differs from that of FreeBSD. I've talked about this in the past (see paragraph starting with "Fast forward to today"). I can dig up John's original response (talking about how the VM needs to be improved in FreeBSD to accomplish the equivalent of what Solaris does) if need be. http://lists.freebsd.org/pipermail/freebsd-fs/2010-May/008598.html -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Wed Aug 11 03:15:09 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CA6061065678 for ; Wed, 11 Aug 2010 03:15:09 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 822278FC24 for ; Wed, 11 Aug 2010 03:15:09 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEAJOyYUyDaFvO/2dsb2JhbACDFZ4Oqy+SAoEmgyFzBIlA X-IronPort-AV: E=Sophos;i="4.55,350,1278302400"; d="scan'208";a="87996372" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 10 Aug 2010 23:15:08 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 8917BB3F2A; Tue, 10 Aug 2010 23:15:08 -0400 (EDT) Date: Tue, 10 Aug 2010 23:15:08 -0400 (EDT) From: Rick Macklem To: Attila Nagy Message-ID: <853573529.515333.1281496508531.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4C614A7D.8030509@fsn.hu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [24.65.230.102] X-Mailer: Zimbra 6.0.7_GA_2476.RHEL4 (ZimbraWebClient - SAF3 (Mac)/6.0.7_GA_2473.RHEL4_64) Cc: freebsd-fs@freebsd.org Subject: Re: NFS problem: file doesn't appear in file listing, but can be accessed directly X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Aug 2010 03:15:09 -0000 > On 08/08/2010 02:06 AM, Rick Macklem wrote: > >> On 07.08.2010, at 02:35, Rick Macklem wrote: > >> > >> I agree, this must be some kind of server issue then. Still, I > >> wonder [stuff snipped] > Doing a find . in the directory on both FreeBSD and Solaris client > shows > some differences. > So what I have now is: both the normal and the experimental NFSD show > the problem on both FreeBSD and Solaris clients (only the missing file > names are different). > > On Solaris there are 78 files missing, on FreeBSD there are 193, I > haven't checked that they are persistent across reboots or not. Ok, so it seems to be a server issue. Since there are quite a few files missing, I'm surprised others aren't seeing problmes? I suspect it is some interaction between ZFS and the NFS server that is causing this, but that's just a hunch. Possibly a problem w.r.t. how the directory offset cookies are handled. Can you conveniently move the directory to a UFS2 volume and export that, to see if the problem then goes away? Also, what architecture is the server running? (I'm wondering if it might be an endianness or 32/64 bit issue related to the directory offset cookies.? Just wild guesses at this point.) If you can't move the directory to UFS2 or that doesn't fix the problem, all I can think to do is write/run a little program locally on the server that does getdirentries() on the directory, to try and spot something that might confuse the NFS server. (I can write such a program for you, but I'd like to hear if it is a ZFS specific problem first. rick From owner-freebsd-fs@FreeBSD.ORG Wed Aug 11 05:43:56 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3899A1065679 for ; Wed, 11 Aug 2010 05:43:56 +0000 (UTC) (envelope-from mgamsjager@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id D60C78FC14 for ; Wed, 11 Aug 2010 05:43:55 +0000 (UTC) Received: by qwg5 with SMTP id 5so8354877qwg.13 for ; Tue, 10 Aug 2010 22:43:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:cc:content-type; bh=GRBC6+k2fF5SHHT5Pct+BvH6Lu6h0Y98roEkgI+3zb4=; b=IZidGiKbXpnhmM0qo2uW/E8q4YGBXHYbk0aCfVXGTR5V5CjMNZafQoyUIG2T8srkan UejIaNAShG+/m14F7jTl3agGSkIdCqbyNr3lhEqKtTegZiC5zR81antAdv+7ZfglF0rR CUQReEG+IHEbpCPOd6CtbApNhYIhjp9PaNZrI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; b=cc5ojl0vhEp0mtnOd9JRxi7QKBITV/2vtDDCkp8hJQKfHdmY/YjKy6WKCg+pJ775Wd cD+vGUTTtZ9FencAjz5/TseN0gJHwgDGVb/ris1Be+tp22/7xszhcZBQRau8O42kvQ0M 2+uE9DlPwhbKZ3S6KUKX1g9W1dE6PvQOWKPrg= Received: by 10.229.181.210 with SMTP id bz18mr9216915qcb.43.1281505435223; Tue, 10 Aug 2010 22:43:55 -0700 (PDT) MIME-Version: 1.0 Received: by 10.229.184.73 with HTTP; Tue, 10 Aug 2010 22:43:24 -0700 (PDT) In-Reply-To: <4C61F9CB.1010506@ukr.net> References: <20100810214418.GA28288@tolstoy.tols.org> <4C61F9CB.1010506@ukr.net> From: Matthias Gamsjager Date: Wed, 11 Aug 2010 07:43:24 +0200 Message-ID: To: "Vladislav V. Prodan" Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-fs@freebsd.org Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Aug 2010 05:43:56 -0000 > Please, insert into /boot/loader.conf only this options: > vm.kmem_size="999M" > vm.kmem_size_max="999M" > vfs.zfs.arc_max="160M" > vfs.zfs.prefetch_disable=1 > > And after reboot, please run dd... vfs.zfs.arc_max="160M" should be enough to limit the size of the arc. and I'm not sure if that ( vm.kmem_size="999M" ,vm.kmem_size_max="999M") is such a good idea. From owner-freebsd-fs@FreeBSD.ORG Wed Aug 11 06:05:58 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7800C1065672 for ; Wed, 11 Aug 2010 06:05:58 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta12.emeryville.ca.mail.comcast.net (qmta12.emeryville.ca.mail.comcast.net [76.96.27.227]) by mx1.freebsd.org (Postfix) with ESMTP id 57E208FC12 for ; Wed, 11 Aug 2010 06:05:57 +0000 (UTC) Received: from omta03.emeryville.ca.mail.comcast.net ([76.96.30.27]) by qmta12.emeryville.ca.mail.comcast.net with comcast id stiH1e0020b6N64ACu5xeC; Wed, 11 Aug 2010 06:05:57 +0000 Received: from koitsu.dyndns.org ([98.248.41.155]) by omta03.emeryville.ca.mail.comcast.net with comcast id su5w1e0073LrwQ28Pu5wuh; Wed, 11 Aug 2010 06:05:57 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 483B49B425; Tue, 10 Aug 2010 23:05:56 -0700 (PDT) Date: Tue, 10 Aug 2010 23:05:56 -0700 From: Jeremy Chadwick To: Matthias Gamsjager Message-ID: <20100811060556.GA60029@icarus.home.lan> References: <20100810214418.GA28288@tolstoy.tols.org> <4C61F9CB.1010506@ukr.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-fs@freebsd.org Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Aug 2010 06:05:58 -0000 On Wed, Aug 11, 2010 at 07:43:24AM +0200, Matthias Gamsjager wrote: > > Please, insert into /boot/loader.conf only this options: > > vm.kmem_size="999M" > > vm.kmem_size_max="999M" > > vfs.zfs.arc_max="160M" > > vfs.zfs.prefetch_disable=1 > > > > And after reboot, please run dd... > > vfs.zfs.arc_max="160M" should be enough to limit the size of the arc. That depends on exactly what date/time your kernel sources are, and what tag/release you're running. The ARC limiting feature was changed at one point in time (from a "high watermark" to a hard-set limit). The OP didn't include uname -a output, so we don't know what exact version of the kernel/system he's running. > and I'm not sure if that ( vm.kmem_size="999M" > ,vm.kmem_size_max="999M") is such a good idea. Yes, it is a good idea. And you shouldn't need to adjust vm.kmem_size_max, only vm.kmem_size. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Wed Aug 11 06:09:15 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 06C4A106567B for ; Wed, 11 Aug 2010 06:09:15 +0000 (UTC) (envelope-from marco@tolstoy.tols.org) Received: from tolstoy.tols.org (tolstoy.tols.org [IPv6:2a02:898:0:20::57:1]) by mx1.freebsd.org (Postfix) with ESMTP id 908A78FC13 for ; Wed, 11 Aug 2010 06:09:14 +0000 (UTC) Received: from tolstoy.tols.org (localhost [127.0.0.1]) by tolstoy.tols.org (8.14.4/8.14.4) with ESMTP id o7B69AAe035184 for ; Wed, 11 Aug 2010 06:09:10 GMT (envelope-from marco@tolstoy.tols.org) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.96.1 at tolstoy.tols.org Received: (from marco@localhost) by tolstoy.tols.org (8.14.4/8.14.4/Submit) id o7B69Art035183 for freebsd-fs@freebsd.org; Wed, 11 Aug 2010 06:09:10 GMT (envelope-from marco) Date: Wed, 11 Aug 2010 06:09:10 +0000 From: Marco van Tol To: freebsd-fs@freebsd.org Message-ID: <20100811060910.GA34883@tolstoy.tols.org> Mail-Followup-To: freebsd-fs@freebsd.org References: <20100810214418.GA28288@tolstoy.tols.org> <4C61F9CB.1010506@ukr.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on tolstoy.tols.org Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Aug 2010 06:09:15 -0000 On Wed, Aug 11, 2010 at 07:43:24AM +0200, Matthias Gamsjager wrote: > > Please, insert into /boot/loader.conf only this options: > > vm.kmem_size="999M" > > vm.kmem_size_max="999M" > > vfs.zfs.arc_max="160M" > > vfs.zfs.prefetch_disable=1 > > > > And after reboot, please run dd... > > vfs.zfs.arc_max="160M" should be enough to limit the size of the arc. But thats the point. I don't want to limit its size, I want it to take every available piece of memory it can find, while allowing applications to temporarely use large slabs of that same memory during their runtime. Just like I'm used to with UFS. :) > and I'm not sure if that ( vm.kmem_size="999M" > ,vm.kmem_size_max="999M") is such a good idea. Me neither. I'm pretty sure I will hit the "kmem map too small" scenario real quick. What is it you are testing with those settings Vladislav? Marco -- Ik loop door het heden, naar de toekomst. - Discovery Channel From owner-freebsd-fs@FreeBSD.ORG Wed Aug 11 06:17:13 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BC4C41065689 for ; Wed, 11 Aug 2010 06:17:13 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta10.emeryville.ca.mail.comcast.net (qmta10.emeryville.ca.mail.comcast.net [76.96.30.17]) by mx1.freebsd.org (Postfix) with ESMTP id 97C5E8FC15 for ; Wed, 11 Aug 2010 06:17:13 +0000 (UTC) Received: from omta11.emeryville.ca.mail.comcast.net ([76.96.30.36]) by qmta10.emeryville.ca.mail.comcast.net with comcast id suDY1e0030mlR8UAAuHDYi; Wed, 11 Aug 2010 06:17:13 +0000 Received: from koitsu.dyndns.org ([98.248.41.155]) by omta11.emeryville.ca.mail.comcast.net with comcast id suHC1e0013LrwQ28XuHCUY; Wed, 11 Aug 2010 06:17:12 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 1B9D59B42E; Tue, 10 Aug 2010 23:17:12 -0700 (PDT) Date: Tue, 10 Aug 2010 23:17:12 -0700 From: Jeremy Chadwick To: freebsd-fs@freebsd.org Message-ID: <20100811061712.GA60244@icarus.home.lan> References: <20100810214418.GA28288@tolstoy.tols.org> <4C61F9CB.1010506@ukr.net> <20100811060910.GA34883@tolstoy.tols.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100811060910.GA34883@tolstoy.tols.org> User-Agent: Mutt/1.5.20 (2009-06-14) Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Aug 2010 06:17:13 -0000 On Wed, Aug 11, 2010 at 06:09:10AM +0000, Marco van Tol wrote: > On Wed, Aug 11, 2010 at 07:43:24AM +0200, Matthias Gamsjager wrote: > > > Please, insert into /boot/loader.conf only this options: > > > vm.kmem_size="999M" > > > vm.kmem_size_max="999M" > > > vfs.zfs.arc_max="160M" > > > vfs.zfs.prefetch_disable=1 > > > > > > And after reboot, please run dd... > > > > vfs.zfs.arc_max="160M" should be enough to limit the size of the arc. > > But thats the point. I don't want to limit its size, I want it to take > every available piece of memory it can find, while allowing applications > to temporarely use large slabs of that same memory during their runtime. As I described in my previous post (not sure if you've seen it yet), ZFS on FreeBSD / the VM, at this moment in time, doesn't support this ability. Solaris offers this. For the time being, you're going to have to adjust /boot/loader.conf as described below. That's the bottom line. > > and I'm not sure if that ( vm.kmem_size="999M" > > ,vm.kmem_size_max="999M") is such a good idea. > > Me neither. I'm pretty sure I will hit the "kmem map too small" > scenario real quick. What is it you are testing with those settings > Vladislav? You need to adjust/increase vm.kmem_size to ensure you don't experience the "kmem map too small" condition, and make sure that vfs.zfs.arc_max is set to something **lower** than that amount. E.g.: vm.kmem_size="2048M" vfs.zfs.arc_max="1536M" Folks need to remember that vm.kmem_size affects more than just ZFS ARC. It's best to give yourself breathing room. I've been harping about this problem for way too long, and complaining in equal amount that something official needs to be written/documented about tuning. The FreeBSD Wiki for ZFS tuning isn't sufficient as we all know, and as I've stated before I'm willing to write something up but I need open communication with folks familiar with both ZFS and the VM to help in explaining the interaction on a technical level. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Wed Aug 11 06:20:10 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6FD60106566B for ; Wed, 11 Aug 2010 06:20:10 +0000 (UTC) (envelope-from marco@tolstoy.tols.org) Received: from tolstoy.tols.org (tolstoy.tols.org [IPv6:2a02:898:0:20::57:1]) by mx1.freebsd.org (Postfix) with ESMTP id 04C8F8FC18 for ; Wed, 11 Aug 2010 06:20:09 +0000 (UTC) Received: from tolstoy.tols.org (localhost [127.0.0.1]) by tolstoy.tols.org (8.14.4/8.14.4) with ESMTP id o7B6K7P3035341 for ; Wed, 11 Aug 2010 06:20:07 GMT (envelope-from marco@tolstoy.tols.org) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.96.1 at tolstoy.tols.org Received: (from marco@localhost) by tolstoy.tols.org (8.14.4/8.14.4/Submit) id o7B6K60q035340 for freebsd-fs@freebsd.org; Wed, 11 Aug 2010 06:20:06 GMT (envelope-from marco) Date: Wed, 11 Aug 2010 06:20:06 +0000 From: Marco van Tol To: freebsd-fs@freebsd.org Message-ID: <20100811062006.GB34883@tolstoy.tols.org> Mail-Followup-To: freebsd-fs@freebsd.org References: <20100810214418.GA28288@tolstoy.tols.org> <20100811014919.GA52992@icarus.home.lan> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100811014919.GA52992@icarus.home.lan> User-Agent: Mutt/1.4.2.3i X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on tolstoy.tols.org Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Aug 2010 06:20:10 -0000 On Tue, Aug 10, 2010 at 06:49:19PM -0700, Jeremy Chadwick wrote: > On Tue, Aug 10, 2010 at 09:44:18PM +0000, Marco van Tol wrote: > > [...] > > > > All in all this looks like a close attempt at zfs memory being auto > > tuned while using maximum amount of memory. The only problem is, nobody > > else is doing it like this so its very likely that this is not the smart > > thing to do. > > I'm not sure what "nobody else is doing it like this" means, but Solaris > 10 behaves exactly as you describe -- the ARC takes up as much memory as > it can. When an application or other piece of the kernel wants memory > which the ARC can free up, it releases memory. There's no tuning > required on Solaris; it "just works". > > At my day job (where we use Solaris 10), when we introduced ZFS into the > picture, many of our memory usage monitors began firing indicating lack > of free memory due to ARC usage. We had to add some code to our > check_system_memory monitor which called kstat to examine the > zfs:0:arcstats:size and zfs:0:arcstats:c_min properties and then add > that to the overall amount of memory available. Code in question: > > my $zfsarcstatssize = "zfs:0:arcstats:size"; > my $zfsarcstatsmin = "zfs:0:arcstats:c_min"; > my $zfsarcsize; > my $zfsarcmin = 0; > if ($OS eq 'solaris' and -x $kstat) { > my @dump = `$kstat -p $zfsarcstatssize $zfsarcstatsmin`; > unless ($?) { > foreach my $dump (@dump) { > chomp $dump; > my ($label, $size) = split(/\t/, $dump); > if ($label eq $zfsarcstatssize) { > $zfsarcsize = sprintf("%.2f", $size/1024/1024); > } > if ($label eq $zfsarcstatsmin) { > $zfsarcmin = sprintf("%.2f", $size/1024/1024); > } > } > $real_available += $zfsarcsize - $zfsarcmin if (defined $zfsarcsize and $zfsarcsize > $zfsarcmin); > } > } > > I believe OpenSolaris behaves the same as Solaris 10 in this regard. > Their VM model differs from that of FreeBSD. > > I've talked about this in the past (see paragraph starting with "Fast > forward to today"). I can dig up John's original response (talking > about how the VM needs to be improved in FreeBSD to accomplish the > equivalent of what Solaris does) if need be. > > http://lists.freebsd.org/pipermail/freebsd-fs/2010-May/008598.html Thank you for your reply. I have to go to work now first, but I will read more closely what you're writing later today. One thing I realized by the way is that I messed something up in the test 4 "run it all together" scenario. I wrote I used value "1_000_000_000", but that will cause a perl process that is too large. Use 500_000_000 instead and what I wrote happens like I wrote it, together with strongly fluctuation arc sizes as it runs. As expected and desired. By the way, uname -a output for the machine running the tests is (line wrapped): FreeBSD hostname.tld 8.1-PRERELEASE FreeBSD 8.1-PRERELEASE #3: Wed Jul 28 22:38:28 CEST 2010 root@hostname.tld:/usr/obj/usr/src/sys/MAGICA amd64 The RELENG tag is: RELENG_8 Thanks! Marco -- Als de redding het hoogst is, is de nood nabij! From owner-freebsd-fs@FreeBSD.ORG Wed Aug 11 14:37:07 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D5936106567A; Wed, 11 Aug 2010 14:37:07 +0000 (UTC) (envelope-from jh@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id ABC888FC0A; Wed, 11 Aug 2010 14:37:07 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o7BEb73c085506; Wed, 11 Aug 2010 14:37:07 GMT (envelope-from jh@freefall.freebsd.org) Received: (from jh@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o7BEb7nI085502; Wed, 11 Aug 2010 14:37:07 GMT (envelope-from jh) Date: Wed, 11 Aug 2010 14:37:07 GMT Message-Id: <201008111437.o7BEb7nI085502@freefall.freebsd.org> To: freebsd@alistairphipps.com, jh@FreeBSD.org, freebsd-fs@FreeBSD.org From: jh@FreeBSD.org Cc: Subject: Re: kern/145778: [zfs] [panic] panic in zfs_fuid_map_id (known issue fixed in opensolaris) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Aug 2010 14:37:07 -0000 Synopsis: [zfs] [panic] panic in zfs_fuid_map_id (known issue fixed in opensolaris) State-Changed-From-To: open->closed State-Changed-By: jh State-Changed-When: Wed Aug 11 14:37:07 UTC 2010 State-Changed-Why: Duplicate of kern/148709. http://www.freebsd.org/cgi/query-pr.cgi?pr=145778 From owner-freebsd-fs@FreeBSD.ORG Wed Aug 11 19:25:42 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D9601106566C for ; Wed, 11 Aug 2010 19:25:42 +0000 (UTC) (envelope-from marco@tolstoy.tols.org) Received: from tolstoy.tols.org (tolstoy.tols.org [IPv6:2a02:898:0:20::57:1]) by mx1.freebsd.org (Postfix) with ESMTP id 5B0A88FC08 for ; Wed, 11 Aug 2010 19:25:42 +0000 (UTC) Received: from tolstoy.tols.org (localhost [127.0.0.1]) by tolstoy.tols.org (8.14.4/8.14.4) with ESMTP id o7BJPcsf045085 for ; Wed, 11 Aug 2010 19:25:38 GMT (envelope-from marco@tolstoy.tols.org) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.96.1 at tolstoy.tols.org Received: (from marco@localhost) by tolstoy.tols.org (8.14.4/8.14.4/Submit) id o7BJPbQQ045084 for freebsd-fs@freebsd.org; Wed, 11 Aug 2010 19:25:37 GMT (envelope-from marco) Date: Wed, 11 Aug 2010 19:25:37 +0000 From: Marco van Tol To: freebsd-fs@freebsd.org Message-ID: <20100811192537.GA44635@tolstoy.tols.org> Mail-Followup-To: freebsd-fs@freebsd.org References: <20100810214418.GA28288@tolstoy.tols.org> <20100811014919.GA52992@icarus.home.lan> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100811014919.GA52992@icarus.home.lan> User-Agent: Mutt/1.4.2.3i X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on tolstoy.tols.org Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Aug 2010 19:25:43 -0000 On Tue, Aug 10, 2010 at 06:49:19PM -0700, Jeremy Chadwick wrote: > On Tue, Aug 10, 2010 at 09:44:18PM +0000, Marco van Tol wrote: > > [...] > > > > All in all this looks like a close attempt at zfs memory being auto > > tuned while using maximum amount of memory. The only problem is, nobody > > else is doing it like this so its very likely that this is not the smart > > thing to do. > > I'm not sure what "nobody else is doing it like this" means, but Solaris > 10 behaves exactly as you describe -- the ARC takes up as much memory as > it can. When an application or other piece of the kernel wants memory > which the ARC can free up, it releases memory. There's no tuning > required on Solaris; it "just works". Hm by "nobody else is doing it like this" I meant on FreeBSD. I also meant to say that what you will typically see people advising/doing is setting arc_max to a value much less then their total physical memory. What I have done instead is set arc_max to nearly all my physical memory (1792G out of a total of 2GB). In combination with that I set kmem_size to 1.5x physical memory to account for kernel fragmentation. Here is a post that touches that subject: http://lists.freebsd.org/pipermail/freebsd-fs/2010-January/007614.html What I am glad to see happening so far, but I haven't given the system any production levels of stress, is that the arc happely takes up nearly all my memory, and just surrenders it to programs that happen to run, taking it back after they finish. Very nice. (I'll feed it bonny++ and see what happens) My mail wasn't intended to make any complaints, much the contrairy. I am a happy camper ever since I changed the test system to those settings. I think/hope that was clear. :) > At my day job (where we use Solaris 10), when we introduced ZFS into the > picture, many of our memory usage monitors began firing indicating lack > of free memory due to ARC usage. We had to add some code to our > check_system_memory monitor which called kstat to examine the > zfs:0:arcstats:size and zfs:0:arcstats:c_min properties and then add > that to the overall amount of memory available. Code in question: > > my $zfsarcstatssize = "zfs:0:arcstats:size"; > my $zfsarcstatsmin = "zfs:0:arcstats:c_min"; > my $zfsarcsize; > my $zfsarcmin = 0; > if ($OS eq 'solaris' and -x $kstat) { > my @dump = `$kstat -p $zfsarcstatssize $zfsarcstatsmin`; > unless ($?) { > foreach my $dump (@dump) { > chomp $dump; > my ($label, $size) = split(/\t/, $dump); > if ($label eq $zfsarcstatssize) { > $zfsarcsize = sprintf("%.2f", $size/1024/1024); > } > if ($label eq $zfsarcstatsmin) { > $zfsarcmin = sprintf("%.2f", $size/1024/1024); > } > } > $real_available += $zfsarcsize - $zfsarcmin if (defined $zfsarcsize and $zfsarcsize > $zfsarcmin); > } > } Its nice when you can rely on something to act equally to actual free memory, yet provide great speedups at times it can use that memory. > I believe OpenSolaris behaves the same as Solaris 10 in this regard. > Their VM model differs from that of FreeBSD. > > I've talked about this in the past (see paragraph starting with "Fast > forward to today"). I can dig up John's original response (talking > about how the VM needs to be improved in FreeBSD to accomplish the > equivalent of what Solaris does) if need be. > > http://lists.freebsd.org/pipermail/freebsd-fs/2010-May/008598.html I had seen that post before indeed. Thank you. The bottom line of my original mail was: - Hey, I set the arc to just take all my RAM - It grows on reads and writes - It shrinks when programs need the memory it has. - It behaves well in a scenario of combined read/write/other programs. - This is good, really good, what is dangerous about my settings that not everybody just sets the arc to nearly all their physical memory on FreeBSD? That's all, :) Thanks, Marco -- A male gynecologist is like an auto mechanic who never owned a car. - Carrie Snow From owner-freebsd-fs@FreeBSD.ORG Wed Aug 11 20:39:27 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4DEF71065673 for ; Wed, 11 Aug 2010 20:39:27 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id F41B18FC12 for ; Wed, 11 Aug 2010 20:39:26 +0000 (UTC) Received: by vws7 with SMTP id 7so465333vws.13 for ; Wed, 11 Aug 2010 13:39:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type:content-transfer-encoding; bh=Ay3WTDmALffuXobzooaM+Il01zkurx998mVag6I71Zk=; b=MpDI8m/QSfhDnvPvwAMAJmZ+7zgpLWb41URJ52Idz4iTq9OvYeSSwnAUzWTtj9Xg9r aX1UvWRG56cwIFkTlPCj8IPA5Aj4hI1Q1xQsIrQZkByPbU/wff4FjBeMpsvdeR7X63Kt 8uzZoqPHkAW31aQw5rABpkNW6yiwEhdrgJGIc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :content-transfer-encoding; b=rkDWpsLGOLzNIl9LxUj2ARI1ZPnhCrjLgywMbXfyALlZ2lVvLCGNmRJZKH+lwa6ulm RRD4VxCKyCEBVJVxfXJQEXpdXXCuy7EKJJghG4v8BDpJa2ranxPmolsvRKbEjdbReAZq uu7PTGfFvxdRShtruhGX6XZJ/deeT2wBuKkJM= MIME-Version: 1.0 Received: by 10.220.161.201 with SMTP id s9mr11751274vcx.277.1281557332002; Wed, 11 Aug 2010 13:08:52 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.220.49.70 with HTTP; Wed, 11 Aug 2010 13:08:51 -0700 (PDT) In-Reply-To: <20100811192537.GA44635@tolstoy.tols.org> References: <20100810214418.GA28288@tolstoy.tols.org> <20100811014919.GA52992@icarus.home.lan> <20100811192537.GA44635@tolstoy.tols.org> Date: Wed, 11 Aug 2010 13:08:51 -0700 X-Google-Sender-Auth: ASh9V9WZfuG3UWJ5L0TA8ZSc8fI Message-ID: From: Artem Belevich To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Aug 2010 20:39:27 -0000 > The bottom line of my original mail was: > - Hey, I set the arc to just take all my RAM > - It grows on reads and writes > - It shrinks when programs need the memory it has. > - It behaves well in a scenario of combined read/write/other programs. > - This is good, really good, what is dangerous about my settings that > =A0not everybody just sets the arc to nearly all their physical memory on > =A0FreeBSD? The issue is that ARC will give up memory even if there's plenty of it available and sitting in inactive/cache queues. That's particularly nasty in case your system uses some other filesystem besides ZFS. For example try tarring up few gigabytes worth of data from UFS filesystem and see how far your ARC size shrinks. It could be mitigated by setting minimum ARC size to be large enough so ZFS performance does not degrade. The downside is that ARC will not give up memory below its minimum no matter what, so if your APP really needs it, it would have to go to swap. There's a hack floating around that attempts to force kernel into freeing up memory from inactive/cache lists before draining ARC. It does help a bit with this issue, but it's still a hack. --Artem From owner-freebsd-fs@FreeBSD.ORG Wed Aug 11 21:43:06 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0B308106566C for ; Wed, 11 Aug 2010 21:43:06 +0000 (UTC) (envelope-from marco@tolstoy.tols.org) Received: from tolstoy.tols.org (tolstoy.tols.org [IPv6:2a02:898:0:20::57:1]) by mx1.freebsd.org (Postfix) with ESMTP id 92AB78FC0C for ; Wed, 11 Aug 2010 21:43:05 +0000 (UTC) Received: from tolstoy.tols.org (localhost [127.0.0.1]) by tolstoy.tols.org (8.14.4/8.14.4) with ESMTP id o7BLh2C0046753 for ; Wed, 11 Aug 2010 21:43:02 GMT (envelope-from marco@tolstoy.tols.org) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.96.1 at tolstoy.tols.org Received: (from marco@localhost) by tolstoy.tols.org (8.14.4/8.14.4/Submit) id o7BLh27S046752 for freebsd-fs@freebsd.org; Wed, 11 Aug 2010 21:43:02 GMT (envelope-from marco) Date: Wed, 11 Aug 2010 21:43:02 +0000 From: Marco van Tol To: freebsd-fs@freebsd.org Message-ID: <20100811214302.GB44635@tolstoy.tols.org> Mail-Followup-To: freebsd-fs@freebsd.org References: <20100810214418.GA28288@tolstoy.tols.org> <20100811014919.GA52992@icarus.home.lan> <20100811192537.GA44635@tolstoy.tols.org> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on tolstoy.tols.org Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Aug 2010 21:43:06 -0000 On Wed, Aug 11, 2010 at 01:08:51PM -0700, Artem Belevich wrote: > > The bottom line of my original mail was: > > - Hey, I set the arc to just take all my RAM > > - It grows on reads and writes > > - It shrinks when programs need the memory it has. > > - It behaves well in a scenario of combined read/write/other programs. > > - This is good, really good, what is dangerous about my settings that > >  not everybody just sets the arc to nearly all their physical memory on > >  FreeBSD? > > The issue is that ARC will give up memory even if there's plenty of it > available and sitting in inactive/cache queues. That's particularly > nasty in case your system uses some other filesystem besides ZFS. For > example try tarring up few gigabytes worth of data from UFS filesystem > and see how far your ARC size shrinks. It could be mitigated by > setting minimum ARC size to be large enough so ZFS performance does > not degrade. The downside is that ARC will not give up memory below > its minimum no matter what, so if your APP really needs it, it would > have to go to swap. > > There's a hack floating around that attempts to force kernel into > freeing up memory from inactive/cache lists before draining ARC. It > does help a bit with this issue, but it's still a hack. That makes sense Artem, thanks. I think you mean the posts with the perl one-liner I used in my tests as well. (perl variable assignment of 1.5GB in the posts their version) I had seen the posts that mentioned that one and decided to remember the perl hack. :) What I understand from it: - In a UFS/ZFS mixed system - In a scenario where UFS "page cache" took (almost) all available memory - Run a perl one-liner to throw out the UFS active/inactive usage - Kind-off hope you do enough relevant ZFS accesses that you get a good new situation. So, if my worries can shift from fighting with kmem_size and arc_max to fighting with arc_min, that's a fight I like a lot better. Especially on zfs-only systems, I have to admit. Thanks a lot! Marco -- It's fried rice, you plick. -- Lethal Weapon 4 From owner-freebsd-fs@FreeBSD.ORG Wed Aug 11 22:10:14 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7BFEA106566B for ; Wed, 11 Aug 2010 22:10:14 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 27CBC8FC15 for ; Wed, 11 Aug 2010 22:10:13 +0000 (UTC) Received: by vws7 with SMTP id 7so577938vws.13 for ; Wed, 11 Aug 2010 15:10:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type:content-transfer-encoding; bh=2e7d9d46dA8nK1FDd35uuaxgai7N5E+JMKq+VmOlybQ=; b=QytbLFni/S24vpmqzRuOukSjbCJcYF8V0kd05/h/X/wd0ZkC617hn8Z/RrU8i995nW /i0T8dAJCASyd5oN7UvAifxYwnVTaQQbrf20TAgX7bhBLWkNYn51rVQhD6WS8CHYYCV5 V8MMJcdF06eMXHQCMtiso7hmOPbIXRubw4Rho= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :content-transfer-encoding; b=EM92v6zoRI+aEpu8w8xc+eSZ0AULKLR3bcLD5tmAOlcF7v9LUZcElyI5Xt2ri+wFAh To3/fh3UJlp9Zzu3/AbFEiyasI3sWLpEiAd8FStcxJXl9bUObnspMoEC0HQqv5KVGBwC FaUwRlGL5Ciub5fMkgS0UmRL9q4KpgnzcjVTo= MIME-Version: 1.0 Received: by 10.220.96.4 with SMTP id f4mr11885756vcn.267.1281564613074; Wed, 11 Aug 2010 15:10:13 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.220.49.70 with HTTP; Wed, 11 Aug 2010 15:10:12 -0700 (PDT) In-Reply-To: <20100811214302.GB44635@tolstoy.tols.org> References: <20100810214418.GA28288@tolstoy.tols.org> <20100811014919.GA52992@icarus.home.lan> <20100811192537.GA44635@tolstoy.tols.org> <20100811214302.GB44635@tolstoy.tols.org> Date: Wed, 11 Aug 2010 15:10:12 -0700 X-Google-Sender-Auth: IQW6hoAIXNctRbAKEZRPrxBpB_0 Message-ID: From: Artem Belevich To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Aug 2010 22:10:14 -0000 On Wed, Aug 11, 2010 at 2:43 PM, Marco van Tol wrote: >> There's a hack floating around that attempts to force kernel into >> freeing up memory from inactive/cache lists before draining ARC. It >> does help a bit with this issue, but it's still a hack. > > That makes sense Artem, thanks. =A0I think you mean the posts with the > perl one-liner I used in my tests as well. =A0(perl variable assignment o= f > 1.5GB in the posts their version) Well, that perl one-liner is somewhat similar to using guillotine to cure a minor headache. I was actually referring to the patch mentioned in this post: http://old.nabble.com/Re%3A-Serious-zfs-slowdown-when-mixed-with-another-fi= le-system-(ufs-msdosfs-etc.).-p29137467.html The patch itself is here: http://pastebin.com/ZCkzkWcs > I had seen the posts that mentioned that one and decided to remember the > perl hack. :) > > What I understand from it: > - In a UFS/ZFS mixed system > - In a scenario where UFS "page cache" took (almost) all available memory > - Run a perl one-liner to throw out the UFS active/inactive usage > - Kind-off hope you do enough relevant ZFS accesses that you get a good > =A0new situation. > > So, if my worries can shift from fighting with kmem_size and arc_max to > fighting with arc_min, that's a fight I like a lot better. =A0Especially > on zfs-only systems, I have to admit. Non-ZFS filesystems are not the only entities that cause memory to end up on inactive list. It may end up there for number of other reasons. For instance on my ZFS-only box I currently have ~1G worth of it. Filesystem just happens to be the entity that often stashes a lot of data on inactive list and that causes immediate issues for ZFS. --Artem From owner-freebsd-fs@FreeBSD.ORG Thu Aug 12 20:56:29 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 725CA10656A4 for ; Thu, 12 Aug 2010 20:56:29 +0000 (UTC) (envelope-from peterjeremy@acm.org) Received: from mail16.syd.optusnet.com.au (mail16.syd.optusnet.com.au [211.29.132.197]) by mx1.freebsd.org (Postfix) with ESMTP id 007148FC19 for ; Thu, 12 Aug 2010 20:56:28 +0000 (UTC) Received: from server.vk2pj.dyndns.org (c220-239-116-103.belrs4.nsw.optusnet.com.au [220.239.116.103]) by mail16.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id o7CKuPqD006065 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 13 Aug 2010 06:56:27 +1000 X-Bogosity: Ham, spamicity=0.000000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.4/8.14.4) with ESMTP id o7CKuPZx079582 for ; Fri, 13 Aug 2010 06:56:25 +1000 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.4/8.14.4/Submit) id o7CKuPij079581 for freebsd-fs@freebsd.org; Fri, 13 Aug 2010 06:56:25 +1000 (EST) (envelope-from peter) Date: Fri, 13 Aug 2010 06:56:25 +1000 From: Peter Jeremy To: freebsd-fs@freebsd.org Message-ID: <20100812205625.GA79515@server.vk2pj.dyndns.org> References: <20100810214418.GA28288@tolstoy.tols.org> <20100811014919.GA52992@icarus.home.lan> <20100811192537.GA44635@tolstoy.tols.org> <20100811214302.GB44635@tolstoy.tols.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="IS0zKkzwUGydFO0o" Content-Disposition: inline In-Reply-To: <20100811214302.GB44635@tolstoy.tols.org> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.20 (2009-06-14) Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Aug 2010 20:56:29 -0000 --IS0zKkzwUGydFO0o Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2010-Aug-11 21:43:02 +0000, Marco van Tol wrote: >On Wed, Aug 11, 2010 at 01:08:51PM -0700, Artem Belevich wrote: >> There's a hack floating around that attempts to force kernel into >> freeing up memory from inactive/cache lists before draining ARC. It >> does help a bit with this issue, but it's still a hack. > >That makes sense Artem, thanks. I think you mean the posts with the >perl one-liner I used in my tests as well. (perl variable assignment of >1.5GB in the posts their version) I suspect Artem is referring to his patch at http://pastebin.com/ZCkzkWcs which I have tweaked somewhat (see the last patch in=20 http://www.freebsd.org/cgi/query-pr.cgi?pr=3D146410 ). Whilst these patches _are_ hacks, they seem to do a good job of making ZFS and UFS play together. --=20 Peter Jeremy --IS0zKkzwUGydFO0o Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.15 (FreeBSD) iEYEARECAAYFAkxkX/kACgkQ/opHv/APuId4kACgmCipE6aOmqERf0qO7fYAdDA2 Nb8AnRwm+ArRfOgCmYqYfprQD3ggFblc =o5nq -----END PGP SIGNATURE----- --IS0zKkzwUGydFO0o-- From owner-freebsd-fs@FreeBSD.ORG Thu Aug 12 21:20:50 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7BC321065675 for ; Thu, 12 Aug 2010 21:20:50 +0000 (UTC) (envelope-from marco@tolstoy.tols.org) Received: from tolstoy.tols.org (tolstoy.tols.org [IPv6:2a02:898:0:20::57:1]) by mx1.freebsd.org (Postfix) with ESMTP id 0E58D8FC1A for ; Thu, 12 Aug 2010 21:20:49 +0000 (UTC) Received: from tolstoy.tols.org (localhost [127.0.0.1]) by tolstoy.tols.org (8.14.4/8.14.4) with ESMTP id o7CLKjtC062810 for ; Thu, 12 Aug 2010 21:20:45 GMT (envelope-from marco@tolstoy.tols.org) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.96.1 at tolstoy.tols.org Received: (from marco@localhost) by tolstoy.tols.org (8.14.4/8.14.4/Submit) id o7CLKjql062809 for freebsd-fs@freebsd.org; Thu, 12 Aug 2010 21:20:45 GMT (envelope-from marco) Date: Thu, 12 Aug 2010 21:20:45 +0000 From: Marco van Tol To: freebsd-fs@freebsd.org Message-ID: <20100812212045.GB62440@tolstoy.tols.org> Mail-Followup-To: freebsd-fs@freebsd.org References: <20100810214418.GA28288@tolstoy.tols.org> <20100811014919.GA52992@icarus.home.lan> <20100811192537.GA44635@tolstoy.tols.org> <20100811214302.GB44635@tolstoy.tols.org> <20100812205625.GA79515@server.vk2pj.dyndns.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="bp/iNruPH9dso1Pn" Content-Disposition: inline In-Reply-To: <20100812205625.GA79515@server.vk2pj.dyndns.org> User-Agent: Mutt/1.4.2.3i X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on tolstoy.tols.org Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Aug 2010 21:20:50 -0000 --bp/iNruPH9dso1Pn Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 13, 2010 at 06:56:25AM +1000, Peter Jeremy wrote: > On 2010-Aug-11 21:43:02 +0000, Marco van Tol wrote: > >On Wed, Aug 11, 2010 at 01:08:51PM -0700, Artem Belevich wrote: > >> There's a hack floating around that attempts to force kernel into > >> freeing up memory from inactive/cache lists before draining ARC. It > >> does help a bit with this issue, but it's still a hack. > > > >That makes sense Artem, thanks. I think you mean the posts with the > >perl one-liner I used in my tests as well. (perl variable assignment of > >1.5GB in the posts their version) >=20 > I suspect Artem is referring to his patch at http://pastebin.com/ZCkzkWcs > which I have tweaked somewhat (see the last patch in=20 > http://www.freebsd.org/cgi/query-pr.cgi?pr=3D146410 ). >=20 > Whilst these patches _are_ hacks, they seem to do a good job of > making ZFS and UFS play together. Oh, goodlooking! Thank you, I'll keep it in mind the coming period for when I run into a system that has both filesystems. Thanks! Marco --=20 Linux is for windows haters, BSD for Unix lovers --bp/iNruPH9dso1Pn Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxkZa0ACgkQbdVEUcCKvoL7VgCcCL0+445yquhIwikVKcOMCt/k qDUAniRFcFFUxO44KP3FP2QTXhwWRfxl =PYaQ -----END PGP SIGNATURE----- --bp/iNruPH9dso1Pn-- From owner-freebsd-fs@FreeBSD.ORG Fri Aug 13 00:14:48 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A4E8E106566B; Fri, 13 Aug 2010 00:14:48 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 7BD008FC13; Fri, 13 Aug 2010 00:14:48 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o7D0Emxl022647; Fri, 13 Aug 2010 00:14:48 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o7D0EmgP022643; Fri, 13 Aug 2010 00:14:48 GMT (envelope-from linimon) Date: Fri, 13 Aug 2010 00:14:48 GMT Message-Id: <201008130014.o7D0EmgP022643@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-amd64@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/149499: [gmultipath] gmultipath label failded X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Aug 2010 00:14:48 -0000 Old Synopsis: gmultipath label failded New Synopsis: [gmultipath] gmultipath label failded Responsible-Changed-From-To: freebsd-amd64->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Fri Aug 13 00:14:05 UTC 2010 Responsible-Changed-Why: Reclassify. http://www.freebsd.org/cgi/query-pr.cgi?pr=149499 From owner-freebsd-fs@FreeBSD.ORG Fri Aug 13 02:26:40 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D9B621065695 for ; Fri, 13 Aug 2010 02:26:40 +0000 (UTC) (envelope-from jonathan@kc8onw.net) Received: from mail.kc8onw.net (kc8onw.net [206.55.209.81]) by mx1.freebsd.org (Postfix) with ESMTP id B707E8FC15 for ; Fri, 13 Aug 2010 02:26:40 +0000 (UTC) Received: from [10.70.3.198] (c-98-223-164-104.hsd1.in.comcast.net [98.223.164.104]) by mail.kc8onw.net (Postfix) with ESMTPSA id AAD171B997; Thu, 12 Aug 2010 22:07:26 -0400 (EDT) Message-ID: <4C64A8D3.3090706@kc8onw.net> Date: Thu, 12 Aug 2010 22:07:15 -0400 From: Jonathan User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.8) Gecko/20100802 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Attila Nagy References: <4C5810D6.9060605@fsn.hu> <4C582533.4070009@fsn.hu> In-Reply-To: <4C582533.4070009@fsn.hu> Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: NFS problem: file doesn't appear in file listing, but can be accessed directly X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Aug 2010 02:26:40 -0000 Just a note but I ran into a problem like this with ZFS and Samba a few years back http://www.mail-archive.com/freebsd-stable@freebsd.org/msg92482.html I don't if it's related but it's probably worth checking. Jonathan On 8/3/2010 10:18 AM, Attila Nagy wrote: > On 08/03/10 14:51, Attila Nagy wrote: >> The strange thing is this, happening on the NFS mount, client side: >> # ls -la 1083536654.80433.be03,S=7592 >> -rw------- 1 mail mail 7592 May 3 2004 1083536654.80433.be03,S=7592 >> # ls | grep 1083536654 >> ls doesn't find that file... > Additional info: > - upgrading the client to 8-STABLE doesn't help > - rebooting the client doesn't help > - on the server side, there is zfs beneath the mount From owner-freebsd-fs@FreeBSD.ORG Fri Aug 13 02:54:26 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C3BB510656A3; Fri, 13 Aug 2010 02:54:26 +0000 (UTC) (envelope-from mjacob@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 9AECE8FC17; Fri, 13 Aug 2010 02:54:26 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o7D2sQMr078202; Fri, 13 Aug 2010 02:54:26 GMT (envelope-from mjacob@freefall.freebsd.org) Received: (from mjacob@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o7D2sQ5x078198; Fri, 13 Aug 2010 02:54:26 GMT (envelope-from mjacob) Date: Fri, 13 Aug 2010 02:54:26 GMT Message-Id: <201008130254.o7D2sQ5x078198@freefall.freebsd.org> To: mjacob@FreeBSD.org, freebsd-fs@FreeBSD.org, mjacob@FreeBSD.org From: mjacob@FreeBSD.org Cc: Subject: Re: kern/149499: [gmultipath] gmultipath label failed X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Aug 2010 02:54:26 -0000 Old Synopsis: [gmultipath] gmultipath label failded New Synopsis: [gmultipath] gmultipath label failed Responsible-Changed-From-To: freebsd-fs->mjacob Responsible-Changed-By: mjacob Responsible-Changed-When: Fri Aug 13 02:53:11 UTC 2010 Responsible-Changed-Why: I do gmultipath maintenance http://www.freebsd.org/cgi/query-pr.cgi?pr=149499 From owner-freebsd-fs@FreeBSD.ORG Fri Aug 13 10:16:32 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 41BA9106564A; Fri, 13 Aug 2010 10:16:32 +0000 (UTC) (envelope-from thomas@gibfest.dk) Received: from mail.tyknet.dk (mail.tyknet.dk [213.150.42.155]) by mx1.freebsd.org (Postfix) with ESMTP id EC4088FC08; Fri, 13 Aug 2010 10:16:31 +0000 (UTC) Received: from [10.32.67.58] (fw.int.webpartner.dk [213.150.34.98]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.tyknet.dk (Postfix) with ESMTPSA id CD45B638E06; Fri, 13 Aug 2010 12:16:30 +0200 (CEST) X-DKIM: OpenDKIM Filter v1.1.2 mail.tyknet.dk CD45B638E06 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=gibfest.dk; s=default; t=1281694590; bh=wfjl9khL6d3B+Zn3U3ggSEb8LVCZVFAxR9l5VBWFwNA=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=aGRzGXESVhRc8V4i1fJsrz78FX6PR0cxo3w1RvXfshtnZvpk7roaSwAaHAkHY5CmN Sz1pW0AnwLcXENAMNNxKqB8m+61ohFDuP0wFhDJkZqkie/wE0OyuCw5hEgsH/2LUJf Z1e2VNWkbvvqHeXafdVHnVs50o2Qh96a91L6MP3M= Message-ID: <4C651B7E.5000805@gibfest.dk> Date: Fri, 13 Aug 2010 12:16:30 +0200 From: Thomas Steen Rasmussen User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.11) Gecko/20100711 Lightning/1.0b1 Thunderbird/3.0.6 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4C57E20E.2030908@gibfest.dk> <20100806135001.GF1710@garage.freebsd.pl> <4C5ECA78.6010803@gibfest.dk> <20100810075528.GA1754@garage.freebsd.pl> <4C61CF4D.4060009@gibfest.dk> In-Reply-To: <4C61CF4D.4060009@gibfest.dk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: HAST initial sync speed X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Aug 2010 10:16:32 -0000 On 11-08-2010 00:14, Thomas Steen Rasmussen wrote: > On 10-08-2010 09:55, Pawel Jakub Dawidek wrote: >> It does make sense, but HAST is not doing this:) Could you start with >> veryfing if synchronization is so slow also when you have only one >> resource configured? >> > > I can now confirm that synchronization is also slow with only one HAST > resource. > It is currently running with 2.5 megabits per second... around > 300kilobytes per sec > is written to the disk according to gstat. > An iperf test shows a network potential of about a gigabit (wire speed). > Just a quick update, it is still working on syncing the one HAST resource configured the other day: [root@server1 ~]# date && hastctl status Thu Aug 12 14:11:18 CEST 2010 hasthd4: role: primary provname: hasthd4 localpath: /dev/label/hd4 extentsize: 2097152 keepdirty: 64 remoteaddr: 192.168.0.15 replication: memsync status: complete dirty: 102651396096 bytes [root@server1 ~]# date && hastctl status Fri Aug 13 09:48:06 CEST 2010 hasthd4: role: primary provname: hasthd4 localpath: /dev/label/hd4 extentsize: 2097152 keepdirty: 64 remoteaddr: 192.168.0.15 replication: memsync status: complete dirty: 80425779200 bytes Just under 20 gigabytes in just under 20 hours. Any suggestions are appreciated, Thomas Steen Rasmussen From owner-freebsd-fs@FreeBSD.ORG Fri Aug 13 19:15:55 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 634771065674 for ; Fri, 13 Aug 2010 19:15:55 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 52F388FC16 for ; Fri, 13 Aug 2010 19:15:43 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 8D2C645DF4; Fri, 13 Aug 2010 21:15:41 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 256A245E13; Fri, 13 Aug 2010 21:15:36 +0200 (CEST) Date: Fri, 13 Aug 2010 21:15:21 +0200 From: Pawel Jakub Dawidek To: Andriy Bakay Message-ID: <20100813191521.GB2006@garage.freebsd.pl> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="1UWUbFP1cBYEclgG" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: Converting sysinstalled FreeBSD into ZFS-only server. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Aug 2010 19:15:55 -0000 --1UWUbFP1cBYEclgG Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Aug 09, 2010 at 04:22:42PM -0400, Andriy Bakay wrote: >=20 > 1. In your post you are using dedicated partition for swap. Did it provide > any advantages versus swap on ZFS volume? There is a good chance to deadlock your system when you have swap based on ZVOL, as to operat on ZVOL ZFS needs to allocate some memory, which you probably don't have much when you're swapping. > 2. You are suggesting to set 'vm_kmem_size' value to 150% of RAM. What > pros. and cons. against following formula: >=20 > vm_kmem_size =3D RAM / 2 > vfs.zfs.arc_max =3D vm_kmem_size - 512M Well, I prefer to have as much address space as I have RAM, when you count VM fragmentation in, 150% should be enough to be able to allocate even entire memory for kernel. Of course it all depends on what you run, etc. This is setting I use and I had no problems with it. > 3. Could you be more verbose about your ZFS layout, what major advantages > it provide against for example the following: > http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot. I found it very interesting > but I need more info. I added some notes why I use and recommend such layout at the end of the post. From my experience the fact that file systems are cheap in ZFS doesn't mean it is to have too many of them. For example I don't like to have all /var/ subdirectories are separate ZFS datasets. There are strong reasons to separate some of them (to turn on compression, for example), but not all of them. Of course if you don't like my reasoning or you have different needs, feel free to use whatever layout you feel fits best for you:) I'm also open to comments on the layout I proposed. I use it for quite a while now and I tried different ones before too, but this one I simply find the best. If our official installer will support creating ZFS-only install, I'll be forcing this layout, so if you think something is _very_ wrong about it, let me know. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --1UWUbFP1cBYEclgG Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxlmckACgkQForvXbEpPzTHxgCgh7GpJHcorBoFg8F3Gjq5aMUZ 8qoAoLOJ4qmUK3gOp2Jvz8JIuV509erm =fhOH -----END PGP SIGNATURE----- --1UWUbFP1cBYEclgG-- From owner-freebsd-fs@FreeBSD.ORG Fri Aug 13 21:50:05 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D8DA21065673 for ; Fri, 13 Aug 2010 21:50:05 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id AD7758FC14 for ; Fri, 13 Aug 2010 21:50:05 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o7DLo5mX055489 for ; Fri, 13 Aug 2010 21:50:05 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o7DLo5wP055488; Fri, 13 Aug 2010 21:50:05 GMT (envelope-from gnats) Date: Fri, 13 Aug 2010 21:50:05 GMT Message-Id: <201008132150.o7DLo5wP055488@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Martin Matuska Cc: Subject: Re: kern/144234: [zfs] Cannot boot machine with recent gptzfsboot code [regression] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Martin Matuska List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Aug 2010 21:50:05 -0000 The following reply was made to PR kern/144234; it has been noted by GNATS. From: Martin Matuska To: bug-followup@FreeBSD.org, jamesbrandongooch@gmail.com Cc: Subject: Re: kern/144234: [zfs] Cannot boot machine with recent gptzfsboot code [regression] Date: Fri, 13 Aug 2010 23:42:04 +0200 Could you please try with recent 9-CURRENT, 8-STABLE or try the patch from: kern/148655 ? From owner-freebsd-fs@FreeBSD.ORG Sat Aug 14 04:13:48 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DE7C810656A6 for ; Sat, 14 Aug 2010 04:13:48 +0000 (UTC) (envelope-from andriy@irbisnet.com) Received: from smtp104.rog.mail.re2.yahoo.com (smtp104.rog.mail.re2.yahoo.com [206.190.36.82]) by mx1.freebsd.org (Postfix) with SMTP id 9A21E8FC1D for ; Sat, 14 Aug 2010 04:13:48 +0000 (UTC) Received: (qmail 55033 invoked from network); 14 Aug 2010 04:13:47 -0000 Received: from smtp.irbisnet.com (andriy@99.235.226.221 with login) by smtp104.rog.mail.re2.yahoo.com with SMTP; 13 Aug 2010 21:13:47 -0700 PDT X-Yahoo-SMTP: dz9sigaswBA5kWoYWVTZrGHmIs2vaKgG1w-- X-YMail-OSG: i6a6KNIVM1k3tTJI8PschhiK.CEUEuOiZQ00V.rytlOiZGU 3M9_zvRUu_JJVjJUr2yjWCEw_.AeB8H5Zo8VHsUYx2FFR40AJ8hF99MJNTo4 KgYUKLIlQnzEOaLfnAya2Jc2MLoy2GWHDTAurGTwV9XiTpKrWJlGAsiB.mlq CopkC2a0LnitOpunIf301D4GNqw4V7H_t.9YAMVIC8AwEmGISCn_6P2L6Cnf UFEyJ_MQ9LLXz.V8kdQYUN8omrE95s6YbY0DY0bQU4gD6oRrV3QL.4oDGPxb E094A70ewlQV4Mgqf6wNNkrmWDGsQ0MW3SnIVo9ZsvtUIb4_xIjEiWhek6TA yjmXuKCTkzdkc7HtT0DDTUu7lU_iZ8PsQ3mdDp20Vw40nSBMx63fV8AK1AE1 mS8ki3gAJ0YENhyfUjJunbpWTnapXpkkb3RvLK2qquH_E X-Yahoo-Newman-Property: ymail-3 Received: from prime.irbisnet.com (prime.irbisnet.vpn [10.78.76.4]) by smtp.irbisnet.com (Postfix) with ESMTPSA id 181441140C; Sat, 14 Aug 2010 00:13:40 -0400 (EDT) Message-ID: <4C6617F2.7080807@irbisnet.com> Date: Sat, 14 Aug 2010 00:13:38 -0400 From: Andriy Bakay User-Agent: Thunderbird 2.0.0.23 (X11/20091223) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <20100813191521.GB2006@garage.freebsd.pl> In-Reply-To: <20100813191521.GB2006@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Converting sysinstalled FreeBSD into ZFS-only server. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Aug 2010 04:13:49 -0000 Pawel Jakub Dawidek wrote: > On Mon, Aug 09, 2010 at 04:22:42PM -0400, Andriy Bakay wrote: >> 1. In your post you are using dedicated partition for swap. Did it provide >> any advantages versus swap on ZFS volume? > > There is a good chance to deadlock your system when you have swap based > on ZVOL, as to operat on ZVOL ZFS needs to allocate some memory, which > you probably don't have much when you're swapping. > Just to clarify. It means I can only use swap based on ZVOL only in non-production environment? Could this problem be solved or it is something fundamental (for FreeBSD) here? As far as I know on Solaris ZVOL based swap works without problems. >> 2. You are suggesting to set 'vm_kmem_size' value to 150% of RAM. What >> pros. and cons. against following formula: >> >> vm_kmem_size = RAM / 2 >> vfs.zfs.arc_max = vm_kmem_size - 512M > > Well, I prefer to have as much address space as I have RAM, when you > count VM fragmentation in, 150% should be enough to be able to allocate > even entire memory for kernel. Of course it all depends on what you run, > etc. This is setting I use and I had no problems with it. > I have FreeBSD 8.0-RELEASE-p2 amd64 with 4G RAM, when I set vm_kmem_size to 6G my vfs.zfs.arc_max became 5G. Is it normal for this kind of setup? Could such ARC value become a problem, because I only have 4G RAM? >> 3. Could you be more verbose about your ZFS layout, what major advantages >> it provide against for example the following: >> http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot. I found it very interesting >> but I need more info. > > I added some notes why I use and recommend such layout at the end of the > post. From my experience the fact that file systems are cheap in ZFS > doesn't mean it is to have too many of them. For example I don't like to > have all /var/ subdirectories are separate ZFS datasets. There are > strong reasons to separate some of them (to turn on compression, for > example), but not all of them. > > Of course if you don't like my reasoning or you have different needs, > feel free to use whatever layout you feel fits best for you:) > > I'm also open to comments on the layout I proposed. I use it for quite a > while now and I tried different ones before too, but this one I simply > find the best. If our official installer will support creating ZFS-only > install, I'll be forcing this layout, so if you think something is > _very_ wrong about it, let me know. > Your notes in the blog (http://blogs.freebsdish.org/pjd/2010/08/06/from-sysinstall-to-zfs-only-configuration/) sounds very reasonable. So, should /var/db be compressed as well? In your layout you create home dataset under /usr: system/usr/home. maybe it is more logical to create it one level upper: system/home or in this case you just follow standard FreeBSD filesystem layout? Thanks for your answers. Andriy From owner-freebsd-fs@FreeBSD.ORG Sat Aug 14 14:56:39 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 14D46106564A for ; Sat, 14 Aug 2010 14:56:39 +0000 (UTC) (envelope-from andriy@irbisnet.com) Received: from smtp105.rog.mail.re2.yahoo.com (smtp105.rog.mail.re2.yahoo.com [206.190.36.83]) by mx1.freebsd.org (Postfix) with SMTP id 8E22C8FC13 for ; Sat, 14 Aug 2010 14:56:38 +0000 (UTC) Received: (qmail 43105 invoked from network); 14 Aug 2010 14:56:37 -0000 Received: from smtp.irbisnet.com (andriy@99.235.226.221 with login) by smtp105.rog.mail.re2.yahoo.com with SMTP; 14 Aug 2010 07:56:37 -0700 PDT X-Yahoo-SMTP: dz9sigaswBA5kWoYWVTZrGHmIs2vaKgG1w-- X-YMail-OSG: ymSbwDkVM1kRAYUXGSEwOLcWikWGBdQwPbG1o30LzoU_9u1 VfY_z.9PfkBacQRPVNvGw5q4hg4lupze4U0toXOLNHMyNqB5RtQcnVRYSGNO eJq63MEOp94s5iysN8at3mesEHpgeeeqxg2DyjRk7v4aYrEXCH5a49fYO325 xXNUkYP9w2QJA3k8vOVoxcnfyZC0SHrzIPBifKKGVrB4O8_l.xRiVLSLxfWT qfuKeHzIIKfI6s35ONQ1Xj0_BS158eQLLAim8hKdDtWcJU2ur2hV5yiX4bTW hN84Ec7lLvofn2ZfsLWK9lYk2fDKyE2GUmLx6dp7i_VH0s3Y7GqTF3vuaIan 85wbbeVm4TQ3xWLiP6URpY5E.LGeZDVKNr5QSn_r9OUHxkE7XoH3WAoaHDNU F10WRfy8Mq6pW8g-- X-Yahoo-Newman-Property: ymail-3 Received: from prime.irbisnet.com (prime.irbisnet.vpn [10.78.76.4]) by smtp.irbisnet.com (Postfix) with ESMTPSA id 3EE0F1140C; Sat, 14 Aug 2010 10:56:36 -0400 (EDT) Message-ID: <4C66AEA0.2030507@irbisnet.com> Date: Sat, 14 Aug 2010 10:56:32 -0400 From: Andriy Bakay User-Agent: Thunderbird 2.0.0.23 (X11/20091223) MIME-Version: 1.0 To: mm@freebsd.org, freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: kern/148655: [zfs] Booting from a degraded raidz no longer works in 8-STABLE [regression] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Aug 2010 14:56:39 -0000 Hi Martin, Does you have any plans to merge (errata notice) your kern/148655 fix to FreeBSD 8.1-RELEASE branch? Thanks, Andriy From owner-freebsd-fs@FreeBSD.ORG Sat Aug 14 15:17:56 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CB9621065694 for ; Sat, 14 Aug 2010 15:17:56 +0000 (UTC) (envelope-from andriy@irbisnet.com) Received: from smtp101.rog.mail.re2.yahoo.com (smtp101.rog.mail.re2.yahoo.com [206.190.36.79]) by mx1.freebsd.org (Postfix) with SMTP id 5609F8FC23 for ; Sat, 14 Aug 2010 15:17:56 +0000 (UTC) Received: (qmail 42664 invoked from network); 14 Aug 2010 15:17:55 -0000 Received: from smtp.irbisnet.com (andriy@99.235.226.221 with login) by smtp101.rog.mail.re2.yahoo.com with SMTP; 14 Aug 2010 08:17:55 -0700 PDT X-Yahoo-SMTP: dz9sigaswBA5kWoYWVTZrGHmIs2vaKgG1w-- X-YMail-OSG: kJDDESYVM1mlcizeQNBTzY9cesrydRDlrfcPVtzjrnEqq4c maTYNUtyGXP.sHPTlE_e.pd_cZj5o2dkcm8cmNT_nSlt8tYN_CyqQCsP.PLf dlXqug6JxqCZOJgfRxgrr_yXhK0W3abIJ1vC60vsG5Hv7.jM6ajOGmCSG8Od qkE4Bgna4ltHLKEVWfW4vBqYdAabvDNeWFeHX2lfPCXZZttA2b..XCthGJdW 1sOp3z7iZk65dSzuboiaHBF0tum5dHTfZIF9XwXRI6SCb5ZATXjxDgJcff01 MSV4WJqtA5nYodc3A8NBOX62LTaY1aQKEnHcPrz4BlMX6bQFXjCN9vG.bxWe NRnwRecJIVUVyufcTcGaWEqXM1Kr8zsyKLJk0wJNMQIt_7CSfcJR_SoNjEoM hwXWR7bMDuwkrmdPoOTdPBWXwciwBWB5HOBYQAZA0LhknRMeg X-Yahoo-Newman-Property: ymail-3 Received: from prime.irbisnet.com (prime.irbisnet.vpn [10.78.76.4]) by smtp.irbisnet.com (Postfix) with ESMTPSA id D6DD11140C; Sat, 14 Aug 2010 11:17:54 -0400 (EDT) Message-ID: <4C66B3A1.7040204@irbisnet.com> Date: Sat, 14 Aug 2010 11:17:53 -0400 From: Andriy Bakay User-Agent: Thunderbird 2.0.0.23 (X11/20091223) MIME-Version: 1.0 To: Pawel Jakub Dawidek Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: GELI suspend/resume X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Aug 2010 15:17:56 -0000 Hi Pawel, Sorry for bothering you and maybe my question does not belong to this mailing list. Do you have any update for GELI suspend/resume feature which you described here: http://blogs.freebsdish.org/pjd/2007/09/28/geli-suspendresume/ Thanks, Andriy From owner-freebsd-fs@FreeBSD.ORG Sat Aug 14 19:12:17 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F65A1065670 for ; Sat, 14 Aug 2010 19:12:17 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 8C9B68FC0A for ; Sat, 14 Aug 2010 19:12:16 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id D3EAD45E86; Sat, 14 Aug 2010 21:12:14 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id D5B6B45683; Sat, 14 Aug 2010 21:12:09 +0200 (CEST) Date: Sat, 14 Aug 2010 21:11:53 +0200 From: Pawel Jakub Dawidek To: Andriy Bakay Message-ID: <20100814191153.GD2006@garage.freebsd.pl> References: <4C66B3A1.7040204@irbisnet.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="M38YqGLZlgb6RLPS" Content-Disposition: inline In-Reply-To: <4C66B3A1.7040204@irbisnet.com> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: GELI suspend/resume X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Aug 2010 19:12:17 -0000 --M38YqGLZlgb6RLPS Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Aug 14, 2010 at 11:17:53AM -0400, Andriy Bakay wrote: > Hi Pawel, >=20 > Sorry for bothering you and maybe my question does not belong to this=20 > mailing list. Do you have any update for GELI suspend/resume feature=20 > which you described here: >=20 > http://blogs.freebsdish.org/pjd/2007/09/28/geli-suspendresume/ Well, the thing was mostly working, but there were still some problems to fix. In the meantime I changed my laptop and on the new one suspend/resume didn't work anymore, so I kinda lost interested in GELI suspend/resume project. I recently changed my laptop again, but suspend/resume still doesn't work... --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --M38YqGLZlgb6RLPS Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxm6ngACgkQForvXbEpPzSAcQCg1gcSmJxmGZAaHA3AoHZHj2nv kBoAn32b4Suq9tPvFKd0tztN5RZ7HbPv =fBlK -----END PGP SIGNATURE----- --M38YqGLZlgb6RLPS--