From owner-freebsd-fs@FreeBSD.ORG Sun Jun 21 00:37:00 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CD06A1065670 for ; Sun, 21 Jun 2009 00:37:00 +0000 (UTC) (envelope-from mat.macy@gmail.com) Received: from yw-out-2324.google.com (yw-out-2324.google.com [74.125.46.29]) by mx1.freebsd.org (Postfix) with ESMTP id 81FC08FC0C for ; Sun, 21 Jun 2009 00:37:00 +0000 (UTC) (envelope-from mat.macy@gmail.com) Received: by yw-out-2324.google.com with SMTP id 9so1119426ywe.13 for ; Sat, 20 Jun 2009 17:36:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=l8D8urkX4O43UsPfqFdN+atwas3+gqQtxkU1fyWDQXU=; b=wCbLz+JFectnjBi3h2J8UazFfA2s/Yqb8CJluULaTQ02fJvi5K4HWd0dyabBHhLNqz evz2CaC1YFcLKSybKZvLao6QXG+T3/f/Q7ZZxP5qOXnSwuVKLv7z+5A2+SXMEfK3UWn2 4WeCdGHZaD4CBJ7+MXAq+8eQ1ZG3gCUQhGfbk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=wcayiYm/VUR8fVcO+LUg4MYoV5dR9xZEOtbt/C7sv2OcuMIJmenELTTx31d8jDX+W4 /4b8Ot4x/4AlVCSl6QkAYhfCajuNZTjtEnVHRHMYXJ2VYCh25CM48caGq6MyeUDilADV IOVDMXTxa6ygsKHjkxOe6uk8+9pwY5CzsKkN4= MIME-Version: 1.0 Sender: mat.macy@gmail.com Received: by 10.100.32.13 with SMTP id f13mr5987600anf.36.1245544619557; Sat, 20 Jun 2009 17:36:59 -0700 (PDT) In-Reply-To: <1245527381.26909.82.camel@phoenix.blechhirn.net> References: <1245519413.26909.60.camel@phoenix.blechhirn.net> <3c1674c90906201050w15e4cd5dpae76cd70d64b4e92@mail.gmail.com> <1245525965.26909.69.camel@phoenix.blechhirn.net> <3c1674c90906201232x63ddee19yf91aeac30f3401bb@mail.gmail.com> <1245527381.26909.82.camel@phoenix.blechhirn.net> Date: Sat, 20 Jun 2009 17:36:59 -0700 X-Google-Sender-Auth: 7dc2090f468dd964 Message-ID: <3c1674c90906201736o335b4a3dv862139dc55f2211f@mail.gmail.com> From: Kip Macy To: mister.olli@googlemail.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Unable to delete files on ZFS volume X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jun 2009 00:37:01 -0000 > >> - How much churn has there been on the file system? > not sure what you mean with 'churn' (there seem to be no translation to > german that makes sense ;-)) > The closest I can come to defining it would be "fluctuation of contents". By this I was asking if you repeatedly created and deleted files. I didn't think it would be this easy to provoke this. I've tried provoking this with a handful of large files without success. I'm guessing that ZFS isn't as careful about bounding metadata creation as it is data creation. My focus is, and for the foreseeable future will be, fixing FreeBSD integration issues. As this is a fundamental ZFS space management issue I don't foresee fixing it any time soon if at all unless it is addressed by upgrading to v15 or v16. I hope that truncating some of the files will free up space for deletion. Cheers, Kip From owner-freebsd-fs@FreeBSD.ORG Sun Jun 21 01:08:36 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9A53A1065670 for ; Sun, 21 Jun 2009 01:08:36 +0000 (UTC) (envelope-from mat.macy@gmail.com) Received: from mail-yx0-f200.google.com (mail-yx0-f200.google.com [209.85.210.200]) by mx1.freebsd.org (Postfix) with ESMTP id 50F658FC14 for ; Sun, 21 Jun 2009 01:08:36 +0000 (UTC) (envelope-from mat.macy@gmail.com) Received: by yxe38 with SMTP id 38so975321yxe.3 for ; Sat, 20 Jun 2009 18:08:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=a4BJse2OTvSJoNEySkGxdSX9+t/r+kLd4Uq1s+zw0hg=; b=mbE4wUpiQr8LClVsw2l1+9CthTWSVyWxMWQmbRZSpZyECEH3ROTX2fjtSIFyY+HeUH d0E+zZjacWJRWB/uRm0N/Zmqu6pYyvRSuAtmfKsqUHseBJssDRwvSnNTOW5MHDG5T6I/ lvWWmrLCl0O77bPktH1m8xHQNz0zUTOVWtXLI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=PLSnqqEdzJZF9ZNK4vVlPe1JayV9HeEV+3G3O4K6DptSFWuetKcQfrDC943eDOJe0p 4mTr5EEi6fwIc62G8g2Y25cFfwvE4y9q2KXXBJm220+2vSKtSOj/u+B2fq32ad+xd967 fgeMkRjr7ZbQLN2Yo/iyQycfDPOnXXW+WQiiA= MIME-Version: 1.0 Sender: mat.macy@gmail.com Received: by 10.100.195.15 with SMTP id s15mr5852309anf.18.1245546515611; Sat, 20 Jun 2009 18:08:35 -0700 (PDT) In-Reply-To: <20090620231130.GA88907@owl.midgard.homeip.net> References: <20090620231130.GA88907@owl.midgard.homeip.net> Date: Sat, 20 Jun 2009 18:08:35 -0700 X-Google-Sender-Auth: d6003c8698070c40 Message-ID: <3c1674c90906201808t1854dd46n82213fbd0c1c254c@mail.gmail.com> From: Kip Macy To: Erik Trulsson Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: ufs2 / softupdates / ZFS / disk write cache X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jun 2009 01:08:36 -0000 > > My guess is that it will be quite noticable, but that is only a guess. > (Keep in mind that UFS+softupdates does quite a bit of write-caching on its > own, so just switching to ZFS is unlikely to improve write performance > significantly compared to using UFS.) That all depends on how much the drive relies on the write cache for batching writes to disk. Soft updates does a lot of small random writes for metadata updates which will likely be heavily penalized by the absence of write caching. On my SSD, which unfortunately turned out to be camera grade flash, with FFS the system was unusable when doing large numbers of metadata updates, svn checkouts would take hours. I postulated that ZFS would map well to the large erase blocks, so I destroyed /usr and recreated a zpool in its place. I now get random write performance better than FFS, "I lived happily ever after." I don't know if ZFS will provide the same benefit in your situation. My point is just that FFS+SU and ZFS are "apples and oranges." Please note that I've taken -stable off of the the CC, ZFS has been getting a lot of mailing list traffic lately and I've been hearing groans from certain quarters about it drowning out other discussions. Let's try to keep the discussions to freebsd-fs. Thanks, Kip From owner-freebsd-fs@FreeBSD.ORG Sun Jun 21 02:18:41 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0C266106566C for ; Sun, 21 Jun 2009 02:18:41 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.245]) by mx1.freebsd.org (Postfix) with ESMTP id B4DFA8FC12 for ; Sun, 21 Jun 2009 02:18:40 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: by an-out-0708.google.com with SMTP id c3so1114449ana.13 for ; Sat, 20 Jun 2009 19:18:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=zKi4tlT7swuDCfrpmD6GKndxMzdfaWjRhEukODozAvA=; b=SyLYhh7dFkF+39UXB7396yJ8HQt+UVHr3p5F1GXzsZYGzJY6VAWlDkZNr5IDPBdIqR LCFkNpjPICVTrTou96vGUKy7mKGv/JYXhKgowGT9AObZBlsKt2HZcwQJHjByVgVRJfPv T4dEsX7ev/7uKt9/gPcBGtYfROMq7tiEUYT5g= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=ix+8lfLk3MwnUN5y+1C7FcrdOoNGP4G1IvGDkyw8eBsr0h2/yYtX3p4XWNygNspo6F bc42WFLEmOZ3MN9PV5BVjm36HocZGEYXBpFLnQf8GtyJcFY8glWHiDz9JaCwio5WZC8o 52dSGfckm56997+HK20QDb2t0WSP0ufwY71ZE= MIME-Version: 1.0 Received: by 10.100.126.15 with SMTP id y15mr6079406anc.71.1245550720009; Sat, 20 Jun 2009 19:18:40 -0700 (PDT) In-Reply-To: <3c1674c90906201808t1854dd46n82213fbd0c1c254c@mail.gmail.com> References: <20090620231130.GA88907@owl.midgard.homeip.net> <3c1674c90906201808t1854dd46n82213fbd0c1c254c@mail.gmail.com> Date: Sun, 21 Jun 2009 05:18:39 +0300 Message-ID: From: Dan Naumov To: Kip Macy Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, Erik Trulsson Subject: Re: ufs2 / softupdates / ZFS / disk write cache X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jun 2009 02:18:41 -0000 I decided to do some performance tests of my own, "bonnie -s 4096" was used to obtain the results. Note that these results should be used to compare "write cache on" to "write cache off" and not to compare UFS2 vs ZFS, as the testing was done on different parts of the same physical disk (the UFS2 partition resides on the first 16gb of disk and ZFS pool takes the remaining ~1,9tb) and I am also using rather conservative ZFS tunables. UFS2 with write cache: -------Sequential Output-------- ---Sequential Input-- --Rand= om-- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seek= s--- Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec = %CPU 4096 55457 95.9 91630 46.7 36264 37.5 46565 74.0 84751 33.7 164.3 = 10.3 UFS2 without write cache: -------Sequential Output-------- ---Sequential Input-- --Rand= om-- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seek= s--- Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec = %CPU 4096 4938 46.9 4685 18.0 4288 21.8 17453 34.0 74232 31.6 165.0 = 9.9 As we can clearly see, the performance diffence between having disk cache enabled and disabled is _ENORMOUS_. In the case of sequential block write on UFS2, the performance loss is a staggering 94,89%. More surprinsingly, even reading seems to be affected in a noticable way, per char reads suffer a 62,62% penalty while block reads take a 12,42% hit. Moving on to testing ZFS with and without disk cache enabled: ZFS with write cache (384M ARC, 1GB max kmem): -------Sequential Output-------- ---Sequential Input-- --Rand= om-- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seek= s--- Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec = %CPU 4096 25972 66.1 45026 40.6 34269 36.0 46371 86.5 93973 34.6 84.5 = 8.5 ZFS without write cache (384M ARC, 1GB max kmem): -------Sequential Output-------- ---Sequential Input-- --Rand= om-- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seek= s--- Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec = %CPU 4096 2399 6.7 2258 3.5 2290 3.9 34380 66.1 85971 32.8 56.7 = 6.1 =09 Uh oh.... After some digging around, I found the following quote: "ZFS is designed to work with storage devices that manage a disk-level cache. ZFS commonly asks the storage device to ensure that data is safely placed on stable storage by requesting a cache flush." at http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide I guess this might be somewhat related to why in the "disk cache disabled" scenario, ZFS suffers bigger losses than UFS2. It is quite obvious at this point that disabling disk cache in order have softupdates live in harmony with disks "lying" about whether disk cache contents have actually been committed to the disk in not in any way, shape or form a viable solution to the problem. On a sidenote, is there any way I can test whether *MY* disk is truthful about writing cache to disk or not? In the past (this was during my previous foray into the FreeBSD world, circa-2001/2002) I have suffered severe data corruption (leading to an unbootable system) using UFS2 + softupdates on 2 different occasions due to power losses and this past experience has me very worried about the proper way to configure my system to avoid such incidents in the future. - Sincerely, Dan Naumov On Sun, Jun 21, 2009 at 4:08 AM, Kip Macy wrote: >> >> My guess is that it will be quite noticable, but that is only a guess. >> (Keep in mind that UFS+softupdates does quite a bit of write-caching on = its >> own, so just switching to ZFS is unlikely to improve write performance >> significantly compared to using UFS.) > > > That all depends on how much the drive relies on the write cache for > batching writes to disk. Soft updates does a lot of small random > writes for metadata updates which will likely be heavily penalized by > the absence of write caching. On my SSD, which unfortunately turned > out to be camera grade flash, with FFS the system was unusable when > doing large numbers of metadata updates, svn checkouts would take > hours. I postulated that ZFS would map well to the large erase blocks, > so I destroyed /usr and recreated a zpool in its place. I now get > random write performance =A0better than FFS, "I lived happily ever > after." > > I don't know if ZFS will provide the same benefit in your situation. > My point is just that FFS+SU and ZFS are "apples and oranges." > > Please note that I've taken -stable off of the the CC, ZFS has been > getting a lot of mailing list traffic lately and I've been hearing > groans from certain quarters about it drowning out other discussions. > Let's try to keep the discussions to freebsd-fs. > > > Thanks, > Kip From owner-freebsd-fs@FreeBSD.ORG Sun Jun 21 02:48:50 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5DB321065687 for ; Sun, 21 Jun 2009 02:48:50 +0000 (UTC) (envelope-from nhoyle@hoyletech.com) Received: from mout.perfora.net (mout.perfora.net [74.208.4.195]) by mx1.freebsd.org (Postfix) with ESMTP id 35F7B8FC0A for ; Sun, 21 Jun 2009 02:48:50 +0000 (UTC) (envelope-from nhoyle@hoyletech.com) Received: from [192.168.1.10] (pool-96-231-140-65.washdc.fios.verizon.net [96.231.140.65]) by mrelay.perfora.net (node=mrus0) with ESMTP (Nemesis) id 0MKp8S-1MID6p0k0U-000g2q; Sat, 20 Jun 2009 22:48:48 -0400 Message-ID: <4A3D9F87.5090108@hoyletech.com> Date: Sun, 21 Jun 2009 02:48:39 +0000 From: Nathanael Hoyle User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: Dan Naumov References: <20090620231130.GA88907@owl.midgard.homeip.net> <3c1674c90906201808t1854dd46n82213fbd0c1c254c@mail.gmail.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V01U2FsdGVkX1+uvPTi2/wIpukdON/NbHzUKgji7zgIIN1yvh3 LsUAyCBho/QzmvPPO2vycJ35SXJ/g80sJ/lDvL/MNawt8vfgsr J1fksr1wMtQkBXdnE3YftfNLqJfEutC Cc: freebsd-fs@freebsd.org, Kip Macy , Erik Trulsson Subject: Re: ufs2 / softupdates / ZFS / disk write cache X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jun 2009 02:48:50 -0000 Dan Naumov wrote: > I decided to do some performance tests of my own, "bonnie -s 4096" was > used to obtain the results. Note that these results should be used to > compare "write cache on" to "write cache off" and not to compare UFS2 > vs ZFS, as the testing was done on different parts of the same > physical disk (the UFS2 partition resides on the first 16gb of disk > and ZFS pool takes the remaining ~1,9tb) and I am also using rather > conservative ZFS tunables. > > > UFS2 with write cache: > -------Sequential Output-------- ---Sequential Input-- --Random-- > -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks--- > Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU > 4096 55457 95.9 91630 46.7 36264 37.5 46565 74.0 84751 33.7 164.3 10.3 > > UFS2 without write cache: > -------Sequential Output-------- ---Sequential Input-- --Random-- > -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks--- > Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU > 4096 4938 46.9 4685 18.0 4288 21.8 17453 34.0 74232 31.6 165.0 9.9 > > > As we can clearly see, the performance diffence between having disk > cache enabled and disabled is _ENORMOUS_. In the case of sequential > block write on UFS2, the performance loss is a staggering 94,89%. More > surprinsingly, even reading seems to be affected in a noticable way, > per char reads suffer a 62,62% penalty while block reads take a 12,42% > hit. Moving on to testing ZFS with and without disk cache enabled: > > > ZFS with write cache (384M ARC, 1GB max kmem): > -------Sequential Output-------- ---Sequential Input-- --Random-- > -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks--- > Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU > 4096 25972 66.1 45026 40.6 34269 36.0 46371 86.5 93973 34.6 84.5 8.5 > > ZFS without write cache (384M ARC, 1GB max kmem): > -------Sequential Output-------- ---Sequential Input-- --Random-- > -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks--- > Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU > 4096 2399 6.7 2258 3.5 2290 3.9 34380 66.1 85971 32.8 56.7 6.1 > > > Uh oh.... After some digging around, I found the following quote: "ZFS > is designed to work with storage devices that manage a disk-level > cache. ZFS commonly asks the storage device to ensure that data is > safely placed on stable storage by requesting a cache flush." at > http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide I > guess this might be somewhat related to why in the "disk cache > disabled" scenario, ZFS suffers bigger losses than UFS2. > > It is quite obvious at this point that disabling disk cache in order > have softupdates live in harmony with disks "lying" about whether disk > cache contents have actually been committed to the disk in not in any > way, shape or form a viable solution to the problem. On a sidenote, is > there any way I can test whether *MY* disk is truthful about writing > cache to disk or not? > > In the past (this was during my previous foray into the FreeBSD world, > circa-2001/2002) I have suffered severe data corruption (leading to an > unbootable system) using UFS2 + softupdates on 2 different occasions > due to power losses and this past experience has me very worried about > the proper way to configure my system to avoid such incidents in the > future. > > > - Sincerely, > Dan Naumov > > > > Dan, Top posting on mailing lists is bad (and not the preferred convention for this list). The performance numbers are startling, and good to have. You could also try setting the 'sync' flag on the FFS+SU mount to see what that looks like, it should give a small extra measure of protection. Since that mount shouldn't be write-heavy, I wouldn't expect much (perceived) performance hit (though the bonnie numbers may be ugly). As Peter Jeremy responded in your question about whether or not your proposed configuration looked sane (your post from the 14th), one solid strategy is to have an *offline* copy of your root filesystem. This ensures that outstanding disk writes cannot leave that instance in an unusable form, and helps protect you from all the evilness that can occur to online/mounted filesystems. On Linux systems where the kernel image and grub config usually reside in /boot, I usually make that a separate partition and set 'noauto' on the /etc/fstab so that it is never mounted except when I'm installing a new kernel or updating my grub config. -Nathanael From owner-freebsd-fs@FreeBSD.ORG Sun Jun 21 09:27:41 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 92529106564A for ; Sun, 21 Jun 2009 09:27:41 +0000 (UTC) (envelope-from erikt@midgard.homeip.net) Received: from ch-smtp02.sth.basefarm.net (ch-smtp02.sth.basefarm.net [80.76.149.213]) by mx1.freebsd.org (Postfix) with ESMTP id 1C6CE8FC12 for ; Sun, 21 Jun 2009 09:27:41 +0000 (UTC) (envelope-from erikt@midgard.homeip.net) Received: from c83-255-48-78.bredband.comhem.se ([83.255.48.78]:60487 helo=falcon.midgard.homeip.net) by ch-smtp02.sth.basefarm.net with esmtp (Exim 4.69) (envelope-from ) id 1MIJKt-0005sN-8F for freebsd-fs@freebsd.org; Sun, 21 Jun 2009 11:27:39 +0200 Received: (qmail 66113 invoked from network); 21 Jun 2009 11:27:36 +0200 Received: from owl.midgard.homeip.net (10.1.5.7) by falcon.midgard.homeip.net with ESMTP; 21 Jun 2009 11:27:36 +0200 Received: (qmail 92746 invoked by uid 1001); 21 Jun 2009 11:27:36 +0200 Date: Sun, 21 Jun 2009 11:27:36 +0200 From: Erik Trulsson To: Dan Naumov Message-ID: <20090621092736.GA92656@owl.midgard.homeip.net> References: <20090620231130.GA88907@owl.midgard.homeip.net> <3c1674c90906201808t1854dd46n82213fbd0c1c254c@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.19 (2009-01-05) X-Originating-IP: 83.255.48.78 X-Scan-Result: No virus found in message 1MIJKt-0005sN-8F. X-Scan-Signature: ch-smtp02.sth.basefarm.net 1MIJKt-0005sN-8F ca067a144d9fa50af6db86823b1baf6d Cc: freebsd-fs@freebsd.org, Kip Macy Subject: Re: ufs2 / softupdates / ZFS / disk write cache X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jun 2009 09:27:41 -0000 On Sun, Jun 21, 2009 at 05:18:39AM +0300, Dan Naumov wrote: > Uh oh.... After some digging around, I found the following quote: "ZFS > is designed to work with storage devices that manage a disk-level > cache. ZFS commonly asks the storage device to ensure that data is > safely placed on stable storage by requesting a cache flush." at > http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide I > guess this might be somewhat related to why in the "disk cache > disabled" scenario, ZFS suffers bigger losses than UFS2. If that quote is correct (and I have no real reason to doubt it) then it should probably be safe to enable the disk's write cache when used with ZFS. (That would make sense since UFS/FFS was originally designed to work with an older generation of disks that did not do any significant amount of write-caching (partly due to having very little cache on them), while ZFS has been designed to be used on modern hardware, and to be reliable even on cheap consumer-grade disks.) > > It is quite obvious at this point that disabling disk cache in order > have softupdates live in harmony with disks "lying" about whether disk > cache contents have actually been committed to the disk in not in any > way, shape or form a viable solution to the problem. On a sidenote, is > there any way I can test whether *MY* disk is truthful about writing > cache to disk or not? If you have IDE/SATA disks they will "lie". SCSI/SAS disks won't. SATA disks using NCQ should probably also be safe -- too bad FreeBSD does not support NCQ yet. > > In the past (this was during my previous foray into the FreeBSD world, > circa-2001/2002) I have suffered severe data corruption (leading to an > unbootable system) using UFS2 + softupdates on 2 different occasions > due to power losses and this past experience has me very worried about > the proper way to configure my system to avoid such incidents in the > future. > > > - Sincerely, > Dan Naumov > > -- Erik Trulsson ertr1013@student.uu.se From owner-freebsd-fs@FreeBSD.ORG Sun Jun 21 09:45:31 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D296E1065675; Sun, 21 Jun 2009 09:45:31 +0000 (UTC) (envelope-from remko@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id A89C38FC13; Sun, 21 Jun 2009 09:45:31 +0000 (UTC) (envelope-from remko@FreeBSD.org) Received: from freefall.freebsd.org (remko@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n5L9jVPY009823; Sun, 21 Jun 2009 09:45:31 GMT (envelope-from remko@freefall.freebsd.org) Received: (from remko@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n5L9jVOS009819; Sun, 21 Jun 2009 09:45:31 GMT (envelope-from remko) Date: Sun, 21 Jun 2009 09:45:31 GMT Message-Id: <200906210945.n5L9jVOS009819@freefall.freebsd.org> To: remko@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: remko@FreeBSD.org Cc: Subject: Re: bin/135710: mount(8): mount -t tmpfs does not follow 'size' option X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jun 2009 09:45:32 -0000 Synopsis: mount(8): mount -t tmpfs does not follow 'size' option Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: remko Responsible-Changed-When: Sun Jun 21 09:45:21 UTC 2009 Responsible-Changed-Why: reassign to fs team http://www.freebsd.org/cgi/query-pr.cgi?pr=135710 From owner-freebsd-fs@FreeBSD.ORG Sun Jun 21 10:03:18 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 19DEA1065673 for ; Sun, 21 Jun 2009 10:03:18 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.240]) by mx1.freebsd.org (Postfix) with ESMTP id C22628FC16 for ; Sun, 21 Jun 2009 10:03:17 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: by an-out-0708.google.com with SMTP id c3so1155767ana.13 for ; Sun, 21 Jun 2009 03:03:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=eS5v+624J8AJ31P3TQEUOVUPdu9xYyX4MjFHpVV8HrQ=; b=twKxU/O6DWNfdyzWpKhA7Bo0Z2xXZDYCBoJoe6fW2HWCkY6/96cBd1a0iiWoMyFFoH DY8vt+LNx3Q+tFhHt1jkVu8ODCU460nRfV3Zlmz4hnUQFA0R8kLvtLg8vCrAyiZunEwl qIqYJQKhNJKcT3E7IIybYwTBKekJ/hJS8qxIo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=bxA5zCytrYqTkrjO1IxAmMYBrZ1a/OOmD+wWlJGdJRfr6attjo1KL2T4DZolLvce1b EVlQ27zZc5/yLks3iaWbLVqkAj7xFzjHnCF2Hhil+bcZJenw9TPZhtPwsQreIwUyKreJ XCCcdCC7Fy2M9lOu0nZ8AVSGdsee4e7QZJuSw= MIME-Version: 1.0 Received: by 10.100.107.8 with SMTP id f8mr6343603anc.197.1245578597027; Sun, 21 Jun 2009 03:03:17 -0700 (PDT) In-Reply-To: <20090621092736.GA92656@owl.midgard.homeip.net> References: <20090620231130.GA88907@owl.midgard.homeip.net> <3c1674c90906201808t1854dd46n82213fbd0c1c254c@mail.gmail.com> <20090621092736.GA92656@owl.midgard.homeip.net> Date: Sun, 21 Jun 2009 13:03:17 +0300 Message-ID: From: Dan Naumov To: Erik Trulsson Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, Kip Macy Subject: Re: ufs2 / softupdates / ZFS / disk write cache X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jun 2009 10:03:18 -0000 On Sun, Jun 21, 2009 at 12:27 PM, Erik Trulsson wro= te: > On Sun, Jun 21, 2009 at 05:18:39AM +0300, Dan Naumov wrote: >> Uh oh.... After some digging around, I found the following quote: "ZFS >> is designed to work with storage devices that manage a disk-level >> cache. ZFS commonly asks the storage device to ensure that data is >> safely placed on stable storage by requesting a cache flush." at >> http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide I >> guess this might be somewhat related to why in the "disk cache >> disabled" scenario, ZFS suffers bigger losses than UFS2. > > If that quote is correct (and I have no real reason to doubt it) then > it should probably be safe to enable the disk's write cache when used wit= h > ZFS. =A0(That would make sense since UFS/FFS was originally designed to w= ork > with an older generation of disks that did not do any significant amount > of write-caching (partly due to having very little cache on them), while > ZFS has been designed to be used on modern hardware, and to be reliable e= ven > on cheap consumer-grade disks.) Actually, now that I think of it, this could be pretty big. If using ZFS on a disk will cause the disk to flush the cache every 5 seconds, wouldn't that mean that the sections of the cache that hold data from the UFS partition get flushed to disk as well, mostly eleminating the entire "disk cache lying =3D softupdates inconsistent" problem altogether? The most important part of this is obviously, whether the "ZFS forces cache flushes every 5 seconds) thing works in all cases (like mine, where I use ZFS on a slice) and not only those where ZFS is given direct access to the disk. Anyone knowledgable in the ways of FreeBSD ZFS implementation care to chip in? :) Sincerely, Dan Naumov From owner-freebsd-fs@FreeBSD.ORG Sun Jun 21 10:41:59 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C0759106564A for ; Sun, 21 Jun 2009 10:41:59 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.243]) by mx1.freebsd.org (Postfix) with ESMTP id 774C38FC17 for ; Sun, 21 Jun 2009 10:41:59 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: by an-out-0708.google.com with SMTP id c3so1158683ana.13 for ; Sun, 21 Jun 2009 03:41:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=ckIbPN9E+QmLwZV5M3vQuJ3dxK4SrTRB68RcJu0ymXk=; b=FzlI83m0piVVPVFdhd9b8lZfzaOl70q3hm+c538wzAxe2NQ+vUXwZsaJ8Tx+K302hT cAlhc4cdRy4l2ku1ewPU3VNsxrPxU0mV8c0A+I47x0ChFTHIeLpOElMMp2Ch1DvDDixF Y3SKY8o4d9C4BXMH4yLqd2r7VuGGq2ekHHI2E= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=FY1uGtzRBG9/fpB2P5OWqTwNiY9CtgMZSgBpDTBb/PrHnPp3zcX2rFyAdclqq6tny0 5Vq19Ap//SLc/+OmnpWHJkpDzahyuq3fFKTx99y3EMmIhiWTgDw1I1Q8EwPhjVIL9rY9 99yp0s2JhE9l9Ysfa1V7DeCo216xZbTKGRR5E= MIME-Version: 1.0 Received: by 10.100.141.15 with SMTP id o15mr6425249and.20.1245580918652; Sun, 21 Jun 2009 03:41:58 -0700 (PDT) In-Reply-To: References: <20090620231130.GA88907@owl.midgard.homeip.net> <3c1674c90906201808t1854dd46n82213fbd0c1c254c@mail.gmail.com> <20090621092736.GA92656@owl.midgard.homeip.net> Date: Sun, 21 Jun 2009 13:41:58 +0300 Message-ID: From: Dan Naumov To: Erik Trulsson Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, Kip Macy Subject: Re: ufs2 / softupdates / ZFS / disk write cache X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jun 2009 10:42:00 -0000 On Sun, Jun 21, 2009 at 1:03 PM, Dan Naumov wrote: > On Sun, Jun 21, 2009 at 12:27 PM, Erik Trulsson w= rote: >> On Sun, Jun 21, 2009 at 05:18:39AM +0300, Dan Naumov wrote: >>> Uh oh.... After some digging around, I found the following quote: "ZFS >>> is designed to work with storage devices that manage a disk-level >>> cache. ZFS commonly asks the storage device to ensure that data is >>> safely placed on stable storage by requesting a cache flush." at >>> http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide I >>> guess this might be somewhat related to why in the "disk cache >>> disabled" scenario, ZFS suffers bigger losses than UFS2. >> >> If that quote is correct (and I have no real reason to doubt it) then >> it should probably be safe to enable the disk's write cache when used wi= th >> ZFS. =A0(That would make sense since UFS/FFS was originally designed to = work >> with an older generation of disks that did not do any significant amount >> of write-caching (partly due to having very little cache on them), while >> ZFS has been designed to be used on modern hardware, and to be reliable = even >> on cheap consumer-grade disks.) > > Actually, now that I think of it, this could be pretty big. If using > ZFS on a disk will cause the disk to flush the cache every 5 seconds, > wouldn't that mean that the sections of the cache that hold data from > the UFS partition get flushed to disk as well, mostly eleminating the > entire "disk cache lying =3D softupdates inconsistent" problem > altogether? The most important part of this is obviously, whether the > "ZFS forces cache flushes every 5 seconds) thing works in all cases > (like mine, where I use ZFS on a slice) and not only those where ZFS > is given direct access to the disk. Anyone knowledgable in the ways of > FreeBSD ZFS implementation care to chip in? :) Actually, if it is possible for ZFS to issue "flush the cache NOW" commands directly to disk every 5 seconds by default (value tunable) I see 2 potential options/changes that would make the life of "UFS2+softupdates on SATA disks" users a whole lot easier. One option would be to add this same functionality to softupdates, making softupdates force a disk cache flush to ensure consistency. Another option would be to have a loader.conf tunable where you could enable and manually adjust the time intervals of forced disk cache flushes (without any regard for actual filesystem used). The latter option is a bit uglier, but still a LOT less ugly than suggesting people disable disk cache altogether ending up with 2-4MB/s write speeds on modern hardware. Should I perhaps Should I perhaps do a "proposed change" send-PR regarding either option? - Sincerely, Dan Naumov From owner-freebsd-fs@FreeBSD.ORG Sun Jun 21 11:45:18 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3DEDF106564A for ; Sun, 21 Jun 2009 11:45:18 +0000 (UTC) (envelope-from andrew@modulus.org) Received: from email.octopus.com.au (email.octopus.com.au [122.100.2.232]) by mx1.freebsd.org (Postfix) with ESMTP id EE8208FC18 for ; Sun, 21 Jun 2009 11:45:17 +0000 (UTC) (envelope-from andrew@modulus.org) Received: by email.octopus.com.au (Postfix, from userid 1002) id 1F345172B9; Sun, 21 Jun 2009 21:45:39 +1000 (EST) X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on email.octopus.com.au X-Spam-Level: X-Spam-Status: No, score=-1.4 required=10.0 tests=ALL_TRUSTED autolearn=failed version=3.2.3 Received: from [10.20.30.102] (60.218.233.220.static.exetel.com.au [220.233.218.60]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: admin@email.octopus.com.au) by email.octopus.com.au (Postfix) with ESMTP id E59051723B; Sun, 21 Jun 2009 21:45:34 +1000 (EST) Message-ID: <4A3E1C72.6000406@modulus.org> Date: Sun, 21 Jun 2009 21:41:38 +1000 From: Andrew Snow User-Agent: Thunderbird 2.0.0.6 (X11/20070926) MIME-Version: 1.0 To: Dan Naumov , freebsd-fs@freebsd.org References: <20090620231130.GA88907@owl.midgard.homeip.net> <3c1674c90906201808t1854dd46n82213fbd0c1c254c@mail.gmail.com> <20090621092736.GA92656@owl.midgard.homeip.net> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: ufs2 / softupdates / ZFS / disk write cache X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jun 2009 11:45:18 -0000 Dan Naumov wrote: > Actually, if it is possible for ZFS to issue "flush the cache NOW" > commands directly to disk every 5 seconds by default (value tunable) I > see 2 potential options/changes that would make the life of > "UFS2+softupdates on SATA disks" users a whole lot easier. I believe you might be barking up the wrong tree here. The issue with cache flushing is not that changes are written to disk - that happens anyway, eventually, even if you don't explicitly flush the cache. The big issue is that things get written to disk in the correct order so that the system can recover from an unexpected crash. Even if the cache gets flushed every 5 seconds, things on UFS could get still get corrupted between T=0 and 4.9s, if the system writes something and expects it to be written syncronously and it is isn't. The only truly safe bet here is to turn disable the ATA write cache tunable. Folks who need to maximize safety and can't afford the performance hit of no write cache need to do what they always have had to do in the past - buy a controller card with battery-backed cached. - Andrew From owner-freebsd-fs@FreeBSD.ORG Sun Jun 21 17:33:34 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 69B94106570D for ; Sun, 21 Jun 2009 17:33:34 +0000 (UTC) (envelope-from numisemis@yahoo.com) Received: from web37308.mail.mud.yahoo.com (web37308.mail.mud.yahoo.com [209.191.90.251]) by mx1.freebsd.org (Postfix) with SMTP id 0CE398FC1C for ; Sun, 21 Jun 2009 17:33:33 +0000 (UTC) (envelope-from numisemis@yahoo.com) Received: (qmail 23287 invoked by uid 60001); 21 Jun 2009 17:33:33 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1245605613; bh=4kFoac7B3x5eusNu9+/+g6SPvJ/xhGDhhy4pAeqoxvM=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding; b=wno8/GFqYD9Oo5gu75I+MZwZxrJJZguUvlWsKgBLDQWR1quH6dlCKs7GrehXZut+jzIQmqgU7+Npxkp26rkFrv+/5NOBk8XKI35/DHhU/0Xc980KaFmqKL2CgABBZ92umIMmDEi+Y2TtzT24hR/hY2Sg+XJv5XNvcvrzO9AqQ/c= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding; b=ClM32QQwHZzYXZTORHpz13FnJ4NUmIvzvZoJQQiuwj+h/1J73nkcDnmx3MeWMqP+obG2iw9p3yEeZ/5+cDSq1hBSHvd5XhH6e+8P3tydyqK/IIbp2NXghQEQv0UVVsna9WnUELXnqp2b0H2+SQRLm8R5H3H55EB2F2nFuCzNvFY=; Message-ID: <570433.20373.qm@web37308.mail.mud.yahoo.com> X-YMail-OSG: rfHAlKUVM1lK6kmAvLNBz2eqvwybm9SAF8K3kLcuYgTYR1dSY5J_sTm6_PSMYRy1FYRxURBCRxNVnCY0y5kAhdK7qnMYXV5rjkvMEAzE4IzAqHHHDxh9_qBZd4JaWP6cWjLjEehjZCpuUD5G__PupIzaSR6EEzrR8E4kp3j990ImvMSULBAYjKFYccBfMyifYG1SnQvGGvljClnAiaRRGpNbQBoqqvLD1BoV9rTGgMsEeF4nqMpMqmhSzFiwaOkbvy5oLJdFniU4CIuADpO_XfKGf6kRWp0518ctaE3wnj3kqMpVTeIjQaCS6ga601Irlt.oz4ICwlr3oOJikFH_BfdCIRaPCYc4ZQNb_X_nYs4u_R56Lt7_kns2MLw- Received: from [87.248.121.54] by web37308.mail.mud.yahoo.com via HTTP; Sun, 21 Jun 2009 10:33:33 PDT X-Mailer: YahooMailWebService/0.7.289.15 Date: Sun, 21 Jun 2009 10:33:33 -0700 (PDT) From: =?utf-8?B?xaBpbXVuIE1pa2VjaW4=?= To: Andrew Snow MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-fs@freebsd.org" Subject: Re: ufs2 / softupdates / ZFS / disk write cache X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jun 2009 17:33:35 -0000 21. lip. 2009., u 13:41, Andrew Snow napisao:=0A> Folk= s who need to maximize safety and can't afford the performance =0A> hit of= no write cache need to do what they always have had to do in =0A> the pas= t - buy a controller card with battery-backed cached.=0A=0AOr:=0AB) use SCS= I instead of ATA disks=0AC) use UFS+gjournal instead of UFS+SU=0AD) use ZFS= instead of UFS+SU=0A=0A=0A=0A=0A From owner-freebsd-fs@FreeBSD.ORG Sun Jun 21 18:06:04 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4A2A81065675; Sun, 21 Jun 2009 18:06:04 +0000 (UTC) (envelope-from mister.olli@googlemail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.158]) by mx1.freebsd.org (Postfix) with ESMTP id 876EB8FC08; Sun, 21 Jun 2009 18:06:03 +0000 (UTC) (envelope-from mister.olli@googlemail.com) Received: by fg-out-1718.google.com with SMTP id e12so414443fga.12 for ; Sun, 21 Jun 2009 11:06:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:subject:from:reply-to:to:cc :in-reply-to:references:content-type:date:message-id:mime-version :x-mailer:content-transfer-encoding; bh=L8Ovnieyg+D/BnLnqTipN5BF9UwWfWYy9fb1f6cTv1w=; b=BIskiPD8oh+2mVO8SpB+3PfgZUYVKHKZPoG20tDwolVGdVzgDMZKhd4J8wvtMowjov vu+B743gehf7xyPm8+BAZ9UTW5pLA5sOcWosyOUyGaoMr/sVX5WAuHyAioI8+yKXe+j+ f8XMNo5Iu4m5574K6dxipHfZy/R6MxcKbGqQ4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=subject:from:reply-to:to:cc:in-reply-to:references:content-type :date:message-id:mime-version:x-mailer:content-transfer-encoding; b=HtFqeI8KcyTHoJvVkpHmQ+IV/iy0Ca1o/k13FqTToQ7UpE/qt6fagxOqw5r904gIhP c9sMHgTIyFERmBDZucEfaCBuaYKLOUj/t8L5cGcrNMxqoz/tdy2dKVSAqvwdI3FCM/ug l/OfF+Wz1BrBpMquD3AsokBjf1OjM5XcBOIrw= Received: by 10.86.49.16 with SMTP id w16mr5549952fgw.67.1245607562559; Sun, 21 Jun 2009 11:06:02 -0700 (PDT) Received: from ?10.30.1.211? (vpn-or.studi-planet.com [78.47.172.52]) by mx.google.com with ESMTPS id d6sm7119849fga.19.2009.06.21.11.06.01 (version=SSLv3 cipher=RC4-MD5); Sun, 21 Jun 2009 11:06:01 -0700 (PDT) From: Mister Olli To: Kip Macy In-Reply-To: <3c1674c90906201736o335b4a3dv862139dc55f2211f@mail.gmail.com> References: <1245519413.26909.60.camel@phoenix.blechhirn.net> <3c1674c90906201050w15e4cd5dpae76cd70d64b4e92@mail.gmail.com> <1245525965.26909.69.camel@phoenix.blechhirn.net> <3c1674c90906201232x63ddee19yf91aeac30f3401bb@mail.gmail.com> <1245527381.26909.82.camel@phoenix.blechhirn.net> <3c1674c90906201736o335b4a3dv862139dc55f2211f@mail.gmail.com> Content-Type: text/plain Date: Sun, 21 Jun 2009 20:05:50 +0200 Message-Id: <1245607551.4757.18.camel@phoenix.blechhirn.net> Mime-Version: 1.0 X-Mailer: Evolution 2.24.5 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Unable to delete files on ZFS volume X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: mister.olli@googlemail.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jun 2009 18:06:04 -0000 hi, On Sat, 2009-06-20 at 17:36 -0700, Kip Macy wrote: > > [snip] > > The closest I can come to defining it would be "fluctuation of > contents". By this I was asking if you repeatedly created and deleted > files. ok, I understand now. There's actually no real fluctuation, since I only add content, but do not delete anything. > I didn't think it would be this easy to provoke this. I've tried > provoking this with a handful of large files without success. I'm > guessing that ZFS isn't as careful about bounding metadata creation as > it is data creation. > > My focus is, and for the foreseeable future will be, fixing FreeBSD > integration issues. As this is a fundamental ZFS space management > issue I don't foresee fixing it any time soon if at all unless it is > addressed by upgrading to v15 or v16. Is that possible already? From my understanding 8-CURRENT creates pools with version13 atm. > I hope that truncating some of the files will free up space for deletion. would be nice to test that, but I just recognized that my xen play machine is down due to power supply problems. Since I don't have the time to fix that, regard this problem as 'unimorrtant' ;-)) Anyway I will give you feedback, when I managed to repair it. Regards, --- Mr. Olli From owner-freebsd-fs@FreeBSD.ORG Sun Jun 21 20:54:06 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C11C9106568C for ; Sun, 21 Jun 2009 20:54:06 +0000 (UTC) (envelope-from andrew@modulus.org) Received: from email.octopus.com.au (email.octopus.com.au [122.100.2.232]) by mx1.freebsd.org (Postfix) with ESMTP id 828E98FC29 for ; Sun, 21 Jun 2009 20:54:06 +0000 (UTC) (envelope-from andrew@modulus.org) Received: by email.octopus.com.au (Postfix, from userid 1002) id 17D6C173C9; Mon, 22 Jun 2009 06:54:28 +1000 (EST) X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on email.octopus.com.au X-Spam-Level: X-Spam-Status: No, score=-1.4 required=10.0 tests=ALL_TRUSTED autolearn=failed version=3.2.3 Received: from [10.1.50.60] (ppp121-44-41-14.lns10.syd7.internode.on.net [121.44.41.14]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: admin@email.octopus.com.au) by email.octopus.com.au (Postfix) with ESMTP id C873617275; Mon, 22 Jun 2009 06:54:19 +1000 (EST) Message-ID: <4A3E9D81.1060406@modulus.org> Date: Mon, 22 Jun 2009 06:52:17 +1000 From: Andrew Snow User-Agent: Thunderbird 2.0.0.14 (X11/20080523) MIME-Version: 1.0 To: =?UTF-8?B?xaBpbXVuIE1pa2VjaW4=?= , freebsd-fs@freebsd.org References: <570433.20373.qm@web37308.mail.mud.yahoo.com> In-Reply-To: <570433.20373.qm@web37308.mail.mud.yahoo.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: Subject: Re: ufs2 / softupdates / ZFS / disk write cache X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jun 2009 20:54:07 -0000 Å imun Mikecin wrote: > 21. lip. 2009., u 13:41, Andrew Snow napisao: >> Folks who need to maximize safety and can't afford the performance >> hit of no write cache need to do what they always have had to do in >> the past - buy a controller card with battery-backed cached. > > Or: > B) use SCSI instead of ATA disks > C) use UFS+gjournal instead of UFS+SU > D) use ZFS instead of UFS+SU All of these solutions still involve disabling of write cache, with a performance hit of varying degrees. (I have tried all of those except gjournal!) From owner-freebsd-fs@FreeBSD.ORG Sun Jun 21 20:59:49 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 02EF3106564A for ; Sun, 21 Jun 2009 20:59:49 +0000 (UTC) (envelope-from mat.macy@gmail.com) Received: from mail-yx0-f200.google.com (mail-yx0-f200.google.com [209.85.210.200]) by mx1.freebsd.org (Postfix) with ESMTP id AAA428FC0A for ; Sun, 21 Jun 2009 20:59:48 +0000 (UTC) (envelope-from mat.macy@gmail.com) Received: by yxe38 with SMTP id 38so1423084yxe.3 for ; Sun, 21 Jun 2009 13:59:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=2w/+SMKAzr7K5Pq8Nq8bzaLUN0eo9HXWICACHXBRrv8=; b=XIsoUpghEaLTYrJl+uiGWJABqcEutFXlXclORtZpGjHr/b8Y4oKp8EM2ZGC6HX30on bkjD2jotKGZ96BCEw6RV8NtwA/6dxVwDiB6isZgfCM+UcoDkRCYeuVnpFwYrb54vdjZ+ Iwa4VsQYFsUr0KuPhnl9AGO0eJISmetf0409A= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=EdzrEFEMPEsmca05T+tyB5C3O5yBaS6ULAj/WyMZvlwzclG1u2PMMh/9ltKZErowUP czMsh+sRLT8lZo6B4z2Na+eu3xsOKjTR7f1hDqiU8lrVCkIN4ZDc91v7C9iwbCPPrR+P YzWrbQz43d1h1tXNOsnzOsuAX03YUgo2JdPfI= MIME-Version: 1.0 Sender: mat.macy@gmail.com Received: by 10.100.45.13 with SMTP id s13mr6932984ans.193.1245617987847; Sun, 21 Jun 2009 13:59:47 -0700 (PDT) In-Reply-To: <20090621092736.GA92656@owl.midgard.homeip.net> References: <20090620231130.GA88907@owl.midgard.homeip.net> <3c1674c90906201808t1854dd46n82213fbd0c1c254c@mail.gmail.com> <20090621092736.GA92656@owl.midgard.homeip.net> Date: Sun, 21 Jun 2009 13:59:47 -0700 X-Google-Sender-Auth: a5f981676ed8b0be Message-ID: <3c1674c90906211359l78e1e953lfe208067aa673873@mail.gmail.com> From: Kip Macy To: Erik Trulsson Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: ufs2 / softupdates / ZFS / disk write cache X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jun 2009 20:59:49 -0000 > If you have IDE/SATA disks they will "lie". =A0SCSI/SAS disks won't. > SATA disks using NCQ should probably also be safe -- too bad FreeBSD > does not support NCQ yet. > NCQ is working in a private branch. I haven't heard an ETA from the develop= er. -Kip From owner-freebsd-fs@FreeBSD.ORG Sun Jun 21 22:25:57 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9CFD41065676; Sun, 21 Jun 2009 22:25:57 +0000 (UTC) (envelope-from ronald-freebsd8@klop.yi.org) Received: from smtp-out2.tiscali.nl (smtp-out2.tiscali.nl [195.241.79.177]) by mx1.freebsd.org (Postfix) with ESMTP id 37C778FC1C; Sun, 21 Jun 2009 22:25:55 +0000 (UTC) (envelope-from ronald-freebsd8@klop.yi.org) Received: from [212.123.145.58] (helo=sjakie.klop.ws) by smtp-out2.tiscali.nl with esmtp id 1MIVU2-0005wH-1U; Mon, 22 Jun 2009 00:25:54 +0200 Received: from 82-170-177-25.ip.telfort.nl (localhost [127.0.0.1]) by sjakie.klop.ws (Postfix) with ESMTP id 7C7D0D189; Mon, 22 Jun 2009 00:25:53 +0200 (CEST) Date: Mon, 22 Jun 2009 00:25:53 +0200 To: "Dan Naumov" , "FreeBSD-STABLE Mailing List" , freebsd-fs@freebsd.org From: "Ronald Klop" Content-Type: text/plain; format=flowed; delsp=yes; charset=us-ascii MIME-Version: 1.0 References: Content-Transfer-Encoding: 7bit Message-ID: In-Reply-To: User-Agent: Opera Mail/9.64 (FreeBSD) Cc: Subject: Re: ufs2 / softupdates / ZFS / disk write cache X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jun 2009 22:25:58 -0000 On Sat, 20 Jun 2009 23:29:26 +0200, Dan Naumov wrote: > I have the following setup: > > A single consumer-grade 2tb SATA disk: Western Digital Green (model > WDC WD20EADS-00R6B0). This disk is setup like this: > > 16gb root partition with UFS2 + softupdates, containing mostly static > things: > /bin /boot /etc /root /sbin /usr /var and such > > a 1,9tb non-redundant zfs pool on top of a slice, it hosts things like: > /DATA, /home, /usr/local, /var/log and such. > > What should I do to ensure (as much as possible) filesystem > consistency of the root filesystem in the case of the power loss? I > know there have been a lot of discussions on the subject of > consumer-level disks literally lying about the state of files in > transit (disks telling the system that files have been written to disk > while in reality they are still in disk's write cache), in turn > throwing softupdates off balance (since softupdates assumes the disks > don't lie about such things), in turn sometimes resulting in severe > data losses in the case of a system power loss during heavy disk IO. > > One of the solutions that was often brought up in the mailing lists is > disabling the actual disk write cache via adding hw.ata.wc=0 to > /boot/loader.conf, FreeBSD 4.3 actually even had this setting by > default, but this was apparently reverted back because some people > have reported a write performance regression on the tune of becoming > 4-6 times slower. So what should I do in my case? Should I disable > disk write cache via the hw.ata.wc tunable? As far as I know, ZFS has > a write cache of it's own and since the ufs2 root filesystem in my > case is mostly static data, I am guessing I "shouldn't" notice that > big of a performance hit. Or am I completely in the wrong here and > setting hw.ata.wc=0 is going to adversely affect the write performance > on both the root partition AND the zfs pool despite zfs using it's own > write cache? > > Another thing I have been pondering is: I do have 2gb of space left > unused on the system (currently being used as swap, I have 2 swap > slices, one 1gb at the very beginning of the disk, the other being 2gb > at the end), which I could turn into a GJOURNAL for the root > filesystem... Using gjournal is a very trusted way for a good balance in consistency and speed. I don't know about any performance impact of having the journal at the other 'end' of the disk than where the fs is. You can try, because switching back is possible. Ronald. From owner-freebsd-fs@FreeBSD.ORG Sun Jun 21 22:36:28 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1228D106564A for ; Sun, 21 Jun 2009 22:36:28 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.247]) by mx1.freebsd.org (Postfix) with ESMTP id C0A328FC0C for ; Sun, 21 Jun 2009 22:36:27 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: by an-out-0708.google.com with SMTP id c3so1248851ana.13 for ; Sun, 21 Jun 2009 15:36:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=+VrQdZE4cmMreA1sNFpzdsi1L+rYV7Ij9FIBZYP2qRE=; b=uTpc2GleK+sWHpFJbg89w9hTbIvG/ccZkDGakfxvmlvIRDn7YeGy40/ILZTnfQ6Isw 3ySDRAHLWt5LwynGHwdFB1lOJuLfRmJe0AwZBHI5LfyTRu1ITjR3pbhchB+ZryWNdRla c/O2xJrC8xUZNmYSwfvgooNABC0PjCs5MjtRc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=ekhzH1s3sZX+/oKkZAojpbHpkLRC4C/VJ4FrlB293LGN4qjyCdIM3yCx+5cx2krFLF yWXrydhYBy93xIR1+OjiwmR/a1EaCM/e9/zYDHVnW9meW/XVWHnnpawweOzt9127TAKG DLd6U07ttJN2lKdVs7AEVbgb9HaaL+UfwG5as= MIME-Version: 1.0 Received: by 10.100.95.11 with SMTP id s11mr7134667anb.112.1245623786760; Sun, 21 Jun 2009 15:36:26 -0700 (PDT) In-Reply-To: References: <570433.20373.qm@web37308.mail.mud.yahoo.com> <4A3E9D81.1060406@modulus.org> Date: Mon, 22 Jun 2009 01:36:26 +0300 Message-ID: From: Dan Naumov To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Subject: Re: ufs2 / softupdates / ZFS / disk write cache X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jun 2009 22:36:28 -0000 ---------- Forwarded message ---------- From: Dan Naumov Date: Mon, Jun 22, 2009 at 1:34 AM Subject: Re: ufs2 / softupdates / ZFS / disk write cache To: Andrew Snow On Sun, Jun 21, 2009 at 11:52 PM, Andrew Snow wrote: > > =8Aimun Mikecin wrote: >> >> 21. lip. 2009., u 13:41, Andrew Snow napisao: >>> >>> Folks who need to maximize safety and can't afford the performance =A0h= it of no write cache need to do what they always have had to do in =A0the p= ast - buy a controller card with battery-backed cached. >> >> Or: >> B) use SCSI instead of ATA disks >> C) use UFS+gjournal instead of UFS+SU >> D) use ZFS instead of UFS+SU > > > All of these solutions still involve disabling of write cache, with a per= formance hit of varying degrees. (I have tried all of those except gjournal= !) Why would using UFS+gjournal, ZFS or SCSI still involve disabling write cache? Disabling write cache is only suggested for systems using UFS + SU on ATA/PATA disks. This in turn actually raises a new point: Why is UFS + SU still the default option provided by the sysinstall installation process? A simple truth is that most FreeBSD users on a modern system are going to be using SATA disks. Another simple truth is that most new users are going to go with what the default installation process suggests and as shown by the benchmark results a few posts back, noone is going to be disabling the write cache unless they are completely out of their mind (who on earth is going to accept 2-4 MB/s write speeds from a modern disk in 2009?). - Dan Naumov From owner-freebsd-fs@FreeBSD.ORG Sun Jun 21 22:51:31 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 660D9106564A for ; Sun, 21 Jun 2009 22:51:31 +0000 (UTC) (envelope-from andrew@modulus.org) Received: from email.octopus.com.au (email.octopus.com.au [122.100.2.232]) by mx1.freebsd.org (Postfix) with ESMTP id 2532A8FC1B for ; Sun, 21 Jun 2009 22:51:30 +0000 (UTC) (envelope-from andrew@modulus.org) Received: by email.octopus.com.au (Postfix, from userid 1002) id E46BB17D4C; Mon, 22 Jun 2009 08:51:52 +1000 (EST) X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on email.octopus.com.au X-Spam-Level: X-Spam-Status: No, score=-1.4 required=10.0 tests=ALL_TRUSTED autolearn=failed version=3.2.3 Received: from [10.1.50.60] (ppp121-44-41-14.lns10.syd7.internode.on.net [121.44.41.14]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: admin@email.octopus.com.au) by email.octopus.com.au (Postfix) with ESMTP id 6E1CE1717D; Mon, 22 Jun 2009 08:51:46 +1000 (EST) Message-ID: <4A3EB902.8080503@modulus.org> Date: Mon, 22 Jun 2009 08:49:38 +1000 From: Andrew Snow User-Agent: Thunderbird 2.0.0.14 (X11/20080523) MIME-Version: 1.0 To: Dan Naumov , freebsd-fs@freebsd.org References: <570433.20373.qm@web37308.mail.mud.yahoo.com> <4A3E9D81.1060406@modulus.org> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: ufs2 / softupdates / ZFS / disk write cache X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jun 2009 22:51:31 -0000 Dan Naumov wrote: >>> Or: >>> B) use SCSI instead of ATA disks >>> C) use UFS+gjournal instead of UFS+SU >>> D) use ZFS instead of UFS+SU >> All of these solutions still involve disabling of write cache, with a performance hit of varying degrees. (I have tried all of those except gjournal!) B) SCSI drives come with write caching disabled by default. But here, the performance loss is partially made up by Tagged Command Queueing and faster spindle speeds C) gjournal needs to flush the disk cache regularly to maintain consistence. It doesn't need to do it as often but on a write-heavy system it isn't ideal for performance because it flushes everything in the cache and not just the journal. D) ZFS - same as (C) > who on earth is going to accept 2-4 MB/s write speeds from > a modern disk in 2009? eg. remote headless systems which don't do much (DNS server) :-) - Andrew From owner-freebsd-fs@FreeBSD.ORG Sun Jun 21 22:57:21 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 97689106564A for ; Sun, 21 Jun 2009 22:57:21 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: from yw-out-2324.google.com (yw-out-2324.google.com [74.125.46.30]) by mx1.freebsd.org (Postfix) with ESMTP id 4BDB68FC24 for ; Sun, 21 Jun 2009 22:57:21 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: by yw-out-2324.google.com with SMTP id 9so1274236ywe.13 for ; Sun, 21 Jun 2009 15:57:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=o4hQJlA1mBE06J8PuDaiAH/MRe7g8NWWX9DPUjjRW8A=; b=I5v/5wP1aAbCRFnaMC0ZxO73+XVdEa/J0661L5+/s4INPPBmDKafKPlp8fkicjwXyg fqskBHiFZMAvyeOw7xSQ/pTPwNvOfldT3adzRAfahcPYoPodEWG/o0XSOxOdZFdOYFKZ IGLmn8Xo7tpmWKY8HwQ6eNcA19aPMIKlGSwvI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=N5ANwGKFtQH9wFqAQIMeTnyKk28iQw9xz1C2uRedvpL93V2WzHmoYqm9qVUEVj7spE E8zTUjx/dGM65i+JVI/lKZ/pwOG4acYmRsFxd5w6zznoe/3fWCvxzpeYgbXqRZaSd/gS cp6UHMpalU6F2hwL8nAFFV4JLVpRoXSlNC2/8= MIME-Version: 1.0 Received: by 10.100.251.6 with SMTP id y6mr7192796anh.44.1245625040447; Sun, 21 Jun 2009 15:57:20 -0700 (PDT) In-Reply-To: <570433.20373.qm@web37308.mail.mud.yahoo.com> References: <570433.20373.qm@web37308.mail.mud.yahoo.com> Date: Mon, 22 Jun 2009 01:57:20 +0300 Message-ID: From: Dan Naumov To: =?windows-1252?Q?=8Aimun_Mikecin?= Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "freebsd-fs@freebsd.org" Subject: Re: ufs2 / softupdates / ZFS / disk write cache X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jun 2009 22:57:22 -0000 2009/6/21 =8Aimun Mikecin > > 21. lip. 2009., u 13:41, Andrew Snow napisao: > > Folks who need to maximize safety and can't afford the performance > > hit of no write cache need to do what they always have had to do in > > the past - buy a controller card with battery-backed cached. > > Or: > B) use SCSI instead of ATA disks > C) use UFS+gjournal instead of UFS+SU > D) use ZFS instead of UFS+SU Actually I think a need a few clarifications regarding ZFS: 1) Does FreeBSD honor the "flush the cache to disk now" commands issued by ZFS to the harrdive only when ZFS is used directly on top of a disk device directly or does this also work when ZFS is used on top of a slice/partition? 2) If we compare ZFS vs UFS+SU while using a regular "lying" SATA disk (wit= h write cache enabled) under heavy IO followed by a power loss. Which one is going to recover better and why? Sincerely, - Dan Naumov From owner-freebsd-fs@FreeBSD.ORG Mon Jun 22 01:17:57 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1BF7C1065675 for ; Mon, 22 Jun 2009 01:17:57 +0000 (UTC) (envelope-from nhoyle@hoyletech.com) Received: from mout.perfora.net (mout.perfora.net [74.208.4.194]) by mx1.freebsd.org (Postfix) with ESMTP id DA1E78FC1E for ; Mon, 22 Jun 2009 01:17:56 +0000 (UTC) (envelope-from nhoyle@hoyletech.com) Received: from [192.168.1.10] (pool-96-231-140-65.washdc.fios.verizon.net [96.231.140.65]) by mrelay.perfora.net (node=mrus1) with ESMTP (Nemesis) id 0MKpCa-1MIYAN18Jf-000cfM; Sun, 21 Jun 2009 21:17:51 -0400 Message-ID: <4A3EDBB8.6010402@hoyletech.com> Date: Mon, 22 Jun 2009 01:17:44 +0000 From: Nathanael Hoyle User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: Andrew Snow , Dan Naumov , freebsd-fs@freebsd.org References: <570433.20373.qm@web37308.mail.mud.yahoo.com> <4A3E9D81.1060406@modulus.org> <4A3EB902.8080503@modulus.org> In-Reply-To: <4A3EB902.8080503@modulus.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V01U2FsdGVkX18LytBHB1A3wkUKo9MCbbRzIt9mSuoI3OMs/OP WnrlfxbOQhyOJHn10i63ZyNluCHj4NRElj1ty9cZ7pKiPGWSXn q3UcoOpeZPZsGSDXb21Jr4q1Ulg/ars Cc: Subject: Re: ufs2 / softupdates / ZFS / disk write cache X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jun 2009 01:17:57 -0000 Andrew Snow wrote: > Dan Naumov wrote: >>>> Or: >>>> B) use SCSI instead of ATA disks >>>> C) use UFS+gjournal instead of UFS+SU >>>> D) use ZFS instead of UFS+SU >>> All of these solutions still involve disabling of write cache, with >>> a performance hit of varying degrees. (I have tried all of those >>> except gjournal!) > > B) SCSI drives come with write caching disabled by default. But here, > the performance loss is partially made up by Tagged Command Queueing > and faster spindle speeds > > C) gjournal needs to flush the disk cache regularly to maintain > consistence. It doesn't need to do it as often but on a write-heavy > system it isn't ideal for performance because it flushes everything in > the cache and not just the journal. > > D) ZFS - same as (C) > As a minor nitpick to point D, IIRC it is possible to explicitly place the ZIL on a different device than the pool it is for. In this case, if the ZIL is on a dedicated device, then it is possible to flush only the ZIL, rather than all data pending in cache for the zpool. I realize it's a minor distinction / special case, but the option is worth mentioning. -Nathanael From owner-freebsd-fs@FreeBSD.ORG Mon Jun 22 01:22:10 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9CA0B1065724 for ; Mon, 22 Jun 2009 01:22:10 +0000 (UTC) (envelope-from nhoyle@hoyletech.com) Received: from mout.perfora.net (mout.perfora.net [74.208.4.194]) by mx1.freebsd.org (Postfix) with ESMTP id 671808FC08 for ; Mon, 22 Jun 2009 01:22:10 +0000 (UTC) (envelope-from nhoyle@hoyletech.com) Received: from [192.168.1.10] (pool-96-231-140-65.washdc.fios.verizon.net [96.231.140.65]) by mrelay.perfora.net (node=mrus0) with ESMTP (Nemesis) id 0MKp8S-1MIYEW47Up-000fyL; Sun, 21 Jun 2009 21:22:09 -0400 Message-ID: <4A3EDCBA.7010204@hoyletech.com> Date: Mon, 22 Jun 2009 01:22:02 +0000 From: Nathanael Hoyle User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: Dan Naumov References: <570433.20373.qm@web37308.mail.mud.yahoo.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-Provags-ID: V01U2FsdGVkX1+/DYqODbZyTnlrlBYH54F0GUYxqvpk1aKm6Lw JmQRrRHjY8KvWnX75yf6xx6Kp7aplQ3+EDpxvPhW5I+Hst29J1 q2fwE0STf7JfHbGFTzRtPtDApnsnPTm Cc: "freebsd-fs@freebsd.org" Subject: Re: ufs2 / softupdates / ZFS / disk write cache X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jun 2009 01:22:11 -0000 Dan Naumov wrote: > 2009/6/21 Šimun Mikecin > > >> 21. lip. 2009., u 13:41, Andrew Snow napisao: >> >>> Folks who need to maximize safety and can't afford the performance >>> hit of no write cache need to do what they always have had to do in >>> the past - buy a controller card with battery-backed cached. >>> >> Or: >> B) use SCSI instead of ATA disks >> C) use UFS+gjournal instead of UFS+SU >> D) use ZFS instead of UFS+SU >> > > > Actually I think a need a few clarifications regarding ZFS: > > 1) Does FreeBSD honor the "flush the cache to disk now" commands issued by > ZFS to the harrdive only when ZFS is used directly on top of a disk device > directly or does this also work when ZFS is used on top of a > slice/partition? > 2) If we compare ZFS vs UFS+SU while using a regular "lying" SATA disk (with > write cache enabled) under heavy IO followed by a power loss. Which one is > going to recover better and why? > > > Sincerely, > - Dan Naumov > ZFS should recover better I believe. It's copy-on-write semantics mean that you always have a valid, intact (even if not the most recent) copy of the data. I believe with soft updates it is still possible to have partially written metadata updates cause problems. I'm not as much of an expert on soft updates semantics however, so I'll defer to those who are to correct me if I'm off-base. -Nathanael From owner-freebsd-fs@FreeBSD.ORG Mon Jun 22 01:42:54 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6B410106566C for ; Mon, 22 Jun 2009 01:42:54 +0000 (UTC) (envelope-from nhoyle@hoyletech.com) Received: from mout.perfora.net (mout.perfora.net [74.208.4.195]) by mx1.freebsd.org (Postfix) with ESMTP id 35E188FC0A for ; Mon, 22 Jun 2009 01:42:53 +0000 (UTC) (envelope-from nhoyle@hoyletech.com) Received: from [192.168.1.10] (pool-96-231-140-65.washdc.fios.verizon.net [96.231.140.65]) by mrelay.perfora.net (node=mrus0) with ESMTP (Nemesis) id 0MKp8S-1MIYYK1112-000g8S; Sun, 21 Jun 2009 21:42:37 -0400 Message-ID: <4A3EE185.9040205@hoyletech.com> Date: Mon, 22 Jun 2009 01:42:29 +0000 From: Nathanael Hoyle User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: remko@FreeBSD.org References: <200906210945.n5L9jVOS009819@freefall.freebsd.org> In-Reply-To: <200906210945.n5L9jVOS009819@freefall.freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V01U2FsdGVkX1/AM22FI3iviTgC4cvshlteL+u+LJWI4imv0W3 ge8EkJuwXi3mEVGIIIvnvo12Z+Ell3yLFsgqpJ/yiWPpSeahDS 8aGdW3To9bq01l8y91OoPYNTYFTFYci Cc: freebsd-fs@FreeBSD.org, mikej@paymentallianceintl.com, freebsd-bugs@FreeBSD.org Subject: Re: bin/135710: mount(8): mount -t tmpfs does not follow 'size' option X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jun 2009 01:42:54 -0000 remko@FreeBSD.org wrote: > Synopsis: mount(8): mount -t tmpfs does not follow 'size' option > > Responsible-Changed-From-To: freebsd-bugs->freebsd-fs > Responsible-Changed-By: remko > Responsible-Changed-When: Sun Jun 21 09:45:21 UTC 2009 > Responsible-Changed-Why: > reassign to fs team > > http://www.freebsd.org/cgi/query-pr.cgi?pr=135710 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > Quoting Peter Snyder in "tmpfs: A Virtual Memory File System", in which he details the design for Solaris tmpfs, which served as the foundation for NetBSD's tmpfs, which was in turn integrated into FreeBSD: "Instead of allocating a fixed amount of memory for exclusive use as a file system, tmpfs file system size is dynamic depending on use, allowing the system to decide the optimal use of memory." And from the FreeBSD tmpfs(5) man page, "*size* - maximum size (in bytes) for the file system", note the use of the word "maximum" versus say, "initial". I would attempt to actually populate/fill the created file system, and unless it returns as being full with less than the size specified worth of data, I believe the behavior observed is as intended. -Nathanael From owner-freebsd-fs@FreeBSD.ORG Mon Jun 22 11:06:54 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 613E71065673 for ; Mon, 22 Jun 2009 11:06:54 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 4D9E38FC29 for ; Mon, 22 Jun 2009 11:06:54 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n5MB6s2k018004 for ; Mon, 22 Jun 2009 11:06:54 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n5MB6r0P018000 for freebsd-fs@FreeBSD.org; Mon, 22 Jun 2009 11:06:53 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 22 Jun 2009 11:06:53 GMT Message-Id: <200906221106.n5MB6r0P018000@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jun 2009 11:06:54 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o bin/135710 fs mount(8): mount -t tmpfs does not follow 'size' option o kern/135594 fs [zfs] Single dataset unresponsive with Samba o kern/135546 fs [zfs] zfs.ko module doesn't ignore zpool.cache filenam o kern/135480 fs [zfs] panic: lock &arg.lock already initialized o kern/135469 fs [ufs] [panic] kernel crash on md operation in ufs_dirb o kern/135412 fs [zfs] [nfs] zfs(v13)+nfs and open(..., O_WRONLY|O_CREA o bin/135314 fs [zfs] assertion failed for zdb(8) usage o kern/135050 fs [zfs] ZFS clears/hides disk errors on reboot o kern/135039 fs [zfs] mkstemp() fails over NFS when server uses ZFS (7 f kern/134496 fs [zfs] [panic] ZFS pool export occasionally causes a ke o kern/134491 fs [zfs] Hot spares are rather cold... o kern/133980 fs [panic] [ffs] panic: ffs_valloc: dup alloc o kern/133676 fs [smbfs] [panic] umount -f'ing a vnode-based memory dis o kern/133614 fs [smbfs] [panic] panic: ffs_truncate: read-only filesys o kern/133373 fs [zfs] umass attachment causes ZFS checksum errors, dat o kern/133174 fs [msdosfs] [patch] msdosfs must support utf-encoded int f kern/133150 fs [zfs] Page fault with ZFS on 7.1-RELEASE/amd64 while w o kern/133134 fs [zfs] Missing ZFS zpool labels f kern/133020 fs [zfs] [panic] inappropriate panic caused by zfs. Pani o kern/132960 fs [ufs] [panic] panic:ffs_blkfree: freeing free frag o kern/132597 fs [tmpfs] [panic] tmpfs-related panic while interrupting o kern/132551 fs [zfs] ZFS locks up on extattr_list_link syscall o kern/132397 fs reboot causes filesystem corruption (failure to sync b o kern/132337 fs [zfs] [panic] kernel panic in zfs_fuid_create_cred o kern/132331 fs [ufs] [lor] LOR ufs and syncer o kern/132237 fs [msdosfs] msdosfs has problems to read MSDOS Floppy o kern/132145 fs [panic] File System Hard Crashes f kern/132068 fs [zfs] page fault when using ZFS over NFS on 7.1-RELEAS o kern/131995 fs [nfs] Failure to mount NFSv4 server o kern/131360 fs [nfs] poor scaling behavior of the NFS server under lo o kern/131342 fs [nfs] mounting/unmounting of disks causes NFS to fail o bin/131341 fs makefs: error "Bad file descriptor" on the mount poin o kern/131086 fs [ext2fs] [patch] mkfs.ext2 creates rotten partition o kern/130979 fs [smbfs] [panic] boot/kernel/smbfs.ko o kern/130920 fs [msdosfs] cp(1) takes 100% CPU time while copying file o kern/130229 fs [iconv] usermount fails on fs that need iconv o kern/130210 fs [nullfs] Error by check nullfs o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129488 fs [smbfs] Kernel "bug" when using smbfs in smbfs_smb.c: o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/129148 fs [zfs] [panic] panic on concurrent writing & rollback o kern/129059 fs [zfs] [patch] ZFS bootloader whitelistable via WITHOUT f kern/128829 fs smbd(8) causes periodic panic on 7-RELEASE o kern/128633 fs [zfs] [lor] lock order reversal in zfs o kern/128514 fs [zfs] [mpt] problems with ZFS and LSILogic SAS/SATA Ad f kern/128173 fs [ext2fs] ls gives "Input/output error" on mounted ext3 o kern/127659 fs [tmpfs] tmpfs memory leak o kern/127492 fs [zfs] System hang on ZFS input-output o kern/127420 fs [gjournal] [panic] Journal overflow on gmirrored gjour o kern/127213 fs [tmpfs] sendfile on tmpfs data corruption o kern/127029 fs [panic] mount(8): trying to mount a write protected zi o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file s kern/125738 fs [zfs] [request] SHA256 acceleration in ZFS o kern/125644 fs [zfs] [panic] zfs unfixable fs errors caused panic whe f kern/125536 fs [ext2fs] ext 2 mounts cleanly but fails on commands li o kern/125149 fs [nfs] [panic] changing into .zfs dir from nfs client c f kern/124621 fs [ext3] [patch] Cannot mount ext2fs partition f bin/124424 fs [zfs] zfs(8): zfs list -r shows strange snapshots' siz o kern/123939 fs [msdosfs] corrupts new files o kern/122888 fs [zfs] zfs hang w/ prefetch on, zil off while running t o kern/122380 fs [ffs] ffs_valloc:dup alloc (Soekris 4801/7.0/USB Flash o kern/122173 fs [zfs] [panic] Kernel Panic if attempting to replace a o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o kern/122047 fs [ext2fs] [patch] incorrect handling of UF_IMMUTABLE / o kern/122038 fs [tmpfs] [panic] tmpfs: panic: tmpfs_alloc_vp: type 0xc o bin/121898 fs [nullfs] pwd(1)/getcwd(2) fails with Permission denied o bin/121779 fs [ufs] snapinfo(8) (and related tools?) only work for t o kern/121770 fs [zfs] ZFS on i386, large file or heavy I/O leads to ke o bin/121366 fs [zfs] [patch] Automatic disk scrubbing from periodic(8 o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha f kern/120991 fs [panic] [fs] [snapshot] System crashes when manipulati o kern/120483 fs [ntfs] [patch] NTFS filesystem locking changes o kern/120482 fs [ntfs] [patch] Sync style changes between NetBSD and F o bin/120288 fs zfs(8): "zfs share -a" does not send SIGHUP to mountd f kern/119735 fs [zfs] geli + ZFS + samba starting on boot panics 7.0-B o kern/118912 fs [2tb] disk sizing/geometry problem with large array o misc/118855 fs [zfs] ZFS-related commands are nonfunctional in fixit o kern/118713 fs [minidump] [patch] Display media size required for a k o kern/118320 fs [zfs] [patch] NFS SETATTR sometimes fails to set file o bin/118249 fs mv(1): moving a directory changes its mtime o kern/118107 fs [ntfs] [panic] Kernel panic when accessing a file at N o bin/117315 fs [smbfs] mount_smbfs(8) and related options can't mount o kern/117314 fs [ntfs] Long-filename only NTFS fs'es cause kernel pani o kern/117158 fs [zfs] zpool scrub causes panic if geli vdevs detach on o bin/116980 fs [msdosfs] [patch] mount_msdosfs(8) resets some flags f o kern/116913 fs [ffs] [panic] ffs_blkfree: freeing free block p kern/116608 fs [msdosfs] [patch] msdosfs fails to check mount options o kern/116583 fs [ffs] [hang] System freezes for short time when using o kern/116170 fs [panic] Kernel panic when mounting /tmp o kern/115645 fs [snapshots] [panic] lockmgr: thread 0xc4c00d80, not ex o bin/115361 fs [zfs] mount(8) gets into a state where it won't set/un o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o kern/113852 fs [smbfs] smbfs does not properly implement DFS referral o bin/113838 fs [patch] [request] mount(8): add support for relative p o kern/113180 fs [zfs] Setting ZFS nfsshare property does not cause inh o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/111843 fs [msdosfs] Long Names of files are incorrectly created o kern/111782 fs [ufs] dump(8) fails horribly for large filesystems s bin/111146 fs [2tb] fsck(8) fails on 6T filesystem o kern/109024 fs [msdosfs] mount_msdosfs: msdosfs_iconv: Operation not o kern/109010 fs [msdosfs] can't mv directory within fat32 file system o bin/107829 fs [2TB] fdisk(8): invalid boundary checking in fdisk / w o kern/106030 fs [ufs] [panic] panic in ufs from geom when a dead disk o kern/105093 fs [ext2fs] [patch] ext2fs on read-only media cannot be m o kern/104406 fs [ufs] Processes get stuck in "ufs" state under persist f kern/104133 fs [ext2fs] EXT2FS module corrupts EXT2/3 filesystems o kern/103035 fs [ntfs] Directories in NTFS mounted disc images appear o kern/101324 fs [smbfs] smbfs sometimes not case sensitive when it's s o kern/99290 fs [ntfs] mount_ntfs ignorant of cluster sizes o kern/97377 fs [ntfs] [patch] syntax cleanup for ntfs_ihash.c o kern/95222 fs [iso9660] File sections on ISO9660 level 3 CDs ignored o kern/94849 fs [ufs] rename on UFS filesystem is not atomic o kern/94769 fs [ufs] Multiple file deletions on multi-snapshotted fil o kern/94733 fs [smbfs] smbfs may cause double unlock o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D o kern/92272 fs [ffs] [hang] Filling a filesystem while creating a sna f kern/91568 fs [ufs] [panic] writing to UFS/softupdates DVD media in o kern/91134 fs [smbfs] [patch] Preserve access and modification time a kern/90815 fs [smbfs] [patch] SMBFS with character conversions somet o kern/89991 fs [ufs] softupdates with mount -ur causes fs UNREFS o kern/88657 fs [smbfs] windows client hang when browsing a samba shar o kern/88266 fs [smbfs] smbfs does not implement UIO_NOCOPY and sendfi o kern/87859 fs [smbfs] System reboot while umount smbfs. o kern/86587 fs [msdosfs] rm -r /PATH fails with lots of small files o kern/85326 fs [smbfs] [panic] saving a file via samba to an overquot o kern/84589 fs [2TB] 5.4-STABLE unresponsive during background fsck 2 o kern/80088 fs [smbfs] Incorrect file time setting on NTFS mounted vi o kern/77826 fs [ext2fs] ext2fs usb filesystem will not mount RW o kern/73484 fs [ntfs] Kernel panic when doing `ls` from the client si o bin/73019 fs [ufs] fsck_ufs(8) cannot alloc 607016868 bytes for ino o kern/71774 fs [ntfs] NTFS cannot "see" files on a WinXP filesystem o kern/68978 fs [panic] [ufs] crashes with failing hard disk, loose po o kern/65920 fs [nwfs] Mounted Netware filesystem behaves strange o kern/65901 fs [smbfs] [patch] smbfs fails fsx write/truncate-down/tr o kern/61503 fs [smbfs] mount_smbfs does not work as non-root o kern/55617 fs [smbfs] Accessing an nsmb-mounted drive via a smb expo o kern/51685 fs [hang] Unbounded inode allocation causes kernel to loc o kern/51583 fs [nullfs] [patch] allow to work with devices and socket o kern/36566 fs [smbfs] System reboot with dead smb mount and umount o kern/18874 fs [2TB] 32bit NFS servers export wrong negative values t 145 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon Jun 22 12:19:02 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2A2F71065672 for ; Mon, 22 Jun 2009 12:19:02 +0000 (UTC) (envelope-from numisemis@yahoo.com) Received: from web37308.mail.mud.yahoo.com (web37308.mail.mud.yahoo.com [209.191.90.251]) by mx1.freebsd.org (Postfix) with SMTP id D88A08FC19 for ; Mon, 22 Jun 2009 12:19:01 +0000 (UTC) (envelope-from numisemis@yahoo.com) Received: (qmail 67957 invoked by uid 60001); 22 Jun 2009 12:19:01 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1245673141; bh=sRUAZV2X+KIS2XUY5rpmuXPq1FlF1zBaZ4tdU+yBt54=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=cjHfq/2tgUXKTmzTmIMsIZY6kzidUzUD3ahqa37xr0DCxVoey8xfsBurnL17HLmNDQwFtbhuyr/DpaE324pnImyvJLXZ3fhoGeZ06HpqFN3DTpADRLgV9hXKcAKC8fQpjKMPgRBQ/ajmMKl+pNe7kHvrahFPbaA+YYvR7aMrbFg= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=Jm3s96ACi5fC1Fmql90zOnXyIcMx20xBq/d+P2KAN+TUb5ppl3ynS3Fg9/jwpMMntprsbUa3vm1HQRvPHUsDIJ4nb3/V3hzHkQGjrRdlVODPM1g9+I/+VFqS7KZk1QGxNFcLY8NfkJFCb8fTxi45cr1MsIdFqKpCfIn8SEvz9YU=; Message-ID: <289445.67836.qm@web37308.mail.mud.yahoo.com> X-YMail-OSG: BEZVl3UVM1laqlBd5_31FKyNs3CDmAzEbRby_424qFBj87gBWkUD4b6oL7c2Ddc.WY.vtZicmdQuoUnpcGb6phNQEag2_ryhd3tKwa1JXf5vTQPQSJuVgxERGOSpfukXub2RZUiYBLcsnB7I4DWspd0rPbtZbDIfp71ePQbXvFrj5paNd5AmTyNTcD1zJQVsrAtjomlly.KhlGhaKFS.E6EICW.MLCmAE1_.p7C0StE1qkIfkUs7tkG9ZPVR5T_6quoTkUOOts7_d0Zip76jKkfcmFYYmbqcaGhFOMrwRxUxwzndKLPsDFr716AHwmG_ee3LCg-- Received: from [87.252.131.19] by web37308.mail.mud.yahoo.com via HTTP; Mon, 22 Jun 2009 05:19:00 PDT X-Mailer: YahooMailRC/1277.43 YahooMailWebService/0.7.289.15 References: <570433.20373.qm@web37308.mail.mud.yahoo.com> Date: Mon, 22 Jun 2009 05:19:00 -0700 (PDT) From: Simun Mikecin To: Dan Naumov In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: ufs2 / softupdates / ZFS / disk write cache X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jun 2009 12:19:02 -0000 Dan Naumov wrote: > Actually I think a need a few clarifications regarding ZFS: > 1) Does FreeBSD honor the "flush the cache to disk now" commands issued by ZFS to the harrdive only when ZFS is used directly on top of a disk device directly or does this also work when ZFS is used on top of a slice/partition? > 2) If we compare ZFS vs UFS+SU while using a regular "lying" SATA disk (with write cache enabled) under heavy IO followed by a power loss. Which one is going to recover better and why? 1) AFAIK on FreeBSD (in contrary to Solaris) there is no difference wheter you use a whole disk or a slice/partition. 2) I wouldn't use UFS+SU on [S]ATA disks because your background fsck will simetimes give up stating something like "unexpected softupdates inconsistency" (unless you had disabled write cache, which you don't really want to) and you will have to do a manual foreground fsck yourself. The choice should (in my opinion) be: ZFS for amd64, UFS+gjournal for i386. Both (ZFS and UFS+gjournal) will not have any recovery penalty if you have write cache enabled. If you have a controller with battery backup cache, you could even run ZFS with disabled cache flush (i don't know wheter it can be disabled on gjournal), but I'm not sure that you will get any real word performance improvement by doing it. From owner-freebsd-fs@FreeBSD.ORG Mon Jun 22 12:44:37 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 306C71065675 for ; Mon, 22 Jun 2009 12:44:37 +0000 (UTC) (envelope-from numisemis@yahoo.com) Received: from web37305.mail.mud.yahoo.com (web37305.mail.mud.yahoo.com [209.191.90.248]) by mx1.freebsd.org (Postfix) with SMTP id E1F9D8FC13 for ; Mon, 22 Jun 2009 12:44:36 +0000 (UTC) (envelope-from numisemis@yahoo.com) Received: (qmail 69468 invoked by uid 60001); 22 Jun 2009 12:44:36 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1245674676; bh=chNWlw/RHA6ZKqx0ZNHE3Gbs835BhISNi3NYoB5a858=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=5DrFGIkB73YgnWnbe2BUImG4O66r5rzw35SDyzyYe45exfWSefpSLMGi076+5CDXSzDkMoROVikz2bWhOOThQv9N3FpQKZbKU3EZ76K9cCvp4B68694H7NctMlsypTBrXvyT9xC97DMfRuJYoz/z9uuSbi+xCT4mX9k7dhEEqeM= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=2nrosbDw20rAXDYOtWoDDOuUkqfOJRnrWCNjT7z+HzaNnod4prmzWwZH7jSOsL4/PdtVNlYPC/ef3F9z4FX8GuoWqIjOoX0vYOrkJSgawg5S3QTeAKr8tC3bUzlNnPmj82vKEGStCIxLb7+C/cNASLuQkErdIIUZVL3pdtwGM9Q=; Message-ID: <457065.69200.qm@web37305.mail.mud.yahoo.com> X-YMail-OSG: KDjwnkkVM1kSWC0DwOtm5jACjtBWy0q9F8IVB6tWnnsAfbVOFAdaj8P0YIZMjFGOKKdUQuUd9QcD7MNhNazNcjNoO16saLEUOi8oTC6zk63vS1gP8LtaF_CkGlgPwlw_67T__nDZrJa8Gh0PoYJqZdDkBW1Tlr6iB6ofEC6F1dpbcJBIOfy_Zv9Ncav2Rn_OuYS51CkDd.LNPKHVx_1CCpeLxiDopQGENXECNy6RXr0Y8eJgHT5KA3L.C6JPiRbvCcrRZR2h7rvqSXUqORWZx6FU1zu0ROSDsNKfO1ZwAea5sk_FU9WKDDeKssdKHSxU40SE2A-- Received: from [87.252.131.19] by web37305.mail.mud.yahoo.com via HTTP; Mon, 22 Jun 2009 05:44:36 PDT X-Mailer: YahooMailRC/1277.43 YahooMailWebService/0.7.289.15 References: <570433.20373.qm@web37308.mail.mud.yahoo.com> <4A3E9D81.1060406@modulus.org> <4A3EB902.8080503@modulus.org> Date: Mon, 22 Jun 2009 05:44:36 -0700 (PDT) From: Simun Mikecin To: Andrew Snow In-Reply-To: <4A3EB902.8080503@modulus.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-fs@freebsd.org Subject: Re: ufs2 / softupdates / ZFS / disk write cache X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jun 2009 12:44:37 -0000 Andrew Snow wrote: > B) SCSI drives come with write caching disabled by default. But here, the performance loss is partially made up by Tagged Command Queueing and faster spindle speeds SCSI drives get configured by the controller BIOS and/or kernel driver. It usually means that both TCQ and write cache become enabled. It is considered safe to enable write cache if TCQ is enabled. The same would be valid for SATA with NCQ enabled. But FreeBSD does not (yet) support SATA NCQ. From owner-freebsd-fs@FreeBSD.ORG Mon Jun 22 13:06:44 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 28B92106566C for ; Mon, 22 Jun 2009 13:06:44 +0000 (UTC) (envelope-from numisemis@yahoo.com) Received: from web37304.mail.mud.yahoo.com (web37304.mail.mud.yahoo.com [209.191.90.247]) by mx1.freebsd.org (Postfix) with SMTP id D8F288FC17 for ; Mon, 22 Jun 2009 13:06:43 +0000 (UTC) (envelope-from numisemis@yahoo.com) Received: (qmail 94515 invoked by uid 60001); 22 Jun 2009 13:06:43 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1245676003; bh=m1zyiDrryplvFU3JyreKyk15iAU8vqtkTriJ7odmdtQ=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=JHHMDhqDevvylWHvY0vPhEdr33BEw0T6G2FKPr3TXd+kzpOSJU+96AxDsc8Vc43GvqLQ9YlMnLcrP4tfKx/GVzw3qPgaXIptpHRjRcDZ/VYW1TaKWwSAlgkc3vS5YafINpzbPR9oSSwCCkvDbCyTkuSDLPkytwrhSyiKqZ+x7x4= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=cNyPgAgxpfrQBGca9eFLuQp7OiRA1ZIxcIcKikCWhLGtsP6STRHjiX7ndOBpjdiqeyHPxMCd+gPLPyNp5LE2vbVkuMKqzeT+8m7GFHc7p+PC7MmxKDFP5+qW7QykZFJ2Pp4l/IICSEXE9XAV3HH5swvRfj1YOXuusrutZxR8IFw=; Message-ID: <303630.93763.qm@web37304.mail.mud.yahoo.com> X-YMail-OSG: 3AyDROYVM1msTupjQ6DPblfE3I2bJTuGhXYzwDeYlDi2m3W0iGuYKTXR_cd7fqhpH4__ppvfLwWssP15mTMv4iqNPPWXBJz3kDD6t0xcN3jpLbDnANPH4IQRIN1sIxbpBGSmhfoQhCY.Za7ymahN91TEVu.u2fkNRjeYyctOM24mjHTibBL9HdrJr48nf6osj7s_nD59s.8U0RWP0ZobFo7uiDN8pAluFSt.duHqNd1cn5ta8B9lxF_6RdzKt2KXD925TcwWnjThrk3nrAhfcjuAlZZfRHQZpZ0_lgnRUPaDKvF7pQvfDZhiyRQLEwKD3ccmjg-- Received: from [87.252.131.19] by web37304.mail.mud.yahoo.com via HTTP; Mon, 22 Jun 2009 06:06:43 PDT X-Mailer: YahooMailRC/1277.43 YahooMailWebService/0.7.289.15 References: <570433.20373.qm@web37308.mail.mud.yahoo.com> <4A3E9D81.1060406@modulus.org> <4A3EB902.8080503@modulus.org> Date: Mon, 22 Jun 2009 06:06:43 -0700 (PDT) From: Simun Mikecin To: Andrew Snow In-Reply-To: <4A3EB902.8080503@modulus.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-fs@freebsd.org Subject: Re: ufs2 / softupdates / ZFS / disk write cache X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jun 2009 13:06:44 -0000 Andrew Snow wrote: > All of these solutions still involve disabling of write cache, with a performance hit of varying degrees. (I have tried all of those except gjournal!) Can you please elaborate this statement? AFAIK all of these solutions are absolutelly safe to use with write cache enabled: - SCSI disks are safe with TCQ enabled (not sure when TCQ is disabled) - UFS+gjournal is safe because of regular disk cache flushing - ZFS is safe by design From owner-freebsd-fs@FreeBSD.ORG Mon Jun 22 13:21:23 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 458A3106566C; Mon, 22 Jun 2009 13:21:23 +0000 (UTC) (envelope-from gavin@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 1A8EB8FC21; Mon, 22 Jun 2009 13:21:23 +0000 (UTC) (envelope-from gavin@FreeBSD.org) Received: from freefall.freebsd.org (gavin@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n5MDLMd4029512; Mon, 22 Jun 2009 13:21:22 GMT (envelope-from gavin@freefall.freebsd.org) Received: (from gavin@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n5MDLMh6029508; Mon, 22 Jun 2009 13:21:22 GMT (envelope-from gavin) Date: Mon, 22 Jun 2009 13:21:22 GMT Message-Id: <200906221321.n5MDLMh6029508@freefall.freebsd.org> To: mandrews@bit0.com, gavin@FreeBSD.org, freebsd-fs@FreeBSD.org, gavin@FreeBSD.org From: gavin@FreeBSD.org Cc: Subject: Re: kern/135039: [zfs] mkstemp() fails over NFS when server uses ZFS (7-stable only) [regression] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jun 2009 13:21:23 -0000 Synopsis: [zfs] mkstemp() fails over NFS when server uses ZFS (7-stable only) [regression] State-Changed-From-To: open->closed State-Changed-By: gavin State-Changed-When: Mon Jun 22 13:18:35 UTC 2009 State-Changed-Why: Close this PR, kern/135412 is a duplicate of this, but contains a simple test case Responsible-Changed-From-To: freebsd-fs->gavin Responsible-Changed-By: gavin Responsible-Changed-When: Mon Jun 22 13:18:35 UTC 2009 Responsible-Changed-Why: Track http://www.freebsd.org/cgi/query-pr.cgi?pr=135039 From owner-freebsd-fs@FreeBSD.ORG Mon Jun 22 13:21:48 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 49008106568A for ; Mon, 22 Jun 2009 13:21:48 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: from mail-yx0-f200.google.com (mail-yx0-f200.google.com [209.85.210.200]) by mx1.freebsd.org (Postfix) with ESMTP id 0014C8FC16 for ; Mon, 22 Jun 2009 13:21:47 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: by yxe38 with SMTP id 38so1977412yxe.3 for ; Mon, 22 Jun 2009 06:21:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=MLgQQIAo+Yum2NYQ6gzSGqOzor/30A0UOTqKrLcJd28=; b=qzEsnXwxUeLQSu/90FJH9x+a0O3Y0rAjeUnJHKmbZk6sPCbxMdwBGBTSxb/vM2przb 2haKTJIA1LGgTr5F30tV72hD9ysVxSTE4B2I4im+8xbQu4YRvMpBcY9KVvKaiiAh0+LS Jj/ToTitXIK9RSCcpONnyLrCUh/l3nYj+42f4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=p26kHWUKTl4Rq3nLs4+3ybMDvq1PKoAcvTkuoG/OXevFNRRxksK0NCR6CmuypCgcI4 2Z1QyRK8bAxZQ2EjNYP284p7kJceZIYybRXaQC1CBoIEycPzm7mS/PYbxCnNsrPMcSgn eFLWLFlPP8mVyrAVU72jxV7pkU/7xXOSgPvcY= MIME-Version: 1.0 Received: by 10.100.141.15 with SMTP id o15mr8141531and.20.1245676508090; Mon, 22 Jun 2009 06:15:08 -0700 (PDT) In-Reply-To: <289445.67836.qm@web37308.mail.mud.yahoo.com> References: <570433.20373.qm@web37308.mail.mud.yahoo.com> <289445.67836.qm@web37308.mail.mud.yahoo.com> Date: Mon, 22 Jun 2009 16:15:08 +0300 Message-ID: From: Dan Naumov To: Simun Mikecin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: ufs2 / softupdates / ZFS / disk write cache X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jun 2009 13:21:48 -0000 On Mon, Jun 22, 2009 at 3:19 PM, Simun Mikecin wrote: > 2) I wouldn't use UFS+SU on [S]ATA disks because your background fsck will > simetimes give up stating something like "unexpected softupdates > inconsistency" (unless you had disabled write cache, which you don't really > want to) and you will have to do a manual foreground fsck yourself. > The choice should (in my opinion) be: ZFS for amd64, UFS+gjournal for i386. > Both (ZFS and UFS+gjournal) will not have any recovery penalty if you have > write cache enabled. You seem to be thinking the way I am thinking :) My biggest concern is for the new users coming to FreeBSD, most of whom are going to end up with having UFS2+SU on their SATA disks, which is not the best defaults to have. I wonder if people running away screaming from sysinstall code has anything to do with why gjournal and zfs are not suggested as an option during the system installation procedure. Is the general consensus that adding these options to the install will only happen if/when FreeBSD moves on to a new installer system? Out of curiosity, how many beers would folks have to chime in for somebody knowledgeable enough to implement direct support for gjournal and zfs into sysinstall? :) > If you have a controller with battery backup cache, you could even run ZFS > with disabled cache flush (i don't know wheter it can be disabled on > gjournal), but I'm not sure that you will get any real word performance > improvement by doing it. In their own documentation, SUN recommends against disabling cache flushing in most cases, as the performance gain is going to be very negligible. However, there is a special case with "smart" SAN devices, where SUN strongly recommends disabling cache flushes because otherwise you are going to suffer through serious performance losses. - Sincerely, Dan Naumov From owner-freebsd-fs@FreeBSD.ORG Mon Jun 22 14:10:04 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 25AA61065670 for ; Mon, 22 Jun 2009 14:10:04 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 14E5D8FC08 for ; Mon, 22 Jun 2009 14:10:04 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n5MEA3Ft061415 for ; Mon, 22 Jun 2009 14:10:03 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n5MEA3uQ061414; Mon, 22 Jun 2009 14:10:03 GMT (envelope-from gnats) Date: Mon, 22 Jun 2009 14:10:03 GMT Message-Id: <200906221410.n5MEA3uQ061414@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Gavin Atkinson Cc: Subject: Re: kern/135412: ZFS issue X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Gavin Atkinson List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jun 2009 14:10:04 -0000 The following reply was made to PR kern/135412; it has been noted by GNATS. From: Gavin Atkinson To: bug-followup@FreeBSD.org Cc: Danny Braniss Subject: Re: kern/135412: ZFS issue Date: Mon, 22 Jun 2009 14:30:27 +0100 Other useful information (from PR kern/135039): This appears to only affect 7-STABLE since the ZFS merge, but doesn't affect -HEAD. After the recent import of ZFS v13 into 7-STABLE, an mkstemp() call from an NFS client to a ZFS-backed NFS server will fail: the syscall returns EIO and the server will have created a 0-byte file with 000 permissions. This breaks not just mktemp but also mv, tar, rsync... Kip Macy said there's a flags check that is too strict, in email message <3c1674c90905280025i17039257l573838d33d8493fd@mail.gmail.com> Otherwise, use cp and rm instead of mv, or use scp instead of NFS, or use UFS2 on the server To submitter: did you upgrade your on-disk pools to v13, or is running the new code and v6 pools enough to show the problem? What is the output of "zpool upgrade"? From owner-freebsd-fs@FreeBSD.ORG Mon Jun 22 15:20:04 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B8BAF106568A for ; Mon, 22 Jun 2009 15:20:04 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id A71FA8FC21 for ; Mon, 22 Jun 2009 15:20:04 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n5MFK4Cu014925 for ; Mon, 22 Jun 2009 15:20:04 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n5MFK4Xr014924; Mon, 22 Jun 2009 15:20:04 GMT (envelope-from gnats) Date: Mon, 22 Jun 2009 15:20:04 GMT Message-Id: <200906221520.n5MFK4Xr014924@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Danny Braniss Cc: Subject: Re: kern/135412: ZFS issue X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Danny Braniss List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jun 2009 15:20:05 -0000 The following reply was made to PR kern/135412; it has been noted by GNATS. From: Danny Braniss To: Gavin Atkinson Cc: bug-followup@FreeBSD.org Subject: Re: kern/135412: ZFS issue Date: Mon, 22 Jun 2009 17:53:44 +0300 > Other useful information (from PR kern/135039): > > This appears to only affect 7-STABLE since the ZFS merge, but doesn't > affect -HEAD. > > After the recent import of ZFS v13 into 7-STABLE, an mkstemp() call from > an NFS client to a ZFS-backed NFS server will fail: the syscall returns > EIO and the server will have created a 0-byte file with 000 permissions. > This breaks not just mktemp but also mv, tar, rsync... > > Kip Macy said there's a flags check that is too strict, in email message > <3c1674c90905280025i17039257l573838d33d8493fd@mail.gmail.com> > Otherwise, use cp and rm instead of mv, or use scp instead of NFS, or > use UFS2 on the server > > To submitter: did you upgrade your on-disk pools to v13, or is running > the new code and v6 pools enough to show the problem? What is the > output of "zpool upgrade"? it happens ONLY because zfs is v13, irrelevant if the pools have been upgraded, created, or not upgraded. so yes, just running a newer -stable is enough to show the problem. Since I am using ZFS + NFS, and providing service to several hundred users, telling them not to use tar, svn, rsync, etc is not an option :-) also, NFS V2 is broken/un-maintained. danny > From owner-freebsd-fs@FreeBSD.ORG Mon Jun 22 16:47:18 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 652511065673 for ; Mon, 22 Jun 2009 16:47:18 +0000 (UTC) (envelope-from kabaev@gmail.com) Received: from mail-qy0-f173.google.com (mail-qy0-f173.google.com [209.85.221.173]) by mx1.freebsd.org (Postfix) with ESMTP id 15F038FC12 for ; Mon, 22 Jun 2009 16:47:17 +0000 (UTC) (envelope-from kabaev@gmail.com) Received: by qyk3 with SMTP id 3so3928766qyk.3 for ; Mon, 22 Jun 2009 09:47:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:date:from:to:cc:subject :message-id:in-reply-to:references:x-mailer:mime-version :content-type; bh=jgS8c23UyxYjskwMfbKUVgXovhT8TO7yRliSgzqoaAg=; b=Ozr6OvzED7MyGxdbVHB6ivDAI3Aww6p+pHPqc/Q3IpdizCzMeuVEorXMvM96HuKDN6 /bfzSpf7IjP7jeUggoIOEy8hyRRKE1qWD0TfqfOdShpY3P7HYd6SnNmCMCVR/wTwvj/6 7+KZVXMNalxhsozBeq19PdmnCO7qt624YQiW8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:in-reply-to:references:x-mailer :mime-version:content-type; b=Y2GX+lj8/qfxNhg3PpRnp7yNMoAZbEee1EIGxL9Paa7Q8ygOh7p2qu1DjyiagLoWAe VfrbcLZPHfevoKYuNhqRPB0tWpdq/mz3bGoOFXI1ph5MsM72eNXpvF5fpNtVvlqUv7QB Lc+uCTB04PGUZn4v2ORDkIVCrvlaykZlGBN/A= Received: by 10.224.60.138 with SMTP id p10mr5105749qah.235.1245687352675; Mon, 22 Jun 2009 09:15:52 -0700 (PDT) Received: from kan.dnsalias.net (c-98-217-224-113.hsd1.ma.comcast.net [98.217.224.113]) by mx.google.com with ESMTPS id 26sm202067qwa.4.2009.06.22.09.15.50 (version=SSLv3 cipher=RC4-MD5); Mon, 22 Jun 2009 09:15:51 -0700 (PDT) Date: Mon, 22 Jun 2009 12:15:43 -0400 From: Alexander Kabaev To: Danny Braniss Message-ID: <20090622121543.7bc4204b@kan.dnsalias.net> In-Reply-To: <200906221520.n5MFK4Xr014924@freefall.freebsd.org> References: <200906221520.n5MFK4Xr014924@freefall.freebsd.org> X-Mailer: Claws Mail 3.7.1 (GTK+ 2.16.2; i386-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_//aJlV1b5ykcff53dcgmmpKR"; protocol="application/pgp-signature" Cc: freebsd-fs@FreeBSD.org Subject: Re: kern/135412: ZFS issue X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jun 2009 16:47:18 -0000 --Sig_//aJlV1b5ykcff53dcgmmpKR Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Mon, 22 Jun 2009 15:20:04 GMT Danny Braniss wrote: > also, NFS V2 is broken/un-maintained. Hi Danny, could you please tell me what to look for and I will take a stab at reproducing/fixing it. --=20 Alexander Kabaev --Sig_//aJlV1b5ykcff53dcgmmpKR Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (FreeBSD) iD8DBQFKP641Q6z1jMm+XZYRAhCwAJ903dY6uQk1+fxJJdRhrlvZTpZ/TQCgtqb5 8T1V+mVELvf+GwyLJ7ht3wI= =9IWB -----END PGP SIGNATURE----- --Sig_//aJlV1b5ykcff53dcgmmpKR-- From owner-freebsd-fs@FreeBSD.ORG Mon Jun 22 17:41:23 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A0C2D1065670 for ; Mon, 22 Jun 2009 17:41:23 +0000 (UTC) (envelope-from update+2soyojoy@facebookmail.com) Received: from mx-out.facebook.com (outcampmail001.snc1.tfbnw.net [69.63.178.186]) by mx1.freebsd.org (Postfix) with ESMTP id 74F1E8FC18 for ; Mon, 22 Jun 2009 17:41:23 +0000 (UTC) (envelope-from update+2soyojoy@facebookmail.com) DKIM-Signature: v=1; a=rsa-sha1; d=facebookmail.com; s=q1-2009b; c=relaxed/relaxed; q=dns/txt; i=@facebookmail.com; t=1245686995; h=From:Subject:Date:To:MIME-Version:Content-Type; bh=DTemPV/b71JgR+QFpzAD5d/jH5s=; b=FuJ+wyjUV5UdSgNqFndUKJUtopRa0FlqwDDKjAWwGlEVDeRNnHyl2BQcNtUJIife lc5suoAiWvAx/r9AWYLTbg==; Received: from [10.18.255.175] ([10.18.255.175:31301] helo=localhost.localdomain) by mta007.snc1.facebook.com (envelope-from ) (ecelerity 2.2.2.37 r(28805/28844)) with ESMTP id D0/F0-26877-3DCAF3A4; Mon, 22 Jun 2009 09:09:55 -0700 X-Facebook: from zuckmail ([]) by localhost.localdomain with local (ZuckMail); Date: Mon, 22 Jun 2009 09:09:55 -0700 To: Freebsd-fs From: Facebook Message-ID: <5e72a609e648a6086bf07ff5177e7e12@localhost.localdomain> X-Priority: 3 X-Mailer: ZuckMail [version 1.00] X-Facebook-Camp: reminder_email X-Facebook-Notify: reminder_email; mailid=a9aa01G66ef269cG81e63G46 Errors-To: update+2soyojoy@facebookmail.com X-FACEBOOK-PRIORITY: 1 MIME-Version: 1.0 Content-Type: text/plain; charset = "UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Reminder: Martin invited you to join Facebook... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Facebook List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jun 2009 17:41:23 -0000 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D To sign up for Facebook, follow the link below: http://www.facebook.com/r.php?re=3Ddd6bdc1df057a0e4e623e1a3655eb135&mid=3Da9aa01G66ef269cG81e63G46 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Hi Freebsd-fs, The following person recently invited you to be their friend on Facebook: Martin Hepworth Other people you may know on Facebook: George Horton (Los Angeles, CA) Steve Pirnie Phil Wagner Don Wershba (New York, NY) Nick Fragile (West Midlands) Jonathan Scott Headland Facebook is a great place to keep in touch with friends, post photos, videos and create events. But first you need to join! Sign up today to create a profile and connect with the people you know. Thanks, The Facebook Team To sign up for Facebook, follow the link below: http://www.facebook.com/r.php?re=3Ddd6bdc1df057a0e4e623e1a3655eb135&mid=3Da9aa01G66ef269cG81e63G46 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D This message was intended for freebsd-fs@freebsd.org. If you do not wish to receive this type of email from Facebook in the future, please click on the link below to unsubscribe. http://www.facebook.com/o.php?c&k=3D1e3cd0&u=3D1726949020&mid=3Da9aa01G66ef269cG81e63G46 Facebook's offices are located at 1601 S. California Ave., Palo Alto, CA 94304. From owner-freebsd-fs@FreeBSD.ORG Mon Jun 22 20:52:21 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 703BB106566C; Mon, 22 Jun 2009 20:52:21 +0000 (UTC) (envelope-from remko@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 45D018FC0A; Mon, 22 Jun 2009 20:52:21 +0000 (UTC) (envelope-from remko@FreeBSD.org) Received: from freefall.freebsd.org (remko@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n5MKqLVL078864; Mon, 22 Jun 2009 20:52:21 GMT (envelope-from remko@freefall.freebsd.org) Received: (from remko@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n5MKqLD6078860; Mon, 22 Jun 2009 20:52:21 GMT (envelope-from remko) Date: Mon, 22 Jun 2009 20:52:21 GMT Message-Id: <200906222052.n5MKqLD6078860@freefall.freebsd.org> To: mikej@paymentallianceintl.com, remko@FreeBSD.org, freebsd-fs@FreeBSD.org From: remko@FreeBSD.org Cc: Subject: Re: bin/135710: mount(8): mount -t tmpfs does not follow 'size' option X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jun 2009 20:52:21 -0000 Synopsis: mount(8): mount -t tmpfs does not follow 'size' option State-Changed-From-To: open->closed State-Changed-By: remko State-Changed-When: Mon Jun 22 20:52:20 UTC 2009 State-Changed-Why: Nathanael gave a pretty good explaination which I follow; the behaviour is intended. http://www.freebsd.org/cgi/query-pr.cgi?pr=135710 From owner-freebsd-fs@FreeBSD.ORG Mon Jun 22 23:35:24 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B8EB4106564A; Mon, 22 Jun 2009 23:35:24 +0000 (UTC) (envelope-from mikej@paymentallianceintl.com) Received: from mx2.confluenttech.com (mx2.confluentasp.com [216.26.153.14]) by mx1.freebsd.org (Postfix) with ESMTP id 4125D8FC13; Mon, 22 Jun 2009 23:35:20 +0000 (UTC) (envelope-from mikej@paymentallianceintl.com) Received: from calvin.pai.local (calvin.pai.local [10.0.6.33]) by mx2.confluenttech.com (8.14.1/8.13.8) with ESMTP id n5MLNn7E072620; Mon, 22 Jun 2009 17:23:54 -0400 (EDT) (envelope-from mikej@paymentallianceintl.com) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.4325 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Date: Mon, 22 Jun 2009 17:23:50 -0400 Importance: normal Priority: normal Message-ID: In-Reply-To: <200906222052.n5MKqLD6078860@freefall.freebsd.org> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: bin/135710: mount(8): mount -t tmpfs does not follow 'size' option Thread-Index: Acnze2EXxA2DL8cdT9Sr35HOV1HqVgABCqsg References: <200906222052.n5MKqLD6078860@freefall.freebsd.org> From: "Michael Jung" To: , Cc: Subject: RE: bin/135710: mount(8): mount -t tmpfs does not follow 'size' option X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jun 2009 23:35:25 -0000 Thank you both for your kind responses. I've been reading man pages since 2.2 and my brain just didn't get what my eyes read. --mikej Michael Jung Payment Alliance International 11857 Commonwealth Drive Louisville, KY 40299 502-212-4045 Work Voice 502-212-4004 Work Facsimile -----Original Message----- From: remko@FreeBSD.org [mailto:remko@FreeBSD.org] Sent: Monday, June 22, 2009 4:52 PM To: Michael Jung; remko@FreeBSD.org; freebsd-fs@FreeBSD.org Subject: Re: bin/135710: mount(8): mount -t tmpfs does not follow 'size' option Synopsis: mount(8): mount -t tmpfs does not follow 'size' option State-Changed-From-To: open->closed State-Changed-By: remko State-Changed-When: Mon Jun 22 20:52:20 UTC 2009 State-Changed-Why: Nathanael gave a pretty good explaination which I follow; the behaviour is intended. http://www.freebsd.org/cgi/query-pr.cgi?pr=135710 CONFIDENTIALITY NOTE: This message is intended only for the use of the individual or entity to whom it is addressed and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, please notify us by telephone at (502) 212-4001 or notify us at PAI , Dept. 99, 11857 Commonwealth Drive, Louisville, KY 40299. Thank you. From owner-freebsd-fs@FreeBSD.ORG Tue Jun 23 07:11:53 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 528581065670 for ; Tue, 23 Jun 2009 07:11:53 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.16.84]) by mx1.freebsd.org (Postfix) with ESMTP id 04FDD8FC16 for ; Tue, 23 Jun 2009 07:11:52 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from pampa.cs.huji.ac.il ([132.65.80.32]) by kabab.cs.huji.ac.il with esmtp id 1MIz1W-00028f-Hh; Tue, 23 Jun 2009 08:58:26 +0300 X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.2 To: Alexander Kabaev In-reply-to: <20090622121543.7bc4204b@kan.dnsalias.net> References: <200906221520.n5MFK4Xr014924@freefall.freebsd.org> <20090622121543.7bc4204b@kan.dnsalias.net> Comments: In-reply-to Alexander Kabaev message dated "Mon, 22 Jun 2009 12:15:43 -0400." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 23 Jun 2009 08:58:26 +0300 From: Danny Braniss Message-ID: Cc: freebsd-fs@FreeBSD.org Subject: Re: kern/135412: ZFS issue X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jun 2009 07:11:53 -0000 > On Mon, 22 Jun 2009 15:20:04 GMT > Danny Braniss wrote: > > > also, NFS V2 is broken/un-maintained. > > Hi Danny, > > could you please tell me what to look for and I will take a stab at > reproducing/fixing it. Hi Alexander, removing a file from a read-only NFS/V2 will hang the client. The main reason that V2 is still around, IMHO, is that diskless boot relies on it, so fixing boot to use NFS/v3 would be more productive. Cheers, danny From owner-freebsd-fs@FreeBSD.ORG Tue Jun 23 10:01:54 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A26091065679 for ; Tue, 23 Jun 2009 10:01:54 +0000 (UTC) (envelope-from numisemis@yahoo.com) Received: from web37303.mail.mud.yahoo.com (web37303.mail.mud.yahoo.com [209.191.90.246]) by mx1.freebsd.org (Postfix) with SMTP id 4994F8FC13 for ; Tue, 23 Jun 2009 10:01:53 +0000 (UTC) (envelope-from numisemis@yahoo.com) Received: (qmail 39047 invoked by uid 60001); 23 Jun 2009 10:01:53 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1245751313; bh=EJwEOrpymaLzFjro0G1IuWfXnayKtVxMrFHzcwYdVXM=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type; b=R7wOyMqriGci/0AqIeYwfctCvfCioKgdzW89FWH3/T+W9efboOCnOMzoI4RINkNwXrASxCPqTkvCjZiLOL8OSRxBsewS+VGimQtic3L2lKRl2SdMLmDsfQU5wNe+yIVUa0g15/8u1F1sknaYVcZgM8mEnCTTqqbPA4TV2z/AsvE= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type; b=WS89mG64T7NquBZ8Z9B6s9RejS1s5+v+8SH4dAzzHSlaiRo4bfIpcA81tN7AkmDn8750ixLpfkvbJTjgajXFq5ApOOFX8LzUUGZB1PMUn4T3UjO+1MLOuDu3p3qza79N3IxHDyed/srB2eFn8NT1JWs+4O06Nzcx1jWMxx+SFAU=; Message-ID: <288239.1376.qm@web37303.mail.mud.yahoo.com> X-YMail-OSG: J1bIEokVM1m58nPb2hRSoV8aFBYP9AGAtUBjtr2t.qTMNAhCZTysdM75iEcQcz5sqHcerWstJHuMwu5XG3IyyvfOVyvvJ9jf77O0G6Fp3tgJQM6t4tEh.pRT5mgCLmCgCKeG6Z5_MplXMsWQjCVQRYGcl89lv72qN0fYqhImIcqFH3ywgJH014StqEYz.BqS.DnmmK6r8N8k3TXLjjql_dGKPglcnzB4H5pck2A3kH95Vai.x1x1Sr_gbuPEN0DVQo0XyD7dvEGfYj75J245ZxfN9bZVeqYsmw6SR.BsN0t8GZzLiLIIFCFCjSDFgf6Y_AREsOSA Received: from [213.147.110.159] by web37303.mail.mud.yahoo.com via HTTP; Tue, 23 Jun 2009 03:01:53 PDT X-Mailer: YahooMailRC/1277.43 YahooMailWebService/0.7.289.15 Date: Tue, 23 Jun 2009 03:01:53 -0700 (PDT) From: Simun Mikecin To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: 7.2-STABLE: swap on ZFS v13 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jun 2009 10:01:54 -0000 Hi! Before ZFS v13 import to the 7-STABLE branch, it was a well known fact that swap on ZFS volume (zvol) will not work. Recently, I tried to use swap on ZFS v13 (7-STABLE/amd64). It seems that it works. I used 4k for the volblocksize as that is equivalent to hardware page size. I tried with compression enabled and disabled. Enabling compression reduces number of bytes that need to be swapped in and out to the zvol. zfs create -o volblocksize=4k -o compression=on -o org.freebsd:swap=on -V 4g tank/swap swapon /dev/zvol/tank/swap Is there anybody else who tried 7-STABLE/amd64 with swap on ZFS v13? Does it work or not? From owner-freebsd-fs@FreeBSD.ORG Tue Jun 23 10:10:47 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2D56C106566C for ; Tue, 23 Jun 2009 10:10:47 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.249]) by mx1.freebsd.org (Postfix) with ESMTP id DE3328FC17 for ; Tue, 23 Jun 2009 10:10:46 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: by an-out-0708.google.com with SMTP id b38so24681ana.13 for ; Tue, 23 Jun 2009 03:10:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=EGZFfhp5c3NxJL1dO2drSYyclOTLsD1sXaS0oMMBfr4=; b=Z1jj9Vj8ZJvu7pMxaYuR+j/1XUuKe/Ou6lBnK7CIXnkel88DkxUETFFxiyrhIqhvpq L/5nrI1pydnBI1qYC+iggSXDpaqwg8jLTVNd+YJ9Xsccel6N/y3kOHL2kEiR8oUAdj6c 0s3CQVP9ogTMg71U+dRwLM8yrmtbMafKXwbjQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=fT5o++vPZiB8Gk+KqKBqSM42NxcL/jCWk0qnVEmAuofZvlibpaqOwOUNccBPRW/Du4 oGy4WG8BdDOClnfdtLZ9Ah/Y/CmBYGmuM5+bNao8z4Rmsg5l5knj+nzDgmB7yI2fizSs GFhyxuU7GaEUximR3JSDkcdvbSAohTZiFVh3s= MIME-Version: 1.0 Received: by 10.100.42.14 with SMTP id p14mr9591133anp.199.1245751846191; Tue, 23 Jun 2009 03:10:46 -0700 (PDT) In-Reply-To: <288239.1376.qm@web37303.mail.mud.yahoo.com> References: <288239.1376.qm@web37303.mail.mud.yahoo.com> Date: Tue, 23 Jun 2009 13:10:46 +0300 Message-ID: From: Dan Naumov To: Simun Mikecin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: 7.2-STABLE: swap on ZFS v13 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jun 2009 10:10:47 -0000 Pardon my perhaps stupid question, but what exactly does the "-o org.freebsd:swap=on" option do? Couldn't find anything related during a quick glance at the zfs manpage (on a 7.2-release/amd64 system with zfsv6). - Sincerely, Dan Naumov On Tue, Jun 23, 2009 at 1:01 PM, Simun Mikecin wrote: > > Hi! > > Before ZFS v13 import to the 7-STABLE branch, it was a well known fact that swap on ZFS volume (zvol) will not work. > Recently, I tried to use swap on ZFS v13 (7-STABLE/amd64). It seems that it works. > > I used 4k for the volblocksize as that is equivalent to hardware page size. > I tried with compression enabled and disabled. Enabling compression reduces number of bytes that need to be swapped in and out to the zvol. > > zfs create -o volblocksize=4k -o compression=on -o org.freebsd:swap=on -V 4g tank/swap > swapon /dev/zvol/tank/swap > > Is there anybody else who tried 7-STABLE/amd64 with swap on ZFS v13? Does it work or not? > > > > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Tue Jun 23 10:19:38 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4006D106566C for ; Tue, 23 Jun 2009 10:19:38 +0000 (UTC) (envelope-from numisemis@yahoo.com) Received: from web37306.mail.mud.yahoo.com (web37306.mail.mud.yahoo.com [209.191.90.249]) by mx1.freebsd.org (Postfix) with SMTP id D612D8FC1B for ; Tue, 23 Jun 2009 10:19:37 +0000 (UTC) (envelope-from numisemis@yahoo.com) Received: (qmail 77456 invoked by uid 60001); 23 Jun 2009 10:19:37 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1245752377; bh=BvWrylY3ZXqF1j0c/Z+TRyFYiP/S2VoXbQsFYvMSokc=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=ylVlLeNzkd4OINUH3JGBJ8S7v6/Bo5jkw54++J/ENtlXgSyqiX59QpTm6xkym83BqnFMV0pFc8AnajnyFbghZWSsTv0aFyIkYfhEsLQbGCJ3knP27qNHeYymmPZ/CdGlZKL56F3eo/Yd3HWGAQJVwlB8eDpdcwTSLp18daZvJvU= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=yA8hKxJnToI4r4n2GGqAq6jUleaGwxHO8KefPoVOmsIP5E19GId7HaT4i2zwbBvUC9Vylb1fuV0E8Lq8iDd4rV4Vm4ujlRZGoU4EQe8rdPRTiRwcoBaGOFMlDWSfTgX5DVf88IK7eu1p8cKUGtDSAEAmwszW9j1LWPUbbrY/O4A=; Message-ID: <173835.76209.qm@web37306.mail.mud.yahoo.com> X-YMail-OSG: hnoIlvcVM1mdmNLLOWjT_6JshJe1oHxlVadk4Zr2cdLX644pWfXmwhxLjNMbrnrmUyVNiT38QG8wJ44AN2KK8BHyK_IdUe5uVD3_ZeFOZNx8K3.EyzUfoIFfuetQ5h4wwV1ZJ.ahDYQ8hJ7pumOujFt45xkAjb7hJ.nTOi.rtBnXkQFKwP4CoEMrrC6NbRsQkI218v.WInm1xbxwHxNAv11cvG.QC7Qa7FjDa6Hxzm.l3PYxeqkGg9U8zaXwbvuVm52QEnEIN.YOHJ3AbBZh313XoL2maPnt7jZHWo8YDhP6r4eI_NUqPLzrSk1vF_t4d81fptmR Received: from [213.147.110.159] by web37306.mail.mud.yahoo.com via HTTP; Tue, 23 Jun 2009 03:19:37 PDT X-Mailer: YahooMailRC/1277.43 YahooMailWebService/0.7.289.15 References: <288239.1376.qm@web37303.mail.mud.yahoo.com> Date: Tue, 23 Jun 2009 03:19:37 -0700 (PDT) From: Simun Mikecin To: Dan Naumov In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-fs@freebsd.org Subject: Re: 7.2-STABLE: swap on ZFS v13 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jun 2009 10:19:38 -0000 Dan Naumov wrote: > Pardon my perhaps stupid question, but what exactly does the "-o > org.freebsd:swap=on" option do? Couldn't find anything related during > a quick glance at the zfs manpage (on a 7.2-release/amd64 system with > zfsv6). Take a look at /etc/rc.d/zfs. Since /etc/fstab is not a supported way for adding swap on zvols, /etc/rc.d/zfs script (executed during boot if you have zfs_enable="YES" in your /etc/rc.conf[.local]) will execute "swapon" on all zvols that have org.freebsd:swap=on. From owner-freebsd-fs@FreeBSD.ORG Tue Jun 23 10:34:38 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 47832106566C for ; Tue, 23 Jun 2009 10:34:38 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.246]) by mx1.freebsd.org (Postfix) with ESMTP id 0046D8FC08 for ; Tue, 23 Jun 2009 10:34:37 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: by an-out-0708.google.com with SMTP id b38so28220ana.13 for ; Tue, 23 Jun 2009 03:34:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=enLPeV4YQaCqBveqd34x2yNCNdz+EOOyZK59SYkmtr8=; b=X/5xPQRsrwITZMSjkhJ6T8+eQpVg5Rh+qfHq+07CxGc5aBCVvQJUnGnQr3l+gbaEuy k4ioYuuHXjEUjX0h01GlsCXIpY42nNbVJ8o0KsOkJadi+aQO+vnzZYA9mdPfF5ULuqdW G3RBBBSuxqXYPgUYSj6vVJo+POvEZzc6NUiJ8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=PuNn5Q/dQ5RId+gI3bAFrdyfJjpq/VcPqwnNA/Lm8A2nJsG0IgJmUwq47f12Kwlamj Nm4Zko21Gw1uhaA1XeUqglWmMo/xDS0t/+uwpCkcAtRVeh+ZSCx79VmnfJvfkwceJGrc WLdHD8mG+K18Q6o8sXjjocOSpuMSZrwuCXZsI= MIME-Version: 1.0 Received: by 10.100.141.17 with SMTP id o17mr8900462and.160.1245753277418; Tue, 23 Jun 2009 03:34:37 -0700 (PDT) In-Reply-To: <173835.76209.qm@web37306.mail.mud.yahoo.com> References: <288239.1376.qm@web37303.mail.mud.yahoo.com> <173835.76209.qm@web37306.mail.mud.yahoo.com> Date: Tue, 23 Jun 2009 13:34:37 +0300 Message-ID: From: Dan Naumov To: Simun Mikecin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: 7.2-STABLE: swap on ZFS v13 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jun 2009 10:34:38 -0000 On Tue, Jun 23, 2009 at 1:19 PM, Simun Mikecin wrote: > > Dan Naumov wrote: >> Pardon my perhaps stupid question, but what exactly does the "-o >> org.freebsd:swap=on" option do? Couldn't find anything related during >> a quick glance at the zfs manpage (on a 7.2-release/amd64 system with >> zfsv6). > Take a look at /etc/rc.d/zfs. > Since /etc/fstab is not a supported way for adding swap on zvols, /etc/rc.d/zfs script (executed during boot if you have zfs_enable="YES" in your /etc/rc.conf[.local]) will execute "swapon" on all zvols that have org.freebsd:swap=on. Oh, right. Kinda weird though that this is already included in /etc/rc.d/zfs already if "swap on ZFS" is considered even more experimental than ZFS support itself. Is this documented anywhere or is it just one of the things you learn by reading rc.d scripts? :) Out of curiosity, what kind of horrible breakage should I expect if I try enabling swap on a zvol using zfsv6 that comes with 7.2-release? - Dan Naumov From owner-freebsd-fs@FreeBSD.ORG Tue Jun 23 12:24:05 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DE2851065678 for ; Tue, 23 Jun 2009 12:24:05 +0000 (UTC) (envelope-from gary.jennejohn@freenet.de) Received: from mout1.freenet.de (mout1.freenet.de [IPv6:2001:748:100:40::2:3]) by mx1.freebsd.org (Postfix) with ESMTP id 72CC78FC0C for ; Tue, 23 Jun 2009 12:24:05 +0000 (UTC) (envelope-from gary.jennejohn@freenet.de) Received: from [195.4.92.21] (helo=11.mx.freenet.de) by mout1.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.69 #92) id 1MJ52i-0003df-1P; Tue, 23 Jun 2009 14:24:04 +0200 Received: from tc20b.t.pppool.de ([89.55.194.11]:30633 helo=ernst.jennejohn.org) by 11.mx.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.69 #79) id 1MJ52h-0007MR-Lp; Tue, 23 Jun 2009 14:24:04 +0200 Date: Tue, 23 Jun 2009 14:24:02 +0200 From: Gary Jennejohn To: Simun Mikecin Message-ID: <20090623142402.2c8d4b65@ernst.jennejohn.org> In-Reply-To: <288239.1376.qm@web37303.mail.mud.yahoo.com> References: <288239.1376.qm@web37303.mail.mud.yahoo.com> X-Mailer: Claws Mail 3.7.1 (GTK+ 2.16.2; amd64-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: 7.2-STABLE: swap on ZFS v13 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gary.jennejohn@freenet.de List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jun 2009 12:24:06 -0000 On Tue, 23 Jun 2009 03:01:53 -0700 (PDT) Simun Mikecin wrote: > Before ZFS v13 import to the 7-STABLE branch, it was a well known fact that swap on ZFS volume (zvol) will not work. > Recently, I tried to use swap on ZFS v13 (7-STABLE/amd64). It seems that it works. > > I used 4k for the volblocksize as that is equivalent to hardware page size. > I tried with compression enabled and disabled. Enabling compression reduces number of bytes that need to be swapped in and out to the zvol. > > zfs create -o volblocksize=4k -o compression=on -o org.freebsd:swap=on -V 4g tank/swap > swapon /dev/zvol/tank/swap > > Is there anybody else who tried 7-STABLE/amd64 with swap on ZFS v13? Does it work or not? > I doubt that you can generate a crash dump using ZFS as swap, especially if you have compression enabled. But I might be wrong. Does anyone know? --- Gary Jennejohn From owner-freebsd-fs@FreeBSD.ORG Tue Jun 23 12:50:06 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DC7B11065670 for ; Tue, 23 Jun 2009 12:50:06 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.250]) by mx1.freebsd.org (Postfix) with ESMTP id 954748FC0C for ; Tue, 23 Jun 2009 12:50:06 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: by an-out-0708.google.com with SMTP id b38so15932ana.13 for ; Tue, 23 Jun 2009 05:50:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=Uk83IGo4dbyy4I9+0+ZwELF3sqH0HUfIOD0rCI1TJeM=; b=iUs3O/76b295yZz7Ov2p8ETHbUM5WQ/n9OsciICJZhExYVsMU9lBx6PmsZmdaDLf1D 5ioSpgScslaVofjZ4uWIE1wpBC4xYSTT4gjvTIx+1gXLOY8CF0nqvi3Idw8xTCSX60X2 p3XKl7GYgyyOoSc8gRCVKOMTsKsq253Wn+6uM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=RwoA5didnrqiaZubmfWoR41UKpnZYmp9MmIspbtkTKeEbQcMFK0DN9BPI11R5jOv25 n7CIq8B2BidCmZh1uZ+Qfv04l69EMgxpISXSQJU1HX6yzurdvxyaOGeGG6PfmhwlbTjL bcOt6ayt59NaC28b6vfKqFepGWMc9C2M0oAcM= MIME-Version: 1.0 Received: by 10.100.46.3 with SMTP id t3mr42465ant.94.1245761405893; Tue, 23 Jun 2009 05:50:05 -0700 (PDT) In-Reply-To: <20090623142402.2c8d4b65@ernst.jennejohn.org> References: <288239.1376.qm@web37303.mail.mud.yahoo.com> <20090623142402.2c8d4b65@ernst.jennejohn.org> Date: Tue, 23 Jun 2009 15:50:05 +0300 Message-ID: From: Dan Naumov To: gary.jennejohn@freenet.de Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: 7.2-STABLE: swap on ZFS v13 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jun 2009 12:50:07 -0000 On Tue, Jun 23, 2009 at 3:24 PM, Gary Jennejohn wrote: >> Is there anybody else who tried 7-STABLE/amd64 with swap on ZFS v13? Does it work or not? >> > > I doubt that you can generate a crash dump using ZFS as swap, especially > if you have compression enabled. > > But I might be wrong. > > Does anyone know? If this is true and this is the only serious limitation of using a zvol as swap, I could very well see myself moving to using it for swap in the near future (as it would lead to a lot cleaner filesystem layouts on multidisk systems utilizing zfs). If by any odd chance, I do run into a reproducible system crash I have to debug, I could just attach an USB stick and temporarily move the swap to that. - Dan Naumov From owner-freebsd-fs@FreeBSD.ORG Tue Jun 23 18:22:58 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F0E0C1065679 for ; Tue, 23 Jun 2009 18:22:57 +0000 (UTC) (envelope-from mandrews@bit0.com) Received: from magnum.bit0.com (magnum.bit0.com [207.246.88.226]) by mx1.freebsd.org (Postfix) with ESMTP id C123B8FC18 for ; Tue, 23 Jun 2009 18:22:57 +0000 (UTC) (envelope-from mandrews@bit0.com) Received: from localhost (localhost [127.0.0.1]) by magnum.bit0.com (Postfix) with ESMTP id B9A88A9A6 for ; Tue, 23 Jun 2009 14:07:17 -0400 (EDT) X-Virus-Scanned: amavisd-new at bit0.com Received: from magnum.bit0.com ([127.0.0.1]) by localhost (magnum.int.bit0.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5-fNJF6GoIFc for ; Tue, 23 Jun 2009 14:07:12 -0400 (EDT) Received: from beast.int.bit0.com (beast.int.bit0.com [172.27.0.2]) by magnum.bit0.com (Postfix) with ESMTP for ; Tue, 23 Jun 2009 14:07:12 -0400 (EDT) Date: Tue, 23 Jun 2009 14:07:11 -0400 (EDT) From: Mike Andrews X-X-Sender: mandrews@beast.int.bit0.com To: fs@freebsd.org In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Subject: Re: weird problem w/ ZFS not reclaiming freed space X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jun 2009 18:22:58 -0000 On Fri, 19 Jun 2009, Mike Andrews wrote: > Somehow I've managed to get ZFS on one of my machines into a state where it > won't reclaim all space after deleting files AND snapshots off of it: > (this is with 7.2-STABLE amd64, compiled June 10) > > # ls -la /weird > total 4 > drwxr-x--- 2 mysql mysql 2 Jun 19 02:42 . > drwxr-xr-x 29 root wheel 1024 Jun 19 02:44 .. > > # df /weird > Filesystem 1K-blocks Used Avail Capacity Mounted on > scotch/weird 282201472 109151232 173050240 39% /weird > > # zfs list scotch/weird > NAME USED AVAIL REFER MOUNTPOINT > scotch/weird 104G 164G 104G /weird > > # zfs list -t snapshot | grep scotch/weird > > # zfs get all scotch/weird > NAME PROPERTY VALUE SOURCE > scotch/weird type filesystem - > scotch/weird creation Wed Jun 17 1:20 2009 - > scotch/weird used 104G - > scotch/weird available 159G - > scotch/weird referenced 104G - > scotch/weird compressratio 1.00x - > scotch/weird mounted yes - > scotch/weird quota none default > scotch/weird reservation none default > scotch/weird recordsize 128K default > scotch/weird mountpoint /weird local > scotch/weird sharenfs off default > scotch/weird checksum on default > scotch/weird compression off default > scotch/weird atime off local > scotch/weird devices on default > scotch/weird exec off local > scotch/weird setuid off local > scotch/weird readonly off default > scotch/weird jailed off default > scotch/weird snapdir hidden default > scotch/weird aclmode groupmask default > scotch/weird aclinherit restricted default > scotch/weird canmount on default > scotch/weird shareiscsi off default > scotch/weird xattr off temporary > scotch/weird copies 1 default > scotch/weird version 3 - > scotch/weird utf8only off - > scotch/weird normalization none - > scotch/weird casesensitivity sensitive - > scotch/weird vscan off default > scotch/weird nbmand off default > scotch/weird sharesmb off default > scotch/weird refquota none default > scotch/weird refreservation none default > scotch/weird primarycache all default > scotch/weird secondarycache all default > scotch/weird usedbysnapshots 0 - > scotch/weird usedbydataset 104G - > scotch/weird usedbychildren 0 - > scotch/weird usedbyrefreservation 0 - > > > If I then rsync stuff to it, space seems OK, if I continue to rsync to it > every few hours, the used space grows, even if no snapshots are being taken > If I do take snapshots, then change stuff, then delete the snapshots, the > snapshot space does appear to be reclaimed. Also if I 'zfs destroy' the > filesystem, the space is correctly reclaimed, but once I create a new one > and repeat the process, the problem reappears. > > I have not had any luck reproducing this on another machine yet, but > admittedly haven't tried super hard yet. > > Scrubbing the zpool returns no errors. > > I'm guessing zdb is my only hope at debugging this, but as I've never used it > before and as it seems to dump core whenever I try running it, can someone > suggest what I need to check/look for in it? > > I did also have a panic a few days ago that, based on the text, might be > related (I do have the vmdump and core.txt) > > panic: solaris assert: P2PHASE(start, 1ULL << sm->sm_shift) == 0, file: > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/space_map.c, > line: 146 > > ...for which I have a vmdump and a core.txt if anyone wants to look at it. Just to update this and move it to the -fs mailing list, removing the "--sparse" flag from rsync solves this problem, so the bug has something to do with ZFS's handling of sparse files. From owner-freebsd-fs@FreeBSD.ORG Wed Jun 24 03:56:01 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DDAE3106566C for ; Wed, 24 Jun 2009 03:56:01 +0000 (UTC) (envelope-from mat.macy@gmail.com) Received: from yw-out-2324.google.com (yw-out-2324.google.com [74.125.46.29]) by mx1.freebsd.org (Postfix) with ESMTP id 9252E8FC14 for ; Wed, 24 Jun 2009 03:56:01 +0000 (UTC) (envelope-from mat.macy@gmail.com) Received: by yw-out-2324.google.com with SMTP id 9so26064ywe.13 for ; Tue, 23 Jun 2009 20:56:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=EGwLVld2bQR9mVzh21gIMRpwXHkpZyzYLThIvWFEmRI=; b=KoC2bvULZUD+5rkH8r8kMf+ThcbWVpgqAhKitqut9QMpLOy02hngxrkwTv4hU6Qbn8 yGzl1/Pzip7WaoXxLz0SwEcbXNywq0xh9il4TLBU2Ct363fjrflhGlHnRn2E29dSeL7A +4hHRfYlTWsKCa3zwnFMYQVxzngzYisJN0eh8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=dYH3VqkiXwfVts0O0mnJX6mjhXF6msZHbfZXLG3ZeReZJvDGCeFkFPprNo5IwJvLUd 2oicOH9GbZoEJU+PkM4sm+Z2zYs3xwj6itH43cFP/ZkmX8Z+HpdXF5HGYIkCfCIolbPh yNWaBGYUH+P9aKEfOqbUUO5cEeviCxYvkgphs= MIME-Version: 1.0 Sender: mat.macy@gmail.com Received: by 10.100.166.7 with SMTP id o7mr1187067ane.90.1245815760759; Tue, 23 Jun 2009 20:56:00 -0700 (PDT) In-Reply-To: <20090624014855.GB79749@logik.internal.network> References: <20090623003117.GA94466@logik.internal.network> <20090623050707.GB21349@logik.internal.network> <20090624004726.GA25475@blazingdot.com> <20090624014855.GB79749@logik.internal.network> Date: Tue, 23 Jun 2009 20:56:00 -0700 X-Google-Sender-Auth: d05224ab0629baf5 Message-ID: <3c1674c90906232056s2040d6d4u59550b6e5d12a4d0@mail.gmail.com> From: Kip Macy To: xorquewasp@googlemail.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: ZFS filesystem not showing total size? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jun 2009 03:56:02 -0000 freebsd-fs is more appropriate - ideally freebsd-questions but I'm not sure how much your mileage will vary there. On Tue, Jun 23, 2009 at 6:48 PM, wrote: > On 2009-06-23 17:47:26, Marcus Reid wrote: >> Hi, >> >> As a side note, freebsd-hackers is not the correct list for this question. > > I considered it a hackers@ question due to the large "ZFS is an experimental > feature" warning. > > xw > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > -- When bad men combine, the good must associate; else they will fall one by one, an unpitied sacrifice in a contemptible struggle. Edmund Burke From owner-freebsd-fs@FreeBSD.ORG Wed Jun 24 08:42:36 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 31AB31065670 for ; Wed, 24 Jun 2009 08:42:36 +0000 (UTC) (envelope-from rihad@mail.ru) Received: from mx45.mail.ru (mx73.mail.ru [94.100.176.87]) by mx1.freebsd.org (Postfix) with ESMTP id E1EF68FC28 for ; Wed, 24 Jun 2009 08:42:35 +0000 (UTC) (envelope-from rihad@mail.ru) Received: from mx71.mail.ru (mx71.mail.ru [94.100.176.85]) by mx45.mail.ru (mPOP.Fallback_MX) with ESMTP id 758D23A4E0 for ; Wed, 24 Jun 2009 12:14:50 +0400 (MSD) Received: from [217.25.27.27] (port=40114 helo=[217.25.27.27]) by mx71.mail.ru with asmtp id 1MJNd2-000P04-00 for freebsd-fs@freebsd.org; Wed, 24 Jun 2009 12:14:48 +0400 Message-ID: <4A41E073.70902@mail.ru> Date: Wed, 24 Jun 2009 13:14:43 +0500 From: rihad User-Agent: Mozilla-Thunderbird 2.0.0.19 (X11/20090103) MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam: Not detected X-Mras: Ok Subject: Is it preferable to use the sync command? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rihad List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jun 2009 08:42:36 -0000 Hi, all, Having experienced a FreeBSD 5.1 crash due to power failure (despite using a UPS) resulting in massive /etc corruption and data loss, in order to minimize future risks should I: 0) tweak (decrease) these default sysctls: kern.filedelay: 30 kern.dirdelay: 29 kern.metadelay: 28 1) mount the root FS with soft-updates enabled (left as disabled in sysinstall by default due to known reasons) 2) setup a cron job calling /bin/sync every minute I somehow feel that turning soft-updates on would do the trick (it is not normally written to and has plenty of free space anyway). Thanks in advance for any educated tips. From owner-freebsd-fs@FreeBSD.ORG Wed Jun 24 09:28:23 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 10286106564A for ; Wed, 24 Jun 2009 09:28:23 +0000 (UTC) (envelope-from numisemis@yahoo.com) Received: from web37303.mail.mud.yahoo.com (web37303.mail.mud.yahoo.com [209.191.90.246]) by mx1.freebsd.org (Postfix) with SMTP id C3D3B8FC17 for ; Wed, 24 Jun 2009 09:28:22 +0000 (UTC) (envelope-from numisemis@yahoo.com) Received: (qmail 22549 invoked by uid 60001); 24 Jun 2009 09:28:22 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1245835702; bh=i6QwPKOz3E0UPqLKXsaYRiuqjHhuCcYrhy4hqwuOMLo=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=ufArkBXq8gUMf4JPX3SWaQ5ehzHukWtuv0V96ZQ4tUwW93MHD5f4cOWC2Ecd33wL7GjxeteObY9kli9REDbQcBSMem9qb9LPXcjA6egVbPdcajoA6HiW9FbGXg7S3guRhkm9rSpkVfmXTR1QqbKWh634H1EZpuyje9OMdmOpoXc= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=L1TPOcN4J/fR9e6eorGeK4Od0wSDpKoc1n1M3sgXayCzuKR0LxXfxNol06+3W4r6bDW943zNEoSzRw17kGRLGjmtiGMlwhrxraqQnBGPShUhz6niC5x2CuoMoxLmPTyRwasbp99FTHuU32hUpjhUNxDXiLKMSxmIsmuWT39KpR4=; Message-ID: <130804.21855.qm@web37303.mail.mud.yahoo.com> X-YMail-OSG: hJn6nTsVM1nIszhbA.PJqsGl.jADvL8SkDsjzROYbeaDEL_CfZs6qfmWyHyHsLWztMz7VlrC.z.rLPP1A8Db.mNm34wolLa_HsziDpm0XPkERWFmGzMox0n__RHIv5idGOtaPfT9Vsryxj8.YM5Z4iOkqaNtT1UdOCm5tX_tPI1gwH.gdV6WdZcJyfUBACcbigPaOPKVrx.bVGd5Mvk50X7S2LuK6WiDYzUFJI0s_wotmzjJ4IR_ee0I6SoxS7aEhc3R9U_PIfA8fgWh_27xzqBezQ980dSr5wkeuL3rPMcNiJG0jCdsx5wFpAb6wzCvesf5NDgu5tjzV3GqkDvGffk- Received: from [213.147.110.159] by web37303.mail.mud.yahoo.com via HTTP; Wed, 24 Jun 2009 02:28:21 PDT X-Mailer: YahooMailRC/1277.43 YahooMailWebService/0.7.289.15 References: <4A41E073.70902@mail.ru> Date: Wed, 24 Jun 2009 02:28:21 -0700 (PDT) From: Simun Mikecin To: rihad In-Reply-To: <4A41E073.70902@mail.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-fs@freebsd.org Subject: Re: Is it preferable to use the sync command? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jun 2009 09:28:23 -0000 rihad wrote: > Having experienced a FreeBSD 5.1 crash due to power failure (despite using a UPS) resulting in massive /etc corruption and data loss, in order to minimize future risks should I: > 0) tweak (decrease) these default sysctls: > kern.filedelay: 30 > kern.dirdelay: 29 > kern.metadelay: 28 > 1) mount the root FS with soft-updates enabled (left as disabled in sysinstall by default due to known reasons) > 2) setup a cron job calling /bin/sync every minute > I somehow feel that turning soft-updates on would do the trick (it is not normally written to and has plenty of free space anyway). Do you use ATA or SCSI? Turning soft-updates on for SCSI should do the trick. Since there is no support for gjournal and/or ZFS on 5.1, for ATA only real solution would be disabling write-cache (which degrades performance): "sysctl hw.ata.wc=0". From owner-freebsd-fs@FreeBSD.ORG Wed Jun 24 09:48:40 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1BD4D106564A for ; Wed, 24 Jun 2009 09:48:40 +0000 (UTC) (envelope-from rihad@mail.ru) Received: from mx40.mail.ru (mx40.mail.ru [94.100.176.54]) by mx1.freebsd.org (Postfix) with ESMTP id CCEFD8FC12 for ; Wed, 24 Jun 2009 09:48:39 +0000 (UTC) (envelope-from rihad@mail.ru) Received: from [217.25.27.27] (port=57067 helo=[217.25.27.27]) by mx40.mail.ru with asmtp id 1MJP5r-0009zZ-00; Wed, 24 Jun 2009 13:48:40 +0400 Message-ID: <4A41F672.9080900@mail.ru> Date: Wed, 24 Jun 2009 14:48:34 +0500 From: rihad User-Agent: Mozilla-Thunderbird 2.0.0.19 (X11/20090103) MIME-Version: 1.0 To: Simun Mikecin References: <4A41E073.70902@mail.ru> <130804.21855.qm@web37303.mail.mud.yahoo.com> In-Reply-To: <130804.21855.qm@web37303.mail.mud.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam: Not detected X-Mras: Ok Cc: freebsd-fs@freebsd.org Subject: Re: Is it preferable to use the sync command? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jun 2009 09:48:40 -0000 Simun Mikecin wrote: > rihad wrote: >> Having experienced a FreeBSD 5.1 crash due to power failure (despite using a UPS) resulting in massive /etc corruption and data loss, in order to minimize future risks should I: >> 0) tweak (decrease) these default sysctls: >> kern.filedelay: 30 >> kern.dirdelay: 29 >> kern.metadelay: 28 >> 1) mount the root FS with soft-updates enabled (left as disabled in sysinstall by default due to known reasons) >> 2) setup a cron job calling /bin/sync every minute >> I somehow feel that turning soft-updates on would do the trick (it is not normally written to and has plenty of free space anyway). > > > Do you use ATA or SCSI? ATA. > Turning soft-updates on for SCSI should do the trick. But not for ATA? Why I'm asking: other partitions using soft-updates don't seem to have lost any data. > Since there is no support for gjournal and/or ZFS on 5.1, for ATA only real solution would be disabling write-cache (which degrades performance): "sysctl hw.ata.wc=0". > I think this is much easier to do remotely than turning soft-updates on :-) I'll still try both solutions, thanks. From owner-freebsd-fs@FreeBSD.ORG Wed Jun 24 20:27:33 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1937D1065672; Wed, 24 Jun 2009 20:27:33 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: from mail-yx0-f179.google.com (mail-yx0-f179.google.com [209.85.210.179]) by mx1.freebsd.org (Postfix) with ESMTP id B40CE8FC14; Wed, 24 Jun 2009 20:27:32 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: by yxe9 with SMTP id 9so177257yxe.3 for ; Wed, 24 Jun 2009 13:27:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type:content-transfer-encoding; bh=MDAfEgqaM9duuPJecDn0tOomnJJKH2S6z0njixr9xfw=; b=qF1+GiMvnO8Zew0FyW+KPKbPiznSK2snHJyOt8VVQZZzl4msgfVg5tM+WYQQJeKWtV 4gekAT4cw9eztqaF4/CKUdOqGActi56ZfXpbTKuRefRSdsFvxi8f7NqoC0fpudfU2SHH 8Ap9uW9+fWHqxJwTqcU44XFf0hUNpDs+GliKQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; b=kktb62XVjqwcYKIfF7RLPT2j/CIYt78lI/d94s1MXjV9dX5pEMamvAIJ6JMJI4d+1+ iDOzp8XibQIsAkpDI9Yxml97LHGNL6G5klrXUbuOBW/kmAVJmE5BAZX7SQbg3E1PicVG Wb04FUNZuWitcuAQuWSvyFbWAQ/xdYxLcrmfo= MIME-Version: 1.0 Received: by 10.100.91.15 with SMTP id o15mr2349248anb.47.1245875251607; Wed, 24 Jun 2009 13:27:31 -0700 (PDT) Date: Wed, 24 Jun 2009 23:27:31 +0300 Message-ID: From: Dan Naumov To: freebsd-geom@freebsd.org, freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: read benchmarks: ufs/zfs/ext3 raidz/raid5 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jun 2009 20:27:33 -0000 Another FreeBSD person on a forum I frequent did some read benchmarks on his system: Athlon64 3500+ with 2GB DDR2 SDRAM, a WD 250GB system drive, and 5 Seagate Barracuda 750GB SATA-II data drives. ZFS and UFS testing was done using FreeBSD 7.2-RELEASE amd64, and ext3 testing was done using Ubuntu Server 8.04-LTS amd64. The used disks do not support NCQ, so there is no "NCQ advantage" on the Linux side. Random Access reads, 5MB chunks: http://virtual.tehinterweb.net/livejournal/2009-06-22_zfs_diskperf/zfs-diskperf-untuned-5mb.png Random Access reads, 1MB chunks: http://virtual.tehinterweb.net/livejournal/2009-06-22_zfs_diskperf/zfs-diskperf-untuned-1mb.png Random Access reads, 5MB chunks (big list): http://virtual.tehinterweb.net/livejournal/raid_performance/raid-diskperf-5mb-all.png Here is the original forum discussion thread: http://episteme.arstechnica.com/eve/forums/a/tpc/f/96509133/m/857002910041 Sincerely, - Dan Naumov From owner-freebsd-fs@FreeBSD.ORG Wed Jun 24 22:35:26 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A53271065687 for ; Wed, 24 Jun 2009 22:35:26 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-gx0-f210.google.com (mail-gx0-f210.google.com [209.85.217.210]) by mx1.freebsd.org (Postfix) with ESMTP id 3760E8FC0A for ; Wed, 24 Jun 2009 22:35:25 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by gxk6 with SMTP id 6so746097gxk.19 for ; Wed, 24 Jun 2009 15:35:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:cc:content-type; bh=e3BnrEIUyUqPPe2BsqO6pKiabbFDJEUabBG+EL4ZYGE=; b=IHl4nAhKa+L7zuiUYJ0hiiQd18Ui8kxdC0GeKELvoN8mh0GCmpfC/28SyU76Pw1l5p 8k5VUZ3yYpsua3rXR6SaEujd01TvKiWDGHsYIGtHqLs+lynVhaZxH1hqWp3rypcnmoS5 UR8LuVS5RmhjLZDwa1vVpTe/I35BCQ8yG9ZOg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:cc:content-type; b=EvYbVaEwJKfEyE/tRsnDTpDVzYnuya2DnAZw19BOXA9Uu4HLE0REIq3rN5zHVPCH65 uFbpXlpRdeMXzYZg4ACGNQTeBDhCI8IiMOIfRgqCimAbTYx/UU8XgxbyJqVWH+tyG4c/ qnT00v1+bcDLx0T00wIRUG2IRCo/5TRPqGD+M= MIME-Version: 1.0 Received: by 10.151.119.1 with SMTP id w1mr3410972ybm.198.1245882925169; Wed, 24 Jun 2009 15:35:25 -0700 (PDT) Date: Wed, 24 Jun 2009 15:35:25 -0700 Message-ID: From: Freddie Cash To: freebsd-cluster@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Fail-over SAN setup: ZFS, NFS, and ...? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jun 2009 22:35:26 -0000 [Not exactly sure which ML this belongs on, as it's related to both clustering and filesystems. If there's a better spot, let me know and I'll update the CC:/reply-to.] We're in the planning stages for building a multi-site, fail-over SAN setup which will be used to provide redundant storage for a virtual machine setup. The setup will be like so: [Server Room 1] . [Server Room 2] ----------------- . ------------------- . [storage server] . [storage server] | . | | . | [storage switch] . [storage switch] \----fibre----/ | . | . | . [storage aggregator] . | . | . /---[switch]---\ . | | | . | [VM box] | . | | | . [VM box] | | . | | [VM box] . | | | . [network switch] . | . | . [internet] Server room 1 and server room 2 are on opposite ends of town (about 3 km) with a dedicated, direct-link, fibre link between them. There will be a set of VM boxes at each site, that use the shared storage, and will act as fail-over for each other. In theory, only 1 server room would ever be active at a time, although we may end up migrating VMs between the two sites for maintenance purposes. We've got the storage server side of things figured out (5U rackmounts with 24 drive bauys, using FreeBSD 7.x and ZFS). We've got the storage switches picked out (HP Procurve 2800 or 2900, depending on if we go with 1 GbE or 10 GbE fibre links between them). We're stuck on the storage aggregator. For a single aggregator box setup, we'd use FreeBSD 7.x with ZFS. The storage servers would each export a single zvol using iSCSI. The storage aggregator would use ZFS to create a pool using a mirrored vdev. To expand the pool, we put in two more storage servers, and add another mirrored vdev to the pool. No biggie. The storage aggregator then uses NFS and/or iSCSI to make storage available to the VM boxes. This is the easy part. However, we'd like to remove the single-point-of-failure that the storage aggregator represents, and have a duplicate of it running at Server Room 1. Right now, we can do this using cold-spares that rsync from the live box every X hours/days. We'd like this to be a live, fail-over spare, though. And this is where we're stuck. What can we use to do this? CARP? Heatbeat? ggate? Should we look at Linux with DRBD or linux-ha or cluster-nfs or similar? Perhaps RedHat Cluster Suite? (We'd prefer not to, as then storage management becomes a nightmare again, requiring mdadm, lvm, and more.) Would a cluster filessytem be needed? AFS or similar? We have next to no knowledge of fail-over clustering when it comes to high-availability and fail-over. Any pointers to things to read online, or tips, or even "don't do that, you're insane" comments greatly appreciated. :) Thanks. -- Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@FreeBSD.ORG Wed Jun 24 23:03:15 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BB7CD1065670 for ; Wed, 24 Jun 2009 23:03:15 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 42D0B8FC17 for ; Wed, 24 Jun 2009 23:03:14 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1MJbUn-0004EA-4e for freebsd-fs@freebsd.org; Wed, 24 Jun 2009 23:03:13 +0000 Received: from 93-138-17-96.adsl.net.t-com.hr ([93.138.17.96]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 24 Jun 2009 23:03:13 +0000 Received: from ivoras by 93-138-17-96.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 24 Jun 2009 23:03:13 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Ivan Voras Date: Thu, 25 Jun 2009 01:02:48 +0200 Lines: 59 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig830E75D9D2AE33084C68CA2F" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 93-138-17-96.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.22 (Windows/20090605) In-Reply-To: X-Enigmail-Version: 0.95.7 Sender: news Cc: freebsd-geom@freebsd.org Subject: Re: read benchmarks: ufs/zfs/ext3 raidz/raid5 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jun 2009 23:03:16 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig830E75D9D2AE33084C68CA2F Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Dan Naumov wrote: > Another FreeBSD person on a forum I frequent did some read benchmarks > on his system: Athlon64 3500+ with 2GB DDR2 SDRAM, a WD 250GB system > drive, and 5 Seagate Barracuda 750GB SATA-II data drives. ZFS and UFS > testing was done using FreeBSD 7.2-RELEASE amd64, and ext3 testing was > done using Ubuntu Server 8.04-LTS amd64. The used disks do not support > NCQ, so there is no "NCQ advantage" on the Linux side. >=20 > Random Access reads, 5MB chunks: > http://virtual.tehinterweb.net/livejournal/2009-06-22_zfs_diskperf/zfs-= diskperf-untuned-5mb.png > Random Access reads, 1MB chunks: > http://virtual.tehinterweb.net/livejournal/2009-06-22_zfs_diskperf/zfs-= diskperf-untuned-1mb.png > Random Access reads, 5MB chunks (big list): > http://virtual.tehinterweb.net/livejournal/raid_performance/raid-diskpe= rf-5mb-all.png >=20 > Here is the original forum discussion thread: > http://episteme.arstechnica.com/eve/forums/a/tpc/f/96509133/m/857002910= 041 This looks about consistent with what I see when comparing ext3 to ZFS and/or UFS on 7.x systems. I have theories and hunches why IO in FreeBSD is slower than on Linux (and for different RAID tools / GEOM classes the reasons are slightly different) but nothing I can back with proof and measurements. I can only confirm that it is consistently slower. I haven't yet tried testing 8 so if you're looking for ideas for testing, try it (disable debugging before you use 8-CURRENT). I think ZFS has some concurrency-enhancing additions there (which will help in case the benchmark was done with a tool testing concurrency; I don't see what was used in the above pages). --------------enig830E75D9D2AE33084C68CA2F Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkpCsKAACgkQldnAQVacBcgLAQCgiVfvtVz9Y1PhH920AgLD/CnA M9IAnAmLI4lV5rqbkOz13uOZysR0tcik =gWQq -----END PGP SIGNATURE----- --------------enig830E75D9D2AE33084C68CA2F-- From owner-freebsd-fs@FreeBSD.ORG Thu Jun 25 00:04:11 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B3442106564A; Thu, 25 Jun 2009 00:04:11 +0000 (UTC) (envelope-from efinleywork@efinley.com) Received: from mail1.etv.net (mail1.etv.net [66.111.113.18]) by mx1.freebsd.org (Postfix) with ESMTP id 9898D8FC13; Thu, 25 Jun 2009 00:04:11 +0000 (UTC) (envelope-from efinleywork@efinley.com) Received: from ef04.etv.net ([74.214.237.51]) by mail1.etv.net with esmtpa (Exim 4.69 (FreeBSD)) (envelope-from ) id 1MJbhV-000JPm-Bi; Wed, 24 Jun 2009 17:16:21 -0600 Message-ID: <4A42B3C7.9000500@efinley.com> Date: Wed, 24 Jun 2009 17:16:23 -0600 From: Elliot Finley User-Agent: Thunderbird 2.0.0.22 (Windows/20090605) MIME-Version: 1.0 To: Freddie Cash References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, freebsd-cluster@freebsd.org Subject: Re: Fail-over SAN setup: ZFS, NFS, and ...? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jun 2009 00:04:12 -0000 Why not take a look at gluster? Freddie Cash wrote: > [Not exactly sure which ML this belongs on, as it's related to both > clustering and filesystems. If there's a better spot, let me know and I'll > update the CC:/reply-to.] > > We're in the planning stages for building a multi-site, fail-over SAN setup > which will be used to provide redundant storage for a virtual machine setup. > The setup will be like so: > [Server Room 1] . [Server Room 2] > ----------------- . ------------------- > . > [storage server] . [storage server] > | . | > | . | > [storage switch] . [storage switch] > \----fibre----/ | > . | > . | > . [storage aggregator] > . | > . | > . /---[switch]---\ > . | | | > . | [VM box] | > . | | | > . [VM box] | | > . | | [VM box] > . | | | > . [network switch] > . | > . | > . [internet] > > Server room 1 and server room 2 are on opposite ends of town (about 3 km) > with a dedicated, direct-link, fibre link between them. There will be a set > of VM boxes at each site, that use the shared storage, and will act as > fail-over for each other. In theory, only 1 server room would ever be > active at a time, although we may end up migrating VMs between the two sites > for maintenance purposes. > > We've got the storage server side of things figured out (5U rackmounts with > 24 drive bauys, using FreeBSD 7.x and ZFS). We've got the storage switches > picked out (HP Procurve 2800 or 2900, depending on if we go with 1 GbE or 10 > GbE fibre links between them). We're stuck on the storage aggregator. > > For a single aggregator box setup, we'd use FreeBSD 7.x with ZFS. The > storage servers would each export a single zvol using iSCSI. The storage > aggregator would use ZFS to create a pool using a mirrored vdev. To expand > the pool, we put in two more storage servers, and add another mirrored vdev > to the pool. No biggie. The storage aggregator then uses NFS and/or iSCSI > to make storage available to the VM boxes. This is the easy part. > > However, we'd like to remove the single-point-of-failure that the storage > aggregator represents, and have a duplicate of it running at Server Room 1. > Right now, we can do this using cold-spares that rsync from the live box > every X hours/days. We'd like this to be a live, fail-over spare, though. > And this is where we're stuck. > > What can we use to do this? CARP? Heatbeat? ggate? Should we look at > Linux with DRBD or linux-ha or cluster-nfs or similar? Perhaps RedHat > Cluster Suite? (We'd prefer not to, as then storage management becomes a > nightmare again, requiring mdadm, lvm, and more.) Would a cluster > filessytem be needed? AFS or similar? > > We have next to no knowledge of fail-over clustering when it comes to > high-availability and fail-over. Any pointers to things to read online, or > tips, or even "don't do that, you're insane" comments greatly appreciated. > :) From owner-freebsd-fs@FreeBSD.ORG Fri Jun 26 07:18:16 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BD491106564A; Fri, 26 Jun 2009 07:18:16 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from mail15.syd.optusnet.com.au (mail15.syd.optusnet.com.au [211.29.132.196]) by mx1.freebsd.org (Postfix) with ESMTP id 4F6D98FC14; Fri, 26 Jun 2009 07:18:16 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from server.vk2pj.dyndns.org (c122-106-216-167.belrs3.nsw.optusnet.com.au [122.106.216.167]) by mail15.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id n5Q7IDr0011915 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 26 Jun 2009 17:18:14 +1000 X-Bogosity: Ham, spamicity=0.000000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.3/8.14.3) with ESMTP id n5Q7ICi4044109; Fri, 26 Jun 2009 17:18:12 +1000 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.3/8.14.3/Submit) id n5Q7IC77044108; Fri, 26 Jun 2009 17:18:12 +1000 (EST) (envelope-from peter) Date: Fri, 26 Jun 2009 17:18:12 +1000 From: peterjeremy@optushome.com.au To: Dan Naumov Message-ID: <20090626071812.GA43965@server.vk2pj.dyndns.org> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="tKW2IUtsqtDRztdT" Content-Disposition: inline In-Reply-To: X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.19 (2009-01-05) Cc: freebsd-fs@freebsd.org, freebsd-geom@freebsd.org Subject: Re: read benchmarks: ufs/zfs/ext3 raidz/raid5 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jun 2009 07:18:17 -0000 --tKW2IUtsqtDRztdT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2009-Jun-24 23:27:31 +0300, Dan Naumov wrote: >Random Access reads, 5MB chunks: >http://virtual.tehinterweb.net/livejournal/2009-06-22_zfs_diskperf/zfs-dis= kperf-untuned-5mb.png >Random Access reads, 1MB chunks: >http://virtual.tehinterweb.net/livejournal/2009-06-22_zfs_diskperf/zfs-dis= kperf-untuned-1mb.png >Random Access reads, 5MB chunks (big list): >http://virtual.tehinterweb.net/livejournal/raid_performance/raid-diskperf-= 5mb-all.png These benchmarks are all fairly meaningless. As a first order approximation, all I/O to a Unix FS should be writes or you don't have enough RAM for your application. A more meaningful benchmark would check writes or a read/write mix with ~90% writes. --=20 Peter Jeremy --tKW2IUtsqtDRztdT Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (FreeBSD) iEYEARECAAYFAkpEdjQACgkQ/opHv/APuIeTswCgnwUYdvRhcnucf6Rrg1qovteD jOEAnR5nn9s7+rzB5O5StMyelmeV4g8G =pxVr -----END PGP SIGNATURE----- --tKW2IUtsqtDRztdT-- From owner-freebsd-fs@FreeBSD.ORG Fri Jun 26 16:53:47 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9842C106564A for ; Fri, 26 Jun 2009 16:53:47 +0000 (UTC) (envelope-from neil@hoggarth.me.uk) Received: from neilhoggarth-2.dsl.easynet.co.uk (neilhoggarth-2.dsl.easynet.co.uk [217.206.124.94]) by mx1.freebsd.org (Postfix) with ESMTP id 1835C8FC1E for ; Fri, 26 Jun 2009 16:53:46 +0000 (UTC) (envelope-from neil@hoggarth.me.uk) Received: from neilhoggarth-2.dsl.easynet.co.uk (localhost [127.0.0.1]) by neilhoggarth-2.dsl.easynet.co.uk (8.14.3/8.14.3) with ESMTP id n5QGOFgS062467 for ; Fri, 26 Jun 2009 17:24:15 +0100 (BST) (envelope-from neil@hoggarth.me.uk) Received: from localhost (njh@localhost) by neilhoggarth-2.dsl.easynet.co.uk (8.14.3/8.14.3/Submit) with ESMTP id n5QGOFpl062464 for ; Fri, 26 Jun 2009 17:24:15 +0100 (BST) (envelope-from neil@hoggarth.me.uk) X-Authentication-Warning: neilhoggarth-2.dsl.easynet.co.uk: njh owned process doing -bs Date: Fri, 26 Jun 2009 17:24:15 +0100 (BST) From: Neil Hoggarth X-X-Sender: njh@neilhoggarth-2.dsl.easynet.co.uk To: freebsd-fs@freebsd.org Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Subject: "zfs upgrade" of a mounted filesystem? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jun 2009 16:53:48 -0000 I have a 7-STABLE system with root-on-ZFS using the recipe from the wiki (http://wiki.freebsd.org/ZFSOnRoot), with a small UFS /boot partition on a flash disk. I updated the system a few days ago and have upgraded the zpool to version 13 and most of the filesystems to version 3, but I can't upgrade the filesystem version for the root filesystem as this is always mounted and it seems that "zfs upgrade" needs to unmount a filesystem to work on it? neilhoggarth-2# zfs upgrade This system is currently running ZFS filesystem version 3. The following filesystems are out of date, and can be upgraded. After being upgraded, these filesystems (and any 'zfs send' streams generated from subsequent snapshots) will no longer be accessible by older software versions. VER FILESYSTEM --- ------------ 1 newtank neilhoggarth-2# zfs upgrade newtank cannot unmount '/': Invalid argument Is there any way to work around this? Or will I need alternate boot/livefs media that incorporates ZFS v13 userland utilities to make the change? Regards, Neil. From owner-freebsd-fs@FreeBSD.ORG Fri Jun 26 17:03:35 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0D1D6106564A for ; Fri, 26 Jun 2009 17:03:35 +0000 (UTC) (envelope-from serenity@exscape.org) Received: from ch-smtp01.sth.basefarm.net (ch-smtp01.sth.basefarm.net [80.76.149.212]) by mx1.freebsd.org (Postfix) with ESMTP id BEA828FC14 for ; Fri, 26 Jun 2009 17:03:34 +0000 (UTC) (envelope-from serenity@exscape.org) Received: from c83-253-252-234.bredband.comhem.se ([83.253.252.234]:53704 helo=mx.exscape.org) by ch-smtp01.sth.basefarm.net with esmtp (Exim 4.69) (envelope-from ) id 1MKEpl-00061m-4r for freebsd-fs@freebsd.org; Fri, 26 Jun 2009 19:03:31 +0200 Received: from [192.168.1.5] (macbookpro [192.168.1.5]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx.exscape.org (Postfix) with ESMTPSA id 3C3E75ED8B for ; Fri, 26 Jun 2009 19:03:28 +0200 (CEST) Message-Id: <25306F0C-437C-4746-98F1-427E335AF8EB@exscape.org> From: Thomas Backman To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Fri, 26 Jun 2009 19:03:25 +0200 X-Mailer: Apple Mail (2.935.3) X-Originating-IP: 83.253.252.234 X-Scan-Result: No virus found in message 1MKEpl-00061m-4r. X-Scan-Signature: ch-smtp01.sth.basefarm.net 1MKEpl-00061m-4r 34f68bb335917fae41c6c503762c9437 Subject: ZFS send/recv: weird stream size? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jun 2009 17:03:35 -0000 Hi all, First off: not subscribed, please make sure I'm cc:ed or such. I found a minor oddity today when running my backup script. It copies a pool into another, using "zfs send -R -I $LASTSNAP tank@$CURRSNAP | zfs recv -Fvd slave" (the variables should explain themselves). Anyway, I got this in the middle: receiving incremental stream of tank/var/crash@backup-20090626-1505 into slave/var/crash@backup-20090626-1505 received 1.28GB stream in 51 seconds (25.6MB/sec) tank/var/crash is a compressed filesystem, using lzjb. Now, the only notable change in var/crash between backups was the addition of vmcore.22: [root@chaos /var/crash]# du -sch *22* 33K core.txt.22 1.0K info.22 34M kernel.debug.22 459M vmcore.22 493M total [root@chaos /var/crash]# du -schA *22* 88K core.txt.22 512B info.22 57M kernel.debug.22 2.2G vmcore.22 2.3G total So, if the new files' compressed size is 493MB, and the real size is 2.3GB, how come zfs send/recv transfers 1.28 GB? Other, possibly interesting (although I doubt it) stats: [root@chaos ~]# zfs get compress,compressratio tank/var/crash NAME PROPERTY VALUE SOURCE tank/var/crash compression lzjb local tank/var/crash compressratio 2.90x - [root@chaos ~]# du -shc /var/crash 1.0G /var/crash 1.0G total [root@chaos ~]# du -shcA /var/crash 6.9G /var/crash 6.9G total Regards, Thomas From owner-freebsd-fs@FreeBSD.ORG Fri Jun 26 17:36:01 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5CE961065672 for ; Fri, 26 Jun 2009 17:36:00 +0000 (UTC) (envelope-from nbari@k9.cx) Received: from exprod7og106.obsmtp.com (exprod7og106.obsmtp.com [64.18.2.165]) by mx1.freebsd.org (Postfix) with SMTP id B05B88FC17 for ; Fri, 26 Jun 2009 17:35:59 +0000 (UTC) (envelope-from nbari@k9.cx) Received: from source ([209.85.218.209]) by exprod7ob106.postini.com ([64.18.6.12]) with SMTP ID DSNKSkUG/FLmDIjAP4H7V/KzicCGmEsDGpXd@postini.com; Fri, 26 Jun 2009 10:36:00 PDT Received: by bwz5 with SMTP id 5so2646912bwz.23 for ; Fri, 26 Jun 2009 10:35:55 -0700 (PDT) MIME-Version: 1.0 Received: by 10.204.71.68 with SMTP id g4mr3876918bkj.81.1246036052740; Fri, 26 Jun 2009 10:07:32 -0700 (PDT) Date: Fri, 26 Jun 2009 12:07:32 -0500 Message-ID: <143912190906261007q265a92b6gcf8958e635334df8@mail.gmail.com> From: Nicolas de Bari Embriz Garcia Rojas To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: zfs nfs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jun 2009 17:36:01 -0000 Hi all, i updated my sources to the lates stable for fixing the bce0 (lagg) problem, but when doing so the zfs sources where also updated, so I had to rebuild/install world, after doing so and restarting the server, the NFS mount point now are only read only and can not write to any shared nfs point. any ides on how to fix this ? regards -- > nbari.tel From owner-freebsd-fs@FreeBSD.ORG Fri Jun 26 23:13:31 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9E3BE1065670 for ; Fri, 26 Jun 2009 23:13:31 +0000 (UTC) (envelope-from gallasch@free.de) Received: from mail.avalon-networks.com (mail.avalon-networks.com [91.204.6.143]) by mx1.freebsd.org (Postfix) with ESMTP id B58A68FC0C for ; Fri, 26 Jun 2009 23:13:30 +0000 (UTC) (envelope-from gallasch@free.de) Received: (qmail 52512 invoked from network); 27 Jun 2009 00:46:48 +0200 Received: by simscan 1.4.0 ppid: 52508, pid: 52509, t: 0.3158s scanners: clamav: 0.95.1/m:51/d:9510 Received: from orwell.free.de (kai@avalon-networks.com@91.204.4.66) by mail.avalon-networks.com with SMTP; 27 Jun 2009 00:46:48 +0200 Message-ID: <4A454FD6.1080005@free.de> Date: Sat, 27 Jun 2009 00:46:46 +0200 From: Kai Gallasch User-Agent: Thunderbird 2.0.0.22 (Macintosh/20090605) MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <4A42B3C7.9000500@efinley.com> In-Reply-To: <4A42B3C7.9000500@efinley.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Subject: Re: Fail-over SAN setup: ZFS, NFS, and ...? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jun 2009 23:13:31 -0000 > Freddie Cash wrote: >> [Not exactly sure which ML this belongs on, as it's related to both >> clustering and filesystems. If there's a better spot, let me know and >> I'll >> update the CC:/reply-to.] >> >> We're in the planning stages for building a multi-site, fail-over SAN >> setup Elliot Finley wrote: > Why not take a look at gluster? Quite interesting project - http://www.gluster.org/ But sadly: http://www.gluster.org/docs/index.php/Whats_New_v2.0 [..] Known Issues Some known issues and pending activities stalled for upcoming releases. * Distribute translator: uses 64bit inode numbers, as FreeBSD doesn't support 64bit inodes. Distribute is seen to not work on FreeBSD --Kai. From owner-freebsd-fs@FreeBSD.ORG Fri Jun 26 23:34:09 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 37DF0106564A for ; Fri, 26 Jun 2009 23:34:09 +0000 (UTC) (envelope-from mat.macy@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.241]) by mx1.freebsd.org (Postfix) with ESMTP id E56EA8FC0A for ; Fri, 26 Jun 2009 23:34:08 +0000 (UTC) (envelope-from mat.macy@gmail.com) Received: by an-out-0708.google.com with SMTP id d14so615321and.13 for ; Fri, 26 Jun 2009 16:34:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=S6wJoSkFfom7KeF3yEuWOykH7UOc7S2b0N5heSidv+M=; b=xoXhsTo9/iLryXA43U50c/JN1oBBvYW1rT4G+XSBuWANA5EPBLr7qW566EtPEtfyFH A6U543xjCo4HsoF0l2YnD+Lq4sA0UJ4TdVGnBYxF7y0sFy/4XJesQwqpmDd9VlpjBzpv rVhDWrzMbnnUF6h8x5Qai4h8PduJEC5naV7jg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=UJeI/CDQn+fqRooMJhZUWGX9GXTZOlo7VHU1eN93kJVYsHt45RGpG2uJ24lo/QwOTU kc3KQwv6+DeoWMt6Sneodg11++ORzM9fe+6fNBsddKMGYFRoDPl1IdsVzFYYeEE3NTsn FqejF5TRiMBgema0MiuqdLigLQc8wUr3HAdWA= MIME-Version: 1.0 Sender: mat.macy@gmail.com Received: by 10.100.123.12 with SMTP id v12mr5750299anc.21.1246059248275; Fri, 26 Jun 2009 16:34:08 -0700 (PDT) In-Reply-To: <4A454FD6.1080005@free.de> References: <4A42B3C7.9000500@efinley.com> <4A454FD6.1080005@free.de> Date: Fri, 26 Jun 2009 16:34:08 -0700 X-Google-Sender-Auth: 9ed62c404973ad4f Message-ID: <3c1674c90906261634l22910f54r31bbb8ec972d5ebf@mail.gmail.com> From: Kip Macy To: Kai Gallasch Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: Fail-over SAN setup: ZFS, NFS, and ...? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jun 2009 23:34:09 -0000 On Fri, Jun 26, 2009 at 3:46 PM, Kai Gallasch wrote: >> Freddie Cash wrote: >>> [Not exactly sure which ML this belongs on, as it's related to both >>> clustering and filesystems. =A0If there's a better spot, let me know an= d >>> I'll >>> update the CC:/reply-to.] >>> >>> We're in the planning stages for building a multi-site, fail-over SAN >>> setup > > Elliot Finley wrote: >> Why not take a look at gluster? > > Quite interesting project - http://www.gluster.org/ > > But sadly: > http://www.gluster.org/docs/index.php/Whats_New_v2.0 > > [..] > Known Issues > Some known issues and pending activities stalled for upcoming releases. > =A0 * Distribute translator: uses 64bit inode numbers, as FreeBSD doesn't > support 64bit inodes. Distribute is seen to not work on FreeBSD ino_t is still a 32-bit type, but it should be safe to make the following change to _types.h: @@ -43,7 +43,7 @@ typedef __uint64_t __fsfilcnt_t; typedef __uint32_t __gid_t; typedef __int64_t __id_t; /* can hold a gid_t, pid_t, or uid_t */ -typedef __uint32_t __ino_t; /* inode number */ +typedef __uint64_t __ino_t; /* inode number */ typedef long __key_t; /* IPC key (for Sys V IPC) = */ typedef __int32_t __lwpid_t; /* Thread ID (a.k.a. LWP) *= / typedef __uint16_t __mode_t; /* permissions */ Cheers, Kip From owner-freebsd-fs@FreeBSD.ORG Fri Jun 26 23:36:55 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C6D2E106564A; Fri, 26 Jun 2009 23:36:55 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: from mail-yx0-f181.google.com (mail-yx0-f181.google.com [209.85.210.181]) by mx1.freebsd.org (Postfix) with ESMTP id 6ED4E8FC12; Fri, 26 Jun 2009 23:36:55 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: by yxe11 with SMTP id 11so1582119yxe.3 for ; Fri, 26 Jun 2009 16:36:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type:content-transfer-encoding; bh=JGZppjjZhOYM/bf7ELTBW4DwjtExjQORB8KgC8wSEkU=; b=J4gSyAWfJuLIxHKTIdYgLtpHRb3A/+m5QXpVHrw95Zb3JBCFASPwSaeDgIsrceDZTM 1Eas7KsGBpZecy3H5/hAeZMlumk/qD426eVNi0dz5oyLeJ4uehk7xxe0IlFXt//xqaMH h7X4SMKtFlqfmGL9NZ8DYITtTyHqpiYK4tcdY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; b=oAifyxGsMFm8un2tnUsUNTiZkNvNTEQ19PvYkDB8jt9YQD7qT+1rvS8sQ4jt7THcLp gff8lmZdsiWz833kU98IIoYlP+SDijBGOUL2u6StCi6CGufK/Ed3Vr7K3IRfJYXGu9Nr juKHp5IjQ7wXm1OqpKD8PexymNH9n8YaG7YRM= MIME-Version: 1.0 Received: by 10.100.226.6 with SMTP id y6mr1325873ang.104.1246059414495; Fri, 26 Jun 2009 16:36:54 -0700 (PDT) Date: Sat, 27 Jun 2009 02:36:54 +0300 Message-ID: From: Dan Naumov To: freebsd-fs@freebsd.org, freebsd-geom@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: read/write benchmarking: UFS2 vs ZFS vs EXT3 vs ZFS RAIDZ vs Linux MDRAID X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jun 2009 23:36:56 -0000 To continue the subject of filesystem benchmarking (search the list for READ results posted a few days ago), here are some write and read/write results: Methodology: /data/5M and /data/1M have 5GB of data each in randomly-ordered chunks 5MB and 1MB in size, respectively. /data/zero.bin is a contiguous 8GB file. A process writes a burst of 5MB to a random location in /data/zero.bin once per second; other processes read chunks from /data/1M or /data/5M as appropriate (and as fast as possible) until the entire 5G dataset is read. Contiguous Write Performance: http://virtual.tehinterweb.net/livejournal/2009-06-22_zfs_diskperf/zfs-diskperf-contig-write.png Random Access Read/Write (5mb read chunks): http://virtual.tehinterweb.net/livejournal/2009-06-22_zfs_diskperf/zfs-diskperf-5MB-readwrite.png Random Access Read/Write (1mb read chunks): http://virtual.tehinterweb.net/livejournal/2009-06-22_zfs_diskperf/zfs-diskperf-1MB-readwrite.png These results are from the following forum thread: http://episteme.arstechnica.com/eve/forums/a/tpc/f/96509133/m/857002910041/p/4 Sincerely, - Dan Naumov From owner-freebsd-fs@FreeBSD.ORG Sat Jun 27 04:02:34 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5DF231065672 for ; Sat, 27 Jun 2009 04:02:34 +0000 (UTC) (envelope-from nork@FreeBSD.org) Received: from sakura.ninth-nine.com (unknown [IPv6:2001:2f0:104:80a0:21b:78ff:fe37:f1cf]) by mx1.freebsd.org (Postfix) with ESMTP id D95318FC17 for ; Sat, 27 Jun 2009 04:02:33 +0000 (UTC) (envelope-from nork@FreeBSD.org) Received: from nadesico.ninth-nine.com (ns1.ninth-nine.com [219.127.74.121] (may be forged)) (authenticated bits=0) by sakura.ninth-nine.com (8.14.3/8.14.3/NinthNine) with ESMTP id n5R42PV6063328; Sat, 27 Jun 2009 13:02:30 +0900 (JST) (envelope-from nork@FreeBSD.org) Date: Sat, 27 Jun 2009 13:02:24 +0900 From: Norikatsu Shigemura To: Neil Hoggarth Message-Id: <20090627130224.2dc662ac.nork@FreeBSD.org> In-Reply-To: References: X-Mailer: Sylpheed 2.6.0 (GTK+ 2.16.2; i386-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org Subject: Re: "zfs upgrade" of a mounted filesystem? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Jun 2009 04:02:34 -0000 Hi Neil. On Fri, 26 Jun 2009 17:24:15 +0100 (BST) Neil Hoggarth wrote: > Or will I need alternate boot/livefs media that incorporates ZFS v13 > userland utilities to make the change? Yes. I did zfs-upgrade by livefs. From owner-freebsd-fs@FreeBSD.ORG Sat Jun 27 22:25:08 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 65B761065670 for ; Sat, 27 Jun 2009 22:25:08 +0000 (UTC) (envelope-from rc_lenzi@yahoo.com.br) Received: from web51002.mail.re2.yahoo.com (web51002.mail.re2.yahoo.com [206.190.38.133]) by mx1.freebsd.org (Postfix) with SMTP id E95A68FC18 for ; Sat, 27 Jun 2009 22:25:07 +0000 (UTC) (envelope-from rc_lenzi@yahoo.com.br) Received: (qmail 32356 invoked by uid 60001); 27 Jun 2009 21:58:26 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com.br; s=s1024; t=1246139906; bh=UHBBn6GIwYkpyUNqKQiZoLUQSqK96m4I33gphXeXgjQ=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=kmzVUuVwlRz5nzIZ/hB5qaEMNWk6vbnla/P/AYn1v9mjNh3F1Q4gc4rz6orGVOA/OC2EGsHVzTzv5O+ggjt1KkWHvtb+ZSbW2rWgsbrUDitOWwkNfJwS2XES8sgVjVUxJWt4PTfSKOEK2OT3GjwUfDUOdx2w0Xitxc5iZ5X3zkg= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com.br; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=V47A0NKjvsZGNATUUHykrETBI/zh+N1/kfNd3A+ESnNa0bIiWM6uOvk7QsJYiMd43r1o+zXhUSaM1c7clSs4E7FlUw6c52/pXuLNgM0IkdyxtA56CPVcZKQIwARQ72UJXW+t31wj1qGuMXCLyUUitj54sLEvfF0auELVNK0e4VU=; Message-ID: <380571.31027.qm@web51002.mail.re2.yahoo.com> X-YMail-OSG: YcdRE2YVM1lnVFGMgr4RZBvkDe3PqLYmJwYcxJ0fy3B2rQpuXXsg5hSaOcCrzg8QQ8IhV62AVxSVe0e_svPzhgONfcfe3xN.32Ajixx3NtHxEelUdV7Yh4TmCfcUjniGKS2K_cun4g0_wKSKKCbSZKwyrRoWy0jT.3CPA2.8M_b8tTzq4uMLBXySV9SloyYe07nlXUOSS1UIxgmb9u1GXJGfl7hJyC.Ii8UMMzaQ14AZv94hD9kuqYxFZpMQl4reXaTtejnExM8NA8_7gC3pvKSuEnqh7isdT3gBE6GQjmrZ7h13cEmkTVegGR3sRPfa6ymBftgNVOFzL6.FNW9qTnci5q34j7iQ.xUOii34PoYtrQ_yhe.xDtovaw-- Received: from [189.26.153.35] by web51002.mail.re2.yahoo.com via HTTP; Sat, 27 Jun 2009 14:58:26 PDT X-Mailer: YahooMailRC/1277.43 YahooMailWebService/0.7.289.15 Date: Sat, 27 Jun 2009 14:58:26 -0700 (PDT) From: Rafael Caesar Lenzi To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Adding more disk's on ZFS array X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Jun 2009 22:25:08 -0000 Hi!=0AHow i can add more disks on ZFS raid0 or raid5 array?=0A=0AThanks!=0A= Rafael Lenzi=0A=0A=0A=0A _____________________________________________= _______________________________________=0AVeja quais s=E3o os assuntos do m= omento no Yahoo! +Buscados=0Ahttp://br.maisbuscados.yahoo.com From owner-freebsd-fs@FreeBSD.ORG Sat Jun 27 22:27:44 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 73E401065676 for ; Sat, 27 Jun 2009 22:27:44 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: from mail-yx0-f196.google.com (mail-yx0-f196.google.com [209.85.210.196]) by mx1.freebsd.org (Postfix) with ESMTP id 31B318FC1B for ; Sat, 27 Jun 2009 22:27:44 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: by yxe34 with SMTP id 34so2875435yxe.28 for ; Sat, 27 Jun 2009 15:27:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=68vdnp4ylF2df9jOHqU4CDVq59p5EKWKtu1NXvVLEsU=; b=KFJjL4WSysf5gctV0LDDGRRC/OuFHHw5MF66fzGUGMGA9wBf6IvPZU6EP6LO0Qq/AC 7ny12Es79ZYqTsOxhCfgG2Tca6Og8uPYSaFhHtzeLaiyyfxwpwmy34RkJmlJA90VgXnU pneQYO3qjniz5k8YsFC/mMmjkrR5a9OTwItmk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=K6fcHdp66wAhJUCYZrlK76QJYNyqwYlHRP4apQPXp2A7YCoPAkP1qIkH/zU3c0qQeB nqnwgTuXbQq5v6qqD2si5Il5dq5kIhBip1wAjFGpURHz8jC0bIFTE1yjuLNNaG6WwnK9 54SCbVjb0sNEuuvoqY8chasXzEBR63WGLkOew= MIME-Version: 1.0 Received: by 10.100.58.1 with SMTP id g1mr6880201ana.181.1246141663695; Sat, 27 Jun 2009 15:27:43 -0700 (PDT) In-Reply-To: <380571.31027.qm@web51002.mail.re2.yahoo.com> References: <380571.31027.qm@web51002.mail.re2.yahoo.com> Date: Sun, 28 Jun 2009 01:27:43 +0300 Message-ID: From: Dan Naumov To: Rafael Caesar Lenzi Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Adding more disk's on ZFS array X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Jun 2009 22:27:44 -0000 On Sun, Jun 28, 2009 at 12:58 AM, Rafael Caesar Lenzi wrote: > > Hi! > How i can add more disks on ZFS raid0 or raid5 array? > > Thanks! > Rafael Lenzi raid0 (stripe): zpool add POOLNAME DEVICENAME raid5 (raidz): you can't