From owner-freebsd-fs@FreeBSD.ORG Sun May 16 00:13:54 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D46841065670 for ; Sun, 16 May 2010 00:13:54 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from QMTA11.westchester.pa.mail.comcast.net (qmta11.westchester.pa.mail.comcast.net [76.96.59.211]) by mx1.freebsd.org (Postfix) with ESMTP id 8437B8FC08 for ; Sun, 16 May 2010 00:13:53 +0000 (UTC) Received: from omta10.westchester.pa.mail.comcast.net ([76.96.62.28]) by QMTA11.westchester.pa.mail.comcast.net with comcast id Hzua1e0050cZkys5B0DuhL; Sun, 16 May 2010 00:13:54 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta10.westchester.pa.mail.comcast.net with comcast id J0Ds1e00D3S48mS3W0DtVX; Sun, 16 May 2010 00:13:54 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 9D5B59B419; Sat, 15 May 2010 17:13:51 -0700 (PDT) Date: Sat, 15 May 2010 17:13:51 -0700 From: Jeremy Chadwick To: Kaya Saman Message-ID: <20100516001351.GA50879@icarus.home.lan> References: <4BEF2F9C.7080409@netscape.net> <4BEF3137.4080203@netscape.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4BEF3137.4080203@netscape.net> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-fs@freebsd.org Subject: Re: Quick ZFS mirroring question for non-mirrored pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2010 00:13:55 -0000 On Sun, May 16, 2010 at 02:41:43AM +0300, Kaya Saman wrote: > Ok I think I've got what I want by using the 'attach' command: > > from here: http://prefetch.net/blog/index.php/2007/01/04/adding-a-mirror-to-a-device-in-a-zfs-pool/ > > rd1# zpool attach zpool1 /mnt/disk1 /mnt/disk3 > rd1# zpool attach zpool1 /mnt/disk2 /mnt/disk4 > rd1# zpool status zpool1 > pool: zpool1 > state: ONLINE > scrub: resilver completed after 0h0m with 0 errors on Sun May 16 > 02:36:58 2010 > config: > > NAME STATE READ WRITE CKSUM > zpool1 ONLINE 0 0 0 > mirror ONLINE 0 0 0 > /mnt/disk1 ONLINE 0 0 0 > /mnt/disk3 ONLINE 0 0 0 > mirror ONLINE 0 0 0 > /mnt/disk2 ONLINE 0 0 0 96.5K resilvered > /mnt/disk4 ONLINE 0 0 0 15.3M resilvered What you have here is the equivalent of RAID-10. It might be more helpful to look at the above as a "stripe of mirrors". In this situation, you might be better off with raidz1 (RAID-5 in concept). You should get better actual I/O performance due to ZFS distributing the I/O workload across 4 disks rather than 2. At least that's how I understand it. > and also space is ok being ~256MB: > > rd1# zpool list > NAME SIZE USED AVAIL CAP HEALTH ALTROOT > zpool1 246M 32.2M 214M 13% ONLINE - > > although not sure where 10MB went as all files in this pool are > 128MB so I should get 256MB no?? I don't have this problem: testbox# zpool create mypool mirror da1 da2 mirror da3 da4 testbox# zpool list NAME SIZE USED AVAIL CAP HEALTH ALTROOT mypool 254G 75K 254G 0% ONLINE - testbox# zpool status pool: mypool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM mypool ONLINE 0 0 0 mirror ONLINE 0 0 0 da1 ONLINE 0 0 0 da2 ONLINE 0 0 0 mirror ONLINE 0 0 0 da3 ONLINE 0 0 0 da4 ONLINE 0 0 0 errors: No known data errors And after creating a 32MByte file: testbox# dd if=/dev/urandom of=/mypool/file bs=1024 count=32768 32768+0 records in 32768+0 records out 33554432 bytes transferred in 1.522111 secs (22044669 bytes/sec) testbox# ls -l /mypool/file -rw-r--r-- 1 root wheel 33554432 May 15 17:12 /mypool/file testbox# zpool list NAME SIZE USED AVAIL CAP HEALTH ALTROOT mypool 254G 32.1M 254G 0% ONLINE - -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Sun May 16 00:51:19 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DC07C106566C for ; Sun, 16 May 2010 00:51:19 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from blade.simplesystems.org (blade.simplesystems.org [65.66.246.74]) by mx1.freebsd.org (Postfix) with ESMTP id A12A18FC08 for ; Sun, 16 May 2010 00:51:18 +0000 (UTC) Received: from freddy.simplesystems.org (freddy.simplesystems.org [65.66.246.65]) by blade.simplesystems.org (8.13.8+Sun/8.13.8) with ESMTP id o4G0pHZZ019886; Sat, 15 May 2010 19:51:17 -0500 (CDT) Date: Sat, 15 May 2010 19:51:17 -0500 (CDT) From: Bob Friesenhahn X-X-Sender: bfriesen@freddy.simplesystems.org To: Jeremy Chadwick In-Reply-To: <20100516001351.GA50879@icarus.home.lan> Message-ID: References: <4BEF2F9C.7080409@netscape.net> <4BEF3137.4080203@netscape.net> <20100516001351.GA50879@icarus.home.lan> User-Agent: Alpine 2.01 (GSO 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (blade.simplesystems.org [65.66.246.90]); Sat, 15 May 2010 19:51:17 -0500 (CDT) Cc: freebsd-fs@freebsd.org Subject: Re: Quick ZFS mirroring question for non-mirrored pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2010 00:51:19 -0000 On Sat, 15 May 2010, Jeremy Chadwick wrote: > > What you have here is the equivalent of RAID-10. It might be more > helpful to look at the above as a "stripe of mirrors". > > In this situation, you might be better off with raidz1 (RAID-5 in > concept). You should get better actual I/O performance due to ZFS > distributing the I/O workload across 4 disks rather than 2. At least > that's how I understand it. That would be a reasonable assumption but actual evidence suggests otherwise. For sequential I/O, mirrors and raidz1 seem to offer roughly similar performance, except that mirrors win for reads and raidz1 often win for writes. The mirror configuration definitely wins as soon as there are many seeks or multi-user activity. The reason why mirrors still do well for sequential I/O is that there is still load-sharing across the vdevs (smart "striping") but in full 128K blocks whereas the raidz1 config needs to break the 128K blocks into smaller blocks which are striped across the disks in the vdev. Breaking the data into smaller chunks for raidz multiplies the disk IOPS required. Disk seeks are slow. The main reason to choose raidz1 is for better space efficiency but mirrors offer more performance. For an interesting set of results, see the results summary of "Bob's method" at "http://www.nedharvey.com/". The only way to be sure for your own system is to create various pool configurations and test with something which represents your expected work load. As long as the pool is not the boot pool, zfs makes such testing quite easy. Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From owner-freebsd-fs@FreeBSD.ORG Sun May 16 01:01:34 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DB5211065673 for ; Sun, 16 May 2010 01:01:34 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta09.emeryville.ca.mail.comcast.net (qmta09.emeryville.ca.mail.comcast.net [76.96.30.96]) by mx1.freebsd.org (Postfix) with ESMTP id C13D68FC08 for ; Sun, 16 May 2010 01:01:34 +0000 (UTC) Received: from omta23.emeryville.ca.mail.comcast.net ([76.96.30.90]) by qmta09.emeryville.ca.mail.comcast.net with comcast id J0wB1e0021wfjNsA911bqZ; Sun, 16 May 2010 01:01:35 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta23.emeryville.ca.mail.comcast.net with comcast id J11a1e0073S48mS8j11aMZ; Sun, 16 May 2010 01:01:35 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id A905B9B419; Sat, 15 May 2010 18:01:33 -0700 (PDT) Date: Sat, 15 May 2010 18:01:33 -0700 From: Jeremy Chadwick To: Bob Friesenhahn Message-ID: <20100516010133.GA52593@icarus.home.lan> References: <4BEF2F9C.7080409@netscape.net> <4BEF3137.4080203@netscape.net> <20100516001351.GA50879@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-fs@freebsd.org Subject: Re: Quick ZFS mirroring question for non-mirrored pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2010 01:01:34 -0000 On Sat, May 15, 2010 at 07:51:17PM -0500, Bob Friesenhahn wrote: > On Sat, 15 May 2010, Jeremy Chadwick wrote: > >What you have here is the equivalent of RAID-10. It might be more > >helpful to look at the above as a "stripe of mirrors". > > > >In this situation, you might be better off with raidz1 (RAID-5 in > >concept). You should get better actual I/O performance due to ZFS > >distributing the I/O workload across 4 disks rather than 2. At least > >that's how I understand it. > > That would be a reasonable assumption but actual evidence suggests > otherwise. For sequential I/O, mirrors and raidz1 seem to offer > roughly similar performance, except that mirrors win for reads and > raidz1 often win for writes. The mirror configuration definitely > wins as soon as there are many seeks or multi-user activity. > > The reason why mirrors still do well for sequential I/O is that > there is still load-sharing across the vdevs (smart "striping") but > in full 128K blocks whereas the raidz1 config needs to break the > 128K blocks into smaller blocks which are striped across the disks > in the vdev. Breaking the data into smaller chunks for raidz > multiplies the disk IOPS required. Disk seeks are slow. > > The main reason to choose raidz1 is for better space efficiency but > mirrors offer more performance. > > For an interesting set of results, see the results summary of "Bob's > method" at "http://www.nedharvey.com/". > > The only way to be sure for your own system is to create various > pool configurations and test with something which represents your > expected work load. As long as the pool is not the boot pool, zfs > makes such testing quite easy. Thanks Bob. You're absolutely right. I'd never seen/read said data results before, nor had I read the below material until now; quite interesting and educational. http://blogs.sun.com/roch/entry/when_to_and_not_to -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Sun May 16 01:29:42 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 101D31065678 for ; Sun, 16 May 2010 01:29:42 +0000 (UTC) (envelope-from SamanKaya@netscape.net) Received: from imr-da06.mx.aol.com (imr-da06.mx.aol.com [205.188.169.203]) by mx1.freebsd.org (Postfix) with ESMTP id C24F38FC1F for ; Sun, 16 May 2010 01:29:41 +0000 (UTC) Received: from mtaout-mb04.r1000.mx.aol.com (mtaout-mb04.r1000.mx.aol.com [172.29.41.68]) by imr-da06.mx.aol.com (8.14.1/8.14.1) with ESMTP id o4G1TRD7010709 for ; Sat, 15 May 2010 21:29:27 -0400 Received: from [172.16.0.66] (unknown [212.156.209.87]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mtaout-mb04.r1000.mx.aol.com (MUA/Third Party Client Interface) with ESMTPSA id 87825E0000B0 for ; Sat, 15 May 2010 21:29:26 -0400 (EDT) Message-ID: <4BEF4A73.8060905@netscape.net> Date: Sun, 16 May 2010 04:29:23 +0300 From: Kaya Saman User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.1.8) Gecko/20100227 Lightning/1.0b1 Thunderbird/3.0.3 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <4BEF2F9C.7080409@netscape.net> <4BEF3137.4080203@netscape.net> <20100516001351.GA50879@icarus.home.lan> In-Reply-To: x-aol-global-disposition: G X-AOL-SCOLL-SCORE: 0:2:486950624:93952408 X-AOL-SCOLL-URL_COUNT: 0 x-aol-sid: 3039ac1d29444bef4a7671a1 X-AOL-IP: 212.156.209.87 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: Quick ZFS mirroring question for non-mirrored pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2010 01:29:42 -0000 Many thanks guys for providing so much valuable input and knowledge!!! I really appreciate all your advice and knowledge. Please excuse my naivety but the statement below: On 05/16/2010 03:51 AM, Bob Friesenhahn wrote: > As long as the pool is not the boot pool, zfs makes such testing quite > easy. I was under the impression that one needed a UFS2 filesystem in order to be able to boot FreeBSD as that is the only FS available upon install..... unlike Solaris10/OpenSolaris which creates the ZFS filesystem upon install. The plan I originally conceived was to use a 40GB solid state disk as the / (root) directory comprising of all descending file systems, eg. /usr, /proc, /lib etc... using the UFS2 FS ....and then use ZFS for the storage portion of my server using 2TB Western Digital RE4 Enterprise SATA drives. Since it's a simple home based server and not a massive enterprise grade environment performance is not too much of an issue. However, system backups are and without funding for a spare system or DAS or SAN solution the only real option I have is to use a RAID0 esq based setup so if one or both the primary drives go offline then at least I have all my data backed up and still available. Regards, Kaya On Sat, May 15, 2010 at 07:51:17PM -0500, Bob Friesenhahn wrote: > > On Sat, 15 May 2010, Jeremy Chadwick wrote: > >> > >What you have here is the equivalent of RAID-10. It might be more >> > >helpful to look at the above as a "stripe of mirrors". >> > > >> > >In this situation, you might be better off with raidz1 (RAID-5 in >> > >concept). You should get better actual I/O performance due to ZFS >> > >distributing the I/O workload across 4 disks rather than 2. At least >> > >that's how I understand it. >> > > > > That would be a reasonable assumption but actual evidence suggests > > otherwise. For sequential I/O, mirrors and raidz1 seem to offer > > roughly similar performance, except that mirrors win for reads and > > raidz1 often win for writes. The mirror configuration definitely > > wins as soon as there are many seeks or multi-user activity. > > > > The reason why mirrors still do well for sequential I/O is that > > there is still load-sharing across the vdevs (smart "striping") but > > in full 128K blocks whereas the raidz1 config needs to break the > > 128K blocks into smaller blocks which are striped across the disks > > in the vdev. Breaking the data into smaller chunks for raidz > > multiplies the disk IOPS required. Disk seeks are slow. > > > > The main reason to choose raidz1 is for better space efficiency but > > mirrors offer more performance. > > > > For an interesting set of results, see the results summary of "Bob's > > method" at"http://www.nedharvey.com/". > > > > The only way to be sure for your own system is to create various > > pool configurations and test with something which represents your > > expected work load. As long as the pool is not the boot pool, zfs > > makes such testing quite easy. > Thanks Bob. You're absolutely right. I'd never seen/read said data results before, nor had I read the below material until now; quite interesting and educational. http://blogs.sun.com/roch/entry/when_to_and_not_to -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Sun May 16 01:40:14 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3D7051065672 for ; Sun, 16 May 2010 01:40:14 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.152]) by mx1.freebsd.org (Postfix) with ESMTP id C9A098FC14 for ; Sun, 16 May 2010 01:40:13 +0000 (UTC) Received: by fg-out-1718.google.com with SMTP id 22so637200fge.13 for ; Sat, 15 May 2010 18:40:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=l9GaxT5iucXVxPNUyvwJNuqn9+zFyvC5NozuOB9nzxM=; b=duyzAOxniMXo6RfYTy+cwJlYqO7DZdxEtil0YNhYYb9w1t6Kaqn/LBPfFV3GlRizBn kd4l/bxMCJL8kKkpRp6Vg3icWUErR9rAQmabqSQaF2UadV3+5/Lyw039toJu7rIWT0ee RqMPsRaN+jdM6ICQ0OkJ/Jz72v7RlngbyNQME= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=QMsojao3Bj0wuiUk8CzP8jlfLLm9OiXbe17dRF817VItOz18n2/aRIiY4sQ+2gdLoF Th/p6QtdMM5iILi/mfF88HmEf1jhv/hwqNNoYJxjB0Bm2T7OIiU6mnOV06kOMbDplsER QHqjYEdu0KKJ0cc+Ogw1i3WGD5nrbugHkbqsM= MIME-Version: 1.0 Received: by 10.204.131.198 with SMTP id y6mr57498bks.4.1273974011987; Sat, 15 May 2010 18:40:11 -0700 (PDT) Received: by 10.204.66.3 with HTTP; Sat, 15 May 2010 18:40:11 -0700 (PDT) Date: Sun, 16 May 2010 04:40:11 +0300 Message-ID: From: Dan Naumov To: SamanKaya@netscape.net, freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Subject: RE: Quick ZFS mirroring question for non-mirrored pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2010 01:40:14 -0000 Hey You can have a "purely ZFS" FreeBSD system, including booting off ZFS. Quite a lot of things are currently completely missing from sysinstall (zfs, gpart, gjournal, glabel) but that's not an indication that they don't exist and/or can't be used :) Look at this installation script: http://anonsvn.h3q.com/projects/freebsd-patches/browser/manageBE/create-zfsboot-gpt_livecd.sh It will make your life easy. - Sincerely, Dan Naumov From owner-freebsd-fs@FreeBSD.ORG Sun May 16 01:50:26 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6154F106566B for ; Sun, 16 May 2010 01:50:26 +0000 (UTC) (envelope-from SamanKaya@netscape.net) Received: from imr-mb01.mx.aol.com (imr-mb01.mx.aol.com [64.12.207.164]) by mx1.freebsd.org (Postfix) with ESMTP id 24CEA8FC12 for ; Sun, 16 May 2010 01:50:25 +0000 (UTC) Received: from mtaout-mb03.r1000.mx.aol.com (mtaout-mb03.r1000.mx.aol.com [172.29.41.67]) by imr-mb01.mx.aol.com (8.14.1/8.14.1) with ESMTP id o4G1oNqa017668 for ; Sat, 15 May 2010 21:50:23 -0400 Received: from [172.16.0.66] (unknown [212.156.209.87]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mtaout-mb03.r1000.mx.aol.com (MUA/Third Party Client Interface) with ESMTPSA id 165E7E0007EC for ; Sat, 15 May 2010 21:50:22 -0400 (EDT) Message-ID: <4BEF4F5C.9010200@netscape.net> Date: Sun, 16 May 2010 04:50:20 +0300 From: Kaya Saman User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.1.8) Gecko/20100227 Lightning/1.0b1 Thunderbird/3.0.3 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit x-aol-global-disposition: G X-AOL-SCOLL-SCORE: 0:2:407615744:93952408 X-AOL-SCOLL-URL_COUNT: 0 x-aol-sid: 3039ac1d29434bef4f5e0bbb X-AOL-IP: 212.156.209.87 Subject: Re: Quick ZFS mirroring question for non-mirrored pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2010 01:50:26 -0000 Thanks so much Dan for that! Will check it out after some sleep - is late round 5am here and need to get some rest now :-) Uh... and I gota learn MS Exchange later on for the migration at work... ouch :-S Regards, Kaya On 05/16/2010 04:40 AM, Dan Naumov wrote: > Hey > > You can have a "purely ZFS" FreeBSD system, including booting off ZFS. > Quite a lot of things are currently completely missing from sysinstall > (zfs, gpart, gjournal, glabel) but that's not an indication that they > don't exist and/or can't be used :) > > Look at this installation script: > http://anonsvn.h3q.com/projects/freebsd-patches/browser/manageBE/create-zfsboot-gpt_livecd.sh > It will make your life easy. > > > - Sincerely, > Dan Naumov > From owner-freebsd-fs@FreeBSD.ORG Sun May 16 01:58:54 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 79688106566B for ; Sun, 16 May 2010 01:58:54 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta10.westchester.pa.mail.comcast.net (qmta10.westchester.pa.mail.comcast.net [76.96.62.17]) by mx1.freebsd.org (Postfix) with ESMTP id 276838FC1D for ; Sun, 16 May 2010 01:58:53 +0000 (UTC) Received: from omta11.westchester.pa.mail.comcast.net ([76.96.62.36]) by qmta10.westchester.pa.mail.comcast.net with comcast id J1T31e0040mv7h05A1yuHE; Sun, 16 May 2010 01:58:54 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta11.westchester.pa.mail.comcast.net with comcast id J1ys1e0083S48mS3X1yt8R; Sun, 16 May 2010 01:58:53 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id E7EBB9B419; Sat, 15 May 2010 18:58:50 -0700 (PDT) Date: Sat, 15 May 2010 18:58:50 -0700 From: Jeremy Chadwick To: Kaya Saman Message-ID: <20100516015850.GA55302@icarus.home.lan> References: <4BEF2F9C.7080409@netscape.net> <4BEF3137.4080203@netscape.net> <20100516001351.GA50879@icarus.home.lan> <4BEF4A73.8060905@netscape.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4BEF4A73.8060905@netscape.net> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-fs@freebsd.org Subject: Re: Quick ZFS mirroring question for non-mirrored pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2010 01:58:54 -0000 On Sun, May 16, 2010 at 04:29:23AM +0300, Kaya Saman wrote: > On 05/16/2010 03:51 AM, Bob Friesenhahn wrote: > >As long as the pool is not the boot pool, zfs makes such testing > >quite easy. > > I was under the impression that one needed a UFS2 filesystem in > order to be able to boot FreeBSD as that is the only FS available > upon install..... unlike Solaris10/OpenSolaris which creates the ZFS > filesystem upon install. > > The plan I originally conceived was to use a 40GB solid state disk > as the / (root) directory comprising of all descending file systems, > eg. /usr, /proc, /lib etc... using the UFS2 FS > > ....and then use ZFS for the storage portion of my server using 2TB > Western Digital RE4 Enterprise SATA drives. I would highly recommend doing exactly what you've described here. In fact, it's what I do on two of my home systems (Intel X25-V 40GB drive used for root, /usr, /var, and /tmp, and ZFS for everything else including /home), and what I plan on our servers one Intel 80GB SSDs drop to a more reasonable price. There are many people here who have gone through the pain (IMHO) of getting ZFS to boot on FreeBSD, and it still isn't as simple nor reliable (in all configurations) as it is on OpenSolaris/Solaris 10. There seem to be a large number of "gotchas" which come up when the administrator least expects/wants it (usually during a failure scenario). Also, please reconsider going with Western Digital RE4 2TB drives. These drives are all "GP" (Green Power) drives, which you do not want. There have been numerous reports on the FreeBSD mailing lists about problems with these drives (repeated head offloading/parking causing problems in RAID arrays), and yes, it applies to Enterprise class drive as well; WD has indirectly confirmed the problem in one user's case by sending him a "fixed" firmware. I can point you to threads if you want to read them. I would recommend you choose WD Caviar Black drives instead (cheaper, benefit from TLER when enabled, and throughput is much higher than GP drives), or another vendor of your choice. Don't ask "Who do you recommend?" because everyone has different experiences/preferences; there's no vendor who's 99% reliable right now. :-) -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Sun May 16 12:44:25 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1DCB1106567A for ; Sun, 16 May 2010 12:44:25 +0000 (UTC) (envelope-from SamanKaya@netscape.net) Received: from imr-ma02.mx.aol.com (imr-ma02.mx.aol.com [64.12.206.40]) by mx1.freebsd.org (Postfix) with ESMTP id D1F068FC14 for ; Sun, 16 May 2010 12:44:24 +0000 (UTC) Received: from mtaout-db01.r1000.mx.aol.com (mtaout-db01.r1000.mx.aol.com [172.29.51.193]) by imr-ma02.mx.aol.com (8.14.1/8.14.1) with ESMTP id o4GCiAIc011924; Sun, 16 May 2010 08:44:10 -0400 Received: from [172.16.0.66] (unknown [212.156.209.87]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mtaout-db01.r1000.mx.aol.com (MUA/Third Party Client Interface) with ESMTPSA id F25F3E0000A0; Sun, 16 May 2010 08:44:09 -0400 (EDT) Message-ID: <4BEFE896.6030508@netscape.net> Date: Sun, 16 May 2010 15:44:06 +0300 From: Kaya Saman User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.1.8) Gecko/20100227 Lightning/1.0b1 Thunderbird/3.0.3 MIME-Version: 1.0 To: Jeremy Chadwick References: <4BEF2F9C.7080409@netscape.net> <4BEF3137.4080203@netscape.net> <20100516001351.GA50879@icarus.home.lan> <4BEF4A73.8060905@netscape.net> <20100516015850.GA55302@icarus.home.lan> In-Reply-To: <20100516015850.GA55302@icarus.home.lan> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit x-aol-global-disposition: G X-AOL-SCOLL-SCORE: 0:2:480869440:93952408 X-AOL-SCOLL-URL_COUNT: 0 x-aol-sid: 3039ac1d33c14befe8995ebe X-AOL-IP: 212.156.209.87 Cc: freebsd-fs@freebsd.org Subject: Re: Quick ZFS mirroring question for non-mirrored pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2010 12:44:25 -0000 Thanks for the response Jeremy! Sorry didn't answer right away but as I said in my previous post it was 5am and I ended up crashing out..... [...] > I would highly recommend doing exactly what you've described here. In > fact, it's what I do on two of my home systems (Intel X25-V 40GB drive > used for root, /usr, /var, and /tmp, and ZFS for everything else > including /home), and what I plan on our servers one Intel 80GB SSDs > drop to a more reasonable price. > Cool :-) It's set then this is how I'm gona go. > There are many people here who have gone through the pain (IMHO) of > getting ZFS to boot on FreeBSD, and it still isn't as simple nor > reliable (in all configurations) as it is on OpenSolaris/Solaris 10. > There seem to be a large number of "gotchas" which come up when the > administrator least expects/wants it (usually during a failure > scenario). > Thanks for the warning, I figured that as ZFS doesn't seem as well developed on BSD then Sun Microsystems version (now Sun Oracle just to be up to date!) > Also, please reconsider going with Western Digital RE4 2TB drives. > These drives are all "GP" (Green Power) drives, which you do not want. > There have been numerous reports on the FreeBSD mailing lists about > problems with these drives (repeated head offloading/parking causing > problems in RAID arrays), and yes, it applies to Enterprise class drive > as well; WD has indirectly confirmed the problem in one user's case by > sending him a "fixed" firmware. I can point you to threads if you want > to read them. > > I would recommend you choose WD Caviar Black drives instead (cheaper, > benefit from TLER when enabled, and throughput is much higher than GP > drives), or another vendor of your choice. Don't ask "Who do you > recommend?" because everyone has different experiences/preferences; > there's no vendor who's 99% reliable right now. :-) > > I checked the Caviar Black out and yes it's cheaper which means better for me if it causes less problems and increases reliability; sorry to keep going on about the reliability issue but I just want the drives to ask a long time as I've had so many drive failures over the past few years it gets irritating not to mention expensive replacing drives every few months or years. I found another alternative too which is a Seagate Barracuda 2TB 5900RPM Low Power drive...?? Don't know what you'd make of it though as you explicitly claimed not to use low-power drives? Quote: " Don't ask "Who do you recommend?" because everyone has different experiences/preferences; there's no vendor who's 99% reliable right now. :-) " Isn't this like asking people what their favorite color is?? There are something like 2^23 potentials lol if talking about a 24bit color matrix! Many thanks for all the advice :-D It really is such a pleasure on mailing lists and forums to be helped by such nice and kind people (this includes everyone that has participated in my post!!) P.s. this is deviating a bit from the subject but just caught the UNIX Systems Administrator part in your signature! It's so much where I wana be you have no idea so I'm really envious right now :-) The incredibly funny part however is after searching far and wide I ended up working for a firm developing MS software for companies as a Systems Engineer.... the only catch is that they're having to teach me how to use Windows as I've never actually used it before since XP days about 8 years ago now and since they're using MS Server 2003/8 and Win 7 I am completely lost as I don't really point click too much for stuff any more. Even though my whole home setup is UNIX/Linux based down the switches, wireless AP's and routers which are all Cisco, 3Com or Foundary. .....that reminds me I gota get rid of the 72" Sun rack sitting in my parents living room LOL! Many thanks, Kaya From owner-freebsd-fs@FreeBSD.ORG Sun May 16 13:00:29 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4055A106566C; Sun, 16 May 2010 13:00:29 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from freefall.freebsd.org (unknown [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 1862B8FC0A; Sun, 16 May 2010 13:00:29 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o4GD0SHs040354; Sun, 16 May 2010 13:00:28 GMT (envelope-from mm@freefall.freebsd.org) Received: (from mm@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o4GD0SC4040344; Sun, 16 May 2010 13:00:28 GMT (envelope-from mm) Date: Sun, 16 May 2010 13:00:28 GMT Message-Id: <201005161300.o4GD0SC4040344@freefall.freebsd.org> To: mm@FreeBSD.org, freebsd-fs@FreeBSD.org, mm@FreeBSD.org From: mm@FreeBSD.org Cc: Subject: Re: kern/140433: [zfs] [panic] panic while replaying ZIL after crash X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2010 13:00:29 -0000 Synopsis: [zfs] [panic] panic while replaying ZIL after crash Responsible-Changed-From-To: freebsd-fs->mm Responsible-Changed-By: mm Responsible-Changed-When: Sun May 16 13:00:28 UTC 2010 Responsible-Changed-Why: I'll take it. http://www.freebsd.org/cgi/query-pr.cgi?pr=140433 From owner-freebsd-fs@FreeBSD.ORG Sun May 16 18:41:02 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 34FA91065677 for ; Sun, 16 May 2010 18:41:02 +0000 (UTC) (envelope-from tsw5@duke.edu) Received: from smtp.duke.edu (smtp-04.oit.duke.edu [152.3.174.85]) by mx1.freebsd.org (Postfix) with ESMTP id F19448FC0A for ; Sun, 16 May 2010 18:41:01 +0000 (UTC) Received: from smtp.duke.edu (localhost.localdomain [127.0.0.1]) by localhost (Postfix) with SMTP id A2B00358E87 for ; Sun, 16 May 2010 14:26:21 -0400 (EDT) Received: from shoggoth.wintermute (cpe-066-057-248-190.nc.res.rr.com [66.57.248.190]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.duke.edu (Postfix) with ESMTP id 5D155358E86 for ; Sun, 16 May 2010 14:26:21 -0400 (EDT) From: Todd Wasson Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Sun, 16 May 2010 14:26:19 -0400 Message-Id: <0B97967D-1057-4414-BBD4-4F1AA2659A5D@duke.edu> To: freebsd-fs@freebsd.org Mime-Version: 1.0 (Apple Message framework v1078) X-Mailer: Apple Mail (2.1078) X-PMX-Version: 5.4.2.338381, Antispam-Engine: 2.6.0.325393, Antispam-Data: 2010.5.16.181815 Subject: zfs drive replacement issues X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2010 18:41:02 -0000 Hi everyone, I've run into some problems replacing a problematic drive = in my pool, and am hopeful someone out there can shed some light on = things for me, since reading previous threads and posts around the net = hasn't helped me so far. The story goes like this: for a couple of = years now (since 7.0-RC something) I've had a pool of four devices: two = 400GB drives and two 400GB slices from 500GB drives. I've recently seen = errors with one of the 400GB drives like this: May 11 21:33:08 newmonkey kernel: ad6: TIMEOUT - READ_DMA retrying (1 = retry left) LBA=3D29369344 May 11 21:33:15 newmonkey kernel: ad6: TIMEOUT - READ_DMA retrying (1 = retry left) LBA=3D58819968 May 11 21:33:23 newmonkey kernel: ad6: TIMEOUT - READ_DMA retrying (1 = retry left) LBA=3D80378624 May 11 21:34:01 newmonkey root: ZFS: vdev I/O failure, zpool=3Dtank = path=3D/dev/ad6 offset=3D262144 size=3D8192 error=3D6 May 11 21:34:01 newmonkey kernel: ad6: FAILURE - device detached ...which also led to a bunch of IO errors showing up for that device in = "zpool status" and prompted me to replace that drive. Since finding a = 400GB drive was a pain, I decided to replace it with at 400GB slice from = a new 500GB drive. This is when I made what I think was the first = critical mistake: I forgot to "zpool offline" it before doing the = replacement, so I just exported the pool, physically replaced the drive, = made a 400GB slice on it with fdisk, and, noticing that it now referred = to the old device by an ID number instead of its "ad6" identifier, did a = "zpool replace tank 10022540361666252397 /dev/ad6s1". This actually prompted a scrub for some reason, and not a resilver. I'm = not sure why. However, I noticed that during a scrub I was seeing a lot = of IO errors in "zpool status" on the new device (making me suspect that = maybe the old drive wasn't bad after all, but I think I'll sort that out = afterwards). Additionally, the device won't resilver, and now it's = stuck in a constant state of "replacing". When I try to "zpool detach" = or "zpool offline" either device (old or new) it says there isn't a = replica and refuses. I've finally resorted to putting the original = drive back in to try and make some progress, and now this is what my = zpool status looks like: pool: tank state: DEGRADED scrub: none requested config: NAME STATE READ WRITE CKSUM tank DEGRADED 0 0 0 raidz1 DEGRADED 0 0 0 ad8 ONLINE 0 0 0 ad10s1 ONLINE 0 0 0 ad12s1 ONLINE 0 0 0 replacing DEGRADED 0 0 8 ad6 ONLINE 0 0 0 1667724779240260276 UNAVAIL 0 204 0 was = /dev/ad6s1 When I do "zpool detach tank 1667724779240260276" it says "cannot detach = 1667724779240260276: no valid replicas". It says the same thing for a = "zpool offline tank 1667724779240260276". Note the IO errors in the new = drive (which is now disconnected), which was ad6s1. It could be a bad = controller, a bad cable, or any number of things, but I can't actually = test it because I can't get rid of the device from the zfs pool. So, does anyone have any suggestions? Can I cancel the "replacing" = operation somehow? Do I have to buy a new device, back up the whole = pool, delete it, and rebuild it? Any help is greatly appreciated! Thanks! Todd= From owner-freebsd-fs@FreeBSD.ORG Sun May 16 19:22:03 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2E79F106570A for ; Sun, 16 May 2010 19:22:03 +0000 (UTC) (envelope-from morganw@chemikals.org) Received: from warped.bluecherry.net (unknown [IPv6:2001:440:eeee:fffb::2]) by mx1.freebsd.org (Postfix) with ESMTP id 810A88FC13 for ; Sun, 16 May 2010 19:22:02 +0000 (UTC) Received: from volatile.chemikals.org (adsl-67-243-123.shv.bellsouth.net [98.67.243.123]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by warped.bluecherry.net (Postfix) with ESMTPSA id 43C528093416; Sun, 16 May 2010 14:22:01 -0500 (CDT) Received: from localhost (morganw@localhost [127.0.0.1]) by volatile.chemikals.org (8.14.4/8.14.4) with ESMTP id o4GJLv5r090111; Sun, 16 May 2010 14:21:57 -0500 (CDT) (envelope-from morganw@chemikals.org) Date: Sun, 16 May 2010 14:21:56 -0500 (CDT) From: Wes Morgan X-X-Sender: morganw@volatile To: Todd Wasson In-Reply-To: <0B97967D-1057-4414-BBD4-4F1AA2659A5D@duke.edu> Message-ID: References: <0B97967D-1057-4414-BBD4-4F1AA2659A5D@duke.edu> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: clamav-milter 0.95.3 at warped X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: zfs drive replacement issues X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2010 19:22:03 -0000 On Sun, 16 May 2010, Todd Wasson wrote: > Hi everyone, I've run into some problems replacing a problematic drive in my pool, and am hopeful someone out there can shed some light on things for me, since reading previous threads and posts around the net hasn't helped me so far. The story goes like this: for a couple of years now (since 7.0-RC something) I've had a pool of four devices: two 400GB drives and two 400GB slices from 500GB drives. I've recently seen errors with one of the 400GB drives like this: > > May 11 21:33:08 newmonkey kernel: ad6: TIMEOUT - READ_DMA retrying (1 retry left) LBA=29369344 > May 11 21:33:15 newmonkey kernel: ad6: TIMEOUT - READ_DMA retrying (1 retry left) LBA=58819968 > May 11 21:33:23 newmonkey kernel: ad6: TIMEOUT - READ_DMA retrying (1 retry left) LBA=80378624 > May 11 21:34:01 newmonkey root: ZFS: vdev I/O failure, zpool=tank path=/dev/ad6 offset=262144 size=8192 error=6 > May 11 21:34:01 newmonkey kernel: ad6: FAILURE - device detached > > ...which also led to a bunch of IO errors showing up for that device in > "zpool status" and prompted me to replace that drive. Since finding a > 400GB drive was a pain, I decided to replace it with at 400GB slice from > a new 500GB drive. This is when I made what I think was the first > critical mistake: I forgot to "zpool offline" it before doing the > replacement, so I just exported the pool, physically replaced the drive, > made a 400GB slice on it with fdisk, and, noticing that it now referred > to the old device by an ID number instead of its "ad6" identifier, did a > "zpool replace tank 10022540361666252397 /dev/ad6s1". > > This actually prompted a scrub for some reason, and not a resilver. > I'm not sure why. However, I noticed that during a scrub I was seeing a > lot of IO errors in "zpool status" on the new device (making me suspect > that maybe the old drive wasn't bad after all, but I think I'll sort > that out afterwards). Additionally, the device won't resilver, and now > it's stuck in a constant state of "replacing". When I try to "zpool > detach" or "zpool offline" either device (old or new) it says there > isn't a replica and refuses. I've finally resorted to putting the > original drive back in to try and make some progress, and now this is > what my zpool status looks like: > > pool: tank > state: DEGRADED > scrub: none requested > config: > > NAME STATE READ WRITE CKSUM > tank DEGRADED 0 0 0 > raidz1 DEGRADED 0 0 0 > ad8 ONLINE 0 0 0 > ad10s1 ONLINE 0 0 0 > ad12s1 ONLINE 0 0 0 > replacing DEGRADED 0 0 8 > ad6 ONLINE 0 0 0 > 1667724779240260276 UNAVAIL 0 204 0 was /dev/ad6s1 > > When I do "zpool detach tank 1667724779240260276" it says "cannot detach > 1667724779240260276: no valid replicas". It says the same thing for a > "zpool offline tank 1667724779240260276". Note the IO errors in the new > drive (which is now disconnected), which was ad6s1. It could be a bad > controller, a bad cable, or any number of things, but I can't actually > test it because I can't get rid of the device from the zfs pool. > > So, does anyone have any suggestions? Can I cancel the "replacing" > operation somehow? Do I have to buy a new device, back up the whole > pool, delete it, and rebuild it? Any help is greatly appreciated! Strange, you should be able to cancel the replacement with detach. There is some kind of DTL issue preventing it. I don't know what it is precisely, but there are known cases where legitimate detach operations are prevented. Try reconnecting the disk you partitioned, leaving the original ad6 still attached, and you should be able to detach the new one. Are you planning on using the leftover space on the 500gb drive for something else? If not, then don't bother partitioning it. Just give the whole device to the pool and it will only use 400gb of it. From owner-freebsd-fs@FreeBSD.ORG Sun May 16 19:46:13 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D6E16106566C for ; Sun, 16 May 2010 19:46:13 +0000 (UTC) (envelope-from tsw5@duke.edu) Received: from smtp.duke.edu (smtp-02.oit.duke.edu [152.3.174.84]) by mx1.freebsd.org (Postfix) with ESMTP id AEF7F8FC17 for ; Sun, 16 May 2010 19:46:13 +0000 (UTC) Received: from smtp.duke.edu (localhost.localdomain [127.0.0.1]) by localhost (Postfix) with SMTP id CA270510E76; Sun, 16 May 2010 15:46:12 -0400 (EDT) Received: from shoggoth.wintermute (cpe-066-057-248-190.nc.res.rr.com [66.57.248.190]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.duke.edu (Postfix) with ESMTP id 9A312510E3D; Sun, 16 May 2010 15:46:12 -0400 (EDT) Mime-Version: 1.0 (Apple Message framework v1078) Content-Type: text/plain; charset=us-ascii From: Todd Wasson In-Reply-To: Date: Sun, 16 May 2010 15:46:12 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <6FADBEDC-FF47-498A-BB17-0FCC163D3C8C@duke.edu> References: <0B97967D-1057-4414-BBD4-4F1AA2659A5D@duke.edu> To: Wes Morgan X-Mailer: Apple Mail (2.1078) X-PMX-Version: 5.4.2.338381, Antispam-Engine: 2.6.0.325393, Antispam-Data: 2010.5.16.193615 Cc: freebsd-fs@freebsd.org Subject: Re: zfs drive replacement issues X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2010 19:46:13 -0000 > Strange, you should be able to cancel the replacement with detach. = There > is some kind of DTL issue preventing it. I don't know what it is > precisely, but there are known cases where legitimate detach = operations > are prevented. >=20 > Try reconnecting the disk you partitioned, leaving the original ad6 = still > attached, and you should be able to detach the new one. The problem with that is that I can't have any more than four SATA = devices in my machine. I'd have to detach one of the other (good) = devices in order to do that. Is that feasible / possible? I would = think at that point I would have only two of the four drives in the pool = and it wouldn't function at all, but if that's not the case I can give = it a try... > Are you planning on using the leftover space on the 500gb drive for > something else? If not, then don't bother partitioning it. Just give = the > whole device to the pool and it will only use 400gb of it. I am, probably. In the other 500GB drives I used the extra 100GB to = make another pool and provide a block device to play around with, so I'd = either add it to that or just make it a scratch filesystem. Thanks for your help! Todd= From owner-freebsd-fs@FreeBSD.ORG Sun May 16 20:35:30 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CE9B91065676 for ; Sun, 16 May 2010 20:35:30 +0000 (UTC) (envelope-from morganw@chemikals.org) Received: from warped.bluecherry.net (unknown [IPv6:2001:440:eeee:fffb::2]) by mx1.freebsd.org (Postfix) with ESMTP id 33F3A8FC16 for ; Sun, 16 May 2010 20:35:30 +0000 (UTC) Received: from volatile.chemikals.org (adsl-67-243-123.shv.bellsouth.net [98.67.243.123]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by warped.bluecherry.net (Postfix) with ESMTPSA id 216A68093416; Sun, 16 May 2010 15:35:27 -0500 (CDT) Received: from localhost (morganw@localhost [127.0.0.1]) by volatile.chemikals.org (8.14.4/8.14.4) with ESMTP id o4GKZNQ8010062; Sun, 16 May 2010 15:35:23 -0500 (CDT) (envelope-from morganw@chemikals.org) Date: Sun, 16 May 2010 15:35:23 -0500 (CDT) From: Wes Morgan X-X-Sender: morganw@volatile To: Todd Wasson In-Reply-To: <6FADBEDC-FF47-498A-BB17-0FCC163D3C8C@duke.edu> Message-ID: References: <0B97967D-1057-4414-BBD4-4F1AA2659A5D@duke.edu> <6FADBEDC-FF47-498A-BB17-0FCC163D3C8C@duke.edu> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: clamav-milter 0.95.3 at warped X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: zfs drive replacement issues X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2010 20:35:30 -0000 On Sun, 16 May 2010, Todd Wasson wrote: > > Strange, you should be able to cancel the replacement with detach. There > > is some kind of DTL issue preventing it. I don't know what it is > > precisely, but there are known cases where legitimate detach operations > > are prevented. > > > > Try reconnecting the disk you partitioned, leaving the original ad6 still > > attached, and you should be able to detach the new one. > > The problem with that is that I can't have any more than four SATA > devices in my machine. I'd have to detach one of the other (good) > devices in order to do that. Is that feasible / possible? I would > think at that point I would have only two of the four drives in the pool > and it wouldn't function at all, but if that's not the case I can give > it a try... I wouldn't recommend that. What about attaching it externally, via eSATA or USB? From owner-freebsd-fs@FreeBSD.ORG Sun May 16 20:37:58 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 53DB9106564A for ; Sun, 16 May 2010 20:37:58 +0000 (UTC) (envelope-from numisemis@yahoo.com) Received: from web112409.mail.gq1.yahoo.com (web112409.mail.gq1.yahoo.com [98.137.26.153]) by mx1.freebsd.org (Postfix) with SMTP id 208848FC18 for ; Sun, 16 May 2010 20:37:57 +0000 (UTC) Received: (qmail 88619 invoked by uid 60001); 16 May 2010 20:37:57 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1274042277; bh=vJCu3wov7Ie7SOPvcW9J2dbDoY8LVvNwBAMgCEc/VIk=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=Q+wlmF2c1KUB7yM4/LN6bSEWlaWErqy1CVgjlPVgGAHs6+emhBVlJGQdSksu1+vuSUNExXfbJfnMqTTUwAMNLcQRPOC6sfrVrZYb8qwpP5KyWeakHJrT3coI8GD61xV1Kww+GodxsYpVq6hwhyJM/tcE067eWx1ihlb8WKfnEFQ= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=GMV5gm7fzguqOmJbQsqBBsmZPBCkrZ7Mu/wpiDVX7kpqmKvgDAQ3BOqTNTshiseMA5++DE6YEcwWFzCWn1nKq434vrr2w2iVdrna96jhAX3SycNzqMmW9c+7tJhduQKgAaycdvaRLZ9FgmnRGG4lFuRwzjfoYG/rdIIDKJG9+Q0=; Message-ID: <657328.88413.qm@web112409.mail.gq1.yahoo.com> X-YMail-OSG: 5i1eb4AVM1lKirK6eRqlZINIOvkhNlRo4yhh0x5bTL1qOAm 6bhNaSOlfmBLDBNfEbvh3mE7BFyR9lbUhqlYmMf6_aUOMiIDHPEfY9J2rtPt hvPCvrOpQ0zPK9C7zXbQ8pq5uSjVF7Ub7ha5cndbhrXJyaPvLkC49woU17Uo FjBuztvltsTpqPKX0GvPHJtW4XQn..8imiJcJMaTBW.OA3Pfx93eJypTolEJ cg_DUWSeGg8pXdqoirVxfTl6EpIUfaejURv0xV3EbOwqEA.BVJr4WMMu3vpC xrZIu8NsL_bR42oCIVGTvHIiZ70vPdvcVFzAjGZuNau648MKVcJ4szPcRDBB Bf3pxU.23SGzR1hXRxfFqaEmXbQe7jA-- Received: from [188.125.24.109] by web112409.mail.gq1.yahoo.com via HTTP; Sun, 16 May 2010 13:37:57 PDT X-Mailer: YahooMailRC/374.4 YahooMailWebService/0.8.103.269680 References: <4BEF2F9C.7080409@netscape.net> <4BEF3137.4080203@netscape.net> <20100516001351.GA50879@icarus.home.lan> <4BEF4A73.8060905@netscape.net> <20100516015850.GA55302@icarus.home.lan> Date: Sun, 16 May 2010 13:37:57 -0700 (PDT) From: Simun Mikecin To: Jeremy Chadwick In-Reply-To: <20100516015850.GA55302@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-fs@freebsd.org Subject: Re: Quick ZFS mirroring question for non-mirrored pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2010 20:37:58 -0000 ----- Original Message ---- > From: Jeremy Chadwick > To: Kaya Saman > Cc: freebsd-fs@freebsd.org > Sent: Sun, May 16, 2010 3:58:50 AM > Subject: Re: Quick ZFS mirroring question for non-mirrored pool > > Also, please reconsider going with Western Digital > RE4 2TB drives. These drives are all "GP" (Green Power) drives, which you do > not want. There have been numerous reports on the FreeBSD mailing lists > about problems with these drives (repeated head offloading/parking > causing problems in RAID arrays), and yes, it applies to Enterprise class > drive as well; WD has indirectly confirmed the problem in one user's case > by sending him a "fixed" firmware. I can point you to threads if you > want to read them. Well, there are two types of WD RE4 enterprise drives: "RE4" and "RE4 GP". Only "RE 4GP" is a "Green Power" drive. "RE4" is a performance drive spinning at 7200rpm (and I believe it is currently fastest enterprise 2TB drive on the market). From owner-freebsd-fs@FreeBSD.ORG Sun May 16 20:45:06 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1FDD1106564A for ; Sun, 16 May 2010 20:45:06 +0000 (UTC) (envelope-from tsw5@duke.edu) Received: from smtp.duke.edu (smtp-01.oit.duke.edu [152.3.174.14]) by mx1.freebsd.org (Postfix) with ESMTP id EA50A8FC13 for ; Sun, 16 May 2010 20:45:05 +0000 (UTC) Received: from smtp.duke.edu (localhost.localdomain [127.0.0.1]) by localhost (Postfix) with SMTP id 00C42308F77; Sun, 16 May 2010 16:45:05 -0400 (EDT) Received: from shoggoth.wintermute (cpe-066-057-248-190.nc.res.rr.com [66.57.248.190]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.duke.edu (Postfix) with ESMTP id C8F41308F75; Sun, 16 May 2010 16:45:04 -0400 (EDT) Mime-Version: 1.0 (Apple Message framework v1078) Content-Type: text/plain; charset=us-ascii From: Todd Wasson In-Reply-To: Date: Sun, 16 May 2010 16:45:04 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <0BE564E4-3B13-41A5-9A38-7D5AE0C983BF@duke.edu> References: <0B97967D-1057-4414-BBD4-4F1AA2659A5D@duke.edu> <6FADBEDC-FF47-498A-BB17-0FCC163D3C8C@duke.edu> To: Wes Morgan X-Mailer: Apple Mail (2.1078) X-PMX-Version: 5.4.2.338381, Antispam-Engine: 2.6.0.325393, Antispam-Data: 2010.5.16.203619 Cc: freebsd-fs@freebsd.org Subject: Re: zfs drive replacement issues X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2010 20:45:06 -0000 >>> Strange, you should be able to cancel the replacement with detach. = There >>> is some kind of DTL issue preventing it. I don't know what it is >>> precisely, but there are known cases where legitimate detach = operations >>> are prevented. >>>=20 >>> Try reconnecting the disk you partitioned, leaving the original ad6 = still >>> attached, and you should be able to detach the new one. >>=20 >> The problem with that is that I can't have any more than four SATA >> devices in my machine. I'd have to detach one of the other (good) >> devices in order to do that. Is that feasible / possible? I would >> think at that point I would have only two of the four drives in the = pool >> and it wouldn't function at all, but if that's not the case I can = give >> it a try... >=20 > I wouldn't recommend that. What about attaching it externally, via = eSATA > or USB? Unfortunately, my machine doesn't support eSATA and I don't have a USB = to SATA adapter. I can pick one up if it'll help me solve this, though, = but one related issue: will the device be identified as the device that = used to be on ad6 (or ad6s1) once it's attached via another means, like = a USB adapter? I suspect it will (or that I can issue a command to tell = the zpool about it, anyway) but I thought I'd double check... Todd= From owner-freebsd-fs@FreeBSD.ORG Sun May 16 21:01:52 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CF439106566B for ; Sun, 16 May 2010 21:01:52 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta11.emeryville.ca.mail.comcast.net (qmta11.emeryville.ca.mail.comcast.net [76.96.27.211]) by mx1.freebsd.org (Postfix) with ESMTP id B2D968FC14 for ; Sun, 16 May 2010 21:01:52 +0000 (UTC) Received: from omta19.emeryville.ca.mail.comcast.net ([76.96.30.76]) by qmta11.emeryville.ca.mail.comcast.net with comcast id JLFJ1e0021eYJf8ABM1t2i; Sun, 16 May 2010 21:01:53 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta19.emeryville.ca.mail.comcast.net with comcast id JM1s1e00H3S48mS01M1suT; Sun, 16 May 2010 21:01:53 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 5EF189B419; Sun, 16 May 2010 14:01:51 -0700 (PDT) Date: Sun, 16 May 2010 14:01:51 -0700 From: Jeremy Chadwick To: Simun Mikecin Message-ID: <20100516210151.GA82487@icarus.home.lan> References: <4BEF2F9C.7080409@netscape.net> <4BEF3137.4080203@netscape.net> <20100516001351.GA50879@icarus.home.lan> <4BEF4A73.8060905@netscape.net> <20100516015850.GA55302@icarus.home.lan> <657328.88413.qm@web112409.mail.gq1.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <657328.88413.qm@web112409.mail.gq1.yahoo.com> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-fs@freebsd.org Subject: Re: Quick ZFS mirroring question for non-mirrored pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2010 21:01:53 -0000 On Sun, May 16, 2010 at 01:37:57PM -0700, Simun Mikecin wrote: > ----- Original Message ---- > > From: Jeremy Chadwick > > To: Kaya Saman > > Cc: freebsd-fs@freebsd.org > > Sent: Sun, May 16, 2010 3:58:50 AM > > Subject: Re: Quick ZFS mirroring question for non-mirrored pool > > > > Also, please reconsider going with Western Digital > > RE4 2TB drives. These drives are all "GP" (Green Power) drives, which you do > > not want. There have been numerous reports on the FreeBSD mailing lists > > about problems with these drives (repeated head offloading/parking > > causing problems in RAID arrays), and yes, it applies to Enterprise class > > drive as well; WD has indirectly confirmed the problem in one user's case > > by sending him a "fixed" firmware. I can point you to threads if you > > want to read them. > > > Well, there are two types of WD RE4 enterprise drives: "RE4" and "RE4 > GP". Only "RE 4GP" is a "Green Power" drive. "RE4" is a performance > drive spinning at 7200rpm (and I believe it is currently fastest > enterprise 2TB drive on the market). You're absolutely right. I reviewed WDC's site before making my statement, but the way they do their layout is a bit confusing. "RE4" appears in the 3rd column, while "RE4-GP" appears in its own column, since they segregate it due to its lower power usage: http://www.wdc.com/en/products/index.asp?cat=2 The price difference between these drives is US$50 (RE4 costing more): WDC RE4: http://www.wdc.com/en/products/Products.asp?DriveID=732 WDC CBk: http://www.wdc.com/en/products/Products.asp?DriveID=733 What do people get in exchange for paying 1/5th more in cost? Basically nothing: * TLER is usable / adjustable on both RE4 and Black drives. It defaults to disabled on the Black. Some tech forum weirdos state WD disabled the ability to adjust the feature on later Black drives; I've personally confirmed that's not true (said weirdos probably forgot to disable AHCI when using the DOS utility to adjust TLER). This is probably the most useful feature for these drives when used in RAID or a RAID-like (ZFS) fashion. * RAFF sounds like marketing schmooze, given that both the RE4 and Black drives already provide an extra-stable motor shaft ("StableTrac"). Yes, I've seen the Youtube video of an administrator showing the effects of vibration on array performance/stability; I've a hard time believing this would matter in the OP's system. * PMR has been in use since 2005 on all types of drives, across multiple vendors. There's nothing amazing about it at this point. * More marketing: the Caviar Black lists "dual processors". Whatever. * As has been proven time and time again, MTBF means jack squat since it's all hypothetical (mathematically calculated based on fab tests). Drives will fail no matter what; that's the entire reason people are using ZFS to begin with. ;-) The only feature that sounds even remotely useful is the "dynamic fly height" feature of the RE4, and it's not worth US$50. I have to wonder if marketing these days has found a way to pray on obsessive-compulsive disorder or what... -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Sun May 16 21:24:00 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CDEA0106566B for ; Sun, 16 May 2010 21:24:00 +0000 (UTC) (envelope-from morganw@chemikals.org) Received: from warped.bluecherry.net (unknown [IPv6:2001:440:eeee:fffb::2]) by mx1.freebsd.org (Postfix) with ESMTP id 5FDF28FC1C for ; Sun, 16 May 2010 21:24:00 +0000 (UTC) Received: from volatile.chemikals.org (adsl-67-243-123.shv.bellsouth.net [98.67.243.123]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by warped.bluecherry.net (Postfix) with ESMTPSA id 823C686D9B26; Sun, 16 May 2010 16:23:59 -0500 (CDT) Received: from localhost (morganw@localhost [127.0.0.1]) by volatile.chemikals.org (8.14.4/8.14.4) with ESMTP id o4GLNugx023337; Sun, 16 May 2010 16:23:56 -0500 (CDT) (envelope-from morganw@chemikals.org) Date: Sun, 16 May 2010 16:23:56 -0500 (CDT) From: Wes Morgan X-X-Sender: morganw@volatile To: Todd Wasson In-Reply-To: <0BE564E4-3B13-41A5-9A38-7D5AE0C983BF@duke.edu> Message-ID: References: <0B97967D-1057-4414-BBD4-4F1AA2659A5D@duke.edu> <6FADBEDC-FF47-498A-BB17-0FCC163D3C8C@duke.edu> <0BE564E4-3B13-41A5-9A38-7D5AE0C983BF@duke.edu> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: clamav-milter 0.95.3 at warped X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: zfs drive replacement issues X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2010 21:24:00 -0000 On Sun, 16 May 2010, Todd Wasson wrote: > >>> Strange, you should be able to cancel the replacement with detach. There > >>> is some kind of DTL issue preventing it. I don't know what it is > >>> precisely, but there are known cases where legitimate detach operations > >>> are prevented. > >>> > >>> Try reconnecting the disk you partitioned, leaving the original ad6 still > >>> attached, and you should be able to detach the new one. > >> > >> The problem with that is that I can't have any more than four SATA > >> devices in my machine. I'd have to detach one of the other (good) > >> devices in order to do that. Is that feasible / possible? I would > >> think at that point I would have only two of the four drives in the pool > >> and it wouldn't function at all, but if that's not the case I can give > >> it a try... > > > > I wouldn't recommend that. What about attaching it externally, via eSATA > > or USB? > > Unfortunately, my machine doesn't support eSATA and I don't have a USB > to SATA adapter. I can pick one up if it'll help me solve this, though, > but one related issue: will the device be identified as the device that > used to be on ad6 (or ad6s1) once it's attached via another means, like > a USB adapter? I suspect it will (or that I can issue a command to tell > the zpool about it, anyway) but I thought I'd double check... Via USB, it will show up as a daX device, and it should be recognized without problem by zfs. You may need to export and import the pool, though. From owner-freebsd-fs@FreeBSD.ORG Mon May 17 01:27:54 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B7B181065670; Mon, 17 May 2010 01:27:54 +0000 (UTC) (envelope-from staalebk@ifi.uio.no) Received: from putsch.kolbu.ws (cl-225.sto-01.se.sixxs.net [IPv6:2001:16d8:ff00:e0::2]) by mx1.freebsd.org (Postfix) with ESMTP id 74F758FC0A; Mon, 17 May 2010 01:27:54 +0000 (UTC) Received: from chiller by putsch.kolbu.ws with local (Exim 4.71 (FreeBSD)) (envelope-from ) id 1ODp7Y-000E89-KS; Mon, 17 May 2010 03:27:52 +0200 Date: Mon, 17 May 2010 03:27:52 +0200 From: =?iso-8859-1?Q?St=E5le?= Kristoffersen To: Pawel Jakub Dawidek Message-ID: <20100517012752.GA3943@putsch.kolbu.ws> References: <20100506012217.GA41806@putsch.kolbu.ws> <20100512102156.GE1703@garage.freebsd.pl> <20100512110803.GF1703@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20100512110803.GF1703@garage.freebsd.pl> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-fs@freebsd.org Subject: Re: Bad hardware + zfs = panic X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 01:27:54 -0000 On 2010-05-12 at 13:08, Pawel Jakub Dawidek wrote: > On Wed, May 12, 2010 at 12:21:56PM +0200, Pawel Jakub Dawidek wrote: > > Well, I don't think it should be possible for vdev to be NULL. > > But if you still have this panic, can you try this patch: > > > > http://people.freebsd.org/~pjd/patches/vdev_mirror.c.patch Yeah, it shouldn't be possible, but I had something in my system that corrupted data in memory, and that can lead to all sorts of problems. I'm not blaming ZFS for not handling 'impossible' situations, but this seemed like something that could be avoided. I'm actually impressed with how well ZFS handled it, my UFS root-fs went ballistic a few times. > It looks like: > > http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6435666 > > The work-around is to remove /boot/zfs/zpool.cache and import the pool > again. Fortunately I found out which file was problematic, and after removing it and recreating it, I've had no more panics. I'm not sure if the tip on that page would have solved the problem, because zpool.cache is removed on my system when exporting the pool. Thanks for the help tho, everything looks to be working now :) -- Ståle Kristoffersen staalebk@ifi.uio.no From owner-freebsd-fs@FreeBSD.ORG Mon May 17 01:36:06 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AD4FB1065673 for ; Mon, 17 May 2010 01:36:06 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-pz0-f174.google.com (mail-pz0-f174.google.com [209.85.222.174]) by mx1.freebsd.org (Postfix) with ESMTP id 7E10B8FC17 for ; Mon, 17 May 2010 01:36:06 +0000 (UTC) Received: by pzk4 with SMTP id 4so2426714pzk.7 for ; Sun, 16 May 2010 18:36:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type; bh=mh5Ya+DVD4lUAgjypMDvmLLaxEPo5hkrJSMRZoRMhMA=; b=Mp95yq+DXdcUHS+Cd9yW34+HC/8dXZiYp1/sZlGuwmZhdUWtbdHZOLqt3AjFuVqv09 XkcTJcmSiBMCMEHC4RdLPvnJPQRl13vefqvURDgJrTFPNl3y4NP3Sq8zKBp+vB7Gz/B+ I2mux/jCxEn2Nwfp0R+o5js+GWMVifn5d6fNY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=qRLFgUvGzteAGrV5G2DUtoNoH5syS94r7cjKm/4JYEPU1rcDv3frmROcCX/fO3yQFA 4sojC67Nf97RPuS6Uj66x3UsTQmIFn1TVKrFQA/PEHwm9Xsf5PBGO4SseQRaplNK4cWb Em1ZjjC0d+GNTImpG31ci1DfHE4TyC0KjNSP8= MIME-Version: 1.0 Received: by 10.142.247.7 with SMTP id u7mr2920622wfh.95.1274060165777; Sun, 16 May 2010 18:36:05 -0700 (PDT) Received: by 10.143.41.20 with HTTP; Sun, 16 May 2010 18:36:05 -0700 (PDT) In-Reply-To: References: <0B97967D-1057-4414-BBD4-4F1AA2659A5D@duke.edu> <6FADBEDC-FF47-498A-BB17-0FCC163D3C8C@duke.edu> <0BE564E4-3B13-41A5-9A38-7D5AE0C983BF@duke.edu> Date: Sun, 16 May 2010 18:36:05 -0700 Message-ID: From: Freddie Cash To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: zfs drive replacement issues X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 01:36:06 -0000 On Sun, May 16, 2010 at 2:23 PM, Wes Morgan wrote: > On Sun, 16 May 2010, Todd Wasson wrote: > > > >>> Strange, you should be able to cancel the replacement with detach. > There > > >>> is some kind of DTL issue preventing it. I don't know what it is > > >>> precisely, but there are known cases where legitimate detach > operations > > >>> are prevented. > > >>> > > >>> Try reconnecting the disk you partitioned, leaving the original ad6 > still > > >>> attached, and you should be able to detach the new one. > > >> > > >> The problem with that is that I can't have any more than four SATA > > >> devices in my machine. I'd have to detach one of the other (good) > > >> devices in order to do that. Is that feasible / possible? I would > > >> think at that point I would have only two of the four drives in the > pool > > >> and it wouldn't function at all, but if that's not the case I can give > > >> it a try... > > > > > > I wouldn't recommend that. What about attaching it externally, via > eSATA > > > or USB? > > > > Unfortunately, my machine doesn't support eSATA and I don't have a USB > > to SATA adapter. I can pick one up if it'll help me solve this, though, > > but one related issue: will the device be identified as the device that > > used to be on ad6 (or ad6s1) once it's attached via another means, like > > a USB adapter? I suspect it will (or that I can issue a command to tell > > the zpool about it, anyway) but I thought I'd double check... > > Via USB, it will show up as a daX device, and it should be recognized > without problem by zfs. You may need to export and import the pool, > though. > > Or, plug the old drive back into the same spot it was plugged into before, so it shows up as ad6. And plug in the new drive via USB. If you manage to get the pool back into a consistent state, then you can export, plug the new drive into the SATA port, and re-import the pool. ZFS will find the drive correctly and adjust things as needed. -- Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@FreeBSD.ORG Mon May 17 03:57:08 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4CAF0106564A; Mon, 17 May 2010 03:57:08 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (unknown [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 102B58FC12; Mon, 17 May 2010 03:57:08 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o4H3v7mh009193; Mon, 17 May 2010 03:57:07 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o4H3v7fD009189; Mon, 17 May 2010 03:57:07 GMT (envelope-from linimon) Date: Mon, 17 May 2010 03:57:07 GMT Message-Id: <201005170357.o4H3v7fD009189@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-i386@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/146528: [zfs] Severe memory leak in ZFS on i386 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 03:57:08 -0000 Old Synopsis: Severe memory leak in ZFS on i386 New Synopsis: [zfs] Severe memory leak in ZFS on i386 Responsible-Changed-From-To: freebsd-i386->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Mon May 17 03:56:07 UTC 2010 Responsible-Changed-Why: Although this may be i386-related, the problem is most likely not in the i386 machine-dependent code, but in ZFS. http://www.freebsd.org/cgi/query-pr.cgi?pr=146528 From owner-freebsd-fs@FreeBSD.ORG Mon May 17 06:11:48 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 39EFE106566C for ; Mon, 17 May 2010 06:11:48 +0000 (UTC) (envelope-from alex.bakhtin@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id C111A8FC16 for ; Mon, 17 May 2010 06:11:47 +0000 (UTC) Received: by fxm19 with SMTP id 19so464785fxm.13 for ; Sun, 16 May 2010 23:11:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=vnqYK4LQlWZopsokcPAnjIM38N154iiKiZUXrCfdKHo=; b=nsClROEwgu8i85V38beRZKXYcKZpnfYMJPkLLfSPELu7wt5ti4Bt7lAPrA2fINjJtv Z7Bv+mraRJfvU+Ktgas/C+6mlJ7YoSZN7ZVXp/PmCx/fLEug7+3bzTuUWTyRtKbGuh0E N5oHZ5zy653r0zWa92xvkvuImgwJfNFvhB9CY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=M2HfCRwVVYbqN7ryxfFmWV8J64/ZfJTtjCXK4hJ0k2KWwk5swmzlQnJmvT7EJ38s8V 4glNahdJbJcLO0JxospmFppMpeuXt4hpGIgsZ+0tr9kt2X/uzxlY+HWJ5dbSUlGsD4cW PdM2sFnWZshSmspCJnHY7ZQPH7Lb75w37mnHo= MIME-Version: 1.0 Received: by 10.204.175.14 with SMTP id v14mr25403bkz.72.1274074957731; Sun, 16 May 2010 22:42:37 -0700 (PDT) Received: by 10.204.71.139 with HTTP; Sun, 16 May 2010 22:42:37 -0700 (PDT) In-Reply-To: <201005130934.o4D9YiJL039462@freefall.freebsd.org> References: <201005130934.o4D9YiJL039462@freefall.freebsd.org> Date: Mon, 17 May 2010 09:42:37 +0400 Message-ID: From: Alex Bakhtin To: pjd@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: kern/145339: [zfs] deadlock after detaching block device from raidz pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 06:11:48 -0000 Pawel, I tested your patch in the following zfs configuration (all on 5x2TB WD20EARS drivers): 1. raidz1 on top of physical disks. 2. raidz1 on top of geli 3. raidz2 on top of physical disks. In all three cases it seems that the problem was fixed - I can't crash zfs in vdev_geom when unplugging the disk. Unfortunately, 3 times I got a deadlock in zfs after plugging vdevs back under load. It happens several seconds after zpool online command. I'm not 100 percent sure that deadlocks are related to this patch, but... I'm going to make some additional testing with patched and not patched kernels. Alex Bakhtin 2010/5/13 : > Synopsis: [zfs] deadlock after detaching block device from raidz pool > > State-Changed-From-To: open->feedback > State-Changed-By: pjd > State-Changed-When: czw 13 maj 2010 09:33:20 UTC > State-Changed-Why: > Could you try this patch: > > =A0 =A0 =A0 =A0http://people.freebsd.org/~pjd/patches/vdev_geom.c.3.patch From owner-freebsd-fs@FreeBSD.ORG Mon May 17 07:10:32 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E9BDA106564A for ; Mon, 17 May 2010 07:10:31 +0000 (UTC) (envelope-from mike.barnardq@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 70AD68FC08 for ; Mon, 17 May 2010 07:10:30 +0000 (UTC) Received: by fxm19 with SMTP id 19so506202fxm.13 for ; Mon, 17 May 2010 00:10:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=U2yzi/jEWCN4A5+HNyXEsmhfzCAZma63krHYWz4W8DU=; b=huF4ZgDs7IDpQMVPquj+Ish8ziYL/zzJMylLP24m1EOjvkGqgRwkhPhTUSfgiOFdYN aLwvB+kCC62B2ppAbz0G6p6N37VEc88uViASlnatgWosxzhgp4wbTg2d63Vb0Ig95N6S elGtbXqnXFIlraAmwpuDhGhmEMdjSglAtTd/g= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=WLBNOVlnqWh3UIHENiOPgiNoadnxPmjtmU8fVmNrurW8lXk9HACtYM7kf6CmqFSenL D+hUVEvPA8vW+f65iCwZS43HMOnAa/P9LlO4GRPnK/Wg+R37cGaTFbDPl9IKMRzLfsFM UwyV+o0ViIWEbs/jgL/OymvcG6JTLUs+ccNII= MIME-Version: 1.0 Received: by 10.223.46.135 with SMTP id j7mr5645490faf.105.1274078610816; Sun, 16 May 2010 23:43:30 -0700 (PDT) Received: by 10.223.105.146 with HTTP; Sun, 16 May 2010 23:43:31 -0700 (PDT) Date: Mon, 17 May 2010 09:43:31 +0300 Message-ID: From: Mike Barnard To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: UFS Journaling gone south X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 07:10:32 -0000 Hi, apologies for cross posting. I had posted this to freebsd-questions before. I believe this should have been the right place to post this. I'm trying out gjournal before I implement if on one server. I require more than 8 partitions, but since I cannot do this, have 9 partitions on one slice, I have created two slices on the disk, da0s1 (100GB) and da0s2 (40GB). On the first slice, I have my usual partitions. On the second slice, I have two partitions, each 20GB (da0s2d and da0s2e) that will be used as the journal providers. I need to journaled two partitions /usr (da0s1f) and /resource (da0s1g). I have done the following to get my data providers and journal providers -- boot into single usermode -- unmount /usr and /resource -- gjournal load -- gjournal label -f da0s1f da0s2d -- gjournal label -f da0s1g da0s2e -- tunefs -J enable -n disable da0s1f.journal -- tunefs -J enable -n disable da0s1g.journal -- mount /dev/da0s1f.journal /usr and mount /dev/da0s1g.journal /resource. Each mount with -o async -- edited fstab to mount the journal devices "/dev/da0s1f.journal /usr ufs rw,async 2 2" "/dev/da0s1g.journal /resource ufs rw,async 2 2" -- edited loader.conf and added geom_journal_load="YES" If I enter ctrl+d, I continue to multi usermode with no problem. However, after a reboot, I get the messages below when it tries to mount the partitions: Root mount waiting for: GJOURNAL GJOURNAL Root mount waiting for: GJOURNAL GJOURNAL GEOM_JOURNAL: Timeout. Journal gjournal 3033687591 cannot be completed GEOM_JOURNAL: Journal 3033687591 : da0s1f contains data. Root mount waiting for: GJOURNAL GJOURNAL Root mount waiting for: GJOURNAL GJOURNAL GEOM_JOURNAL: Timeout. Journal gjournal 107992178 cannot be completed GEOM_JOURNAL: Journal 107992178 : da0s1g contains data. Root mount waiting for: GJOURNAL GJOURNAL Root mount waiting for: GJOURNAL GJOURNAL GEOM_JOURNAL: Timeout. Journal gjournal 3033687591 cannot be completed GEOM_JOURNAL: Journal 3033687591 : ufsid/4bed9437003f40f4 contains data. Root mount waiting for: GJOURNAL GJOURNAL Root mount waiting for: GJOURNAL GJOURNAL GEOM_JOURNAL: Timeout. Journal gjournal 3033687591 cannot be completed GEOM_JOURNAL: Journal 3033687591 : ufsid/4bed9437dfb979f4 contains data. This goes on and on... I cannot go beyond this point. Did I miss something? PS: I started off with a journal provider partition of 5GB and increased all the way to 20GB testing whether my journal sizes were too small. This was after I googled and read that this error will occur if the journal provider size is small. I have attempted this with the journal provider partions on the first slice, da0s1 and also on the second sloce da0s2. All get me the above error. PPS: I am doing this on FreeBSD 8.0 RELEASE AMD64 -- Mike Of course, you might discount this possibility, but remember that one in a million chances happen 99% of the time. ------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Mon May 17 07:37:29 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B5BE41065670 for ; Mon, 17 May 2010 07:37:29 +0000 (UTC) (envelope-from stark@mapper.nl) Received: from smtp-out1.tiscali.nl (smtp-out1.tiscali.nl [195.241.79.176]) by mx1.freebsd.org (Postfix) with ESMTP id 456B48FC13 for ; Mon, 17 May 2010 07:37:29 +0000 (UTC) Received: from [82.170.17.27] (helo=mapper.nl) by smtp-out1.tiscali.nl with esmtp (Exim) (envelope-from ) id 1ODutE-0004jv-7A; Mon, 17 May 2010 09:37:28 +0200 Received: from [10.58.235.50] by mapper.nl with esmtp (Exim 4.69 (FreeBSD)) (envelope-from ) id 1ODutA-000569-Co; Mon, 17 May 2010 09:37:24 +0200 Message-ID: <4BF0F231.9000706@mapper.nl> Date: Mon, 17 May 2010 09:37:21 +0200 From: Mark Stapper User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: Todd Wasson References: <0B97967D-1057-4414-BBD4-4F1AA2659A5D@duke.edu> In-Reply-To: <0B97967D-1057-4414-BBD4-4F1AA2659A5D@duke.edu> X-Enigmail-Version: 1.0.1 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigA5740296903094CE5EABB9DB" Cc: freebsd-fs@freebsd.org Subject: Re: zfs drive replacement issues X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 07:37:29 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigA5740296903094CE5EABB9DB Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 16/05/2010 20:26, Todd Wasson wrote: > Hi everyone, I've run into some problems replacing a problematic drive = in my pool, and am hopeful someone out there can shed some light on thing= s for me, since reading previous threads and posts around the net hasn't = helped me so far. The story goes like this: for a couple of years now (s= ince 7.0-RC something) I've had a pool of four devices: two 400GB drives = and two 400GB slices from 500GB drives. I've recently seen errors with o= ne of the 400GB drives like this: > > May 11 21:33:08 newmonkey kernel: ad6: TIMEOUT - READ_DMA retrying (1 r= etry left) LBA=3D29369344 > May 11 21:33:15 newmonkey kernel: ad6: TIMEOUT - READ_DMA retrying (1 r= etry left) LBA=3D58819968 > May 11 21:33:23 newmonkey kernel: ad6: TIMEOUT - READ_DMA retrying (1 r= etry left) LBA=3D80378624 > May 11 21:34:01 newmonkey root: ZFS: vdev I/O failure, zpool=3Dtank pat= h=3D/dev/ad6 offset=3D262144 size=3D8192 error=3D6 > May 11 21:34:01 newmonkey kernel: ad6: FAILURE - device detached > > ...which also led to a bunch of IO errors showing up for that device in= "zpool status" and prompted me to replace that drive. Since finding a 4= 00GB drive was a pain, I decided to replace it with at 400GB slice from a= new 500GB drive. This is when I made what I think was the first critica= l mistake: I forgot to "zpool offline" it before doing the replacement, s= o I just exported the pool, physically replaced the drive, made a 400GB s= lice on it with fdisk, and, noticing that it now referred to the old devi= ce by an ID number instead of its "ad6" identifier, did a "zpool replace = tank 10022540361666252397 /dev/ad6s1". > > This actually prompted a scrub for some reason, and not a resilver. I'= m not sure why. However, I noticed that during a scrub I was seeing a lo= t of IO errors in "zpool status" on the new device (making me suspect tha= t maybe the old drive wasn't bad after all, but I think I'll sort that ou= t afterwards). Additionally, the device won't resilver, and now it's stu= ck in a constant state of "replacing". When I try to "zpool detach" or "= zpool offline" either device (old or new) it says there isn't a replica a= nd refuses. I've finally resorted to putting the original drive back in = to try and make some progress, and now this is what my zpool status looks= like: > > pool: tank > state: DEGRADED > scrub: none requested > config: > > NAME STATE READ WRITE CKSUM > tank DEGRADED 0 0 0 > raidz1 DEGRADED 0 0 0 > ad8 ONLINE 0 0 0 > ad10s1 ONLINE 0 0 0 > ad12s1 ONLINE 0 0 0 > replacing DEGRADED 0 0 8 > ad6 ONLINE 0 0 0 > 1667724779240260276 UNAVAIL 0 204 0 was /dev= /ad6s1 > > When I do "zpool detach tank 1667724779240260276" it says "cannot detac= h 1667724779240260276: no valid replicas". It says the same thing for a = "zpool offline tank 1667724779240260276". Note the IO errors in the new = drive (which is now disconnected), which was ad6s1. It could be a bad co= ntroller, a bad cable, or any number of things, but I can't actually test= it because I can't get rid of the device from the zfs pool. > > So, does anyone have any suggestions? Can I cancel the "replacing" ope= ration somehow? Do I have to buy a new device, back up the whole pool, d= elete it, and rebuild it? Any help is greatly appreciated! > > Thanks! > > > Todd_______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > =20 Hello, You could try exporting and importing the pool with three disks. Then make sure the "new" drive isn't part of any zpool (low-level format?= ). Then try a "replace" again. Have fun! --------------enigA5740296903094CE5EABB9DB Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkvw8jUACgkQN9xNqOOVnWAhcgCeP7JLQNWTFp97vBPJh29FUuQM b0AAnjWQYqTIIf4bBz8MuYiLK1EiJp2y =zOQN -----END PGP SIGNATURE----- --------------enigA5740296903094CE5EABB9DB-- From owner-freebsd-fs@FreeBSD.ORG Mon May 17 08:33:41 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 011D31065679 for ; Mon, 17 May 2010 08:33:41 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta13.emeryville.ca.mail.comcast.net (qmta13.emeryville.ca.mail.comcast.net [76.96.27.243]) by mx1.freebsd.org (Postfix) with ESMTP id DE30D8FC2B for ; Mon, 17 May 2010 08:33:40 +0000 (UTC) Received: from omta08.emeryville.ca.mail.comcast.net ([76.96.30.12]) by qmta13.emeryville.ca.mail.comcast.net with comcast id JYZh1e0010FhH24ADYZhDb; Mon, 17 May 2010 08:33:41 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta08.emeryville.ca.mail.comcast.net with comcast id JYZg1e0083S48mS8UYZgaC; Mon, 17 May 2010 08:33:41 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 7F82D9B419; Mon, 17 May 2010 01:33:39 -0700 (PDT) Date: Mon, 17 May 2010 01:33:39 -0700 From: Jeremy Chadwick To: Mark Stapper Message-ID: <20100517083339.GA98532@icarus.home.lan> References: <0B97967D-1057-4414-BBD4-4F1AA2659A5D@duke.edu> <4BF0F231.9000706@mapper.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4BF0F231.9000706@mapper.nl> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-fs@freebsd.org, Todd Wasson Subject: Re: zfs drive replacement issues X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 08:33:41 -0000 On Mon, May 17, 2010 at 09:37:21AM +0200, Mark Stapper wrote: > Then make sure the "new" drive isn't part of any zpool (low-level format?). I think zeroing out the first *and* the last few sectors of the disk should be sufficient; I'd say use dd to zero out the first and last 64KBytes of the disk. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Mon May 17 08:42:28 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C0EA51065676 for ; Mon, 17 May 2010 08:42:28 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 775DA8FC0C for ; Mon, 17 May 2010 08:42:28 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1ODvu5-0007zH-SA for freebsd-fs@freebsd.org; Mon, 17 May 2010 10:42:25 +0200 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 17 May 2010 10:42:25 +0200 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 17 May 2010 10:42:25 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org connect(): No such file or directory From: Ivan Voras Date: Mon, 17 May 2010 10:42:17 +0200 Lines: 9 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.9) Gecko/20100512 Thunderbird/3.0.4 In-Reply-To: X-Enigmail-Version: 1.0.1 Subject: Re: UFS Journaling gone south X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 08:42:28 -0000 On 05/17/10 08:43, Mike Barnard wrote: > GEOM_JOURNAL: Journal 3033687591 : ufsid/4bed9437dfb979f4 contains data. > > This goes on and on... I cannot go beyond this point. Did I miss something? It looks like in your setup gjournal interacts badly with glabel. Try using the "-h" switch to gjournal. From owner-freebsd-fs@FreeBSD.ORG Mon May 17 10:09:54 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8E719106564A for ; Mon, 17 May 2010 10:09:54 +0000 (UTC) (envelope-from bf1783@googlemail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 26E1D8FC16 for ; Mon, 17 May 2010 10:09:53 +0000 (UTC) Received: by wyg36 with SMTP id 36so3716319wyg.13 for ; Mon, 17 May 2010 03:09:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:received:reply-to:date :message-id:subject:from:to:cc:content-type; bh=L4RnB2d8AxR/ozsDSFGoU+cuuvFXhU3N5zV9f+UksYk=; b=Pb4hxN8398ntivwQwB3a0AjKBzGn3oYAhBQkg5ZolBwhwvNDnurn7EBeMbrOYoLLdk rRPKrQBmcRO2tlU+8os0wSsZvyH3zF8uU+i40CM2ZKVD2KZWAE+Ue3hOCyTAMpmJn7OL lfJ2TDEIrdiqj2tGk9O8MUVmBeoybCP91zSQA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:reply-to:date:message-id:subject:from:to:cc :content-type; b=jcnq7GdlQ7Rhm3OqP/jaqO2d7Z9INayYo0hzKInNGL8dwZHH39NtiJYsjd46l7tzXn q8fyj0BVuIK0fQ3ZdwmUi0NmftTACWmkmdkkjllICmPeb+WYoZ3hbBIKrg54Czc2Xapx 5ayacW0H2BwtzvUuDqzwhs3wsf5LP8JLZ2yEA= MIME-Version: 1.0 Received: by 10.216.187.204 with SMTP id y54mr2958697wem.136.1274090988962; Mon, 17 May 2010 03:09:48 -0700 (PDT) Received: by 10.216.163.21 with HTTP; Mon, 17 May 2010 03:09:48 -0700 (PDT) Date: Mon, 17 May 2010 10:09:48 +0000 Message-ID: From: "b. f." To: freebsd-fs@FreeBSD.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Subject: Re: UFS Journaling gone south X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: bf1783@gmail.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 10:09:54 -0000 Ivan Voras wrote: >On 05/17/10 08:43, Mike Barnard wrote: > >> GEOM_JOURNAL: Journal 3033687591 : ufsid/4bed9437dfb979f4 contains data. >> >> This goes on and on... I cannot go beyond this point. Did I miss something? > >It looks like in your setup gjournal interacts badly with glabel. Try >using the "-h" switch to gjournal. And since you said in follow-ups to your other post that you were able to create the data providers anew, and that you had zeroed out the disk before creating them, then why not use "newfs -r" with an appropriate number (2 reserved sectors?), in your newfs/gjournal label -f/tunefs -J sequence, to reduce the chance of overwriting something important. b. From owner-freebsd-fs@FreeBSD.ORG Mon May 17 11:06:58 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0CAE71065674 for ; Mon, 17 May 2010 11:06:58 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (unknown [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D51FE8FC27 for ; Mon, 17 May 2010 11:06:57 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o4HB6vX3015724 for ; Mon, 17 May 2010 11:06:57 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o4HB6v5D015722 for freebsd-fs@FreeBSD.org; Mon, 17 May 2010 11:06:57 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 17 May 2010 11:06:57 GMT Message-Id: <201005171106.o4HB6v5D015722@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 11:06:58 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/146528 fs [zfs] Severe memory leak in ZFS on i386 o kern/146502 fs [nfs] FreeBSD 8 NFS Client Connection to Server o kern/146375 fs [nfs] [patch] Typos in macro variables names in sys/fs o kern/145778 fs [zfs] [panic] panic in zfs_fuid_map_id (known issue fi s kern/145712 fs [zfs] cannot offline two drives in a raidz2 configurat s kern/145424 fs [zfs] [patch] move source closer to v15 o kern/145411 fs [xfs] [panic] Kernel panics shortly after mounting an o kern/145309 fs [disklabel]: Editing disk label invalidates the whole o kern/145272 fs [zfs] [panic] Panic during boot when accessing zfs on o kern/145246 fs [ufs] dirhash in 7.3 gratuitously frees hashes when it o kern/145238 fs [zfs] [panic] kernel panic on zpool clear tank o kern/145229 fs [zfs] Vast differences in ZFS ARC behavior between 8.0 o kern/145189 fs [nfs] nfsd performs abysmally under load o kern/144929 fs [ufs] [lor] vfs_bio.c + ufs_dirhash.c o kern/144458 fs [nfs] [patch] nfsd fails as a kld p kern/144447 fs [zfs] sharenfs fsunshare() & fsshare_main() non functi o kern/144416 fs [panic] Kernel panic on online filesystem optimization s kern/144415 fs [zfs] [panic] kernel panics on boot after zfs crash o kern/144234 fs [zfs] Cannot boot machine with recent gptzfsboot code o bin/144214 fs zfsboot fails on gang block after upgrade to zfs v14 o kern/143825 fs [nfs] [panic] Kernel panic on NFS client o kern/143345 fs [ext2fs] [patch] extfs minor header cleanups to better o kern/143212 fs [nfs] NFSv4 client strange work ... o kern/143184 fs [zfs] [lor] zfs/bufwait LOR o kern/142924 fs [ext2fs] [patch] Small cleanup for the inode struct in o kern/142914 fs [zfs] ZFS performance degradation over time o kern/142878 fs [zfs] [vfs] lock order reversal o kern/142597 fs [ext2fs] ext2fs does not work on filesystems with real o kern/142489 fs [zfs] [lor] allproc/zfs LOR o kern/142466 fs Update 7.2 -> 8.0 on Raid 1 ends with screwed raid [re o kern/142401 fs [ntfs] [patch] Minor updates to NTFS from NetBSD o kern/142306 fs [zfs] [panic] ZFS drive (from OSX Leopard) causes two o kern/142068 fs [ufs] BSD labels are got deleted spontaneously o kern/141897 fs [msdosfs] [panic] Kernel panic. msdofs: file name leng o kern/141463 fs [nfs] [panic] Frequent kernel panics after upgrade fro o kern/141305 fs [zfs] FreeBSD ZFS+sendfile severe performance issues ( o kern/141091 fs [patch] [nullfs] fix panics with DIAGNOSTIC enabled o kern/141086 fs [nfs] [panic] panic("nfs: bioread, not dir") on FreeBS o kern/141010 fs [zfs] "zfs scrub" fails when backed by files in UFS2 o kern/140888 fs [zfs] boot fail from zfs root while the pool resilveri o kern/140661 fs [zfs] [patch] /boot/loader fails to work on a GPT/ZFS- o kern/140640 fs [zfs] snapshot crash o kern/140134 fs [msdosfs] write and fsck destroy filesystem integrity o kern/140068 fs [smbfs] [patch] smbfs does not allow semicolon in file o kern/139725 fs [zfs] zdb(1) dumps core on i386 when examining zpool c o kern/139715 fs [zfs] vfs.numvnodes leak on busy zfs o bin/139651 fs [nfs] mount(8): read-only remount of NFS volume does n o kern/139597 fs [patch] [tmpfs] tmpfs initializes va_gen but doesn't u o kern/139564 fs [zfs] [panic] 8.0-RC1 - Fatal trap 12 at end of shutdo o kern/139407 fs [smbfs] [panic] smb mount causes system crash if remot o kern/139363 fs [nfs] diskless root nfs mount from non FreeBSD server o kern/138790 fs [zfs] ZFS ceases caching when mem demand is high o kern/138421 fs [ufs] [patch] remove UFS label limitations o kern/138202 fs mount_msdosfs(1) see only 2Gb f kern/137037 fs [zfs] [hang] zfs rollback on root causes FreeBSD to fr o kern/136968 fs [ufs] [lor] ufs/bufwait/ufs (open) o kern/136945 fs [ufs] [lor] filedesc structure/ufs (poll) o kern/136944 fs [ffs] [lor] bufwait/snaplk (fsync) o kern/136873 fs [ntfs] Missing directories/files on NTFS volume o kern/136865 fs [nfs] [patch] NFS exports atomic and on-the-fly atomic o kern/136470 fs [nfs] Cannot mount / in read-only, over NFS o kern/135546 fs [zfs] zfs.ko module doesn't ignore zpool.cache filenam o kern/135469 fs [ufs] [panic] kernel crash on md operation in ufs_dirb o kern/135050 fs [zfs] ZFS clears/hides disk errors on reboot o kern/134491 fs [zfs] Hot spares are rather cold... o kern/133676 fs [smbfs] [panic] umount -f'ing a vnode-based memory dis o kern/133614 fs [panic] panic: ffs_truncate: read-only filesystem o kern/133174 fs [msdosfs] [patch] msdosfs must support utf-encoded int f kern/133150 fs [zfs] Page fault with ZFS on 7.1-RELEASE/amd64 while w o kern/132960 fs [ufs] [panic] panic:ffs_blkfree: freeing free frag o kern/132397 fs reboot causes filesystem corruption (failure to sync b o kern/132331 fs [ufs] [lor] LOR ufs and syncer o kern/132237 fs [msdosfs] msdosfs has problems to read MSDOS Floppy o kern/132145 fs [panic] File System Hard Crashes o kern/131441 fs [unionfs] [nullfs] unionfs and/or nullfs not combineab o kern/131360 fs [nfs] poor scaling behavior of the NFS server under lo o kern/131342 fs [nfs] mounting/unmounting of disks causes NFS to fail o bin/131341 fs makefs: error "Bad file descriptor" on the mount poin o kern/130979 fs [smbfs] [panic] boot/kernel/smbfs.ko o kern/130920 fs [msdosfs] cp(1) takes 100% CPU time while copying file o kern/130229 fs [iconv] usermount fails on fs that need iconv o kern/130210 fs [nullfs] Error by check nullfs o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129488 fs [smbfs] Kernel "bug" when using smbfs in smbfs_smb.c: o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/129059 fs [zfs] [patch] ZFS bootloader whitelistable via WITHOUT f kern/128829 fs smbd(8) causes periodic panic on 7-RELEASE o kern/127420 fs [gjournal] [panic] Journal overflow on gmirrored gjour o kern/127029 fs [panic] mount(8): trying to mount a write protected zi o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file o kern/125895 fs [ffs] [panic] kernel: panic: ffs_blkfree: freeing free s kern/125738 fs [zfs] [request] SHA256 acceleration in ZFS p kern/124621 fs [ext3] [patch] Cannot mount ext2fs partition f bin/124424 fs [zfs] zfs(8): zfs list -r shows strange snapshots' siz o kern/123939 fs [msdosfs] corrupts new files o kern/122380 fs [ffs] ffs_valloc:dup alloc (Soekris 4801/7.0/USB Flash o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o bin/121898 fs [nullfs] pwd(1)/getcwd(2) fails with Permission denied o bin/121779 fs [ufs] snapinfo(8) (and related tools?) only work for t o bin/121366 fs [zfs] [patch] Automatic disk scrubbing from periodic(8 o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha f kern/120991 fs [panic] [fs] [snapshot] System crashes when manipulati o kern/120483 fs [ntfs] [patch] NTFS filesystem locking changes o kern/120482 fs [ntfs] [patch] Sync style changes between NetBSD and F f kern/119735 fs [zfs] geli + ZFS + samba starting on boot panics 7.0-B o kern/118912 fs [2tb] disk sizing/geometry problem with large array o kern/118713 fs [minidump] [patch] Display media size required for a k o bin/118249 fs mv(1): moving a directory changes its mtime o kern/118107 fs [ntfs] [panic] Kernel panic when accessing a file at N o bin/117315 fs [smbfs] mount_smbfs(8) and related options can't mount o kern/117314 fs [ntfs] Long-filename only NTFS fs'es cause kernel pani o kern/117158 fs [zfs] zpool scrub causes panic if geli vdevs detach on o bin/116980 fs [msdosfs] [patch] mount_msdosfs(8) resets some flags f o conf/116931 fs lack of fsck_cd9660 prevents mounting iso images with o kern/116913 fs [ffs] [panic] ffs_blkfree: freeing free block p kern/116608 fs [msdosfs] [patch] msdosfs fails to check mount options o kern/116583 fs [ffs] [hang] System freezes for short time when using o kern/116170 fs [panic] Kernel panic when mounting /tmp o kern/115645 fs [snapshots] [panic] lockmgr: thread 0xc4c00d80, not ex o bin/115361 fs [zfs] mount(8) gets into a state where it won't set/un o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o kern/113852 fs [smbfs] smbfs does not properly implement DFS referral o bin/113838 fs [patch] [request] mount(8): add support for relative p o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/111843 fs [msdosfs] Long Names of files are incorrectly created o kern/111782 fs [ufs] dump(8) fails horribly for large filesystems s bin/111146 fs [2tb] fsck(8) fails on 6T filesystem o kern/109024 fs [msdosfs] mount_msdosfs: msdosfs_iconv: Operation not o kern/109010 fs [msdosfs] can't mv directory within fat32 file system o bin/107829 fs [2TB] fdisk(8): invalid boundary checking in fdisk / w o kern/106107 fs [ufs] left-over fsck_snapshot after unfinished backgro o kern/106030 fs [ufs] [panic] panic in ufs from geom when a dead disk o kern/104406 fs [ufs] Processes get stuck in "ufs" state under persist o kern/104133 fs [ext2fs] EXT2FS module corrupts EXT2/3 filesystems o kern/103035 fs [ntfs] Directories in NTFS mounted disc images appear o kern/101324 fs [smbfs] smbfs sometimes not case sensitive when it's s o kern/99290 fs [ntfs] mount_ntfs ignorant of cluster sizes o kern/97377 fs [ntfs] [patch] syntax cleanup for ntfs_ihash.c o kern/95222 fs [iso9660] File sections on ISO9660 level 3 CDs ignored o kern/94849 fs [ufs] rename on UFS filesystem is not atomic o kern/94769 fs [ufs] Multiple file deletions on multi-snapshotted fil o kern/94733 fs [smbfs] smbfs may cause double unlock o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D o kern/92272 fs [ffs] [hang] Filling a filesystem while creating a sna f kern/91568 fs [ufs] [panic] writing to UFS/softupdates DVD media in o kern/91134 fs [smbfs] [patch] Preserve access and modification time a kern/90815 fs [smbfs] [patch] SMBFS with character conversions somet o kern/88657 fs [smbfs] windows client hang when browsing a samba shar o kern/88266 fs [smbfs] smbfs does not implement UIO_NOCOPY and sendfi o kern/87859 fs [smbfs] System reboot while umount smbfs. o kern/86587 fs [msdosfs] rm -r /PATH fails with lots of small files o kern/85326 fs [smbfs] [panic] saving a file via samba to an overquot o kern/84589 fs [2TB] 5.4-STABLE unresponsive during background fsck 2 o kern/80088 fs [smbfs] Incorrect file time setting on NTFS mounted vi o kern/73484 fs [ntfs] Kernel panic when doing `ls` from the client si o bin/73019 fs [ufs] fsck_ufs(8) cannot alloc 607016868 bytes for ino o kern/71774 fs [ntfs] NTFS cannot "see" files on a WinXP filesystem o kern/68978 fs [panic] [ufs] crashes with failing hard disk, loose po o kern/65920 fs [nwfs] Mounted Netware filesystem behaves strange o kern/65901 fs [smbfs] [patch] smbfs fails fsx write/truncate-down/tr o kern/61503 fs [smbfs] mount_smbfs does not work as non-root o kern/55617 fs [smbfs] Accessing an nsmb-mounted drive via a smb expo o kern/53137 fs [ffs] [panic] background fscking causing ffs_valloc pa o kern/51685 fs [hang] Unbounded inode allocation causes kernel to loc o kern/51583 fs [nullfs] [patch] allow to work with devices and socket o kern/36566 fs [smbfs] System reboot with dead smb mount and umount o kern/33464 fs [ufs] soft update inconsistencies after system crash o kern/18874 fs [2TB] 32bit NFS servers export wrong negative values t 173 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon May 17 11:47:57 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D76C5106566C for ; Mon, 17 May 2010 11:47:57 +0000 (UTC) (envelope-from mike.barnardq@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 656DC8FC23 for ; Mon, 17 May 2010 11:47:57 +0000 (UTC) Received: by fxm19 with SMTP id 19so804957fxm.13 for ; Mon, 17 May 2010 04:47:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=ln5k55ILcREtUOmn+prcdcdrtcTDSluKnLwbd/JnWuw=; b=MMQ7dVQztQGDfZe1/ixyY7Uz1DUxNIhD94/2vXvUyEk6z5fpgE2eJtoUFTQq0bRVW8 iokf+PqnVaU7sNjUYWS5Yjym+8aCPMITkjLFm8KRELsYSnxO/JyRH4ebAC+g50T9m6Cu 0qfPcU0tADm72RJs2xlzYwduzNEgECOXFcUC0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=BTvp4/s/zQAtLp1K88WvXmsKkZ1fgAV1ysiqMVMveNdclYoy0x2fJ6ImKOaYGddJ0Z QJAzRs5Zi5zFhFFUX5F9n+j+bRR+iyRQQG/LRGfgETUtYF8y0O7KJQ98kUwNvZsQyTNp bApuHjFeCbAfMLBbwp5LBwHuROtwe3lFub8k4= MIME-Version: 1.0 Received: by 10.223.161.204 with SMTP id s12mr6109456fax.103.1274096876223; Mon, 17 May 2010 04:47:56 -0700 (PDT) Received: by 10.223.105.146 with HTTP; Mon, 17 May 2010 04:47:56 -0700 (PDT) In-Reply-To: References: Date: Mon, 17 May 2010 14:47:56 +0300 Message-ID: From: Mike Barnard To: bf1783@gmail.com Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: UFS Journaling gone south X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 11:47:57 -0000 Hi, On Mon, May 17, 2010 at 1:09 PM, b. f. wrote: > > >It looks like in your setup gjournal interacts badly with glabel. Try > >using the "-h" switch to gjournal. > > that seems to have done something... but when I reboot the server, my journal devices seem to have vanished :-s. I can can get into single usermode and an 'ls /dev' shows my disk partitions but no journal devices. Prior to rebooting, I checked and I did have da0s1f.journal and da0s1g.journal and the da0s2d and da0s2e devices, after the reboot and the panic, I check /dev and the seem to have reverted back to da0s1f and da0s1g... and da0s2d and da0s2e have also gone AWOL... :-w > And since you said in follow-ups to your other post that you were able > to create the data providers anew, and that you had zeroed out the > disk before creating them, then why not use "newfs -r" with an > appropriate number (2 reserved sectors?), in your newfs/gjournal > label -f/tunefs -J sequence, to reduce the chance of overwriting > something important. > > fortunately, this is a new installation on which I want to test journal devices. I'll do this again, reserving some sectors on both partitions. -- Mike Of course, you might discount this possibility, but remember that one in a million chances happen 99% of the time. ------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Mon May 17 14:28:01 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 410061065673 for ; Mon, 17 May 2010 14:28:01 +0000 (UTC) (envelope-from gljennjohn@googlemail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id B9A288FC2B for ; Mon, 17 May 2010 14:27:57 +0000 (UTC) Received: by fxm19 with SMTP id 19so1005356fxm.13 for ; Mon, 17 May 2010 07:27:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:date:from:to:cc:subject :message-id:in-reply-to:references:reply-to:x-mailer:mime-version :content-type:content-transfer-encoding; bh=rEMdfBFI8zWtV9ay6ZAaNfV9nmyphBH3KoY7Ugc2SDc=; b=o7+WyE5hY6qVh3H8x2rRKVIc3xJGVIc0Mr14sq0jobxyWMXOGAlFReexsu3Yrq1Ezq Rr+F8ZA9UjrDgmcVOsiRmUxvNdDj4cQqJ9RzmSzjr4jj4gQ1TSqLrNb9UviaQSa4+kw8 tAtcEQx97TZdfBGuRcjEC9q9/AmzvHdttQaRA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :x-mailer:mime-version:content-type:content-transfer-encoding; b=RrYrgx1GsHNwPxlwmbFKVLSA6et+dvKaVNTbVAVfD8AuvAsI9zxzPItasOtz432rmS 2FRBsD/wYWR5+3unfadX0nelDPpQhThdoGnM1GFMSWNn8jLhd83F6g1B0/WkkhbNY3C8 KrOIBMKI8yJIVNWIRa2xcNNBNgBERSm4hV+cg= Received: by 10.223.32.85 with SMTP id b21mr6353196fad.95.1274105068925; Mon, 17 May 2010 07:04:28 -0700 (PDT) Received: from ernst.jennejohn.org (p57AE50EE.dip.t-dialin.net [87.174.80.238]) by mx.google.com with ESMTPS id g10sm25987344fai.0.2010.05.17.07.04.27 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 17 May 2010 07:04:28 -0700 (PDT) Date: Mon, 17 May 2010 16:04:26 +0200 From: Gary Jennejohn To: Jeremy Chadwick Message-ID: <20100517160426.3c54e07e@ernst.jennejohn.org> In-Reply-To: <20100516210151.GA82487@icarus.home.lan> References: <4BEF2F9C.7080409@netscape.net> <4BEF3137.4080203@netscape.net> <20100516001351.GA50879@icarus.home.lan> <4BEF4A73.8060905@netscape.net> <20100516015850.GA55302@icarus.home.lan> <657328.88413.qm@web112409.mail.gq1.yahoo.com> <20100516210151.GA82487@icarus.home.lan> X-Mailer: Claws Mail 3.7.5 (GTK+ 2.18.7; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Quick ZFS mirroring question for non-mirrored pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gljennjohn@googlemail.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 14:28:01 -0000 On Sun, 16 May 2010 14:01:51 -0700 Jeremy Chadwick wrote: > * As has been proven time and time again, MTBF means jack squat since > it's all hypothetical (mathematically calculated based on fab tests). > Drives will fail no matter what; that's the entire reason people are > using ZFS to begin with. ;-) > Hear, hear! I bought a supposedly server-grade SATA drive from Samsung a few years ago. Within a few weeks the spindle broke. I'd never seen that before and was totally flabbergasted. Who'd expect the spindle to break? -- Gary Jennejohn From owner-freebsd-fs@FreeBSD.ORG Mon May 17 14:54:36 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3C91C1065673 for ; Mon, 17 May 2010 14:54:36 +0000 (UTC) (envelope-from roberto@keltia.freenix.fr) Received: from keltia.net (unknown [IPv6:2a01:240:fe5c::41]) by mx1.freebsd.org (Postfix) with ESMTP id E42BC8FC0A for ; Mon, 17 May 2010 14:54:35 +0000 (UTC) Received: from roberto-al.eurocontrol.fr (aran.keltia.net [88.191.250.24]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: roberto) by keltia.net (Postfix/TLS) with ESMTPSA id 205A586DC; Mon, 17 May 2010 16:54:34 +0200 (CEST) Date: Mon, 17 May 2010 16:54:32 +0200 From: Ollivier Robert To: freebsd-fs@freebsd.org, Kaya Saman Message-ID: <20100517145432.GB80288@roberto-al.eurocontrol.fr> References: <4BEF2F9C.7080409@netscape.net> <4BEF3137.4080203@netscape.net> <20100516001351.GA50879@icarus.home.lan> <4BEF4A73.8060905@netscape.net> <20100516015850.GA55302@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100516015850.GA55302@icarus.home.lan> X-Operating-System: MacOS X / Macbook Pro - FreeBSD 7.2 / Dell D820 SMP User-Agent: Mutt/1.5.20 (2009-06-14) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.3 (keltia.net); Mon, 17 May 2010 16:54:34 +0200 (CEST) Cc: Subject: Re: Quick ZFS mirroring question for non-mirrored pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 14:54:36 -0000 According to Jeremy Chadwick: > There are many people here who have gone through the pain (IMHO) of > getting ZFS to boot on FreeBSD, and it still isn't as simple nor It is not as painful as it sounds (cf my own tutorial at [1]) and it offers advantages over UFS+ZFS such as snapshots for the system, pool-size vs filesystem sizes and so on. [1] http://www.keltia.net/howtos/zfsboot -- Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- roberto@keltia.freenix.fr In memoriam to Ondine : http://ondine.keltia.net/ From owner-freebsd-fs@FreeBSD.ORG Mon May 17 15:36:42 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0C7C41065678; Mon, 17 May 2010 15:36:42 +0000 (UTC) (envelope-from alex.bakhtin@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 434718FC1D; Mon, 17 May 2010 15:36:40 +0000 (UTC) Received: by fxm19 with SMTP id 19so1093653fxm.13 for ; Mon, 17 May 2010 08:36:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=s6mO/6cTcDHdEDtJuLGwJCAdUv7kscjsuHbH4HIkXzY=; b=ceFfim9UHXQDD3Cc/XyoGvGIHCxpSgaOLdjWlRZMZuMeAjyDw5Ryhcf18fxC/G5Guw euCWBriA1hoKCCG9gkmvdFiW2Hyx6wW7R2Gx6R4+6c4Awi9uglavhtjL4ZB/DrcVwmF7 gKpAeIO5gDWe2TI8jcRNM8BQ5bGj75Z+z8+HE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=hEt1OLQKqjFXKjfvHip82tVLkiovsBar14vDlUhn+UKj23P0VVj+X5gIjlO4LMHOL1 UYDv+8JLuZS9fIjfKx3jpw7o9VLXJCO2wPXI8Z9V9Ws+KJtPGFM1UOCFpL/+RRszv+nN AJVSwFzHvqkR9zJQAqI+ZBJ0RTbfHw8BbWUqM= MIME-Version: 1.0 Received: by 10.204.33.194 with SMTP id i2mr10200bkd.140.1274110599812; Mon, 17 May 2010 08:36:39 -0700 (PDT) Received: by 10.204.71.139 with HTTP; Mon, 17 May 2010 08:36:39 -0700 (PDT) In-Reply-To: <201005130934.o4D9YiJL039462@freefall.freebsd.org> References: <201005130934.o4D9YiJL039462@freefall.freebsd.org> Date: Mon, 17 May 2010 19:36:39 +0400 Message-ID: From: Alex Bakhtin To: pjd@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, bug-followup@freebsd.org Subject: Re: kern/145339: [zfs] deadlock after detaching block device from raidz pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 15:36:42 -0000 Pawel, I tested your patch in the following zfs configuration (all on 5x2TB WD20EARS drivers): 1. raidz1 on top of physical disks. 2. raidz1 on top of geli 3. raidz2 on top of physical disks. In all three cases it seems that the problem was fixed - I can't crash zfs in vdev_geom when unplugging the disk. Unfortunately, 3 times I got a deadlock in zfs after plugging vdevs back under load. It happens several seconds after zpool online command. I'm not 100 percent sure that deadlocks are related to this patch, but... I'm going to make some additional testing with patched and not patched kernels. 2010/5/13 : > Synopsis: [zfs] deadlock after detaching block device from raidz pool > > State-Changed-From-To: open->feedback > State-Changed-By: pjd > State-Changed-When: czw 13 maj 2010 09:33:20 UTC > State-Changed-Why: > Could you try this patch: > > =A0 =A0 =A0 =A0http://people.freebsd.org/~pjd/patches/vdev_geom.c.3.patch > > It is against most recent HEAD. If it is rejected on 8-STABLE, just grab > entire vdev_geom.c from HEAD and patch this. > > > Responsible-Changed-From-To: freebsd-fs->pjd > Responsible-Changed-By: pjd > Responsible-Changed-When: czw 13 maj 2010 09:33:20 UTC > Responsible-Changed-Why: > I'll take this one. > > http://www.freebsd.org/cgi/query-pr.cgi?pr=3D145339 > From owner-freebsd-fs@FreeBSD.ORG Mon May 17 15:47:02 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 363FB1065674 for ; Mon, 17 May 2010 15:47:02 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 5E05F8FC08 for ; Mon, 17 May 2010 15:47:00 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id SAA00522; Mon, 17 May 2010 18:46:38 +0300 (EEST) (envelope-from avg@icyb.net.ua) Message-ID: <4BF164DD.4060502@icyb.net.ua> Date: Mon, 17 May 2010 18:46:37 +0300 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.24 (X11/20100517) MIME-Version: 1.0 To: Pawel Tyll References: <598841171.20100515193024@nitronet.pl> In-Reply-To: <598841171.20100515193024@nitronet.pl> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Root mount from ZFS takes forever...literally. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 15:47:02 -0000 on 15/05/2010 20:30 Pawel Tyll said the following: > Hi list, > > Has this happened to anyone? > > Boot process "hangs" on Trying to mount root from zfs:tank/root - I > can import the pool from alternate boot source (PXE), and access files > fine, but I can't finish booting the system from ZFS. > > It's a recent 8-STABLE, it happened after upgrading from older > 8-STABLE, but reverting to older kernel and modules doesn't solve > this, so maybe it's unrelated. What messages do you have on screen when this happens? Can you upload a screenshot somewhere and post a link? -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Mon May 17 15:56:46 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F1239106564A for ; Mon, 17 May 2010 15:56:46 +0000 (UTC) (envelope-from the_mix_room@hotmail.com) Received: from bay0-omc4-s27.bay0.hotmail.com (bay0-omc4-s27.bay0.hotmail.com [65.54.190.229]) by mx1.freebsd.org (Postfix) with ESMTP id A548F8FC19 for ; Mon, 17 May 2010 15:56:46 +0000 (UTC) Received: from BAY112-W50 ([65.54.190.199]) by bay0-omc4-s27.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Mon, 17 May 2010 08:44:47 -0700 Message-ID: X-Originating-IP: [129.132.125.29] From: To: Date: Mon, 17 May 2010 15:44:46 +0000 Importance: Normal MIME-Version: 1.0 X-OriginalArrivalTime: 17 May 2010 15:44:47.0151 (UTC) FILETIME=[E0C8CBF0:01CAF5D7] Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Possible problems with file- and memory-backed vdev in ZFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 15:56:47 -0000 Dear list=2C=20 I have been experiencing possible problems with file-backed vdevs in ZFS.=20 I was playing around with them for lack of proper drives=2C trying to get a= quainted with the system.=20 I noticed that my machine would hang when I deleted the file which was incl= uded in the pool # mkfile file{1=2C2=2C3=2C4=2C5} # zpool create tank raidz2 file{1=2C2=2C3=2C4=2C5} # zpool scrub tank -> OK # rm file1 # zpool scrub tank -> FAILURE This does not occur then one of the files is replaced with a partition on a= USB-drive.=20 I have posted more complete information under forums.freebsd.org=20 http://forums.freebsd.org/showthread.php?t=3D13619 Would appreciate if someone could point me in the direction to proceed=2C w= hether to post PR or what to do.=20 Best regards=2C=20 =20 _________________________________________________________________ Hotmail: Free=2C trusted and rich email service. https://signup.live.com/signup.aspx?id=3D60969= From owner-freebsd-fs@FreeBSD.ORG Mon May 17 16:47:06 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 26FD9106566C for ; Mon, 17 May 2010 16:47:06 +0000 (UTC) (envelope-from ptyll@nitronet.pl) Received: from web.nitronet.pl (web.nitronet.pl [195.90.106.5]) by mx1.freebsd.org (Postfix) with ESMTP id DA69B8FC13 for ; Mon, 17 May 2010 16:47:05 +0000 (UTC) Received: from mailnull by web.nitronet.pl with virscan (Exim 4.69 (FreeBSD)) (envelope-from ) id 1OE3T6-000Frv-BT for freebsd-fs@freebsd.org; Mon, 17 May 2010 18:47:04 +0200 Date: Mon, 17 May 2010 18:47:02 +0200 From: Pawel Tyll X-Priority: 3 (Normal) Message-ID: <1554873904.20100517184702@nitronet.pl> To: Andriy Gapon In-Reply-To: <4BF164DD.4060502@icyb.net.ua> References: <598841171.20100515193024@nitronet.pl> <4BF164DD.4060502@icyb.net.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-2 Content-Transfer-Encoding: quoted-printable X-Virus-Scanned: Nitronet.pl X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: ptyll@nitronet.pl X-SA-Exim-Scanned: No (on web.nitronet.pl); SAEximRunCond expanded to false Cc: freebsd-fs@freebsd.org Subject: Re: Root mount from ZFS takes forever...literally. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 16:47:06 -0000 >> Boot process "hangs" on Trying to mount root from zfs:tank/root - I >> can import the pool from alternate boot source (PXE), and access files >> fine, but I can't finish booting the system from ZFS. > What messages do you have on screen when this happens? > Can you upload a screenshot somewhere and post a link? Just mounting root, and nothing happens. System is responsive, attaching/detaching USB keyboard generates messages, and hitting alt+ctrl+del correctly reboots the system. After messing around with fstab (added / entry, bit redundant with loader.conf vfs.root.mountfrom entry) it started to working again, but I can't repeat it now by removing this. Too bad I didn't take a snapshot before fixing :( Strange. :) From owner-freebsd-fs@FreeBSD.ORG Mon May 17 16:53:53 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8BC1B1065677 for ; Mon, 17 May 2010 16:53:53 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta02.emeryville.ca.mail.comcast.net (qmta02.emeryville.ca.mail.comcast.net [76.96.30.24]) by mx1.freebsd.org (Postfix) with ESMTP id 7287A8FC2F for ; Mon, 17 May 2010 16:53:52 +0000 (UTC) Received: from omta01.emeryville.ca.mail.comcast.net ([76.96.30.11]) by qmta02.emeryville.ca.mail.comcast.net with comcast id JcAV1e0060EPchoA2gttS4; Mon, 17 May 2010 16:53:53 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta01.emeryville.ca.mail.comcast.net with comcast id Jgtt1e0043S48mS8MgttdT; Mon, 17 May 2010 16:53:53 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 2B0699B419; Mon, 17 May 2010 09:53:52 -0700 (PDT) Date: Mon, 17 May 2010 09:53:52 -0700 From: Jeremy Chadwick To: Pawel Tyll Message-ID: <20100517165352.GA12092@icarus.home.lan> References: <598841171.20100515193024@nitronet.pl> <4BF164DD.4060502@icyb.net.ua> <1554873904.20100517184702@nitronet.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1554873904.20100517184702@nitronet.pl> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-fs@freebsd.org, Andriy Gapon Subject: Re: Root mount from ZFS takes forever...literally. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 16:53:53 -0000 On Mon, May 17, 2010 at 06:47:02PM +0200, Pawel Tyll wrote: > >> Boot process "hangs" on Trying to mount root from zfs:tank/root - I > >> can import the pool from alternate boot source (PXE), and access files > >> fine, but I can't finish booting the system from ZFS. > > What messages do you have on screen when this happens? > > Can you upload a screenshot somewhere and post a link? > Just mounting root, and nothing happens. System is responsive, > attaching/detaching USB keyboard generates messages, and hitting > alt+ctrl+del correctly reboots the system. > > After messing around with fstab (added / entry, bit redundant with > loader.conf vfs.root.mountfrom entry) it started to working again, but > I can't repeat it now by removing this. Too bad I didn't take a > snapshot before fixing :( > > Strange. :) Are you *absolutely sure* the problem has to do with ZFS? I see you mention PXE booting...... This sounds much like an environment where serial console is set up incorrectly; kernel messages will be seen up to the final "Mounting root..." message, but everything past that goes to VGA console (or serial console, depends on the configuration mishap). -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Mon May 17 17:35:52 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 96C841065675 for ; Mon, 17 May 2010 17:35:52 +0000 (UTC) (envelope-from jamescarmstrong@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4ED438FC20 for ; Mon, 17 May 2010 17:35:52 +0000 (UTC) Received: by gyh20 with SMTP id 20so2767458gyh.13 for ; Mon, 17 May 2010 10:35:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type; bh=smzFwrKXqsO8Q/3LfiuZkc+IqUrsQPqHUJBCy6/mPck=; b=DB4ujH451SKaheqZVpUXk/oQ0vnBHKZlYojjcv25nFb+Qr8FbZU+BncqjQa9ufx5aF mO9lwvnAuGl+L3hN4EeMhmuKMnhfU+RhSo4o87ZoknsNsqeoW8Xm2SNNwK8tWHMd67XM aoglbcz/aO4P95AnZmxIJ240Ehnm1Og/CwR8A= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=jhjonXBcHbBgf3rpDzkncz4v53KAzYN8JsrzTnSKfYUjNS5wOcDpLOzirGJyZloAtj 9xjtFnSOfESoEOHyaOyDzm5IPbOTY2eeD9JO/vzl837uBv58rD3FaMVM8X9GSxJbmgkU eRj6eFkxVuEu0bDdHXWZyMUQFR6H+CeBQrTF8= MIME-Version: 1.0 Received: by 10.151.61.2 with SMTP id o2mr6530720ybk.378.1274115945396; Mon, 17 May 2010 10:05:45 -0700 (PDT) Received: by 10.150.51.16 with HTTP; Mon, 17 May 2010 10:05:37 -0700 (PDT) In-Reply-To: <20100517165352.GA12092@icarus.home.lan> References: <598841171.20100515193024@nitronet.pl> <4BF164DD.4060502@icyb.net.ua> <1554873904.20100517184702@nitronet.pl> <20100517165352.GA12092@icarus.home.lan> Date: Mon, 17 May 2010 10:05:37 -0700 Message-ID: From: James Armstrong To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: Root mount from ZFS takes forever...literally. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 17:35:52 -0000 On Mon, May 17, 2010 at 09:53, Jeremy Chadwick wrote: > On Mon, May 17, 2010 at 06:47:02PM +0200, Pawel Tyll wrote: > > >> Boot process "hangs" on Trying to mount root from zfs:tank/root - I > > >> can import the pool from alternate boot source (PXE), and access files > > >> fine, but I can't finish booting the system from ZFS. > > > What messages do you have on screen when this happens? > > > Can you upload a screenshot somewhere and post a link? > > Just mounting root, and nothing happens. System is responsive, > > attaching/detaching USB keyboard generates messages, and hitting > > alt+ctrl+del correctly reboots the system. > > > > After messing around with fstab (added / entry, bit redundant with > > loader.conf vfs.root.mountfrom entry) it started to working again, but > > I can't repeat it now by removing this. Too bad I didn't take a > > snapshot before fixing :( > > > > Strange. :) > > Are you *absolutely sure* the problem has to do with ZFS? I see you > mention PXE booting...... > > This sounds much like an environment where serial console is set up > incorrectly; kernel messages will be seen up to the final "Mounting > root..." message, but everything past that goes to VGA console (or > serial console, depends on the configuration mishap). > > We've seen a similar issue, Ctrl-T shows a zfs process running, and there is a lot of activity on some of the disks in our zfs array. James From owner-freebsd-fs@FreeBSD.ORG Mon May 17 17:38:43 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 43D421065689 for ; Mon, 17 May 2010 17:38:43 +0000 (UTC) (envelope-from mike.barnardq@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id C7CD08FC0A for ; Mon, 17 May 2010 17:38:42 +0000 (UTC) Received: by fxm19 with SMTP id 19so1245677fxm.13 for ; Mon, 17 May 2010 10:38:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=DD4WlE/H3yjqlfKw2uInPztMj66BkrSf4juEAk9M794=; b=qmIYv3ppJkcE6yQTUAvhSHFLR+j2/Ka7hIzpCazN6yzh9UV1Ij7Ufpm9GsTeZAlS+U B8o9defs8KAv+gfdbTQoUZf8bmK+KmfPVmVoDfWjHMuFu4wIf0HJNiaWtPpfvW/VKFYi g1oJixbyCsSsoPgEnD/sgxpaXDN2QtPfTbPjA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=QOG4q6devJkw0O3qxOOrM+PNHx/2mFtwgIvnGD6JH9cnvuxl2JIBpVmxjZVmHFnDf0 3DCOVWUefHvyMvI/hamaIzHY3TBmJgFOp3AhCZnNoWkzaMScnCi9lLwPMujFpKxgWeXi 09obJ+qstE+PTKViU8Yt4O9naA+MAU038XW+k= MIME-Version: 1.0 Received: by 10.223.17.197 with SMTP id t5mr1828691faa.84.1274117921606; Mon, 17 May 2010 10:38:41 -0700 (PDT) Received: by 10.223.105.146 with HTTP; Mon, 17 May 2010 10:38:41 -0700 (PDT) In-Reply-To: References: Date: Mon, 17 May 2010 20:38:41 +0300 Message-ID: From: Mike Barnard To: bf1783@gmail.com Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: UFS Journaling gone south X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 17:38:43 -0000 On Mon, May 17, 2010 at 2:47 PM, Mike Barnard wrote > > > > Prior to rebooting, I checked and I did have da0s1f.journal and > da0s1g.journal and the da0s2d and da0s2e devices, after the reboot and the > panic, I check /dev and the seem to have reverted back to da0s1f and > da0s1g... and da0s2d and da0s2e have also gone AWOL... :-w > > fortunately, this is a new installation on which I want to test journal > devices. I'll do this again, reserving some sectors on both partitions. > > I must admit, this has me hands down. I have entered custom newfs flags on sysinstall (-r 2, 4, and 8). With a test on these sector sizes to be reserved at the end of the disk, gjournal label da0s1f da0s2d still says that I need to use -f to overwrite. I do so and end up with journal devices, but when I reboot... the journal devices and the journal providers are missing. I have even done -J flag on the newfs options in sysinstall but I end up with the same results... no journal devices are created and I end up with a kernel panic and into single usermode. There are no journal devices created. Mounting the disks fails with: warning: GJOURNAL flag on fs but no gjournal provider below. Am I doing all this wrong, Am I missing something? -- Mike Of course, you might discount this possibility, but remember that one in a million chances happen 99% of the time. ------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Mon May 17 18:17:50 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1F37B1065673 for ; Mon, 17 May 2010 18:17:50 +0000 (UTC) (envelope-from tsw5@duke.edu) Received: from smtp.duke.edu (smtp-01.oit.duke.edu [152.3.174.14]) by mx1.freebsd.org (Postfix) with ESMTP id EAE838FC08 for ; Mon, 17 May 2010 18:17:49 +0000 (UTC) Received: from smtp.duke.edu (localhost.localdomain [127.0.0.1]) by localhost (Postfix) with SMTP id 339A3308F5D; Mon, 17 May 2010 14:17:49 -0400 (EDT) Received: from user-152-3-155-113.wireless.duke.edu (user-152-3-155-113.wireless.duke.edu [152.3.155.113]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.duke.edu (Postfix) with ESMTP id 17A34308DE4; Mon, 17 May 2010 14:17:49 -0400 (EDT) Mime-Version: 1.0 (Apple Message framework v1078) Content-Type: text/plain; charset=us-ascii From: Todd Wasson In-Reply-To: <4BF0F231.9000706@mapper.nl> Date: Mon, 17 May 2010 14:17:48 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <53F15A8B-77DA-4CEF-A790-2902BEC91002@duke.edu> References: <0B97967D-1057-4414-BBD4-4F1AA2659A5D@duke.edu> <4BF0F231.9000706@mapper.nl> To: Mark Stapper X-Mailer: Apple Mail (2.1078) X-PMX-Version: 5.4.2.338381, Antispam-Engine: 2.6.0.325393, Antispam-Data: 2010.5.17.180314 Cc: freebsd-fs@freebsd.org Subject: Re: zfs drive replacement issues X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 18:17:50 -0000 > Hello, > You could try exporting and importing the pool with three disks. > Then make sure the "new" drive isn't part of any zpool (low-level = format?). > Then try a "replace" again. > Have fun! >=20 Hi Mark, I was about to try this, but I just tried putting the "old" = (damaged) drive back in the pool and detaching the "new" drive from the = pool, which I've tried before, but for some reason this time it = succeeded. I was then able to "zpool offline" the old drive, physically = replace it with the new one, and "zpool replace" the old one with the = new one. It just finished successfully resilvering, and apparently = everything is working well. I'm going to initiate a scrub to be sure = that everything is alright, but I'm fairly sure that the problem is = solved. I didn't do anything that I hadn't already tried, so I don't = know why it worked this time, but I'm not complaining. Thanks to = everyone for your help; at the very least, the idea of putting the = original drive back into the machine and mucking around with it led me = in the right direction. Next time I'll be sure to issue an offline = command before replacing a device! Thanks again. Todd= From owner-freebsd-fs@FreeBSD.ORG Mon May 17 18:24:20 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 12A081065724 for ; Mon, 17 May 2010 18:24:20 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta08.westchester.pa.mail.comcast.net (qmta08.westchester.pa.mail.comcast.net [76.96.62.80]) by mx1.freebsd.org (Postfix) with ESMTP id B62228FC17 for ; Mon, 17 May 2010 18:24:14 +0000 (UTC) Received: from omta22.westchester.pa.mail.comcast.net ([76.96.62.73]) by qmta08.westchester.pa.mail.comcast.net with comcast id JcJR1e0081ap0As58iQE7u; Mon, 17 May 2010 18:24:14 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta22.westchester.pa.mail.comcast.net with comcast id JiQC1e00L3S48mS3iiQDZk; Mon, 17 May 2010 18:24:14 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 8DCC59B419; Mon, 17 May 2010 11:24:11 -0700 (PDT) Date: Mon, 17 May 2010 11:24:11 -0700 From: Jeremy Chadwick To: Todd Wasson Message-ID: <20100517182411.GA14251@icarus.home.lan> References: <0B97967D-1057-4414-BBD4-4F1AA2659A5D@duke.edu> <4BF0F231.9000706@mapper.nl> <53F15A8B-77DA-4CEF-A790-2902BEC91002@duke.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53F15A8B-77DA-4CEF-A790-2902BEC91002@duke.edu> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-fs@freebsd.org Subject: Re: zfs drive replacement issues X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 18:24:20 -0000 On Mon, May 17, 2010 at 02:17:48PM -0400, Todd Wasson wrote: > Hi Mark, I was about to try this, but I just tried putting the "old" > (damaged) drive back in the pool and detaching the "new" drive from > the pool, which I've tried before, but for some reason this time it > succeeded. I was then able to "zpool offline" the old drive, > physically replace it with the new one, and "zpool replace" the old > one with the new one. It just finished successfully resilvering, and > apparently everything is working well. I'm going to initiate a scrub > to be sure that everything is alright, but I'm fairly sure that the > problem is solved. I didn't do anything that I hadn't already tried, > so I don't know why it worked this time, but I'm not complaining. > Thanks to everyone for your help; at the very least, the idea of > putting the original drive back into the machine and mucking around > with it led me in the right direction. Next time I'll be sure to > issue an offline command before replacing a device! Todd, Can you provide some details about the controller you have these disks attached to? "dmesg | egrep -i 'ata|ahci'" should be sufficient at this point. SMART statistics for the drive(s) you feel might be bad would also help. I can try my best to describe the conditions of the disk(s) based on output from "smartctl -a /dev/adX". You can install smartmontools from ports/sysutils/smartmontools. Thanks. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Mon May 17 18:41:34 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A48CC106564A for ; Mon, 17 May 2010 18:41:34 +0000 (UTC) (envelope-from tsw5@duke.edu) Received: from smtp.duke.edu (smtp-01.oit.duke.edu [152.3.174.14]) by mx1.freebsd.org (Postfix) with ESMTP id 679FB8FC0A for ; Mon, 17 May 2010 18:41:34 +0000 (UTC) Received: from smtp.duke.edu (localhost.localdomain [127.0.0.1]) by localhost (Postfix) with SMTP id B0CE8309025; Mon, 17 May 2010 14:41:33 -0400 (EDT) Received: from [152.3.137.189] (k.cs.duke.edu [152.3.137.189]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.duke.edu (Postfix) with ESMTP id 8EA2B309047; Mon, 17 May 2010 14:41:33 -0400 (EDT) Message-ID: <4BF18DEA.7030405@duke.edu> Date: Mon, 17 May 2010 14:41:46 -0400 From: Todd Wasson User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.9) Gecko/20100423 Thunderbird/3.0.4 MIME-Version: 1.0 To: Jeremy Chadwick References: <0B97967D-1057-4414-BBD4-4F1AA2659A5D@duke.edu> <4BF0F231.9000706@mapper.nl> <53F15A8B-77DA-4CEF-A790-2902BEC91002@duke.edu> <20100517182411.GA14251@icarus.home.lan> In-Reply-To: <20100517182411.GA14251@icarus.home.lan> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-PMX-Version: 5.4.2.338381, Antispam-Engine: 2.6.0.325393, Antispam-Data: 2010.5.17.183021 Cc: freebsd-fs@freebsd.org Subject: Re: zfs drive replacement issues X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 18:41:34 -0000 > Can you provide some details about the controller you have these disks > attached to? "dmesg | egrep -i 'ata|ahci'" should be sufficient at this > point. Hi Jeremy, sure. As the problem has been resolved, I suppose this is just for curiosity's sake, but here's the dmesg|egrep output: atapci0: port 0xdc00-0xdc07,0xd880-0xd883,0xd800-0xd807,0xd480-0xd483,0xd400-0xd40f irq 16 at device 0.0 on pci2 atapci0: [ITHREAD] ata2: on atapci0 ata2: [ITHREAD] atapci1: port 0x9c00-0x9c07,0x9880-0x9883,0x9800-0x9807,0x9480-0x9483,0x9400-0x940f,0x9080-0x908f irq 22 at device 31.2 on pci0 atapci1: [ITHREAD] ata3: on atapci1 ata3: [ITHREAD] ata4: on atapci1 ata4: [ITHREAD] atapci2: port 0xac00-0xac07,0xa880-0xa883,0xa800-0xa807,0xa480-0xa483,0xa400-0xa40f,0xa080-0xa08f irq 22 at device 31.5 on pci0 atapci2: [ITHREAD] ata5: on atapci2 ata5: [ITHREAD] ata6: on atapci2 ata6: [ITHREAD] ad4: 152627MB at ata2-master UDMA100 ad6: 476940MB at ata3-master SATA300 ad8: 381554MB at ata4-master SATA300 ad10: 476940MB at ata5-master SATA300 ad12: 476940MB at ata6-master SATA300 The motherboard in question is an ASUS P5K-VM, with an Intel ICH9 SATA controller. > SMART statistics for the drive(s) you feel might be bad would also help. > I can try my best to describe the conditions of the disk(s) based on > output from "smartctl -a /dev/adX". You can install smartmontools from > ports/sysutils/smartmontools. The problematic drive is physically disconnected from the machine at the moment, but if you think it would be helpful, I can take the new drive back out, reconnect the old one, and run smartctl on it later this evening. Thanks again. Todd From owner-freebsd-fs@FreeBSD.ORG Mon May 17 18:57:11 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E9CA21065678 for ; Mon, 17 May 2010 18:57:11 +0000 (UTC) (envelope-from ptyll@nitronet.pl) Received: from web.nitronet.pl (web.nitronet.pl [195.90.106.5]) by mx1.freebsd.org (Postfix) with ESMTP id A7FD78FC1B for ; Mon, 17 May 2010 18:57:11 +0000 (UTC) Received: from mailnull by web.nitronet.pl with virscan (Exim 4.69 (FreeBSD)) (envelope-from ) id 1OE5V0-000M0l-G8 for freebsd-fs@freebsd.org; Mon, 17 May 2010 20:57:10 +0200 Date: Mon, 17 May 2010 20:57:10 +0200 From: Pawel Tyll X-Priority: 3 (Normal) Message-ID: <1977671390.20100517205710@nitronet.pl> To: Jeremy Chadwick In-Reply-To: <20100517165352.GA12092@icarus.home.lan> References: <598841171.20100515193024@nitronet.pl> <4BF164DD.4060502@icyb.net.ua> <1554873904.20100517184702@nitronet.pl> <20100517165352.GA12092@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-2 Content-Transfer-Encoding: quoted-printable X-Virus-Scanned: Nitronet.pl X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: ptyll@nitronet.pl X-SA-Exim-Scanned: No (on web.nitronet.pl); SAEximRunCond expanded to false Cc: freebsd-fs@freebsd.org, Andriy Gapon Subject: Re: Root mount from ZFS takes forever...literally. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 18:57:12 -0000 > Are you *absolutely sure* the problem has to do with ZFS? I see you > mention PXE booting...... PXE boot was used as an emergency boot method to fix the problem. > This sounds much like an environment where serial console is set up > incorrectly; kernel messages will be seen up to the final "Mounting > root..." message, but everything past that goes to VGA console (or > serial console, depends on the configuration mishap). I thought so too at first, but then this doesn't explain lack of any disk activity after the message and network unavailability. From owner-freebsd-fs@FreeBSD.ORG Mon May 17 19:51:30 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 25BEC106564A for ; Mon, 17 May 2010 19:51:30 +0000 (UTC) (envelope-from Lassi.Tuura@cern.ch) Received: from CERNMX31.cern.ch (cernmx31.cern.ch [137.138.144.179]) by mx1.freebsd.org (Postfix) with ESMTP id 875988FC08 for ; Mon, 17 May 2010 19:51:29 +0000 (UTC) Received: from cernfe12.cern.ch (137.138.140.37) by cernmxgwlb2.cern.ch (137.138.144.179) with Microsoft SMTP Server (TLS) id 14.0.694.0; Mon, 17 May 2010 21:51:26 +0200 Received: from cmsmac01.cern.ch (137.138.52.75) by smtp.cern.ch (137.138.140.59) with Microsoft SMTP Server (TLS) id 8.2.176.0; Mon, 17 May 2010 21:51:26 +0200 MIME-Version: 1.0 (Apple Message framework v1078) Content-Type: text/plain; charset="us-ascii" From: Lassi Tuura In-Reply-To: <20100504004905.GA12233@icarus.home.lan> Date: Mon, 17 May 2010 21:51:25 +0200 Content-Transfer-Encoding: 7bit Message-ID: <6B548C21-8049-4680-96CA-07166D5102DE@cern.ch> References: <4529AF96-4BFF-4424-B77F-FE5BC8AE43D3@cern.ch> <20100504004905.GA12233@icarus.home.lan> To: Jeremy Chadwick X-Mailer: Apple Mail (2.1078) Keywords: CERN SpamKiller Note: -50 Cc: freebsd-fs@freebsd.org Subject: Re: Instant crash with ZFS + iozone? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2010 19:51:30 -0000 Hi, On May 4, 2010, at 02:49 , Jeremy Chadwick wrote: On Tue, May 04, 2010 at 01:22:35AM +0200, Lassi Tuura wrote: >> I installed 8.0-RELEASE, then 8.0-STABLE 201004 (amd64) on a system >> with 4 * 1TB hard drives for ZFS (2 * Hitachi Deskstar 7K2000, 2 * >> Samsung SpinPoint F3), plus 2GB IDE flash for OS itself (Transcend), >> 4GB ECC RAM, AMD Athlon II X2 235e CPU, Asus M4A78L-M LE motherboard. >> >> I can use the basic system fine. However when I create a ZFS volume >> out of the 4 disks and run iozone on it, the system will reliably die >> within 5 seconds or so, sometimes it takes a little longer, up to a >> minute or so. >> >> By "die" I mean the screen goes completely blank, and it will no >> longer respond to anything - no network, not even ping and any >> existing network connections will die, no keyboard, screen totally >> black without as much as a cursor... nothing. The soft power button >> won't work either. AFAIK the only thing that works is the reset >> button. > > This sounds like a hardware problem to me -- particularly excessive draw > on the PSU, voltage issues, or other whatnots. Especially if the screen > goes black (no cursor, monitor not in power save mode) and requires a > hard reset. Possibly the problem is only witnessed under extreme disk > I/O combined with high CPU usage + memory I/O. It looks like the freezes are caused by PCIe wifi card, TP-LINK TL-WN350GD which is identified as "ath0: mem 0xfebf0000-0xfebfffff irq 21 at device 7.0 on pci3". If I remove the card the system no longer hangs. Swapping PSU, memory or disks does not appear to change this, at least as far as reusing bits from another system did not add or remove freezes. I thought these Atheros chips are quite well supported in FreeBSD, but I am quite new to FreeBSD. Which forum that would be able to help me debug this, why for example it flakes under disk load? The card was by no means expensive, but I'd like to return it while I still can if it's just bad; and obviously, get it to work if this is just some silly conflict issue. Regards, Lassi From owner-freebsd-fs@FreeBSD.ORG Tue May 18 00:18:37 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 93553106566B for ; Tue, 18 May 2010 00:18:37 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 460248FC18 for ; Tue, 18 May 2010 00:18:36 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AowGAN158UuDaFvJ/2dsb2JhbACRdAEBjAJxvX+FEAQ X-IronPort-AV: E=Sophos;i="4.53,250,1272859200"; d="scan'208";a="76706437" Received: from ganges.cs.uoguelph.ca ([131.104.91.201]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 17 May 2010 20:18:35 -0400 Received: from localhost (localhost.localdomain [127.0.0.1]) by ganges.cs.uoguelph.ca (Postfix) with ESMTP id 4765EFB8036 for ; Mon, 17 May 2010 20:18:36 -0400 (EDT) X-Virus-Scanned: amavisd-new at ganges.cs.uoguelph.ca Received: from ganges.cs.uoguelph.ca ([127.0.0.1]) by localhost (ganges.cs.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SY5m+GW0rWG8 for ; Mon, 17 May 2010 20:18:35 -0400 (EDT) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by ganges.cs.uoguelph.ca (Postfix) with ESMTP id 92A91FB801F for ; Mon, 17 May 2010 20:18:35 -0400 (EDT) Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id o4I0Xmx23265 for ; Mon, 17 May 2010 20:33:48 -0400 (EDT) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Mon, 17 May 2010 20:33:48 -0400 (EDT) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: freebsd-fs@freebsd.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Subject: Re: kern/146502 running rpcbind in live DVD X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 May 2010 00:18:37 -0000 This PR (kern/146502) is w.r.t. doing an NFS mount using the live CD/DVD for FreeBSD8. The error he gets is because rpcbind isn't running. I know diddly about the live CD/DVD. Can he start rpcbind easily or does something need to be changed on the live CD/DVD so it can run rpcbind? Thanks for any help with this, rick From owner-freebsd-fs@FreeBSD.ORG Wed May 19 05:06:07 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2124E106564A for ; Wed, 19 May 2010 05:06:07 +0000 (UTC) (envelope-from morganw@chemikals.org) Received: from warped.bluecherry.net (unknown [IPv6:2001:440:eeee:fffb::2]) by mx1.freebsd.org (Postfix) with ESMTP id AAE718FC08 for ; Wed, 19 May 2010 05:06:06 +0000 (UTC) Received: from volatile.chemikals.org (adsl-67-124-127.shv.bellsouth.net [98.67.124.127]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by warped.bluecherry.net (Postfix) with ESMTPSA id 31FC286D9DB2; Wed, 19 May 2010 00:06:05 -0500 (CDT) Received: from localhost (morganw@localhost [127.0.0.1]) by volatile.chemikals.org (8.14.4/8.14.4) with ESMTP id o4J55xkK079568; Wed, 19 May 2010 00:06:00 -0500 (CDT) (envelope-from morganw@chemikals.org) Date: Wed, 19 May 2010 00:05:59 -0500 (CDT) From: Wes Morgan X-X-Sender: morganw@volatile To: Todd Wasson In-Reply-To: <53F15A8B-77DA-4CEF-A790-2902BEC91002@duke.edu> Message-ID: References: <0B97967D-1057-4414-BBD4-4F1AA2659A5D@duke.edu> <4BF0F231.9000706@mapper.nl> <53F15A8B-77DA-4CEF-A790-2902BEC91002@duke.edu> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: clamav-milter 0.95.3 at warped X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: zfs drive replacement issues X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 May 2010 05:06:07 -0000 On Mon, 17 May 2010, Todd Wasson wrote: > > Hello, > > You could try exporting and importing the pool with three disks. > > Then make sure the "new" drive isn't part of any zpool (low-level format?). > > Then try a "replace" again. > > Have fun! > > > > > Hi Mark, I was about to try this, but I just tried putting the "old" > (damaged) drive back in the pool and detaching the "new" drive from the > pool, which I've tried before, but for some reason this time it > succeeded. I was then able to "zpool offline" the old drive, physically > replace it with the new one, and "zpool replace" the old one with the > new one. It just finished successfully resilvering, and apparently > everything is working well. I'm going to initiate a scrub to be sure > that everything is alright, but I'm fairly sure that the problem is > solved. I didn't do anything that I hadn't already tried, so I don't > know why it worked this time, but I'm not complaining. Thanks to > everyone for your help; at the very least, the idea of putting the > original drive back into the machine and mucking around with it led me > in the right direction. Next time I'll be sure to issue an offline > command before replacing a device! I'm not certain that you really always want to do that. When you offline a device in a redundant pool you lose that redundancy. If you have a drive that is completely dead, it is obviously the right thing to do, but otherwise perhaps not. Were you the have another failure during the rebuild, or if there was another error on a different vdev, you wouldn't be able to recover that data because of the missing device. The same reason why offlining and replacing each device in a raidz1 to "grow" it isn't as safe as you might think -- any error could lead to data loss. Just food for thought. From owner-freebsd-fs@FreeBSD.ORG Wed May 19 08:27:10 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 462F31065672 for ; Wed, 19 May 2010 08:27:10 +0000 (UTC) (envelope-from tzim@tzim.net) Received: from golanth.tzim.net (unknown [IPv6:2001:41d0:1:d91f:21c:c0ff:fe4b:cf32]) by mx1.freebsd.org (Postfix) with ESMTP id D94E98FC12 for ; Wed, 19 May 2010 08:27:09 +0000 (UTC) Received: from 12rf.tzim.net ([82.232.60.244] helo=[192.168.0.10]) by golanth.tzim.net with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.71 (FreeBSD)) (envelope-from ) id 1OEecP-0003WM-00 for freebsd-fs@freebsd.org; Wed, 19 May 2010 10:27:09 +0200 Message-ID: <4BF3A0DD.4080404@tzim.net> Date: Wed, 19 May 2010 10:27:09 +0200 From: Arnaud Houdelette User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.9) Gecko/20100317 Lightning/1.0b1 Thunderbird/3.0.4 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Authenticated-User: tzim@tzim.net X-Authenticator: plain Subject: ZFS Recordsize tuning & transmission (bittorent daemon) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 May 2010 08:27:10 -0000 Hi, I'm using the transmission bittorent client on a single drive ZFS pool (name unsafe). Downloading is mostly OK. But moving a downloaded file to an other zpool (zraid, name tank) takes ages. The pools : [carenath] ~> zpool status pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz1 ONLINE 0 0 0 gptid/ffb4e96a-d497-11de-96bd-001d923bc7a0 ONLINE 0 0 0 gptid/9fb111f8-d426-11de-99bc-001d923bc7a0 ONLINE 0 0 0 gptid/0902db4e-d462-11de-96bd-001d923bc7a0 ONLINE 0 0 0 gptid/e3838ce5-d4ed-11de-96bd-001d923bc7a0 ONLINE 0 0 0 errors: No known data errors pool: unsafe state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM unsafe ONLINE 0 0 0 ad0p3 ONLINE 0 0 0 errors: No known data errors zpool iostat during the move shows : zpool iostat 5 capacity operations bandwidth pool used avail read write read write ---------- ----- ----- ----- ----- ----- ----- tank 2.65T 77.7G 6 8 9.65K 346K unsafe 28.4G 118G 14 10 1.77M 214K ---------- ----- ----- ----- ----- ----- ----- tank 2.65T 77.7G 88 0 131K 0 unsafe 28.4G 118G 76 11 9.28M 114K ---------- ----- ----- ----- ----- ----- ----- tank 2.65T 77.7G 102 46 147K 2.53M unsafe 28.4G 118G 79 6 9.81M 34.0K ---------- ----- ----- ----- ----- ----- ----- tank 2.65T 77.6G 2 42 4.30K 2.08M unsafe 28.4G 118G 81 3 10.1M 173K ---------- ----- ----- ----- ----- ----- ----- Why is there 10MB reads on source pool where only ~3MB are written on destination ? ZFS recordsize on both pools are default (128k). But as transmission bittorrent client has no write (nor read) cache, it could mean that data is written is smaller chunks during download. Could this lead to data being stored in many not-full records ? Does those unfull records would have to be read as whole (128k) during the move, which would explain the above difference on read/write ? I'm just making assumptions here, as my understanding of internals of ZFS is limited. Some insights would be appreciated. Thanks. Arnaud Houdelette From owner-freebsd-fs@FreeBSD.ORG Wed May 19 08:47:19 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1AA0E1065673 for ; Wed, 19 May 2010 08:47:19 +0000 (UTC) (envelope-from mgamsjager@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.159]) by mx1.freebsd.org (Postfix) with ESMTP id 9D1B18FC21 for ; Wed, 19 May 2010 08:47:18 +0000 (UTC) Received: by fg-out-1718.google.com with SMTP id 22so1901085fge.13 for ; Wed, 19 May 2010 01:47:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:cc:content-type; bh=TxkIiKajGrP4fbXg620qct58VgIHviIKqhWUCoe1z+M=; b=irlh09cCJCZ3qQ1AVisV4IhzzvhmK77vYw7AnpE0ZAoduZ5fMNeuVROWUQy8dWxQhG 7L3luNbHFqpdRU5qEWpZdq3dRsg+xgcx2xBiAi1M0IoM6b+C2D8qM83HPEo76VZrljwV 1Br5L3HD+rk/2+6u2cBaO9DsyqLM0SylAEpDg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; b=ETCLT94vDtP0VWdMYqUIqDnPrWq/4wj+3AKsBboPjbSERCnpf/oAgLDdO6dp+gqkr5 37V5lavIoWyCWxxI0zdoDX+Qeazdchxyv7m526z/lxpl3pf9Y9l4kLTnZobC+iZoRLiD R655TLTxGRUywlohW4pTlKifax5Et8OloxnWI= Received: by 10.103.78.10 with SMTP id f10mr5847896mul.126.1274258836956; Wed, 19 May 2010 01:47:16 -0700 (PDT) MIME-Version: 1.0 Received: by 10.204.78.194 with HTTP; Wed, 19 May 2010 01:46:46 -0700 (PDT) In-Reply-To: <4BF3A0DD.4080404@tzim.net> References: <4BF3A0DD.4080404@tzim.net> From: Matthias Gamsjager Date: Wed, 19 May 2010 10:46:46 +0200 Message-ID: To: Arnaud Houdelette Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS Recordsize tuning & transmission (bittorent daemon) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 May 2010 08:47:19 -0000 > > ZFS recordsize on both pools are default (128k). But as transmission > bittorrent client has no write (nor read) cache, it could mean that data is > written is smaller chunks during download. Could this lead to data being > stored in many not-full records ? Does those unfull records would have to be > read as whole (128k) during the move, which would explain the above > difference on read/write ? > > I'm just making assumptions here, as my understanding of internals of ZFS is > limited. Some insights would be appreciated. > Well ZFS does not write immediately but waits couple of seconds and write them in 1 single write operation. could you give more info about what freebsd version do you use, hardware used, zfs parameters in /boot/loader.conf and zfs info like compression etc... did you test your pool with Iozone to see if it performance as it should? From owner-freebsd-fs@FreeBSD.ORG Wed May 19 09:23:48 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C1A6A106564A for ; Wed, 19 May 2010 09:23:48 +0000 (UTC) (envelope-from tzim@tzim.net) Received: from golanth.tzim.net (unknown [IPv6:2001:41d0:1:d91f:21c:c0ff:fe4b:cf32]) by mx1.freebsd.org (Postfix) with ESMTP id 5E6088FC24 for ; Wed, 19 May 2010 09:23:48 +0000 (UTC) Received: from 12rf.tzim.net ([82.232.60.244] helo=[192.168.0.10]) by golanth.tzim.net with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.71 (FreeBSD)) (envelope-from ) id 1OEfVD-00043W-87; Wed, 19 May 2010 11:23:47 +0200 Message-ID: <4BF3AE24.9080605@tzim.net> Date: Wed, 19 May 2010 11:23:48 +0200 From: Arnaud Houdelette User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.9) Gecko/20100317 Lightning/1.0b1 Thunderbird/3.0.4 MIME-Version: 1.0 To: Matthias Gamsjager References: <4BF3A0DD.4080404@tzim.net> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Authenticated-User: tzim@tzim.net X-Authenticator: plain Cc: freebsd-fs@freebsd.org Subject: Re: ZFS Recordsize tuning & transmission (bittorent daemon) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 May 2010 09:23:48 -0000 On 19/05/2010 10:46, Matthias Gamsjager wrote: >> ZFS recordsize on both pools are default (128k). But as transmission >> bittorrent client has no write (nor read) cache, it could mean that data is >> written is smaller chunks during download. Could this lead to data being >> stored in many not-full records ? Does those unfull records would have to be >> read as whole (128k) during the move, which would explain the above >> difference on read/write ? >> >> I'm just making assumptions here, as my understanding of internals of ZFS is >> limited. Some insights would be appreciated. >> >> > Well ZFS does not write immediately but waits couple of seconds and > write them in 1 single write operation. > That I understand. But bittorrent writes are really random... I'm unsure that ZFS is able to aggregates the writes. > could you give more info about what freebsd version do you use, > hardware used, zfs parameters in /boot/loader.conf and zfs info like > compression etc... > uname -a FreeBSD carenath.tzim.net 8.0-STABLE FreeBSD 8.0-STABLE #0: Tue May 11 18:29:26 CEST 2010 tzim@carenath.tzim.net:/usr/obj/usr/src/sys/CARENATH amd64 Hardware : Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: AMD Athlon(tm) 64 Processor 3200+ (1995.24-MHz K8-class CPU) Origin = "AuthenticAMD" Id = 0x40ff2 Family = f Model = 4f Stepping = 2 Features=0x78bfbff Features2=0x2001 AMD Features=0xea500800 AMD Features2=0x1d real memory = 1610612736 (1536 MB) avail memory = 1508560896 (1438 MB) - For unsafe: atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xff00-0xff0f at device 20.1 on pci0 ad0: 152627MB at ata0-master UDMA33 - For tank ahci0: port 0xb000-0xb007,0xa000-0xa003,0x9000-0x9007,0x8000-0x8003,0x7000-0x700f mem 0xfe7ff800-0xfe7ffbff irq 22 at device 18.0 on pci0 ada0: ATA-7 SATA 2.x device ada1: ATA-7 SATA 2.x device ada2: ATA-7 SATA 2.x device ada3: ATA-7 SATA 2.x device zfs get all unsafe/dl NAME PROPERTY VALUE SOURCE unsafe/dl type filesystem - unsafe/dl creation Thu Nov 26 10:25 2009 - unsafe/dl used 26.2G - unsafe/dl available 106G - unsafe/dl referenced 12.6G - unsafe/dl compressratio 1.00x - unsafe/dl mounted yes - unsafe/dl quota none default unsafe/dl reservation 1G local unsafe/dl recordsize 128K default unsafe/dl mountpoint /store/dl local unsafe/dl sharenfs off default unsafe/dl checksum on default unsafe/dl compression off default unsafe/dl atime off local unsafe/dl devices on default unsafe/dl exec on default unsafe/dl setuid on default unsafe/dl readonly off default unsafe/dl jailed off default unsafe/dl snapdir hidden default unsafe/dl aclmode groupmask default unsafe/dl aclinherit restricted default unsafe/dl canmount on default unsafe/dl shareiscsi off default unsafe/dl xattr off temporary unsafe/dl copies 1 default unsafe/dl version 3 - unsafe/dl utf8only off - unsafe/dl normalization none - unsafe/dl casesensitivity sensitive - unsafe/dl vscan off default unsafe/dl nbmand off default unsafe/dl sharesmb off default unsafe/dl refquota none default unsafe/dl refreservation none default unsafe/dl primarycache all default unsafe/dl secondarycache all default unsafe/dl usedbysnapshots 13.6G - unsafe/dl usedbydataset 12.6G - unsafe/dl usedbychildren 0 - unsafe/dl usedbyrefreservation 0 - cat /boot/loader.conf ahci_load="YES" zfs_load="YES" vfs.root.mountfrom="zfs:unsafe/root" #vm.kmem_size="512M" vm.kmem_size_max="512M" vfs.zfs.arc_max="150M" vfs.zfs.arc_min="64M" vfs.zfs.vdev.cache.size="10M" vfs.zfs.prefetch_disable="0" Bad performance IS expected on this hardware. This is a home NAS, and the "unsafe" pool is on a laptop 2.5" IDE drive. Still, bad performance would'nt explain the discrepancies between read and write stats (both in zpool io stat and gstat). > did you test your pool with Iozone to see if it performance as it should? > > I did not. I just installed the port. What test should I run to get relevant data ? Thanks. From owner-freebsd-fs@FreeBSD.ORG Wed May 19 09:47:33 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D48E31065674 for ; Wed, 19 May 2010 09:47:33 +0000 (UTC) (envelope-from mgamsjager@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.158]) by mx1.freebsd.org (Postfix) with ESMTP id 613FE8FC13 for ; Wed, 19 May 2010 09:47:32 +0000 (UTC) Received: by fg-out-1718.google.com with SMTP id l26so2963824fgb.13 for ; Wed, 19 May 2010 02:47:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:cc:content-type; bh=4zNslirB/HY07WUWgVnctRHYb8U71RHkfPn/2Vdw0DU=; b=TUUaqkxSGg8jZAYndtnTGqTTeFICdendXsGW3p4KHHSFX+0fCq9ZKwWEBgAHpSEdmZ RjH+Jp2ZTan7nWCoJB8cqebqYb70kKtpeWEfSlucAw87PlRXb0UtqFhn+QfyWsBq8E9f Bb6a4SLkltr5qq2Jd9kj/eRiyQt6+VKxh2KXA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; b=s+UcNCxrKTgdTVvJngUBsJd4DWLCGATdSA/awH/nxVQD3yPmXCuJBWfzAJ/iqKEje7 dcboXaeAYCCNconkMUk6IDJDuQsX9M41wnBy3wb7WkvNWjgwk+4kl+HMwbpXPDtRtxLq l1ppoNvacOJ5SFYtz/ZFR2oqW2JAmq9O7yS+M= Received: by 10.87.61.22 with SMTP id o22mr12575445fgk.50.1274262452149; Wed, 19 May 2010 02:47:32 -0700 (PDT) MIME-Version: 1.0 Received: by 10.86.26.18 with HTTP; Wed, 19 May 2010 02:47:02 -0700 (PDT) In-Reply-To: <4BF3AE24.9080605@tzim.net> References: <4BF3A0DD.4080404@tzim.net> <4BF3AE24.9080605@tzim.net> From: Matthias Gamsjager Date: Wed, 19 May 2010 11:47:02 +0200 Message-ID: To: Arnaud Houdelette Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS Recordsize tuning & transmission (bittorent daemon) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 May 2010 09:47:33 -0000 > cat /boot/loader.conf > ahci_load="YES" > zfs_load="YES" > vfs.root.mountfrom="zfs:unsafe/root" > #vm.kmem_size="512M" > vm.kmem_size_max="512M" > vfs.zfs.arc_max="150M" > vfs.zfs.arc_min="64M" > vfs.zfs.vdev.cache.size="10M" > vfs.zfs.prefetch_disable="0" Your on AMD64, try to remove all zfs stuff in loader.conf. Try the default and see how it works out. How did you come up with these settings anyway? > Bad performance IS expected on this hardware. This is a home NAS, and the > "unsafe" pool is on a laptop 2.5" IDE drive. > Still, bad performance would'nt explain the discrepancies between read and > write stats (both in zpool io stat and gstat). > >> did you test your pool with Iozone to see if it performance as it should? >> >> > > I did not. I just installed the port. What test should I run to get relevant > data ? iozone -r 128k -s 4g -t1 -x should do the trick. From owner-freebsd-fs@FreeBSD.ORG Wed May 19 11:43:13 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 18D5C106566B for ; Wed, 19 May 2010 11:43:13 +0000 (UTC) (envelope-from mike.barnardq@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 88BAF8FC0C for ; Wed, 19 May 2010 11:43:12 +0000 (UTC) Received: by fxm4 with SMTP id 4so914625fxm.13 for ; Wed, 19 May 2010 04:43:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=RFzURuG6q4UgefV6LwAo/Ur4pE7BNwEZTF1ipQV5/AI=; b=YNhBBBG7D9d8Vdk4LGaWJ5/lllUntDOCTYCIGgAQX2l40zhA8HG1t5GxtuPc2hHA1g 5pCqdGZ5t79vPgcAeXm94V5m6cjXmFnPBDd+9Hw5G1gLklwJtWDeMnLEUT/YZnE1kEeK /4iA19IYUNVEyaSI+TyNbctCRGd4GR8QJH3uI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=j+NI/h4o7bP6oncvCLVcQ+v0UMZXbjKIvknPcqMH/VDBdH1IxHz560P2cGXgE2L6X9 I9YCvwaXNbWHXtqfB2TF711k9tAssJaWz2N76MDHTi7YkORr2+lZ3786fEzi99VX9VDX lb39ks94Chev+iFyClvwlgG1s9HSFZiqUxJb8= MIME-Version: 1.0 Received: by 10.223.30.10 with SMTP id s10mr2864920fac.4.1274269391453; Wed, 19 May 2010 04:43:11 -0700 (PDT) Received: by 10.223.105.146 with HTTP; Wed, 19 May 2010 04:43:11 -0700 (PDT) In-Reply-To: References: Date: Wed, 19 May 2010 14:43:11 +0300 Message-ID: From: Mike Barnard To: bf1783@gmail.com Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: UFS Journaling gone south X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 May 2010 11:43:13 -0000 On Mon, May 17, 2010 at 8:38 PM, Mike Barnard wrote: > > > On Mon, May 17, 2010 at 2:47 PM, Mike Barnard wrote >> >> >> >> Prior to rebooting, I checked and I did have da0s1f.journal and >> da0s1g.journal and the da0s2d and da0s2e devices, after the reboot and the >> panic, I check /dev and the seem to have reverted back to da0s1f and >> da0s1g... and da0s2d and da0s2e have also gone AWOL... :-w >> >> fortunately, this is a new installation on which I want to test journal >> devices. I'll do this again, reserving some sectors on both partitions. >> >> > I must admit, this has me hands down. I have entered custom newfs flags on > sysinstall (-r 2, 4, and 8). > > With a test on these sector sizes to be reserved at the end of the disk, > gjournal label da0s1f da0s2d still says that I need to use -f to overwrite. > I do so and end up with journal devices, but when I reboot... the journal > devices and the journal providers are missing. > > I have even done -J flag on the newfs options in sysinstall but I end up > with the same results... no journal devices are created and I end up with a > kernel panic and into single usermode. There are no journal devices created. > Mounting the disks fails with: warning: GJOURNAL flag on fs but no gjournal > provider below. > > Am I doing all this wrong, Am I missing something? > > I have redone this setup with FreeBSD 8.0 STABLE but I get the same results. My da0s1f.journal and da0s1g.journal disappear and I get this: mount: /dev/da0s1g.journal : No such file or directory mount: /dev/da0s1f.journal : No such file or directory mount: /dev/da0s1g.journal : No such file or directory While in single usermode, an ls /dev still shows that da0s2* are not present. Is there a particular reason I'd have the journal devices vanish? Is there a solution for this? -- Mike Of course, you might discount this possibility, but remember that one in a million chances happen 99% of the time. ------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Wed May 19 13:04:04 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 89951106566B for ; Wed, 19 May 2010 13:04:04 +0000 (UTC) (envelope-from mike.barnardq@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 0AA588FC17 for ; Wed, 19 May 2010 13:04:03 +0000 (UTC) Received: by fxm4 with SMTP id 4so1013469fxm.13 for ; Wed, 19 May 2010 06:04:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type; bh=DoYSCHQs7L1Il8tBSGPWqojypL4b0MAo9FI9nXiSvyk=; b=KfJaPLjoVd/RptxkakSq+f7zIrJXQ5dF9sq5eP0+ZIovIIPyS79T4pa33QWgKWmmBq GiKEkXGMZ7We8+ck4FLMKRVJA+0ZKJJJdGPqkCDctslmITL2tOnDkc9ZpkmfLVe1uH2B u+o5F3OgkTdDRyoq9lozPGHruXltV97MfL1CU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=gdy+fcL19lsgJNie2RCZH59WNYbp8TTQW9AVdJ6+kxC7/D7vvOibpNl063ZcljDlm4 TL8yfV0Ue/gKr/0rvPeUlrtgfw+p4buje12E2PHYB208gXW7gdFWDCpSmavTQV4qon5B YfHmpda7HWn8d4NebLIfdapRCi+nNze0glHV8= MIME-Version: 1.0 Received: by 10.223.65.18 with SMTP id g18mr3715366fai.32.1274274242768; Wed, 19 May 2010 06:04:02 -0700 (PDT) Received: by 10.223.105.146 with HTTP; Wed, 19 May 2010 06:04:02 -0700 (PDT) In-Reply-To: References: Date: Wed, 19 May 2010 16:04:02 +0300 Message-ID: From: Mike Barnard To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: UFS Journaling gone south X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 May 2010 13:04:04 -0000 Am I on the wrong mailing list with this question... should I bump it over to another... if so, which one, freebsd-geom? On Wed, May 19, 2010 at 2:43 PM, Mike Barnard wrote: > > > On Mon, May 17, 2010 at 8:38 PM, Mike Barnard wrote: > >> >> >> On Mon, May 17, 2010 at 2:47 PM, Mike Barnard wrote >>> >>> >>> >>> Prior to rebooting, I checked and I did have da0s1f.journal and >>> da0s1g.journal and the da0s2d and da0s2e devices, after the reboot and the >>> panic, I check /dev and the seem to have reverted back to da0s1f and >>> da0s1g... and da0s2d and da0s2e have also gone AWOL... :-w >>> >>> fortunately, this is a new installation on which I want to test journal >>> devices. I'll do this again, reserving some sectors on both partitions. >>> >>> >> I must admit, this has me hands down. I have entered custom newfs flags on >> sysinstall (-r 2, 4, and 8). >> >> With a test on these sector sizes to be reserved at the end of the disk, >> gjournal label da0s1f da0s2d still says that I need to use -f to overwrite. >> I do so and end up with journal devices, but when I reboot... the journal >> devices and the journal providers are missing. >> >> I have even done -J flag on the newfs options in sysinstall but I end up >> with the same results... no journal devices are created and I end up with a >> kernel panic and into single usermode. There are no journal devices created. >> Mounting the disks fails with: warning: GJOURNAL flag on fs but no gjournal >> provider below. >> >> Am I doing all this wrong, Am I missing something? >> >> > I have redone this setup with FreeBSD 8.0 STABLE but I get the same > results. My da0s1f.journal and da0s1g.journal disappear and I get this: > > mount: /dev/da0s1g.journal : No such file or directory > mount: /dev/da0s1f.journal : No such file or directory > mount: /dev/da0s1g.journal : No such file or directory > > While in single usermode, an ls /dev still shows that da0s2* are not > present. > > Is there a particular reason I'd have the journal devices vanish? Is there > a solution for this? > > > > > -- > Mike > > Of course, you might discount this possibility, but remember that one in > a million chances happen 99% of the time. > ------------------------------------------------------------ > -- Mike Of course, you might discount this possibility, but remember that one in a million chances happen 99% of the time. ------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Wed May 19 13:32:52 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 91EEC1065675 for ; Wed, 19 May 2010 13:32:52 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 4D5438FC1B for ; Wed, 19 May 2010 13:32:51 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1OEjOE-0007Dr-Nt for freebsd-fs@freebsd.org; Wed, 19 May 2010 15:32:50 +0200 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 19 May 2010 15:32:50 +0200 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 19 May 2010 15:32:50 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org connect(): No such file or directory From: Ivan Voras Date: Wed, 19 May 2010 15:32:42 +0200 Lines: 19 Message-ID: References: <4BF3A0DD.4080404@tzim.net> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.9) Gecko/20100518 Thunderbird/3.0.4 In-Reply-To: <4BF3A0DD.4080404@tzim.net> X-Enigmail-Version: 1.0.1 Subject: Re: ZFS Recordsize tuning & transmission (bittorent daemon) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 May 2010 13:32:52 -0000 On 05/19/10 10:27, Arnaud Houdelette wrote: > Hi, > > I'm using the transmission bittorent client on a single drive ZFS pool > (name unsafe). Downloading is mostly OK. > But moving a downloaded file to an other zpool (zraid, name tank) takes > ages. You will generally observe the same problem with any file system you can use because torrent (and other similar p2p) downloads target random chunks of data in the files. In most other "less smart" file systems (e.g. UFS) you can preallocate the space (not by using sparse files but by really writing zeroes or random junk in the length of the total file size), which you can't in ZFS (or technically, can but you won't benefit from it). You cannot "escape" from this feature of ZFS no matter what you do (except by using expensive hardware). From owner-freebsd-fs@FreeBSD.ORG Wed May 19 13:39:42 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B503E1065676 for ; Wed, 19 May 2010 13:39:42 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id F35628FC18 for ; Wed, 19 May 2010 13:39:41 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1OEjUp-0002JB-K2 for freebsd-fs@freebsd.org; Wed, 19 May 2010 15:39:39 +0200 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 19 May 2010 15:39:39 +0200 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 19 May 2010 15:39:39 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org connect(): No such file or directory From: Ivan Voras Date: Wed, 19 May 2010 15:39:31 +0200 Lines: 16 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.9) Gecko/20100518 Thunderbird/3.0.4 In-Reply-To: X-Enigmail-Version: 1.0.1 Subject: Re: UFS Journaling gone south X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 May 2010 13:39:42 -0000 On 05/19/10 15:04, Mike Barnard wrote: > Am I on the wrong mailing list with this question... should I bump it over > to another... if so, which one, freebsd-geom? You have written too much and done too much for anyone to correctly follow. I suggest you start over: 1. Destroy all your data & journal partitions/slices/devices. Be sure all metadata is also destroyed. 2. Configure /boot/loader.conf to load gjournal 3. Use gjournal "-h" switch when creating gjournal devices 4. Use newfs "-J" switch when creating file systems on the gjournal devices 5. Use "async" mount option when mounting those file systems Or, if fsck times are your main concern, wait for UFS+SUJ to be MFC-ed (which should happen before 8.2, maybe earlier). From owner-freebsd-fs@FreeBSD.ORG Wed May 19 14:53:56 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 01735106566C for ; Wed, 19 May 2010 14:53:56 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from blade.simplesystems.org (blade.simplesystems.org [65.66.246.74]) by mx1.freebsd.org (Postfix) with ESMTP id BE24F8FC08 for ; Wed, 19 May 2010 14:53:55 +0000 (UTC) Received: from freddy.simplesystems.org (freddy.simplesystems.org [65.66.246.65]) by blade.simplesystems.org (8.13.8+Sun/8.13.8) with ESMTP id o4JErs03024839; Wed, 19 May 2010 09:53:54 -0500 (CDT) Date: Wed, 19 May 2010 09:53:54 -0500 (CDT) From: Bob Friesenhahn X-X-Sender: bfriesen@freddy.simplesystems.org To: Arnaud Houdelette In-Reply-To: <4BF3A0DD.4080404@tzim.net> Message-ID: References: <4BF3A0DD.4080404@tzim.net> User-Agent: Alpine 2.01 (GSO 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (blade.simplesystems.org [65.66.246.90]); Wed, 19 May 2010 09:53:54 -0500 (CDT) Cc: freebsd-fs@freebsd.org Subject: Re: ZFS Recordsize tuning & transmission (bittorent daemon) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 May 2010 14:53:56 -0000 On Wed, 19 May 2010, Arnaud Houdelette wrote: > > ZFS recordsize on both pools are default (128k). But as transmission > bittorrent client has no write (nor read) cache, it could mean that data is > written is smaller chunks during download. Could this lead to data being > stored in many not-full records ? Does those unfull records would have to be > read as whole (128k) during the move, which would explain the above > difference on read/write ? Zfs always writes full blocks. If bittorrent updates just part of the blocks, then the whole block will be re-written to a new location via COW. This causes file fragmentation. The file fragmentation will be worse if the pool is very full. There should be less fragmentation if the system has quite a lot of RAM for buffering the writes. With a lot of RAM, zfs will write less often (waiting as long as 30 seconds). If your filesystem is only used for bittorrent and if bittorrent always uses the same block size, then it may be useful to set the receiving filesystem to use the bittorrent block size. If the bittorrent block size is variable, then this tuning is not possible. Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From owner-freebsd-fs@FreeBSD.ORG Wed May 19 15:44:16 2010 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1EEB81065670; Wed, 19 May 2010 15:44:16 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id CDF138FC16; Wed, 19 May 2010 15:44:15 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 52A5246BC5; Wed, 19 May 2010 11:44:15 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 57F228A01F; Wed, 19 May 2010 11:44:14 -0400 (EDT) From: John Baldwin To: fs@freebsd.org Date: Wed, 19 May 2010 11:44:00 -0400 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201005191144.00382.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Wed, 19 May 2010 11:44:14 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Rick Macklem , Robert Watson Subject: [PATCH] Better handling of stale filehandles in open() in the NFS client X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 May 2010 15:44:16 -0000 One of the things the NFS client does to provide close-to-open consistency is that the client mandates that at least one ACCESS or GETATTR RPC is sent over the wire as part of every open(2) system call. However, we currently only enforce that during nfs_open() (VOP_OPEN()). If nfs_open() encounters a stale file handle, it fails the open(2) system call with ESTALE. A much nicer user experience is for nfs_lookup() to actually send the ACCESS or GETATTR RPC instead. If that RPC fails with ESTALE, then nfs_lookup() will send a LOOKUP RPC which will find the new file handle (assuming a rename has caused the file handle for a given filename to change) and the open(2) will succeed with the new file handle. I believe that this in fact used to happen quite often until I merged a change from Yahoo! which stopped flushing cached attributes during nfs_close(). With that change an open() -> close() -> open() sequence in quick succession will now use cached attributes during the lookup and only notice a stale filehandle in nfs_open(). This can lead to some astonishing behavior. To reproduce, run 'cat /some/file' in an loop every 2 seconds or so on an NFS client. In another window, login to the NFS server and replace /some/file with /some/otherfile using mv(1). The next cat in the NFS client window will usually fail with ESTALE. The subsequent cat will work as it will relookup the filename and find the new filehandle. The fix I came up with is to modify the NFS client lookup routine. Before we trust a hit in the namecache, we check the attributes to see if we should trust the namecache hit. What my patch does is to force that attribute check to send a GETATTR or ACCESS RPC over the wire instead of using cached attributes when doing a lookup on the last component of an ISOPEN lookup (so a lookup for open(2) or execve(2)). This forces the ESTALE error to occur during the VOP_LOOKUP() stage of open(2) instead of VOP_OPEN(). Thoughts? Index: nfs_vnops.c =================================================================== --- nfs_vnops.c (revision 208707) +++ nfs_vnops.c (working copy) @@ -875,7 +875,7 @@ struct mbuf *mreq, *mrep, *md, *mb; long len; nfsfh_t *fhp; - struct nfsnode *np; + struct nfsnode *np, *newnp; int error = 0, attrflag, fhsize; int v3 = NFS_ISV3(dvp); struct thread *td = cnp->cn_thread; @@ -901,8 +901,24 @@ * change time of the file matches our cached copy. * Otherwise, we discard the cache entry and fallback * to doing a lookup RPC. + * + * To better handle stale file handles, clear the + * attribute cache of this node if it is a leaf + * component and part of an open() call before + * fetching the attributes. This should allow stale + * file handles to be detected here where we can fall + * back to a LOOKUP RPC to recover rather than having + * nfs_open() detect the stale file handle and failing + * open(2) with ESTALE. */ newvp = *vpp; + if ((cnp->cn_flags & (ISLASTCN | ISOPEN)) == + (ISLASTCN | ISOPEN)) { + newnp = VTONFS(newvp); + mtx_lock(&newnp->n_mtx); + newnp->n_attrstamp = 0; + mtx_unlock(&newnp->n_mtx); + } if (!VOP_GETATTR(newvp, &vattr, cnp->cn_cred, td) && vattr.va_ctime.tv_sec == VTONFS(newvp)->n_ctime) { nfsstats.lookupcache_hits++; -- John Baldwin From owner-freebsd-fs@FreeBSD.ORG Wed May 19 17:59:55 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3BBF6106566B for ; Wed, 19 May 2010 17:59:55 +0000 (UTC) (envelope-from tsw5@duke.edu) Received: from smtp.duke.edu (smtp-03.oit.duke.edu [152.3.174.16]) by mx1.freebsd.org (Postfix) with ESMTP id 11EE58FC0A for ; Wed, 19 May 2010 17:59:54 +0000 (UTC) Received: from smtp.duke.edu (localhost.localdomain [127.0.0.1]) by localhost (Postfix) with SMTP id 1752E8204; Wed, 19 May 2010 13:59:54 -0400 (EDT) Received: from [152.3.137.189] (k.cs.duke.edu [152.3.137.189]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.duke.edu (Postfix) with ESMTP id F213E8052; Wed, 19 May 2010 13:59:53 -0400 (EDT) Message-ID: <4BF4272A.9020204@duke.edu> Date: Wed, 19 May 2010 14:00:10 -0400 From: Todd Wasson User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.9) Gecko/20100423 Thunderbird/3.0.4 MIME-Version: 1.0 To: Wes Morgan References: <0B97967D-1057-4414-BBD4-4F1AA2659A5D@duke.edu> <4BF0F231.9000706@mapper.nl> <53F15A8B-77DA-4CEF-A790-2902BEC91002@duke.edu> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-PMX-Version: 5.4.2.338381, Antispam-Engine: 2.6.0.325393, Antispam-Data: 2010.5.19.174815 Cc: freebsd-fs@freebsd.org Subject: Re: zfs drive replacement issues X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 May 2010 17:59:55 -0000 > I'm not certain that you really always want to do that. When you offline a > device in a redundant pool you lose that redundancy. If you have a drive > that is completely dead, it is obviously the right thing to do, but > otherwise perhaps not. Were you the have another failure during the > rebuild, or if there was another error on a different vdev, you wouldn't > be able to recover that data because of the missing device. The same > reason why offlining and replacing each device in a raidz1 to "grow" it > isn't as safe as you might think -- any error could lead to data loss. > > Just food for thought. I'm guessing the implication here is that it's better to connect the new drive in addition to the old one, then, like via USB or eSATA interface? In a machine with no extra interfaces, as was the case for me, offlining it seemed to be the only choice. Maybe it's just time to invest in a USB SATA cable, just in case... Thanks! Todd From owner-freebsd-fs@FreeBSD.ORG Thu May 20 00:27:05 2010 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8D02B106566C for ; Thu, 20 May 2010 00:27:05 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 3F4238FC15 for ; Thu, 20 May 2010 00:27:05 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AvsEACMY9EuDaFvG/2dsb2JhbACeAXG9e4J2AYIZBA X-IronPort-AV: E=Sophos;i="4.53,266,1272859200"; d="scan'208";a="77016758" Received: from amazon.cs.uoguelph.ca ([131.104.91.198]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 19 May 2010 19:56:56 -0400 Received: from localhost (localhost.localdomain [127.0.0.1]) by amazon.cs.uoguelph.ca (Postfix) with ESMTP id E8C1F210193; Wed, 19 May 2010 19:56:56 -0400 (EDT) X-Virus-Scanned: amavisd-new at amazon.cs.uoguelph.ca Received: from amazon.cs.uoguelph.ca ([127.0.0.1]) by localhost (amazon.cs.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PF53E3H7nOxh; Wed, 19 May 2010 19:56:55 -0400 (EDT) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by amazon.cs.uoguelph.ca (Postfix) with ESMTP id 67CA42101DF; Wed, 19 May 2010 19:56:54 -0400 (EDT) Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id o4K0CBs10768; Wed, 19 May 2010 20:12:12 -0400 (EDT) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Wed, 19 May 2010 20:12:10 -0400 (EDT) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: John Baldwin In-Reply-To: <201005191144.00382.jhb@freebsd.org> Message-ID: References: <201005191144.00382.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Rick Macklem , Robert Watson , fs@freebsd.org Subject: Re: [PATCH] Better handling of stale filehandles in open() in the NFS client X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 May 2010 00:27:05 -0000 On Wed, 19 May 2010, John Baldwin wrote: > One of the things the NFS client does to provide close-to-open consistency is > that the client mandates that at least one ACCESS or GETATTR RPC is sent over > the wire as part of every open(2) system call. However, we currently only > enforce that during nfs_open() (VOP_OPEN()). If nfs_open() encounters a stale > file handle, it fails the open(2) system call with ESTALE. > > A much nicer user experience is for nfs_lookup() to actually send the ACCESS > or GETATTR RPC instead. If that RPC fails with ESTALE, then nfs_lookup() will > send a LOOKUP RPC which will find the new file handle (assuming a rename has > caused the file handle for a given filename to change) and the open(2) will > succeed with the new file handle. I believe that this in fact used to happen > quite often until I merged a change from Yahoo! which stopped flushing cached > attributes during nfs_close(). With that change an open() -> close() -> > open() sequence in quick succession will now use cached attributes during the > lookup and only notice a stale filehandle in nfs_open(). > > This can lead to some astonishing behavior. To reproduce, run 'cat > /some/file' in an loop every 2 seconds or so on an NFS client. In another > window, login to the NFS server and replace /some/file with /some/otherfile > using mv(1). The next cat in the NFS client window will usually fail with > ESTALE. The subsequent cat will work as it will relookup the filename and > find the new filehandle. > Not astonishing at all:-) That's just NFS not having any cache coherency protocol. (Many moons ago, I tried via nqnfs, but nobody cared.:-) Btw, many server's don't change a file handle upon a rename and it was once considered bad form to do so, but nowadays some don't and some do. > The fix I came up with is to modify the NFS client lookup routine. Before we > trust a hit in the namecache, we check the attributes to see if we should > trust the namecache hit. What my patch does is to force that attribute check > to send a GETATTR or ACCESS RPC over the wire instead of using cached > attributes when doing a lookup on the last component of an ISOPEN lookup (so a > lookup for open(2) or execve(2)). This forces the ESTALE error to occur > during the VOP_LOOKUP() stage of open(2) instead of VOP_OPEN(). > > Thoughts? > It sounds fine but seems like it's going to increase the Getattr RPC cnt since nfs_open() invalidates the attribute cache for some cases? Did you happen to try something like a "make buildworld" with and without the patch and compare RPC counts? I'd say sounds great so long as the RPC counts don't go up much. If they do, I suspect somebody won't be happy. (When I talked to Alfred last week, all Juniper cares about is build performance and doesn't care diddly w.r.t. coherence between multiple clients/client and server.) Have fun with it, rick From owner-freebsd-fs@FreeBSD.ORG Thu May 20 09:14:20 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 49EEF1065676 for ; Thu, 20 May 2010 09:14:20 +0000 (UTC) (envelope-from staale@kristoffersen.ws) Received: from putsch.kolbu.ws (cl-225.sto-01.se.sixxs.net [IPv6:2001:16d8:ff00:e0::2]) by mx1.freebsd.org (Postfix) with ESMTP id 0CEBD8FC2F for ; Thu, 20 May 2010 09:14:20 +0000 (UTC) Received: from chiller by putsch.kolbu.ws with local (Exim 4.71 (FreeBSD)) (envelope-from ) id 1OEoVw-000IDz-9J for freebsd-fs@freebsd.org; Wed, 19 May 2010 21:01:08 +0200 Date: Wed, 19 May 2010 21:01:08 +0200 From: =?iso-8859-1?Q?St=E5le?= Kristoffersen To: freebsd-fs@freebsd.org Message-ID: <20100519190108.GA41394@putsch.kolbu.ws> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit User-Agent: Mutt/1.5.18 (2008-05-17) Subject: zdb crash X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 May 2010 09:14:20 -0000 As I posted about a while ago I had memory corruption, after fixing the problem I ran a scrub that found a few errors that I fixed. I then started zdb -cvv to check the metadata. Thats 8 days ago. It didn't print out anything, just consumed 600MB of memory and 5% of the cpu. This morning I saw that it had crashed: # zdb -cvv media Traversing all blocks to verify checksums and verify nothing leaked ... zdb_blkptr_cb: Got error 86 reading <441, 8198, 0, 1ff51> -- skipping zdb_blkptr_cb: Got error 86 reading <447, 0, 1, 0> -- skipping error: zfs: freeing free segment (offset=4372277157888 size=8192) [1] 16661 abort zdb -cvv media And in the log file I saw: pid 16661 (zdb), uid 0: exited on signal 6 I take it something is wrong, but what? And how do I fix it? The file system seems to be working great. -- Ståle Kristoffersen staalebk@ifi.uio.no From owner-freebsd-fs@FreeBSD.ORG Thu May 20 13:33:15 2010 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A0D3A106566C; Thu, 20 May 2010 13:33:15 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 5BC868FC1C; Thu, 20 May 2010 13:33:15 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id E7E9F46BA1; Thu, 20 May 2010 09:33:14 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id D26FD8A021; Thu, 20 May 2010 09:33:13 -0400 (EDT) From: John Baldwin To: Rick Macklem Date: Thu, 20 May 2010 09:22:17 -0400 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <201005191144.00382.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201005200922.17245.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Thu, 20 May 2010 09:33:14 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.4 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Rick Macklem , Robert Watson , fs@freebsd.org Subject: Re: [PATCH] Better handling of stale filehandles in open() in the NFS client X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 May 2010 13:33:15 -0000 On Wednesday 19 May 2010 8:12:10 pm Rick Macklem wrote: > > On Wed, 19 May 2010, John Baldwin wrote: > > > One of the things the NFS client does to provide close-to-open consistency is > > that the client mandates that at least one ACCESS or GETATTR RPC is sent over > > the wire as part of every open(2) system call. However, we currently only > > enforce that during nfs_open() (VOP_OPEN()). If nfs_open() encounters a stale > > file handle, it fails the open(2) system call with ESTALE. > > > > A much nicer user experience is for nfs_lookup() to actually send the ACCESS > > or GETATTR RPC instead. If that RPC fails with ESTALE, then nfs_lookup() will > > send a LOOKUP RPC which will find the new file handle (assuming a rename has > > caused the file handle for a given filename to change) and the open(2) will > > succeed with the new file handle. I believe that this in fact used to happen > > quite often until I merged a change from Yahoo! which stopped flushing cached > > attributes during nfs_close(). With that change an open() -> close() -> > > open() sequence in quick succession will now use cached attributes during the > > lookup and only notice a stale filehandle in nfs_open(). > > > > This can lead to some astonishing behavior. To reproduce, run 'cat > > /some/file' in an loop every 2 seconds or so on an NFS client. In another > > window, login to the NFS server and replace /some/file with /some/otherfile > > using mv(1). The next cat in the NFS client window will usually fail with > > ESTALE. The subsequent cat will work as it will relookup the filename and > > find the new filehandle. > > > > Not astonishing at all:-) That's just NFS not having any cache coherency > protocol. (Many moons ago, I tried via nqnfs, but nobody cared.:-) > Btw, many server's don't change a file handle upon a rename and it was > once considered bad form to do so, but nowadays some don't and some do. True, though I guess that implies that CTO doesn't cover renames, only open and close of a given filehandle. It's probably non-obvious to many users of NFS though. > > The fix I came up with is to modify the NFS client lookup routine. Before we > > trust a hit in the namecache, we check the attributes to see if we should > > trust the namecache hit. What my patch does is to force that attribute check > > to send a GETATTR or ACCESS RPC over the wire instead of using cached > > attributes when doing a lookup on the last component of an ISOPEN lookup (so a > > lookup for open(2) or execve(2)). This forces the ESTALE error to occur > > during the VOP_LOOKUP() stage of open(2) instead of VOP_OPEN(). > > > > Thoughts? > > > > It sounds fine but seems like it's going to increase the Getattr RPC cnt > since nfs_open() invalidates the attribute cache for some cases? It doesn't change the RPC count because of changes that Mohan added to the NFS client a while ago so that nfs_open() doesn't invalide the attribute cache during nfs_open() if it was already updated via nfs_lookup() during the same system call. With Mohan's changes in place, all this change does is move the GETATTR/ACCESS RPC earlier in the case of a namecache hit. -- John Baldwin From owner-freebsd-fs@FreeBSD.ORG Fri May 21 00:29:55 2010 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3CFF9106566C; Fri, 21 May 2010 00:29:55 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id A549E8FC13; Fri, 21 May 2010 00:29:54 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AvsEAHRw9UuDaFvK/2dsb2JhbACeHXG/HoJcgjYE X-IronPort-AV: E=Sophos;i="4.53,274,1272859200"; d="scan'208";a="77162935" Received: from fraser.cs.uoguelph.ca ([131.104.91.202]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 20 May 2010 20:29:53 -0400 Received: from localhost (localhost.localdomain [127.0.0.1]) by fraser.cs.uoguelph.ca (Postfix) with ESMTP id B27E9109C2E5; Thu, 20 May 2010 20:29:53 -0400 (EDT) X-Virus-Scanned: amavisd-new at fraser.cs.uoguelph.ca Received: from fraser.cs.uoguelph.ca ([127.0.0.1]) by localhost (fraser.cs.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id M6ENSUL+MRzn; Thu, 20 May 2010 20:29:53 -0400 (EDT) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by fraser.cs.uoguelph.ca (Postfix) with ESMTP id 463DD109C2E1; Thu, 20 May 2010 20:29:53 -0400 (EDT) Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id o4L0jD416381; Thu, 20 May 2010 20:45:13 -0400 (EDT) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Thu, 20 May 2010 20:45:13 -0400 (EDT) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: John Baldwin In-Reply-To: <201005200922.17245.jhb@freebsd.org> Message-ID: References: <201005191144.00382.jhb@freebsd.org> <201005200922.17245.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Rick Macklem , Robert Watson , fs@freebsd.org Subject: Re: [PATCH] Better handling of stale filehandles in open() in the NFS client X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 May 2010 00:29:55 -0000 On Thu, 20 May 2010, John Baldwin wrote: > > It doesn't change the RPC count because of changes that Mohan added to the > NFS client a while ago so that nfs_open() doesn't invalide the attribute > cache during nfs_open() if it was already updated via nfs_lookup() during > the same system call. With Mohan's changes in place, all this change does > is move the GETATTR/ACCESS RPC earlier in the case of a namecache hit. > Well, it sounds like a good theory. Something like: - VOP_LOOKUP() locks the vnode, which is then passed to VOP_OPEN() and since it is locked, other threads can't perform VOPs on the vp. I ran a single pass of a kernel "make cleandepend; make depend; make" here (1 without the patch and one with the patch). Now, it could be random variation, but since the other RPC counts changed by < 1%, I suspect not. (I'll run another pass of each to see how much variation I see w.r.t. Getattr.) Here's the counts for the 5 RPCs I think might be interesting: RPC Count without patch Count with patch Getattr 590936 625987 +5.9% Lookup 157194 157528 Access 59040 59690 +1.1% Read 70585 70586 Write 112531 112530 I let you know what another pass of each gives, but it looks like it has caused an increase in RPC cnts. I don't know if the increase is enough to deter adding the patch, but it might be worth exploring it more? rick From owner-freebsd-fs@FreeBSD.ORG Fri May 21 12:13:32 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9EB8F1065678; Fri, 21 May 2010 12:13:32 +0000 (UTC) (envelope-from eugene@imedia.ru) Received: from lynx.imedia.ru (lynx.imedia.ru [212.65.64.254]) by mx1.freebsd.org (Postfix) with ESMTP id 1A3DB8FC15; Fri, 21 May 2010 12:13:30 +0000 (UTC) Received: from badger.imedia.ru (root@badger.imedia.ru [10.167.1.243]) by lynx.imedia.ru (8.14.3/8.14.3/TWINS7_LDAP) with ESMTP id o4LBu0NN001974; Fri, 21 May 2010 15:56:00 +0400 (MSD) (envelope-from eugene@imedia.ru) Received: from badger.imedia.ru (eugene@localhost [127.0.0.1]) by badger.imedia.ru (8.14.3/8.13.1) with ESMTP id o4LBtxls003121; Fri, 21 May 2010 15:55:59 +0400 (MSD) (envelope-from eugene@imedia.ru) Received: from localhost (localhost [[UNIX: localhost]]) by badger.imedia.ru (8.14.3/8.13.8/Submit) id o4LBtxjs003120; Fri, 21 May 2010 15:55:59 +0400 (MSD) (envelope-from eugene@imedia.ru) From: Eugene Mitrofanov Organization: Independent Media Sanoma Magazines To: freebsd-fs@freebsd.org, freebsd-questions@freebsd.org Date: Fri, 21 May 2010 15:55:59 +0400 User-Agent: KMail/1.9.10 X-Origin: badger.imedia.ru MIME-Version: 1.0 Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <201005211555.59622.eugene@imedia.ru> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.0.1 (lynx.imedia.ru [10.167.0.252]); Fri, 21 May 2010 15:56:00 +0400 (MSD) X-Virus-Scanned: clamav-milter 0.96-exp at lynx.imedia.ru X-Virus-Status: Clean Cc: Subject: FreeBSD 8.1-PRERELEASE: property 'jailed' not supported on FreeBSD: permission denied X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Eugene Mitrofanov List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 May 2010 12:13:32 -0000 Hi The command "zfs set jailed=on tank/s1" is failed with the message " property 'jailed' not supported on FreeBSD: permission denied". Output of "zfs get jailed tank/s1" shows me that the property "jailed" is still exists: NAME PROPERTY VALUE SOURCE tank/s1 jailed off default How can I change its value? Thanks. -- EMIT-RIPN, EVM7-RIPE From owner-freebsd-fs@FreeBSD.ORG Fri May 21 15:25:32 2010 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 15579106566C; Fri, 21 May 2010 15:25:32 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 6E8C08FC0C; Fri, 21 May 2010 15:25:31 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AvsEAHhC9kuDaFvI/2dsb2JhbACeH3G9bYUSBA X-IronPort-AV: E=Sophos;i="4.53,278,1272859200"; d="scan'208";a="77237375" Received: from darling.cs.uoguelph.ca ([131.104.91.200]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 21 May 2010 11:25:29 -0400 Received: from localhost (localhost.localdomain [127.0.0.1]) by darling.cs.uoguelph.ca (Postfix) with ESMTP id A50DC9400AA; Fri, 21 May 2010 11:25:29 -0400 (EDT) X-Virus-Scanned: amavisd-new at darling.cs.uoguelph.ca Received: from darling.cs.uoguelph.ca ([127.0.0.1]) by localhost (darling.cs.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SwdGfZijh47V; Fri, 21 May 2010 11:25:28 -0400 (EDT) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by darling.cs.uoguelph.ca (Postfix) with ESMTP id A14C5940174; Fri, 21 May 2010 11:25:28 -0400 (EDT) Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id o4LFeoa27571; Fri, 21 May 2010 11:40:50 -0400 (EDT) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Fri, 21 May 2010 11:40:50 -0400 (EDT) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: John Baldwin In-Reply-To: <201005200922.17245.jhb@freebsd.org> Message-ID: References: <201005191144.00382.jhb@freebsd.org> <201005200922.17245.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Rick Macklem , Robert Watson , fs@freebsd.org Subject: Re: [PATCH] Better handling of stale filehandles in open() in the NFS client X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 May 2010 15:25:32 -0000 On Thu, 20 May 2010, John Baldwin wrote: > > It doesn't change the RPC count because of changes that Mohan added to the > NFS client a while ago so that nfs_open() doesn't invalide the attribute > cache during nfs_open() if it was already updated via nfs_lookup() during > the same system call. With Mohan's changes in place, all this change does > is move the GETATTR/ACCESS RPC earlier in the case of a namecache hit. > I tried another kernel build with and without the patch and an about 6% increase in Getattr RPCs seems real. (The 1% increase in Access RPCs was not related to the patch and appears to have happened because I didn't do an unmount/mount between runs, which results in RPC cnts changing by up to 1%.) I have no idea whether or not an approx. 6% increase in Getattr RPCs for this case is enough of a concern w.r.t. patch that it needs to be looked at further? Btw, I'm doing the tests on a pretty recent -current kernel which appears to do shared vnode locks for read opens, which might explain why the Getattr cnt changes? (You're patch had quite different line #s, so I assume it wasn't against -current?) Have fun with it, rick From owner-freebsd-fs@FreeBSD.ORG Fri May 21 15:51:10 2010 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D3F511065670; Fri, 21 May 2010 15:51:10 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 8F83C8FC1C; Fri, 21 May 2010 15:51:10 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 24F2446B95; Fri, 21 May 2010 11:51:10 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id EECAF8A025; Fri, 21 May 2010 11:51:08 -0400 (EDT) From: John Baldwin To: Rick Macklem Date: Fri, 21 May 2010 10:39:04 -0400 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <201005191144.00382.jhb@freebsd.org> <201005200922.17245.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201005211039.04108.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Fri, 21 May 2010 11:51:08 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.4 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Rick Macklem , Robert Watson , fs@freebsd.org Subject: Re: [PATCH] Better handling of stale filehandles in open() in the NFS client X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 May 2010 15:51:11 -0000 On Thursday 20 May 2010 8:45:13 pm Rick Macklem wrote: > > On Thu, 20 May 2010, John Baldwin wrote: > > > > > It doesn't change the RPC count because of changes that Mohan added to the > > NFS client a while ago so that nfs_open() doesn't invalide the attribute > > cache during nfs_open() if it was already updated via nfs_lookup() during > > the same system call. With Mohan's changes in place, all this change does > > is move the GETATTR/ACCESS RPC earlier in the case of a namecache hit. > > > Well, it sounds like a good theory. Something like: > - VOP_LOOKUP() locks the vnode, which is then passed to VOP_OPEN() and > since it is locked, other threads can't perform VOPs on the vp. No, it's not the lock, it's this thing Mohan added here in nfs_open(): struct thread *td = curthread; if (np->n_ac_ts_syscalls != td->td_syscalls || np->n_ac_ts_tid != td->td_tid || td->td_proc == NULL || np->n_ac_ts_pid != td->td_proc->p_pid) { np->n_attrstamp = 0; KDTRACE_NFS_ATTRCACHE_FLUSH_DONE(vp); } Which used to be an unconditional clearing of n_attrstamp so that the VOP_GETATTR() in nfs_open() would always go over the wire. Now it doesn't clear the attributes if the attribute cache was last updated during the same system call that is invoking nfs_open() by the same thread. Hmm, however, concurrent lookups of the same pathname may cause this test to fail and still clear n_attrstamp since we now use shared vnode locks for pathname lookups. That was true before my change as well. In fact, using shared vnode locks for read-only opens (the MNTK_EXTENDED_SHARED flag) probably does cause Mohan's patch to not work in many cases which probably accounts for the small increase in RPC counts. Try setting the vfs.lookup_shared sysctl to 0 to see if that removes all the extra RPCs. Another change would be to readd the change to flush the attribute cache on close and see if that also brings back extra attribute/access RPCs without my patch (but with lookup_shared left enabled) to verify that shared lookups are the root cause of extra RPCs. > I ran a single pass of a kernel "make cleandepend; make depend; make" > here (1 without the patch and one with the patch). Now, it could be > random variation, but since the other RPC counts changed by < 1%, I > suspect not. (I'll run another pass of each to see how much variation > I see w.r.t. Getattr.) > > Here's the counts for the 5 RPCs I think might be interesting: > RPC Count without patch Count with patch > Getattr 590936 625987 +5.9% > Lookup 157194 157528 > Access 59040 59690 +1.1% > Read 70585 70586 > Write 112531 112530 > > I let you know what another pass of each gives, but it looks like > it has caused an increase in RPC cnts. I don't know if the increase > is enough to deter adding the patch, but it might be worth exploring > it more? > > rick > > -- John Baldwin From owner-freebsd-fs@FreeBSD.ORG Fri May 21 21:03:00 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BCE30106566C; Fri, 21 May 2010 21:03:00 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (unknown [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 8D56B8FC16; Fri, 21 May 2010 21:03:00 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o4LL30KK008932; Fri, 21 May 2010 21:03:00 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o4LL30nT008928; Fri, 21 May 2010 21:03:00 GMT (envelope-from linimon) Date: Fri, 21 May 2010 21:03:00 GMT Message-Id: <201005212103.o4LL30nT008928@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/146708: [ufs] [panic] Kernel panic in softdep_disk_write_complete X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 May 2010 21:03:00 -0000 Old Synopsis: Kernel panic in softdep_disk_write_complete New Synopsis: [ufs] [panic] Kernel panic in softdep_disk_write_complete Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Fri May 21 21:02:10 UTC 2010 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=146708 From owner-freebsd-fs@FreeBSD.ORG Sat May 22 07:05:14 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 33510106566B; Sat, 22 May 2010 07:05:14 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id C670E8FC12; Sat, 22 May 2010 07:05:13 +0000 (UTC) Received: by gyh20 with SMTP id 20so1024891gyh.13 for ; Sat, 22 May 2010 00:05:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:openpgp:content-type:content-transfer-encoding; bh=arz6aK5SGOnix8KjRswa4/vpdsN/SKxDMbrzuX67ZS4=; b=CZEz1pQPB0ZUpcmVJkydZM+JfcVxtdZfxmuaq7gJg0+F6jaydH3/uhl+Qd+1YVMbrD aHh6X4ZHZnko1TyQFio2nXPdBep+fkIV+PlNNGwI5wX0e2hVBykK5frLjgiuUc7veOYP GGtemKqIQNVK7X1WInlSqjy7AZy2qsiOvuCrY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:openpgp:content-type :content-transfer-encoding; b=WP1anXODVnO13R2TF/3tEFLCFZ67/3msr45/BoI9n/L355jxcjwuzbchz3kgRPqZlp iZ/AOSD8DjP2W/4aDjvpzsoc4uPIIxeH5q81/P34xVKS7+KvELL4+uTHCyeREP4aaIO1 cUjVFieT+SVLNBufhfxWQWwUe0QZUDpwJv2NQ= Received: by 10.150.183.11 with SMTP id g11mr3665766ybf.66.1274511912934; Sat, 22 May 2010 00:05:12 -0700 (PDT) Received: from centel.dataix.local (adsl-99-181-129-134.dsl.klmzmi.sbcglobal.net [99.181.129.134]) by mx.google.com with ESMTPS id r27sm26206721ybc.1.2010.05.22.00.05.11 (version=SSLv3 cipher=RC4-MD5); Sat, 22 May 2010 00:05:12 -0700 (PDT) Sender: "J. Hellenthal" Message-ID: <4BF78226.6020403@dataix.net> Date: Sat, 22 May 2010 03:05:10 -0400 From: jhell User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1.9) Gecko/20100515 Thunderbird/3.0.4 MIME-Version: 1.0 To: Eugene Mitrofanov References: <201005211555.59622.eugene@imedia.ru> In-Reply-To: <201005211555.59622.eugene@imedia.ru> X-Enigmail-Version: 1.0.1 OpenPGP: id=89D8547E Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, freebsd-questions@freebsd.org Subject: Re: FreeBSD 8.1-PRERELEASE: property 'jailed' not supported on FreeBSD: permission denied X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 May 2010 07:05:14 -0000 On 05/21/2010 07:55, Eugene Mitrofanov wrote: > Hi > > The command "zfs set jailed=on tank/s1" is failed with the message " > property 'jailed' not supported on FreeBSD: permission denied". > > Output of "zfs get jailed tank/s1" shows me that the property "jailed" is > still exists: > NAME PROPERTY VALUE SOURCE > tank/s1 jailed off default > > How can I change its value? > > Thanks. Simply put, property 'jailed' not supported on FreeBSD. Some features that you may see in a "zfs get all pool" will not work because they are not implemented yet or are not planned to be implemented because they are too *Solaris dependent. See jail(1) for setting up a jail on FreeBSD. -- jhell From owner-freebsd-fs@FreeBSD.ORG Sat May 22 23:40:42 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0FCFF106564A; Sat, 22 May 2010 23:40:42 +0000 (UTC) (envelope-from alex.bakhtin@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3FB668FC0A; Sat, 22 May 2010 23:40:40 +0000 (UTC) Received: by fxm4 with SMTP id 4so2277654fxm.13 for ; Sat, 22 May 2010 16:40:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:cc:references :in-reply-to:subject:date:message-id:mime-version:content-type :content-transfer-encoding:x-mailer:thread-index:content-language; bh=xVpHHIJ34RbvSdQ0J8gyVZdPf5hoeQN511tZ50u2nBg=; b=awtKjlyibNbrtbYms/jk33HfHL2bkxPaAjS/smOuYgnLPKvMRR1Qq4Ndx2rZooW9P6 t7FVXNoW3sELSrFq4lJSg6FHyHoJNodw3NxRBwt8FryqSmltmYiWtkYQrrVcwyRN1UJ4 gv+/6CU+Mpai4LwgP3W+UXPnck0KI/1kgptOk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:cc:references:in-reply-to:subject:date:message-id :mime-version:content-type:content-transfer-encoding:x-mailer :thread-index:content-language; b=eTyzbwtsDInLZZNC0FSGZq/3ju2kYVgLhJCuOHX9FmTSSnlYaiib2+oDDlbtwz3ZWt Fd+6xuJoJVs2lZLRzgWlPA30VOYxulKEuYsQBcBjW8r/Rq8aT/l3vbBZ1EspUTJo2uVL aZp+Bsj9Q9+XgiktPOv9uJOjr9hy5b8ZD/Gcw= Received: by 10.223.144.77 with SMTP id y13mr3226494fau.86.1274571640085; Sat, 22 May 2010 16:40:40 -0700 (PDT) Received: from BAKHTINNB (nateneshi.static.corbina.ru [85.21.246.62]) by mx.google.com with ESMTPS id z12sm11696663fah.21.2010.05.22.16.40.38 (version=SSLv3 cipher=RC4-MD5); Sat, 22 May 2010 16:40:39 -0700 (PDT) From: Alex Bakhtin To: References: <201005130934.o4D9YiJL039462@freefall.freebsd.org> In-Reply-To: Date: Sun, 23 May 2010 03:40:38 +0400 Message-ID: <4bf86b77.0c3fdf0a.45ff.4f6b@mx.google.com> MIME-Version: 1.0 Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: Acr11r3SaNvSr8JYQISoBtL7GpzldgELaWQQ Content-Language: ru Cc: freebsd-fs@freebsd.org, bug-followup@freebsd.org Subject: RE: kern/145339: [zfs] deadlock after detaching block device from raidz pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 May 2010 23:40:42 -0000 Pawel, I made some additional testing. Now I'm 95 percent sure that this deadlock was introduced by this patch. I tried patched and non-patched GENERIC kernel. It seems that it is harder to reproduce this deadlock on raidz1 than raidz2. With raidz2 I tried to detach and reattach back two = disk at the same time, and deadlock is 100% reproducible on patched kernel = but I can't reproduce it on non-patched kernel. How to reproduce: 1. Create raiz2 pool 2. Detach two devices while the pool is idle. 3. Start writing to the pool (dd if=3D/dev/zero of=3D/storage/test = bs=3D1m) 4. atacontrol detach/attach to have disks back. 5. Online two disks at the same time (zpool online storage adX adY). 6. Wait some time (in my testing - several seconds, less than one = minute) - all disk activity would be stopped. After that it's impossible to abort zpool online or dd command. Also it's impossible to reboot without hard-reset. If you need core from deadlocked kernel please let me know. Alex Bakhtin -----Original Message----- From: Alex Bakhtin [mailto:alex.bakhtin@gmail.com]=20 Sent: Monday, May 17, 2010 7:37 PM To: pjd@freebsd.org Cc: freebsd-fs@freebsd.org; bug-followup@freebsd.org Subject: Re: kern/145339: [zfs] deadlock after detaching block device = from raidz pool Pawel, I tested your patch in the following zfs configuration (all on 5x2TB WD20EARS drivers): 1. raidz1 on top of physical disks. 2. raidz1 on top of geli 3. raidz2 on top of physical disks. In all three cases it seems that the problem was fixed - I can't crash zfs in vdev_geom when unplugging the disk. Unfortunately, 3 times I got a deadlock in zfs after plugging vdevs back under load. It happens several seconds after zpool online command. I'm not 100 percent sure that deadlocks are related to this patch, but... I'm going to make some additional testing with patched and not patched kernels. 2010/5/13 : > Synopsis: [zfs] deadlock after detaching block device from raidz pool > > State-Changed-From-To: open->feedback > State-Changed-By: pjd > State-Changed-When: czw 13 maj 2010 09:33:20 UTC > State-Changed-Why: > Could you try this patch: > > =9A =9A =9A = =9Ahttp://people.freebsd.org/~pjd/patches/vdev_geom.c.3.patch > > It is against most recent HEAD. If it is rejected on 8-STABLE, just = grab > entire vdev_geom.c from HEAD and patch this. > > > Responsible-Changed-From-To: freebsd-fs->pjd > Responsible-Changed-By: pjd > Responsible-Changed-When: czw 13 maj 2010 09:33:20 UTC > Responsible-Changed-Why: > I'll take this one. > > http://www.freebsd.org/cgi/query-pr.cgi?pr=3D145339 >